AI Repair Triage for Remote Public Housing

Separate the evidence from the simulation

Before the team could model the problem, they hit a simple constraint: there was no public Northern Territory work-order dataset they could use. Without real historical work orders, they couldn't train and test the system on the kind of repair records a live service would generate. So they created a year of synthetic repair requests.

Some inputs were real, though. The geographic and operational context included 70 communities, house counts, crew hubs, road closures, airstrips, and response-time data. The repair requests and tenant behaviour were simulated. That split matters. Synthetic examples won't necessarily match how people describe faults, and they won't reflect how work orders get recorded day-to-day.

The demo labels its synthetic requests as synthetic rather than pretending they're real cases. That's a product decision as much as a documentation note. If a tool is built on simulated data, it should be clear where a user is interacting with a model that hasn't seen the live world.

Define fairness at the point of ranking

The urgency model had one rule from the start: queue position should depend on the fault, not the person reporting it. A repair shouldn't move up simply because a tenant called rather than used a form, wrote a longer description, or communicated in more fluent English.

That sounds obvious, but it changes what the system needs to learn and what has to be tested. Language models and text classifiers can treat wording, vocabulary, or writing style as if it were severity. A short report can describe something dangerous; a polished report can describe something routine. So the system needs a way to infer urgency from the reported condition without turning someone's ability to describe it into a proxy for priority.

Why the confidence score stayed hidden

The model's ranking performance looked stronger than its calibration. When it reported confidence of at least 90%, it was correct only 56% of the time in the project's evaluation. A 90% figure can look reassuring to a tenant waiting for a repair, but the measured results don't support using that number as a straightforward certainty statement.

So tenants don't see a confidence percentage. Instead, the product gives a plain-English explanation of why the repair is waiting. This isn't an argument that explanations can never work, or that probabilities are always useless. It's about this particular probability: it wasn't reliable enough to display as if it were a trustworthy estimate.

A misleading confidence score can cause harm in two different ways. It can lull someone into thinking the decision is solid when the model is actually uncertain. Or it can create unnecessary anxiety when the number is misunderstood. A dashboard might benefit from a neat percentage, but the person affected by the decision needs information the system can defend.

A model can rank cases usefully and still be unfit to present its own certainty.

That's why calibration belongs in evaluation - especially when outputs are shown to people who are affected. A model can score well on ranking metrics and still produce overconfident estimates. If the product has no honest use for that confidence, hiding it is better than presenting a precise number that isn't dependable.

Keep human review in the safety design

In the demo, 42% of dangerous reports are routed to a human review step before a decision is finalised. The project reports that the complete system - including human review of flagged reports - catches 99.3% of dangerous cases. That difference matters: it describes the system as a whole, not the model acting alone.

Human review isn't a magic fix. It adds workload, and it only helps if reviewers have enough context, time, and authority to question the model. But it gives the workflow a path for cases that shouldn't be left to an automated score. For a safety-critical setup, the real question isn't just whether a model can classify a report. It's what happens when it gets things wrong, and whether someone can intervene before that mistake turns into real delay.

Combine batching with priority rules

Grouping jobs can reduce travel costs, but it also creates a risk: a "convenient" trip might end up pushing an urgent repair later. ReachNT keeps the two decisions connected, but not fused. Batching decides which communities can be covered together. Urgency decides the order of repairs inside each trip.

That separation makes the trade-off explicit. The system can look for efficiencies without letting a routine repair jump ahead of an urgent one just because it happens to fit a route. In short: batching helps save money; ranking decides order. Neither should quietly override the other.

Use the solver to prepare a plan, not own the decision

Weekly schedules are generated with a CP-SAT solver. In the project's evaluation, it produced 99.5% of 26,018 weekly plans as optimal, with a median solve time of 8 milliseconds. The solver optimises cost and fault-days while keeping urgency order as a constraint.

The speed is useful, but it isn't what makes the plan safe. Each weekly plan still needs sign-off from a named coordinator before it goes out. The solver builds schedules from synthetic requests and scheduled distances. A coordinator can know things the model doesn't represent - local access changes, current constraints, and practical considerations that come from being on the ground. And the approval step gives the service a person who can explain why a tenant's repair waited.

That approval only matters if the reviewer can inspect and change what the system proposes. A signature attached to an opaque recommendation can become process theatre. The workflow should be designed so plan review is real and the coordinator can challenge the output.

Treat synthetic metrics as a starting point

The demo's numbers show how the design behaves under simulated conditions. They don't prove the model will make the same predictions on real tenant reports, or that a live service would achieve the simulated costs and repair timelines.

There are three practical reasons to stay cautious:

• The repair requests are synthetic. Real tenants may describe faults differently, and that can change model performance.
• The confidence analysis is based on simulated text. Calibration should be rechecked against genuine work orders before anyone relies on the estimates.
• Travel distances and access are simplified. Flooded roads, seasonal conditions, and local routing decisions can change both cost and feasibility of shared trips.

The team suggested a pilot in the Katherine hub's 16 communities, precisely because it would test some of those assumptions. Before expanding, a real pilot would need to measure how reports are written, how often reviewers override the model, whether planned routes match actual travel, and how waiting times change for different kinds of repair.

Put safety review into multiple layers

A multi-agent AI audit found a bug in the demo: a dangerous life-support repair could be downgraded based on a photo alone. A later reassessment based on an image could potentially change a high-severity repair to routine, which would undermine the system's core priority rule.

The fix was applied at three levels. The portal blocks that downgrade path in the interface, the server validates the change, and the database preserves the original severity so a later edit can't quietly erase it. The underlying lesson for safety-critical software is simple: if you only enforce a safety property in the interface, another client or direct request can bypass it. Safety properties shouldn't depend on one layer always behaving perfectly.

Adversarial review helps for exactly this reason. A system can look coherent while still containing a failure path its builders didn't anticipate. The useful review work is in the flows where an error could change safety or priority - not just the screens that behave correctly in the "happy path." Catching this issue before a tenant encountered it was one of the most consequential outcomes of the project.