Policy WindowAI, evidence & public choice
Menu
Human by choice
Policy Window advocacyWiki · neutral reference ↗

What should a human check actually change?

Human by choice · Policy Window’s advocacy programme

Prepared 12 November 2026

A person checking an AI recommendation can catch an error, hear an objection or explain a decision. Each is a different job. Before relying on a promise of human review, we need to know which job the person can actually perform-and what would show that the arrangement helps.

Consider a fictional repairs service. Software suggests which requests should receive priority. A staff member checks the suggested list before it goes ahead. That description leaves important questions unanswered. Does the person check addresses, reconsider urgency, ask the resident for missing information or have permission to change the order? What happens if they find a problem they cannot resolve?

Keeping a person involved can serve purposes beyond getting more answers right. A service may value an accountable explanation, a chance to contest a decision or a relationship with someone who understands the circumstances. Those purposes deserve their own tests. An accuracy result alone cannot establish that people have a usable route to be heard.

Three studies, different questions

This is a deliberately small comparison of three selected studies, not a survey of all human-review research. Their tasks and measures differ. The point is to identify what each arrangement tested before drawing a lesson for a different workplace.

Better than which alternative?

Vaccaro and colleagues’ 2024 meta-analysis included 74 papers covering 106 experiments published from 2020 to June 2023. Studies had to report performance for people alone, AI alone and their combination. On average, the combinations outperformed people alone but underperformed the better solo option. Tasks and results varied greatly; the selected evidence and possible publication bias limit generalisation. This is a finding about the performance measures studied, not a prediction for every workplace or an answer about the value of being heard. Primary study.

Screening with an assisted workflow

PRAIM’s 2025 report compared AI-supported double reading with standard double reading in German mammography screening, analysing 461,818 women. Its adjusted cancer-detection estimate was 17.6% higher in the AI-assisted group (95% confidence interval for the relative increase: 5.7% to 30.8%). For recalls for further assessment, the confidence interval’s upper limit stayed below the study’s 10% increase margin; a reduction was not established. Radiologists chose the viewer, and visible AI tags affected selection. One feature could ask a reader to reconsider a scan initially judged unsuspicious, but its effect was not isolated. Vara funded and participated in the study. Detection does not establish a mortality benefit. This observational comparison did not test a fully automated service against the assisted one. Primary study.

Changing how advice arrives

Buçinca and colleagues’ 2021 author manuscript analysed 199 online participants choosing meal substitutions with simulated AI advice. Advice appeared after an initial answer, on request, or after a 30-second wait. Grouped together, these designs produced better decisions on wrong-advice questions than immediate-explanation designs. But both AI-assisted categories did worse than no AI on those questions. Across all questions, both AI categories outperformed no AI, and their difference was not significant. The forcing designs were rated more complex than immediate-explanation designs. The study establishes neither that answering first is the best design nor that its findings generalise to workplace decisions. Author manuscript.

Two checks with different jobs

Return to the invented repairs service. Its existing manual route asks staff to read requests and assign a priority. Now consider two possible changes. Neither describes a real service, a tested design or an existing legal entitlement.

In arrangement A, software proposes the priority and staff verify that the request has the right address and has been entered correctly. They can correct those details. They cannot change the priority within this step. A separate merits review might exist elsewhere; this example leaves that question open.

In arrangement B, software proposes the priority, but staff also examine the resident’s original account. Within explicitly assigned powers, they can seek clarification and alter the priority. They can refer a disagreement outside those powers to someone who can decide it. The service must provide time and resources for that work.

A performs a useful clerical check. If the public promise is that a person reassesses urgency, the service needs to show where that further work happens. Giving the first check a broader name would leave the resident with an assurance the described process cannot fulfil.

B creates an opportunity for a substantive change. It can also introduce mistaken judgments, inconsistent treatment and delays. Seeking clarification may disadvantage someone who cannot respond easily. Extra checking can consume resources that would otherwise support repairs. These are reasons to compare the arrangements carefully; no performance benefit is assumed here.

Common Wealth’s 2024 essay on democratising AI distinguishes participation from governing power. Apply that distinction here: a reviewer may change an individual priority while having no power over the criteria or the repair budget. A resident may supply information while having no say in those rules. Giving staff an override button does not itself give workers control over their jobs or residents authority over the service. A proposal to share power must identify those further decisions and who can make them.

An override count cannot settle the question

Here is a separate invented example. In one set of 100 cases, every recommendation is correct and a reviewer agrees with all of them. In another, every recommendation is wrong and a reviewer again agrees with all of them. The agreement rate is identical. The quality of the decisions is completely different.

The example is arithmetic, not evidence about a real tool. It shows why agreement alone cannot reveal whether a reviewer is exercising useful judgment. The same problem applies to counting disagreements: a changed answer can correct an error or introduce one. There is no preferred override rate in this proposal.

Records of agreement and change can still identify cases worth examining. To judge those cases, the organisation needs a defensible account of what counts as a good outcome. Some questions have a reliable reference answer. Others involve incomplete information, conflicting objectives or a judgment about whose needs take priority. An assessment should expose those choices rather than treat an existing score as unquestionable truth.

Look beyond the changed cases. What happened when the recommendation was accepted, when a request never reached the reviewer, or when a person stopped pursuing it? A procedure that works well for persistent, well-resourced users may place a heavier burden on everyone else. Assessing a review arrangement means deciding whose experience counts as evidence.

The next decision belongs to the service

For an organisation considering such a change, our proposal is a specific decision before making a public promise: the responsible service owner and budget holder should agree what the reviewer is expected to change, what authority and resources support that task, and what evidence would make them alter or discontinue the arrangement. These roles must be identified in the actual institution; this paper grants them no new powers.

Compare credible alternatives. Improving the existing manual process may be cheaper or more dependable. A clearly automated step, where permissible and appropriate, may avoid a misleading promise of substantive human assessment. A different assisted workflow may preserve a valued conversation while reducing clerical work. The choice depends on the task, its consequences and the purposes people want the service to serve.

Use existing records if they answer the question. Where they do not, specify the missing evidence and the cost of obtaining it. Do not require a new form merely to demonstrate activity. Changes to the process should consider staff workload, access, waiting time and the distribution of errors, alongside the outcome the tool is designed to improve.

Workers and affected users can help identify failures that a performance measure misses. Their actual involvement needs its own arrangements and authority. An internal analysis supplies neither their views nor their consent; a small discussion would not represent everyone affected.

The difficult objection remains: checking may cost more than the errors it prevents. Human work can be valuable for several reasons, but those reasons need to be stated and weighed. If a service retains a person for answerability or a relationship, it should examine whether people can actually obtain those benefits. If it promises error correction, it should examine the errors. If it claims shared power, it should show which decisions people can control.

That is a more useful promise than the presence of a person somewhere in the process: a defined human contribution, supported by real authority, with consequences that can be questioned.

References