Human Oversight

Decouple Review Actions

Separate “Mark as reviewed” from “Send”, so approval is a real review and not one click.

Phase: Design · Build
Applies to: End users
Oversight: Human in the loop
Key factor: Oversight capability

Overview When to use Questions to ask Example Anti-pattern Anti-patterns to pay attention to References Evidence Related

Overview

“Approve & Send” combines two acts, checking the work and releasing it, so the check becomes a formality. However, for high-consequence decisions, decoupling Review from Approve or Send is strategic and safe, ensuring oversight is not performative and becomes documented review.

When to use

Use when

Users approve AI-prepared results that then go out: customer replies, reports, case decisions, letters, etc.

Not needed when

Low-stakes internal drafts where review is optional. The extra step should match the consequence.

Questions to ask

Ask these at kickoff, when design work starts. Tick them off as your team answers.

Which AI outputs or results go out after approval?
Which decision or output could have potential negative impacts without proper and calibrated oversight?
Which items in it get flagged for review?
Where is the record of who reviewed what, and when?

Example

A claims reviewer finishes an AI-drafted decision letter. Each flagged item is marked as reviewed (or corrected) on its own, and Send unlocks only when all of them are done.

Claim decision letter · ReviewDo

☑ Coverage datesReviewed

☑ Deductible amountCorrected

☐ Exclusion clause citedNot reviewed

Mark as reviewedSend letter · 2 of 3 reviewed

See it in context · full screen

Claims workspace · Decision letterIn context

Decision letter · Claim 55120

Your policy covered the period 1 Mar 2025 to 28 Feb 2026.

A deductible of $500 applies to this claim.

Water damage is excluded under section 7(c) of your policy. 1

1Each flagged item shows where it sits in the letter, so the reviewer reads it in context.
2Items are marked reviewed, or corrected, one at a time. That's the review step.
3Send is a separate step and stays locked until every flagged item is done.
4The record shows who reviewed which item, and when, not just that someone clicked approve.

Anti-pattern: Acknowledgment fatigue

An “OK” prompt on every single output. Within a day, people click it without reading.

Every outputDon't

This content was generated by AI.

Anti-patterns to pay attention to

A single “Approve & Send” action.
AI output can be sent while flagged items are unopened.
The record shows an approval but not what was reviewed.

References

Current guidelines · EU AI Act, ISO/IEC 420013 references

For orientation only, not legal advice. Applies to high-risk systems. EU AI Act links open a readable version of the official text published on EUR-Lex; ISO/IEC 42001 is a paid standard.

EU AI Act Art. 14(1) Systems must be designed so people can effectively oversee them, not just nominally approve them.
EU AI Act Art. 26(2) Deployers assign oversight to people with the necessary competence, training and authority.
ISO/IEC 42001 A.6.2.8 Decide which events are logged to support audit; review steps are among them.

Evidence

Survey & researchSurvey insights, respondent quotes, my research

Survey insightA related idea came from survey respondents. One described it in detail:

“Have a visually arresting/contrasting colour button at the beginning and end of any analysis created using AI and have the user have to click to verify they know the degree of accuracy (or “confidence score”)… that will prevent them from moving on unless they click that they are aware…”
Survey respondent, open-ended answer (lightly trimmed)

From my researchProposed in my paper as a way to turn performative approval into documented review.

The survey is still open. Add your perspective ↗