The oversight model
AI does the work.
People make the call.
Human in the loop is not a checkbox at the end of a workflow. It is a design decision made for every step: how much the agent may do alone, when it must stop, and who is accountable for the result.
Work climbs the autonomy ladder step by step. At the top step, the handover waits for a person.
01 · Suggest
The agent drafts. A person decides and acts.
02 · Approve
The agent prepares the exact action. A person confirms before anything happens.
03 · Act and notify
The agent acts within limits. A person is told and can undo.
04 · Act alone
The agent acts. People review a sample and watch the numbers.
05 · Always human
Some decisions never leave people: hiring, firing, legal positions, anything you would not delegate to a new hire.
Approval limits must be enforced by the system outside the model. An agent told to ask permission in its prompt can be talked out of it; an agent that cannot execute without a signed approval cannot.
01 / Map decisions
List every decision the agent touches.
Oversight starts with an inventory. Break each workflow into the decisions inside it, then ask what happens if each one is wrong, who notices, and how easily it can be undone.
Impact and reversibility
Score every decision on two axes: how much harm a mistake causes, and whether it can be reversed. A draft email and a sent payment sit at opposite corners.
Who owns it today
Record which role makes this decision now and what they check. That person's judgement is what the agent must either match or hand back to.
A procurement workflow splits into eleven decisions. Nine are low-impact and reversible; two (supplier onboarding and payment release) are not.
Every decision in scope has an impact score, a reversibility rating and a named human owner.
02 / Autonomy levels
Give each decision a level.
Autonomy is not on or off. Assign each decision one of a few clear levels, so everyone knows exactly what the agent may do alone and where it must stop.
Four levels
Suggest (agent drafts, person acts). Approve (agent prepares the exact action, person confirms). Act and notify (agent acts, person is told and can undo). Act alone (agent acts, sampled review).
Start low, earn trust
New workflows start at Suggest or Approve. Move a decision up a level only when measured accuracy and override rates justify it, and write down who approved the change.
Refunds under 100 USD move from Approve to Act and notify after eight weeks with a 1% override rate. Refunds above 100 USD stay at Approve.
The autonomy level for every decision lives in configuration, is versioned, and cannot be changed by the agent itself.
03 / Approval design
Make approvals meaningful, not ceremonial.
An approval is only useful if the reviewer can actually judge what they are approving. Rubber-stamp approvals add delay without adding safety.
Proposals stop at the gate. Each is released only after the person above it approves.
Show the exact action
Present the precise change: amount, recipient, record, wording. Bind the approval to that exact action so any edit afterwards requires a new approval.
Give the reasons
Show the evidence the agent used, its confidence, what policy applies and what it is unsure about. A reviewer should be able to decide in one screen.
A supplier payment approval shows the invoice, the matched purchase order, the bank details change history and a flag that the account is new.
Approvals expire, are single-use, and the system rejects execution if the action differs from what was approved.
04 / Escalation
Design when the agent asks for help.
The most valuable thing an agent can do when unsure is stop and ask. Escalation rules decide which cases reach a person, and how fast.
Most requests pass straight through. Uncertain ones rise to a person and return resolved.
Clear triggers
Low confidence, missing information, conflicting sources, policy exceptions, angry or vulnerable customers, and anything outside the agent's tested scope.
Route with context
Send the case to the right person with a summary, what the agent tried and what it needs. Never make the person start over.
A customer mentions a bereavement. The agent stops, tags the case as sensitive and routes it to a senior agent with the conversation summarised.
Escalated cases are measured for time to response and outcome, and the escalation rate is tracked per trigger.
05 / The reviewer's screen
Design for the person, not the agent.
Human oversight fails when reviewers are overloaded or bored. The review interface determines whether people stay sharp or start clicking approve on everything.
Fight approval fatigue
Batch similar low-risk items, prioritise by impact, and occasionally insert known test cases to check reviewers are reading.
One-click outcomes
Approve, edit, reject with reason, or escalate further. Capture the reason every time; it is the most valuable training signal you have.
Reviewers see high-impact items first, with a weekly cap on items per person. Planted test items reveal one reviewer approving without reading.
Average review time, override rate and planted-item catch rate are visible per reviewer and per workflow.
06 / Override & stop
People can always stop the agent.
Oversight means control, not just visibility. Anyone accountable for a workflow must be able to pause, undo or take over at any moment.
Pause and kill switches
Pause one case, one workflow or the whole agent instantly. Pending actions are cancelled, not queued for later.
Undo paths
For every action at Act and notify, define how it is reversed and by whom. If it cannot be reversed, it does not belong at that level.
An operations lead pauses the invoice agent during a supplier dispute. All pending payments are held and the team takes over manually.
The pause control is tested on a schedule, and the time from pause to no new actions is measured.
07 / Accountability
A named person owns every outcome.
Agents do not carry responsibility. Every decision the agent makes or prepares maps to a person or role who is accountable for it.
Ownership map
Each workflow has an owner who sets its autonomy levels, reviews its performance and answers for its outcomes.
Complete records
Log what the agent proposed, who approved or changed it, when, and why. Regulators and customers will ask.
A complaint about a declined claim traces to the agent's draft, the reviewer who confirmed it and the policy version applied.
Any action can be traced from outcome back to proposal, evidence, approver and policy version in minutes.
08 / Learning loop
Every correction improves the system.
Human decisions are data. Edits, rejections and escalations show exactly where the agent is wrong and where autonomy can safely grow.
Corrections from the reviewer circle back into the agent, so each pass is better than the last.
Capture and review
Group corrections by reason each week. Fix prompts, data or tools for common patterns and add the cases to the evaluation set.
Adjust autonomy both ways
Raise a decision's level when it consistently performs, and lower it immediately when override rates rise or the environment changes.
Rejections cluster around one product line with a new returns policy. The policy is added to the agent's sources and that decision drops back to Approve until rates recover.
Override rate per decision is reviewed on a fixed schedule, and every autonomy change is logged with its evidence.
Worked example / finance operations
Choose the right level.
Three illustrative decisions from one finance workflow show how impact and reversibility set the level. Reference examples, not a production policy.
- Decision
- Assign an expense category to a receipt.
- Risk
- Low impact, easily corrected at month end.
- Level
- ACT ALONE / sampled weekly review
- Control
- 5% of categorisations are reviewed each week; override rate above 3% drops the level back to Act and notify.
# Illustrative autonomy policy, not deployable configuration
workflow: accounts_payable
owner: finance_lead
default_level: suggest
categorise_expense: { level: act_alone, review_sample: 5% }
send_reminder: { level: act_and_notify, undo_window: 30m,
key_accounts: approve }
release_payment: { level: approve, bind: exact_action,
above_10000: two_approvers }
change_bank_details: { level: escalate, to: finance_lead }
escalate_when: [low_confidence, missing_data, policy_exception]
on_policy_missing: suggest_and_askWhen a decision is not in the policy, or the policy cannot be loaded, the agent prepares and asks. Unknown never means allowed.
Before production
Ten questions before agents act.
Use these when designing or reviewing any agent workflow. Attach the owner and evidence to each answer.
0 / 10 reviewed in this session
Further reading
Go deeper.
This guide draws on public guidance on human oversight of AI. Requirements differ by sector and jurisdiction; check what applies to you.
Technical guidance, not a certification or a claim that these practices are already in place at your organisation.
Keep people
firmly in charge.
Bring a workflow. We can help map its decisions, set autonomy levels and design the approval and escalation flow.