Human oversight of AI is designed process by process: you decide what a person reviews, at what point, with what information and what they can do if they disagree. In sales and operations, that means placing the checkpoint before the result leaves the company and giving it an owner with time and authority.
This article is part of AI-First hybridisation consultancy.
In short
- Overseeing means being able to understand, correct and stop the agent, not glancing at it now and then.
- Each checkpoint answers four questions: what, when, with what information and with what power.
- You start by reviewing everything and move to samples or exceptions once the corrections allow it.
- In sales, a person reviews each message before it goes out.
- Oversight fails for lack of time, authority or information, hardly ever because of the technology.
What human oversight of AI is
Human oversight of AI means that a specific person can understand what an agent does, correct it and stop it. It has three parts: seeing, deciding and acting. Take one away and what is left is a person watching a screen.
In company hybridisation with AI, each process is divided between agents and people. Oversight is the seam between the two halves, and it is best drawn before the agent is built. Added afterwards, it tends to end up as an «approve» button nobody reads.
The four design decisions
Each checkpoint is defined by answering four questions:
| Decision | Usual options | What works best |
|---|---|---|
| What is reviewed | Everything, a sample or only the exceptions | Everything at first; less once corrections become scarce |
| When | Before the result goes out, or afterwards | Before, if it reaches a client or counts in the accounts |
| With what information | The result alone, or the result with the rule and the data used | Always with the rule that was applied in view |
| What the person can do | Approve, correct, send back or stop the agent | All four, without asking anyone's permission |
The fourth row is the one most often neglected. A person who can only approve or wait is not overseeing anything. They have to be able to correct and, if need be, halt the process.
Three ways to review
Not every process needs the same intensity of review. There are three ways, and you can move from one to another:
- Approve first: the person sees every result and nothing goes out without their sign-off. It is always how you start, and how you stay when an error is expensive.
- Review a sample: the agent works and the person reviews part of the output, picked without looking. It suits a stable process where an error can be put right without harm.
- Review exceptions: the agent sets aside whatever its rules do not cover and the person sees only that. It needs rules that say clearly when to set a case aside.
You move from the first to the others when corrections become rare and always for the same reason. At that point the reason is already a rule, and you write it down. If corrections climb again, you go back to approving first.
How to design it in sales
In sales, the agent's output reaches someone outside the company, so the checkpoint comes before sending. The usual division is this:
- The agent finds companies that match the ideal client. The person reviews the list and removes the ones they do not want to contact.
- The agent drafts the first message. The person reads it, changes it if needed and decides whether it goes out.
- The agent flags when someone replies. The person handles the conversation and the meeting.
- Prices, terms and promises are not delegated: a person always gives them.
This is how the Legathum platform works: a person reviews every message before it is sent, and email is never sent cold.
How to design it in operations
In operations, the result stays inside the company, but it often counts: an invoice recorded, an order confirmed, a report someone will use to decide. The checkpoint comes before that moment.
Take an accountancy practice, an example made up for this article. The agent sorts the invoices clients send in and sets aside those that arrive without a tax number. The head of administration confirms the batch before it goes to the accounts and deals with the ones set aside. For the first few weeks she reviews everything. After that, only the ones set aside and a sample of the rest.
The logic is the same for management reports. The agent prepares the data and flags what falls outside the plan. What to do about each alert is decided by a person.
What makes oversight fail
When oversight does not work, it is nearly always for one of these five reasons:
- The overseer has no time. The review was added to their day without taking anything away.
- They have no authority. They see the error and cannot change the rule or stop the agent.
- They approve out of habit. After a hundred correct results, the next one is approved unread. The EU AI Act calls this automation bias.
- They cannot see the rule. Without knowing what judgement the agent applied, they can only give an opinion on the result.
- Corrections are not written down. The same error is fixed by hand again and again.
The first two are solved when the work is assigned. The last three, when the screen the person reviews on is designed.
What the EU AI Act asks for
For high-risk systems, the EU AI Act says two things about oversight. Article 26 requires it to be assigned to people with the necessary competence, training and authority. Article 14 requires the system to let them understand its capacities and limitations, remain aware of automation bias, disregard or reverse its output and interrupt it safely.
Those obligations concern the uses listed in Annex III, such as selecting candidates for a job, and apply from 2 December 2027. As a checklist they are useful for any agent. They are EU law; other jurisdictions have their own rules.
How to tell whether oversight is working
Four figures, noted from the first day, tell you whether a checkpoint is doing its job:
- How many results are corrected, and for what reason.
- How long each case takes to review.
- How many cases the agent sets aside, and how many of them were right to be set aside.
- How many times the process has been stopped, and what was changed afterwards.
If nobody ever corrects anything, either the agent is perfect or nobody is reading. The second is more common. These figures are compared with the baseline of the process, which we explain in how to measure a process before automating it.
Where to start
By writing down, for a single agent, who reviews, what they review, when, and what they can do. If any of the four answers is blank, the agent should not be working on its own yet. The division of work by area is in AI agents for SMEs.
In the Legathum AI-First Hybridisation Method, each agent has a specification with its judgement, its limits and its human oversight, and nothing is built without it. That is part of the AI-First hybridisation consultancy. In an AI-first company, each checkpoint has a person responsible for it, with the training and authority to correct the agent.
Frequently asked questions
What is human oversight of AI?
It means a specific person can understand what an AI agent does, correct it and stop it, with the time and authority to do so.
Does everything an agent does have to be reviewed?
At first, yes. Once corrections become rare and always for the same reason, you can move to reviewing a sample or only the exceptions.
Where does the checkpoint go in sales?
Before sending. A person reviews each message before it goes out and handles the conversation when the client replies.
Who should oversee an agent?
The owner of the process: a named person who knows the work, has time to review and can correct or stop the agent.
How do I know whether oversight is working?
By noting how many results are corrected and why, how long the review takes and how many cases the agent sets aside. If nothing is ever corrected, the likeliest explanation is that nobody is reading.
Where do I start?
By writing down who reviews, what, when and what they can do, for a single agent. If you want to look at it with your own company's case, the first step is a 15-minute meeting with Deibson Silva.
Sources
- Regulation (EU) 2024/1689 (AI Act), Article 14 (human oversight, paragraph 4). Consolidated text, European Commission AI Act Service Desk.
- AI Act, Article 26 (obligations of deployers of high-risk AI systems, paragraph 2). AI Act Service Desk.
- AI Act, Article 113 (entry into force and application). AI Act Service Desk.
- AI Act, Annex III (high-risk AI systems). AI Act Service Desk.