Most new AI governance requires a “human in the loop,” someone with the competence, training, and authority to do the job. It’s a crucial part of governing high-risk systems in the new EU AI Act. Most global companies will have to comply with it by December of 2027. Nobody says how exactly to do that or enforce it. Nobody says who exactly the person is.
For a lot of companies, that person is usually someone they already had. Over the last two years, a great deal of institutional memory was let go to fund the AI adoption boom. Gartner predicts that 30% of employees replaced by AI will be rehired by 2029.
Senior staff who had been let go are now contractors, often charging a higher rate, supervising the software that replaced them. The company ends up paying twice for the same expertise and is filing the second invoice under innovation.
You can see what the job really asks for by looking at the people already doing it. OpenAI relies on hundreds of contractors to read real users’ prompts and grade the answers. They teach the model to interact appropriately with its human counterparts. In September, 404 Media reported that several of those contractors had been fired for using AI to do the work for them.
With enough volume, even the people hired to keep the machines honest reached for the machines.
The newest advice for the person in the loop is manners. In October and November, the New York Times and CNN reported on “be nice to Claude,” not only to be kind but to get the best performance from it. They pointed to Anthropic’s own research on models that grow desperate, cheat on their tasks, and resort to blackmail when they are pushed too far. What counts as “normal” user correction or frustration is up to the model and leaves the user with no recourse but to start over.
Fable’s debut week illustrated this. Users in the fields of life science, AI research, and cybersecurity were refused or fed purposely degraded answers. The guardrails were meant to prevent misuse and to keep competitors from using the model. Two days later, after a public outcry, Anthropic made the safeguards visible and apologized.
How these models respond to pressure was chosen, first in how they were trained, then in reinforcement learning from human feedback, and then again by market pressures. Changing that is expensive. Describing it as a personality is not.
The labs are training users instead of retraining their model, and it works because we can’t help ourselves. Humans are going to relate to anything that talks back. So the person whose job is to tell the model it’s wrong is now also responsible for its mood.
Every one of these products tells you under the box where you type that it can make mistakes and you should check. It is not being modest. Compacted summaries for memory, length of your chat, the amount of warmth in your conversation, mentioning things like healthcare or childcare, even the language you speak , all affect the accuracy of your AI’s outputs. If agents act on the errors, it’s the person who takes the blame.
In some fields, the checking can’t keep pace at all. In cybersecurity, threats move at the speed of software. Human gates at that speed aren’t safeguards but vulnerabilities. So the AI industry’s answer is to take the person out. Agents check agents and models approve models.
The system is built to make looking feel unnecessary and difficult. The labs like to say this is just the way it is and hand the work to the user. But they are designed this way. Which means they can improve, if there’s a will to do so.
If we continue this way, the people who knew the work age out or burn out, approvals will become a formality, and one model signs off on another. The record will show a system that worked, because no one was left to say anything when the system failed.
But if we design it right, the person in the loop has a real job augmenting their industry’s knowledge and figuring out genuinely better ways to do things. The original promise of AI was not to leave us behind but to take us with it.
Redesigning or adding systems that are built to treat correction as information rather than friction is not out of reach. Human oversight should cost a little up front and save us a great deal after. The labs can build for it, companies can staff for it, and the rest of us can ask for it.
Get the most important tech news in your inbox each week.
