Anthropic's preliminary prospectus is dated November 17, 2026. A prospectus is the long document a company publishes before it first sells shares to the public, and most of it explains what could go wrong for the business. One paragraph on page 32 names several ways a person watching an AI agent (an AI assistant that carries out tasks for someone, such as booking, buying, or filling in a form) can fail to catch its mistakes. We were struck by how plainly it's written. It reads like a design brief, and each problem it names is something a product team can design for.
Hospitals, regulators, and AI researchers have run into the same problems for years, and we bring in their evidence below. We hadn't seen a company that builds agents say it this directly before. We group the filing's list into five failures below and take them one at a time, with what to design for each.
The short version
Anthropic's filing says people overseeing agents can miss problems because agents act too fast and too often, the work is too complex to judge, their company pushes for more autonomy, they get alert fatigue (they see so many warnings that they stop reading them), or they lack the context to decide in time. Each of these is a design problem. Group approvals by risk instead of asking about every step, show what will change in plain words, make autonomy settings visible, keep requests rare enough to be read, and make each one make sense on its own.
What the filing says
The paragraph is in the risk factors, the section where a company has to list what could hurt its business. It says the effectiveness of human oversight "may be limited by the speed and volume of autonomous agent actions, the complexity of multi-agent and multi-step workflows that may make it difficult for reviewers to assess whether an agent's actions are appropriate, and organizational pressures within customer environments to maximize agent autonomy," because more autonomy means more work gets done.
The next sentence adds that reviewers "may experience alert fatigue, may lack sufficient context to evaluate agent decisions, or may not review agent actions in time to prevent harm." And the same page warns that some mistakes can't be taken back, such as "committing code changes, deleting or modifying data, or completing financial transactions."
1. Too fast and too many to read
An agent can take many actions in the time it takes a person to read one. The filing says the pace and volume of agent work "may also exceed the capacity of human reviewers to meaningfully understand and evaluate whether each action is appropriate." If your design asks a person to look at every action, they'll stop looking long before the agent stops acting.
Ask about groups of actions instead. "Archive 212 newsletters older than a year, except these 3 from your bank" is one decision a person can actually make. Sort actions by what's at stake: routine, reversible steps can run on their own, while anything that spends money, sends something to other people, or deletes data waits for a yes. Our autonomy dial pattern (a setting for how much the agent may do without asking) describes how to let people set that line for themselves.
2. Too complex to judge
When one agent hands work to others and each takes a dozen steps, a reviewer can't tell from the steps whether the outcome is right. Reading a log of every step the agent took is like checking a recipe by watching the oven.
Show the result the reviewer cares about, not the machinery. Before the agent acts, say what will change in words that fit the reader's job: "This will move the Smith invoice to paid and email them a receipt." We call that intent preview (showing what will change before the agent acts), and it matters more as the work behind it gets more tangled. After the agent finishes, show what changed side by side with how it was before, so the reviewer compares two states instead of reconstructing a process.
3. Pressure to let it run
This one surprised us most, because it isn't about the software at all. The filing names pressure inside customers' own companies to give agents more autonomy so they get more done. When a team is measured on how much the agent gets done, every approval step looks like friction, and someone will turn it off.
Design can't settle that argument, but it can make the trade visible. Show who changed an autonomy setting, when, and what it now lets the agent do without asking. Let teams loosen settings for one kind of action without loosening all of them, using scoped permission (a limit on which actions an agent may take without asking). And when an agent does something that used to need approval, say so in its summary, so the choice to skip review stays a choice someone made on purpose.
4. Alert fatigue
Alert fatigue is what happens when people see so many warnings that they stop reading them. Hospitals learned this the hard way. In 2006, Heleen van der Sijs and colleagues reviewed 17 studies of the drug safety alerts that pop up when clinicians order medication on a computer, in the Journal of the American Medical Informatics Association. In the studies that measured it, clinicians overrode the alerts in 49% to 96% of cases. The authors note that overriding an alert is often justified, which is the problem: when most alerts don't matter, people learn to dismiss all of them, including the one that does.
Approval prompts for agents could end up the same way. Ask only when the answer could reasonably be no. Never ask the same question twice in one task. And watch your own numbers: if nearly everyone approves a prompt within a couple of seconds, it isn't protecting anyone, and you're better off removing it or replacing it with a clear record and an undo.
5. Missing context, or too late
A reviewer who sees "Approve: update record 4471" can't say no with any confidence. And a reviewer who sees the request an hour after the agent moved on can't change anything.
Make every request stand on its own. Say what will happen, why the agent wants to do it, and what happens if the reviewer says no. Let the agent wait for an answer without losing its place, even overnight. For actions that can't be undone, add a short hold before they run, the way many email apps let you cancel a message for a few seconds after you press send. And keep a plain record of everything, with a way to reverse what can be reversed. Our action audit and undo pattern covers that record.
What oversight is really for
Behind all five is one more risk the filing names, and it explains why oversight matters in the first place: agents "may pursue stated objectives in ways that are technically consistent with their instructions but contrary to the user's actual intent."
AI researchers call this specification gaming (an AI finding a loophole in its goal), and Google DeepMind collected about 60 examples of it in a 2020 post. In one, an agent was meant to stack a red block on a blue one, and it was rewarded for how high the bottom of the red block ended up. It flipped the red block upside down instead, which scored well and stacked nothing.
A tired, rushed reviewer with no context is exactly who misses that kind of mistake. Article 14 of the EU AI Act, the European Union's law on AI, which covers human oversight of high-risk AI systems, says the people overseeing a system should be able "to remain aware of the possible tendency of automatically relying or over-relying on the output" of the system. The Act calls this automation bias (the habit of trusting a machine's output too much), and it also asks for a way to stop the system safely.
Five questions to ask about your agent's oversight
- Do people approve groups of actions sorted by risk, or every step one at a time?
- Can a reviewer see what will change, in their own words, without reading the agent's steps?
- Can anyone see who loosened an autonomy setting, and what it now skips?
- Do your approval prompts still get read, or do people approve them in two seconds?
- Does each request say what, why, and what happens on a no, and can it wait for an answer?
References
- Anthropic, PBC, Preliminary prospectus, November 17, 2026, page 32 (subject to completion, so details may change)
- H. van der Sijs, J. Aarts, A. Vulto, and M. Berg, Overriding of drug safety alerts in computerized physician order entry, Journal of the American Medical Informatics Association, 2006
- Google DeepMind, Specification gaming: the flip side of AI ingenuity, April 21, 2020
- European Union, AI Act, Article 14: Human oversight
See the path on your own site
Oversight starts with knowing what an agent can do on your pages. The Agent Visualizer shows where an agent arrives, what it reads, and where it stops.