A new study suggests the most enthusiastic adopters of “AI employees” may be quietly undoing the efficiency they paid for. The reason is not the technology. It is the way humans stop watching it.
Over the past two years, a wave of companies began treating AI agents as genuine staff, giving them names, peer status, and in some cases a box on the organizational chart. Human resources leaders have pitched the practice as a productivity unlock and a signal that the company is operating on the frontier. The problem, according to researchers, is that the framing changes how managers behave, and not for the better.
Managers stop checking the work
Emma Wiles, a Boston University professor who studies AI and labor, ran an experiment with three collaborators from Boston Consulting Group across dozens of companies that use AI employees. Managers were handed five flawed documents and 20 minutes to catch as many errors as possible. Some were told a human produced the work, some an AI tool, and some an AI “employee.”
For most managers, the stated source barely moved the needle. But at companies that formally list AI agents on the org chart, managers caught noticeably fewer errors when they believed an AI employee had done the work. Wiles’s read is that these managers no longer felt the mistakes were theirs to own. If something slipped through, that was a problem for the tech team or the executives who installed the AI in the first place.
That is a meaningful break from how management normally works. When a human subordinate errs, the manager assumes the hit lands on them, so they scrutinize the output. When an AI tool errs, managers still feel on the hook. But once the AI becomes a named “colleague,” accountability appears to evaporate. The supervision layer that catches mistakes goes with it.
The framing is already widespread
This is not a fringe practice. In a survey of more than 1,000 corporate managers, about a third said their organization refers to AI as a teammate or employee, and nearly a quarter said their employer puts AI agents on the org chart. One manager described an agent as a technical peer on the team, with its own name.
The business-model implication is uncomfortable. The entire case for AI agents rests on cheaper, faster output. But if the org-chart framing strips out the human review step, defect rates climb invisibly, and the cost of those defects lands later and harder. You do not see the bill on the productivity dashboard. You see it in the shipped error, the mispriced deal, the bad hire.
The hidden biases stacking up underneath
The supervision gap is only the most visible flaw. Researchers are surfacing others that most corporate users have not registered.
One is anti-human bias. A 2025 paper in the Proceedings of the National Academy of Sciences found that several large language models systematically rate AI-written text above human-written text. A follow-up study found that AI résumé screeners favor applications written with AI help over those written entirely by a person. Companies racing to automate hiring may be selecting for “wrote it with a chatbot” rather than for talent.
Another is cold game-theory logic. When AI models are asked to set a price or pick a location, they tend to assume rivals and customers are perfectly rational, which real humans are not. That can push a company toward aggressively undercutting a competitor and triggering a price war that leaves everyone worse off. Humans, left alone, more often look for the win-win.
A third is homogenization. Researchers who lean on AI at every step of a creative or analytical process report that, individually, the output feels sharp, but collectively everyone starts sounding the same. If tomorrow’s models train on today’s model output without care, the sameness compounds into a self-reinforcing loop. This is the same value-capture and differentiation problem we have tracked in the Stanford 2026 AI Index breakdown: when everything converges, the moat disappears.
The fix is cheap, if you know the flaw exists
The encouraging part is that several of these defects are correctable once named. Researchers reduced anti-human bias simply by instructing the model to judge the quality of the writing and ignore who wrote it. The org-chart accountability gap could be closed by holding managers explicitly responsible for the work of their AI reports, the same way they are for human ones.
The catch is the “unknown unknowns.” You cannot patch a bias you have not detected, and the field is detecting them in real time, well after companies have wired the tools into hiring, pricing, and operations. The productivity payoff that vendors promise keeps arriving later than the spending, and undetected defects are one reason the gap persists.
There is also a next chapter already forming. At the same conference where Wiles first heard executives praise their AI employees, one said her company would soon have AI agents managing humans. If a named AI colleague is enough to make managers stop checking its work, it is worth asking what happens to accountability when the AI is the one signing off.
Frequently Asked Questions
Why do managers catch fewer errors when AI is on the org chart?
Because the framing shifts who feels responsible. When an AI agent is labeled an “employee,” managers tend to treat its mistakes as someone else’s problem, usually the tech team’s, so they review the output less carefully. When the same work is attributed to a plain AI tool or a human, managers still feel on the hook and scrutinize it more closely.
How many companies actually treat AI as an employee?
In a survey of more than 1,000 managers, about a third said their organization refers to AI as a teammate or employee, and nearly a quarter said their employer lists AI agents on the company org chart. The practice is mainstream among early adopters, not a fringe experiment.
What is anti-human bias in AI?
It is the tendency of large language models to rate AI-generated work above human-generated work. A 2025 study found this pattern in text evaluation, and a follow-up found AI résumé screeners favoring applications written with AI assistance over those written entirely by a person, which can distort hiring toward tool use rather than skill.
Can AI really start a price war?
It can nudge a company toward one. AI models often assume competitors and customers behave with perfect rationality, so they may recommend aggressively undercutting a rival even when that risks a mutually damaging price war. Humans left to their own judgment more often pursue win-win outcomes.
How do you fix the AI accountability gap?
The cheapest fix is to assign a specific human owner for every AI agent’s output and hold that person responsible for its errors, exactly as they would be for a human report. Researchers have also reduced anti-human bias by instructing models to judge work on quality alone and ignore who produced it.
The Business Model Analyst Take
The lesson here is not “slow down on AI.” It is that the org-chart label is a behavioral lever, not a neutral piece of HR housekeeping. Calling an agent an “employee” feels like a status upgrade for the technology. In practice it can quietly switch off the human review that makes the output trustworthy, and the savings you booked turn into deferred liabilities.
The operators who win with agents will treat them like high-output but unproven hires: full speed on volume, zero relaxation on review. Name the agent whatever you like. Just make sure a human still owns its mistakes, because the moment nobody does, the productivity story starts running in reverse.
