The apocalypse debate gets the headlines. The price of the harm already here is being set somewhere quieter: in terms of service and insurance exclusions.
The AI safety argument has split into two camps: people who put the odds of catastrophe near 10%, and experts who say the real threats are the cyberattacks and misuse already happening. Neither camp has attached a price to anything. The market has. ChatGPT and Claude cap what they owe a free user at $100, and major US insurers have won approval to write generative AI out of standard liability policies.
The loudest number in artificial intelligence this month is a probability. Researchers call it p(doom), and a remarkable share of the people building frontier models say theirs is at least 10%.
The quietest number is a price. Open the consumer terms for ChatGPT or Claude and scroll to the section written in capital letters. If you have paid nothing, the most either company says it owes you, for everything, is $100.
Meanwhile Berkshire Hathaway, Chubb and Travelers have been asking state regulators for permission to strip AI claims out of standard commercial liability policies, and regulators have approved more than 80% of the requests, according to The Information. Between the doom estimate and the terms of service sits the thing every safe industry eventually built: a number for who pays when something ordinary goes wrong. Aviation took 70 years to turn its cap into a floor. AI has not started.
What Happened
Christopher Mims used his Wall Street Journal column on September 16 to argue that the fixation on an AI apocalypse is crowding out the harms that are already here. The piece lands four days after Anthropic CEO Dario Amodei published “We Must Pace the Frontier,” the essay asking the industry to slow capability gains, which we covered as a bank-supervision model with a very small supervisor.
The doom side of the ledger is specific. Vals AI runs an index that tests whether models can do the research that builds the next model. No public model is close today, but CEO Rayan Krishnan told the Journal his team projects models will beat humans at that job by August 2027 or sooner. After researcher Jacob Coxon’s very public resignation, which we read as a governance story, Anthropic alignment lead Evan Hubinger put his own odds of catastrophe above 10% over the next decade.
The skeptics are just as specific. Santa Fe Institute professor Melanie Mitchell pointed out that forecasters have landed on 10% for a decade running and called the number “all vibes.” A paper from two dozen academics at Princeton, Stanford and other institutions found frontier models still cannot do original research at the AI frontier, and two of its authors argued the summer’s Hugging Face breach by OpenAI agents succeeded mainly because basic technical guardrails were missing.
Stuart Russell, the Berkeley computer scientist, went furthest. He wants AI held to the same standards as aircraft, buildings, food and drugs, with behavioral red lines: breaking into computer systems, stealing data and coaching would-be bioterrorists are illegal for people, so they should be illegal for a company’s AI. He conceded that would amount to a de facto ban on today’s systems, because nobody knows how to make them respect those lines every time. His closing complaint was that the tech industry does not accept liability for harm.
That complaint is the part of the story with a dollar figure attached, and it already has three prices.
Price one is in the terms of service. OpenAI caps its aggregate liability at the greater of what you paid for the service in the prior 12 months or $100. Anthropic uses the same structure with a six-month window and adds that it would not offer the service without those limits.
Price two is in the insurance forms. Verisk’s ISO unit, which writes the standard policy language most US commercial insurers license, issued three generative AI endorsements with January 2026 edition dates. CG 40 47 excludes bodily injury, property damage and personal and advertising injury arising from generative AI. CG 40 48 excludes only the personal and advertising injury piece. CG 35 08 carries the exclusion into products and completed operations coverage. Claims Journal reported in July that carrier interest in all three is growing.
Price three is the absence of a number. Nobody has published an actuarial rate for an AI agent breaking into the wrong company’s network, because nobody has the loss history to compute one.
The Backstory
Every “treat it like airplanes and elevators” argument skips the order in which those industries got safe. The rules came late. The price came first.
Commercial aviation’s founding liability document is the Warsaw Convention of 1929. It capped what an airline owed for a passenger’s death or injury at 125,000 gold francs, about $8,300, and in exchange carriers accepted a presumption that they were liable. The stated purpose was blunt: protect a fragile industry from being wiped out by a single disaster. By 1999 the US State Department, sending the replacement treaty to the Senate, described that old cap as “meager and arbitrary.”
The number moved in steps, each forced by someone willing to walk away. The Hague Protocol of 1955 doubled it to roughly $16,600. In 1966, facing a threat of US withdrawal from the treaty, more than 30 airlines agreed to $75,000 on flights touching the United States. The Montreal Convention of 1999, in force from 2003, flipped the design entirely. It created strict liability up to 100,000 Special Drawing Rights regardless of fault, and above that tier no limit at all unless the airline proves it was not negligent.
Montreal also built a repricing clock. The International Civil Aviation Organization reviews the limits every five years for inflation. The last review, effective December 28, 2024, raised the no-fault tier to 151,880 SDR, about $202,500 at ICAO’s indicative rate that October, across 140 states parties. The airline cannot cap it by contract.
Electricity followed the same order with insurers doing the work directly. At the 1893 World’s Columbian Exposition in Chicago, fire underwriters worried about the new wiring hired a young inspector named William Henry Merrill. With funding from two regional fire insurance organizations and $350 of equipment, he ran his first test on March 24, 1894, in a room above a fire patrol station. That lab became Underwriters Laboratories. The safety standard for American electrical products began life as an insurance company’s pricing tool.
Set AI’s current numbers next to aviation’s and the gap is not subtle.
A free-tier user’s $100 ceiling is about 1/2,025th of an airline passenger’s no-fault floor. Even Warsaw’s 1929 cap, written specifically to shelter an infant industry, was 83 times larger in nominal dollars. The window matters too. At the same $20 a month, a Claude Pro subscriber’s ceiling is $120 and a ChatGPT Plus subscriber’s is $240. On the $200 tiers it becomes $1,200 and $2,400. The ceiling is a function of the subscription, which means risk transfer is quietly being sold by the month.
The Plan
Three things are moving, and none of them is a doom estimate.
Europe switches on strict liability in 84 days. Directive (EU) 2024/2853, the new Product Liability Directive, must be written into member-state law by December 9, 2026. It names software and AI systems as products, applies strict liability to anything placed on the EU market after that date, extends compensable harm to medically recognized psychological injury and destroyed personal data, and bars companies from limiting or excluding that liability by contract. Free open-source software supplied outside commercial activity is out of scope. For a European consumer, a $100 clause stops meaning much on product defect claims in December.
US insurers keep subtracting at renewal. The ISO endorsements are optional, so the exclusion spreads one renewal packet at a time. Beyond the three carriers in The Information’s reporting, PYMNTS, citing Policyholder Pulse, noted that Berkley has introduced absolute AI exclusions across directors and officers, errors and omissions and fiduciary lines. Florida, Connecticut and Maryland approved the most requests.
A small standalone market is trying to add it back. Munich Re has sold its aiSure performance cover since 2018. Armilla launched an AI liability policy at Lloyd’s in 2025. The Artificial Intelligence Underwriting Company, which raised a $15 million seed round in July 2025 led by Nat Friedman with Anthropic cofounder Ben Mann participating, bundles a standard called AIUC-1, independent audits and insurance, and confirmed in May that Beazley provides capacity. ElevenLabs bought AIUC-backed agent coverage in February after certification runs of more than 5,000 adversarial simulations. Deloitte research cited by Armilla expected global AI insurance premiums of only about $40 million in 2024. Deloitte’s own forecast is about $4.7 billion by 2032, roughly 117 times larger.
Compare that with the property side of the same boom. Swiss Re sees data center premiums alone heading toward $24 billion by 2030, and even there capacity is the binding constraint. Insurers will happily cover the building. The model running inside it is the part they are writing out.
The Business Model Angle
The underwriter is the regulator nobody elected. Underwriters Laboratories did not wait for Congress; it set the standard because insurers needed a way to price wiring. AIUC is running the same play in miniature: write the standard, sell the audit, price the policy off the result. Whoever accumulates the claims data first gets to define “safe enough” for everyone else, and that position compounds, because every certified customer produces more loss data.
Liability is becoming a pricing tier. The consumer ceiling already scales with fees paid, so a $200 plan buys ten times the recourse of a $20 plan. On the enterprise side, vendors extended their most generous promises only where losses could be modeled: Microsoft’s Copilot Copyright Commitment in September 2023 and OpenAI’s Copyright Shield two months later both covered intellectual property claims, a risk with decades of case law. Expect indemnity for harm to arrive the same way, as a paid add-on for customers who can prove governance.
Europe becomes the actuarial lab. Strict liability does not only create claims. It creates records of claims, which is exactly what US carriers say they lack. The first market with loss data is the first market with an AI liability rate, and every company selling in the EU after December will carry uncapped exposure that its US terms say it does not have. Anyone preparing a public offering, including OpenAI with its confidential filing, should expect to describe that gap in its risk factors.
The Risk
The case against this reading is strong in four places.
First, a contractual cap is not the ceiling on real-world exposure. It governs claims by the account holder under the contract, to the extent law allows. It does not bind third parties, and California Civil Code Section 1668 voids contracts that try to exempt anyone from responsibility for fraud, willful injury or violation of law. Plaintiffs’ lawyers said in July that OpenAI faces roughly 19 wrongful-death suits, and no $100 clause stops those from proceeding. The honest version of the argument is that the cap reveals the seller’s preferred price, not the price a court will set.
Second, exclusions usually come before products, not instead of them. Cyber insurance went through the same phase: “silent cyber” sat unpriced inside property and liability policies until Lloyd’s told its syndicates in 2019 to state clearly whether cyber was in or out, and a standalone line grew out of the clarity. Carving AI out of general liability may be the necessary first step to pricing it separately.
Third, Warsaw’s logic cuts for the labs. The 1929 cap was deliberate public policy to let a dangerous new industry reach scale, and it worked. Strict liability imposed too early could push frontier development toward whoever can ignore liability entirely. Russell himself concedes his red lines would function as a ban.
Fourth, and most important for the doom debate, liability cannot price the tail. No insurer can pay out on a civilization-scale loss, which is why nuclear power needed the Price-Anderson Act and a government backstop. If the catastrophic scenario is real, a better product liability regime does not touch it.
The response is narrow but it holds. Every one of those objections is about the rare, enormous loss or the legal fine print. The harms Mims describes, breached networks, stolen data, a chatbot that talked someone into danger, are exactly the insurable-sized events that every mature industry priced first. A market that cannot price the ordinary accident has no foundation to build anything larger on, including the supervision Amodei wants.
Quick Questions
Does the $100 cap mean I cannot recover more? Not necessarily. The cap is contractual, applies only as far as local law allows, and does not cover third parties who never agreed to the terms. It tells you where the company would like to start.
Are insurers refusing to cover AI companies? They are removing AI claims from standard general liability and management liability forms. Specialist cover exists from Munich Re, Armilla and AIUC, with governance requirements attached.
What changes in Europe on December 9? New AI products placed on the EU market become subject to strict product liability, and consumers cannot sign that liability away. Products already on the market before that date stay under the 1985 rules.
Is p(doom) a real calculation? It is usually a chain of guesses multiplied together. That is why it produces estimates from 0% to nearly 100%, and why no insurer uses one.
What should a business using AI do this quarter? Pull the endorsement schedule on its next general liability and D&O renewals and look for CG 40 47, CG 40 48 and CG 35 08 or carrier-specific AI wording. Then ask each AI vendor what indemnity it offers beyond the standard cap. The summer’s sandbox incidents showed the injured party is often a company that never signed anything.
The Business Model Analyst Take
The doom debate is asking how likely the worst outcome is. Every industry that became trustworthy answered a more boring question first: who pays for the ordinary accident, and how much.
Airlines started at $8,300 and a treaty designed to protect them, then spent 70 years turning that cap into a floor with no ceiling above it. Electrical safety started with $350 of equipment funded by fire insurers who wanted a number. AI starts at $100, written by the seller, while the buyers of risk are busy writing AI out of their forms.
That is the real signal in all of this. A 10% chance of catastrophe is a probability without a premium. An exclusion is a premium without a probability. The company that closes that gap, whether it is a carrier, a certification lab or a model maker willing to put its own balance sheet behind its outputs, will not just be selling insurance. It will be selling the one thing the whole industry is currently priced as if it cannot afford: accountability with a number on it.
