In short: The cost of an AI hallucination is rarely the wrong answer itself - it is the commitment you are then held to, and businesses have already paid it in tribunal rulings, clawed-back fees and seven-figure write-offs.
A customer messages your support chat at 11pm asking whether they can return an opened product. The bot answers immediately and confidently: yes, within 60 days, full refund, no restocking fee.
Your actual policy is 14 days, unopened only.
The customer screenshots the reply. By morning it is in your Instagram comments and three more people want the same deal. Nobody wrote that policy. Your AI produced it in one fluent, entirely reasonable-sounding sentence - and you are probably going to honour it, because arguing costs more than the refund.
That is the part most coverage of AI hallucinations misses. The cost is almost never the wrong answer. The cost is the commitment you are then held to.
We have already covered why AI hallucinates, forgets and argues, and the engineering fixes for it. This piece is the invoice. Every figure below comes from a ruling, a regulator, a vendor’s own system card or a published contract value.
1. You own what your bot says - this is settled, not theoretical
In November 2022, Jake Moffatt booked a flight with Air Canada after his grandmother died. The airline’s website chatbot told him he could claim a bereavement discount retroactively, within 90 days of travel. Air Canada’s real policy required approval before flying.
When the airline refused and offered a coupon instead, Moffatt took it to the British Columbia Civil Resolution Tribunal. On 14 February 2024, in Moffatt v. Air Canada, 2024 BCCRT 149, the tribunal found the airline liable for negligent misrepresentation and awarded CA$650.88 plus interest and fees.
The award is small. The reasoning is not. Air Canada argued that the chatbot was a separate legal entity responsible for its own actions. Tribunal member Christopher Rivers called that a remarkable submission and rejected it:
While a chatbot has an interactive component, it is still just a part of Air Canada’s website. It should be obvious to Air Canada that it is responsible for all the information on its website.
Note what did not save the airline. The correct policy was published elsewhere on the same site. The tribunal held that a customer was entitled to rely on the answer the chatbot gave, and a link to the real terms two clicks away was not a defense.
The fix
Policies, prices and entitlements are retrieved, never generated. The model’s job is to phrase what a lookup returned, not to recall it. If the lookup returns nothing, the correct behavior is a handoff, not a guess. A lookup only helps if the records behind it agree with each other, which is why so many confident wrong answers turn out to be a data problem rather than a model problem.
The objection we hear most
“Ours only handles simple FAQs, so this does not apply to us.” Air Canada’s chatbot was also only handling FAQs. Bereavement fare eligibility is an FAQ. The questions that get a company into trouble look identical to the ones it deployed the bot for, which is why the boundary has to sit in the architecture rather than the job description.
2. The invented policy that cost real customers
In April 2025, users of the coding tool Cursor started being logged out when they switched machines. They asked support, and the AI agent explained this was expected under a one-device security policy. There was no such policy - a co-founder had to say so publicly. The logouts were a session-handling bug; the AI invented a policy to explain them, and some subscribers canceled on the strength of it.
No cancellation figure was ever published, so we will not invent one. The lesson is the mechanism: the bot did not fail on an exotic question. It filled a gap in its own knowledge with a plausible story, and that story became the company’s official position. A hallucinated policy is worse than “I do not know” precisely because it sounds like it came from somewhere.
3. Sometimes the cleanup costs more than the tool ever saved
New York City launched an official chatbot for small business owners in October 2023. In March 2024, The Markup logged it telling businesses things that were illegal to act on: that landlords could refuse Section 8 vouchers, that restaurants could go cash-free, and - asked whether an employer could take a cut of workers’ tips - “Yes, you can take a cut of your worker’s tips.”
The city did not take it down. It added disclaimers and narrowed the questions it would answer. On 30 January 2026 the incoming administration killed it, calling it functionally unusable. Reporting put the build at roughly $600,000, with around $500,000 a year to run.
Cost of getting it wrong
Roughly $1.1m spent, disclaimers bolted on, tool switched off, nothing salvaged. The hallucinations themselves cost nothing directly. The two years of patching, defending and finally writing off a tool nobody trusted is where the money went.
This is the pattern in small and mid-sized deployments too. The first wrong answer is cheap. What gets expensive is the permanent tax that follows: a human quietly re-checking everything the AI produces, a growing list of topics the bot is forbidden to discuss, and a team that no longer believes the output. At that point you are paying for the tool and for the work it was meant to remove.
4. Courts have started pricing hallucinations per error
The clearest sign this has moved from embarrassment to budget line is in litigation, where fabricated case citations became routine enough to be tariffed.
The origin case is Mata v. Avianca: on 22 June 2023 a New York federal judge sanctioned two lawyers and their firm $5,000 over a brief containing six entirely fabricated citations, complete with plausible docket numbers and invented quotations.
It scaled from there. Norton Rose Fulbright reported in June 2026 that a public database tracking these incidents had logged over 1,148 US cases. In December 2025 the Oregon Court of Appeals adopted a per-infraction schedule: $500 for every fabricated citation and $1,000 for every fabricated quotation. In March 2026 the Sixth Circuit sanctioned two attorneys $15,000 each, plus opposing costs, over more than two dozen nonexistent or incorrect citations.
Then there is the version that hits a supplier. Deloitte Australia produced a report for the Department of Employment and Workplace Relations under a contract worth around A$440,000. After more than a dozen references turned out not to exist - including a fabricated quotation from a Federal Court judgment - the firm agreed in October 2025 to repay the final instalment, reported at around A$97,000. Its revised report disclosed that a generative AI tool chain had been used.
A client clawed back a fee because the deliverable contained invented sources. If you sell reports, analyzes, proposals or advice, that is your risk model, not a legal-sector curiosity. Generated images carry a version of the same exposure, which is why checking image licensing before you publish matters.
5. Why do AI models make things up instead of saying “I do not know”?
Because confident guessing scores better than admitting ignorance. Most benchmarks reward a correct answer and treat a refusal the same as a wrong one, so a model that guesses outscores a model that abstains. The behavior is trained in, not accidental.
Artificial Analysis’s AA-Omniscience benchmark (16 November 2025) measures this directly, applying no penalty for declining to answer. On its first run, all but three tested models were more likely to hallucinate than to answer correctly - the best bluffed on 28% of what it did not know, some frontier models above 85%. That is not “wrong most of the time”. It means guessing is still the default when a model does not know, and every mitigation worth paying for follows from that one sentence.
A real number from our own deployment
In the D2C deployment where we cut operational costs 68%, the support agent ran at roughly 4% error during its supervised phase - one decision in 25 needed correction. By day 60, after the feedback loop and policy-prompt refinements, that was under 1.5%. We never designed for zero. We designed for who catches the 1.5%, and how fast.
Myth vs Facts
Myth: “Hallucinations are a bug. The next model will fix them.”
Fact: OpenAI’s own system card for o3 and o4-mini (16 April 2025) recorded the newer reasoning model hallucinating on 33% of PersonQA responses against 16% for o1. Capability and honesty are separate axes, and they do not always move together.
Myth: “If the AI gets it wrong, that is the vendor’s problem.”
Fact: In Moffatt v. Air Canada the tribunal rejected the argument that the chatbot was a separate legal entity. You are the publisher of your bot’s output.
Myth: “We only use AI internally, so the risk is low.”
Fact: Deloitte Australia’s invented citations were in a government report, and the client took back the final instalment. Internal work still gets delivered to somebody who can ask for their money back.
Myth: “A disclaimer covers us.”
Fact: Air Canada published the correct policy on the same website and still lost. New York added disclaimers and still wrote the tool off. Disclaimers manage expectations; they do not transfer liability for a specific answer a customer reasonably relied on.
Where you can and cannot let AI answer unsupervised
The useful question is not “is our AI accurate enough?” It is “what does a wrong answer cost here, and who catches it?” Sort by consequence, not by difficulty.
| The task | What a hallucination costs here | The control it needs |
|---|---|---|
| Internal summary or first draft | An hour of rework | Spot checks. No gate needed. |
| Product or how-to question | One wrong expectation, one annoyed customer | Grounded in retrieved documents, source shown |
| Quoting a price, policy or entitlement | A binding commitment you never made (Air Canada) | Looked up from a system of record, or refused |
| Issuing a refund, credit or discount | Cash out of the door, plus dispute handling | Hard cap, human approval above a threshold |
| Anything a regulator or court will read | Sanctions priced per error (Oregon: $500 per fake citation) | Named human sign-off, every time |
| Telling a customer what the law requires | Advice that is illegal to follow (NYC MyCity) | Do not deploy this |
What this means if you are running AI in your business
Every hallucination cost in this article traces back to the same design decision: an AI was allowed to state something it should only have been allowed to look up, or to act where it should only have been allowed to recommend.
That is an architecture problem, and it is fixable without a better model. When we build agents for clients, the boundary is set before anything ships: what the agent may read, what it may say, what it may do alone, and where a person has to nod. It is the same discipline that stops agents being exploited from outside, which we covered in the five ways AI agents get hacked in production. Hallucination is that failure inside out: the bad instruction comes from the model, not an attacker.
- Every policy, price and entitlement answer is retrieved from a system of record, not generated from memory
- The agent can say “I do not know” and hand off, and you have tested that it actually does
- Refunds, credits and discounts sit behind a hard cap the agent cannot exceed
- Every customer-facing answer is logged with the source it used, so any dispute can be reconstructed
- One named person owns the review queue, with hours in their week actually allocated to it
- You can state your current error rate as a number, because you measured it rather than assumed it
If you cannot tick the last one, start there. You cannot price a risk you have never measured, and the businesses that get burned are almost always the ones who found out their error rate from a customer.
Which hallucination costs are you carrying right now?
Tick each one that is true of your current setup.
- Our AI can state a price, policy or delivery promise from memory rather than from a lookup
- We do not know our current error rate as a number
- An AI can move money, issue credit or apply a discount without a hard cap
- Nobody is named as the owner of reviewing AI output
- We could not reconstruct what our AI told a specific customer three months ago
- AI-drafted work reaches clients or regulators without a human checking the sources
If this were your deployment, here is our first move
We would not start by changing your model. We would spend the first week sorting every AI-touched task into the table above, then find the ones in the wrong row - where something that can invent a sentence is allowed to commit your business to it. There are usually two or three, they were never deliberate decisions, and closing them costs far less than the first dispute would.
Air Canada’s exposure was one FAQ answer. New York’s was one chatbot nobody wanted to be the person to switch off.
Hallucination is not a reason to keep AI out of your business. It is a reason to be deliberate about what your AI may say on your behalf, and to know your numbers before someone else finds them. For a straight assessment of where your setup is exposed, talk to us and we will walk your use cases through the same table.
