You ask a question, you get a confident answer, and later you find out it was wrong. Not vague — wrong. A court case that does not exist. A feature the software has never had. A statistic with a decimal point and a source that turns out to be nothing.
This is the single biggest reason people stop trusting AI tools. It is also the thing most easily managed, once you understand what is actually happening.
What “hallucination” means
In AI, a hallucination is output that is fluent, plausible and false. The word is a poor one, because it suggests a malfunction. It is closer to the normal behaviour of the system than to a bug.
The important thing it does not mean: the model is not lying. Lying requires knowing the truth and choosing otherwise. There is no such step here.
Why it happens, in plain language
A language model is built to produce text that fits. Given everything so far, it produces what most plausibly comes next, over and over, very quickly.
Notice what is missing from that description. There is no lookup step. No moment where it checks a fact against a record and confirms it. “Sounds right” and “is right” overlap most of the time, which is why these tools are useful at all — but they are not the same thing, and nothing in the process forces them to match.
Two things make it worse. The model is built to answer, so silence is rarely its default. And it has no reliable sense of its own confidence, so a wild guess and a well-established fact come out in exactly the same tone.
That last point is the real problem. If invented answers arrived hedged and nervous, nobody would be caught out.
Where it bites hardest
Errors are not evenly spread. These are the categories where you should assume a mistake until you have checked:
- Citations and sources. Paper titles, case numbers, page references. Plausible-looking references that do not exist are a classic failure, and they have cost people their jobs.
- Specific numbers. Prices, percentages, dates, version numbers. A precise-looking figure is not evidence of anything.
- Links. URLs are easy to assemble and frequently lead nowhere.
- Quotes. Especially quotes attributed to real, named people.
- Software menus. “Go to Settings, then Advanced, then Integrations” — for an app whose menu has never looked like that.
- Recent events. Anything after the model’s training data, unless it genuinely searched the web for you.
Conversely, it is on much firmer ground rewriting text you supplied, summarizing a document you uploaded, explaining a well-established general concept, or drafting something where you provide the facts.
That is the practical rule: the more the answer depends on knowledge you did not supply, the more likely it is to be invented.
How to catch it in under a minute
- Click every link. Not skim — click. A dead or unrelated link is the fastest tell there is.
- Search one specific claim. Take the most precise detail in the answer and search it on its own. If it exists, you will find it immediately.
- Ask again in a fresh chat. Same question, new conversation, no context. Facts tend to stay stable; invented details tend to drift.
- Ask it to say where this came from. Not proof, but hedging that appears only when challenged is informative.
- Check software steps against the actual software. Open the app. Thirty seconds.
Apply this in proportion to the cost of being wrong. An email to a colleague does not need it. Anything going to a client, into a contract, onto a website, or in front of a regulator does.
How to reduce it before it starts
- Supply the facts yourself. Paste the document, the pricing page, the email thread. A model working from text you provided has far less room to invent.
- Give it permission not to know. Add: “If you are not sure, say so rather than guessing.” It does not eliminate the problem, but it noticeably increases how often you get an admission instead of an invention.
- Ask narrow questions. Broad questions invite confident essays. Narrow ones are easier to verify.
- Use search when the tool offers it. An answer with real, checkable sources attached is a different kind of answer — provided you actually open them.
- Never ask it to recall a specific document from memory. Upload the document instead.
But how often is it actually wrong?
People want a number here, and you will find plenty quoted. We are not going to give you one, because the honest answer is that a single percentage would be meaningless.
Error rates vary enormously by model, by version, by subject, by how the question was asked, and by who ran the test. Benchmarks measure narrow, specific tasks that may look nothing like your work, and many of the headline figures circulating come from companies with an interest in the result. A number from one benchmark tells you very little about whether the answer on your screen is correct.
The useful mental model is not a percentage. It is this: treat every factual claim as unverified until you have checked it, and scale how hard you check to how much it would cost you to be wrong.
This is not a reason to avoid AI
It is a reason to use it for the right jobs. Drafting, rewriting, summarizing, explaining, organizing, getting past a blank page — all of these are genuinely useful and none of them depend on the model being a reliable encyclopedia.
The people who get burned are the ones who treat it as a source of truth. The people who get value treat it as a fast, tireless, slightly unreliable assistant whose work always gets read before it goes out. If you are still working out where to begin, this is a good place to start.
