AI Hallucination Explained: Why AI Makes Things Up and How to Catch It

What is AI hallucination? (meaning in simple words)

AI hallucination is when an AI system generates an answer that is false, made up or unsupported by its sources, while presenting it as if it were true.

The word comes from psychology, where a hallucination means perceiving something that is not there. For AI, it means producing something that is not real: a fact, a date, a quote, a court case, a research paper, a function in a code library.

So what is it called when AI makes up facts? The standard term is hallucination. Some researchers prefer confabulation, a term from neuroscience for filling memory gaps with invented details without meaning to deceive. You may also hear "fabrication" or simply "making things up". They all describe the same problem.

Three features make hallucinations tricky:

• They sound fluent. The grammar is perfect and the tone is confident.
• They mix with real facts. A paragraph can be 90 percent accurate with one invented detail in the middle.
• The model does not flag them. Unless it is specifically built or prompted to show uncertainty, the AI will not say "I am guessing here".

Hallucination is not limited to chatbots. Image generators produce hands with six fingers and signboards with nonsense letters. Speech-to-text tools sometimes insert words nobody said. Any generative AI system can produce output that does not match reality.

Why does AI hallucinate?

AI hallucinates because a large language model (LLM) is trained to predict the next likely piece of text, not to look up and verify facts.

Here is the process in plain steps:

1. Training on patterns. The model learns from huge amounts of text which words tend to follow which. It learns that "The capital of France is" is usually followed by "Paris".
2. Prediction, not retrieval. When you ask a question, the model does not search a database of facts. It generates an answer one token at a time, each time choosing a likely continuation.
3. Gaps get filled smoothly. For famous facts, the most likely continuation is also the correct one. For rare facts (a small college's founding year, a niche court case, a new library version), the model may have seen little or no reliable data, yet it still produces the most plausible-sounding continuation.
4. No built-in truth check. Nothing in basic next-token prediction asks "is this real?". A made-up paper title and a real one can look equally likely to the model.

Training can reward guessing

In 2025, OpenAI published research arguing that common training and evaluation methods reward models for guessing instead of admitting uncertainty. Think of a multiple-choice exam with no negative marking: guessing always scores better than leaving a question blank. If models are graded the same way, they learn to answer confidently even when they should say "I don't know".

Other common causes

• Outdated knowledge. A model only knows what was in its training data up to a cutoff date.
• Missing context. If you ask about "my company's leave policy" without sharing it, the model can only guess.
• Long or messy input. Details buried deep in a very long document can be missed or blended. Our page on the context window explains why.
• Leading questions. "Why did Gandhi meet Einstein in 1931?" assumes a meeting happened. The model may go along with the false premise instead of correcting it.

Types of AI hallucination (with examples)

The four types of AI hallucination beginners meet most often are factual errors, fake citations, made-up code packages and reasoning slips.

Fake citations deserve extra attention

For students, invented references are the most dangerous type, because they go straight into assignments, theses and reports. A model can combine a real professor's name, a real journal and a believable title into something that never existed. Always open the actual source.

Made-up packages are a security risk

A 2024 research paper titled "We Have a Package for You!" studied how code-generating models recommend software packages that do not exist. The risk is real: an attacker can publish malicious code under a commonly hallucinated package name, and a developer who installs it without checking gets compromised. Security researchers sometimes call this "slopsquatting".

Contradicting the source

A fifth type shows up when you give the AI a document. The model summarises a PDF but adds a point the PDF never made, or flips a "not". Researchers call this an unfaithful or ungrounded answer, and it matters a lot for summaries of legal, medical or exam material.

AI hallucination examples in real life

The best-known real-life example of AI hallucination is Mata v. Avianca, a 2023 US court case where lawyers submitted a legal brief containing court decisions that ChatGPT had invented.

What happened, in short:

• A passenger sued the airline Avianca in a federal court in New York over an injury on a flight.
• His lawyers filed a brief citing earlier court decisions to support their argument.
• The airline's lawyers and the judge could not find several of those decisions. They did not exist. One of the lawyers had used ChatGPT for research, and the cases, quotes and citations were fabricated.
• The lawyer later said he had even asked ChatGPT whether the cases were real, and it said yes.
• In June 2023, the judge sanctioned the lawyers and their firm and fined them $5,000.

The case became a global warning because the people involved were trained professionals, and the hallucinations were convincing enough to reach a court filing.

Another public example

In February 2023, a promotional demo of Google's Bard chatbot claimed that the James Webb Space Telescope took the very first picture of a planet outside our solar system. Astronomers quickly pointed out that the first such image came years earlier from a different telescope. The error got wide media coverage within days of the launch.

What should you take from these cases? Not that AI is useless. The lesson is narrower: AI output is a draft, not a source. The lawyers' mistake was not using ChatGPT. It was filing its output without checking a single citation.

Why do AI hallucinations sound so confident?

AI hallucinations sound confident because the model's writing style and its accuracy are produced by the same prediction process, and nothing forces the style to drop when accuracy does.

When a human expert is unsure, their voice changes. They pause, hedge or say "let me check". A language model learned its tone from millions of well-written articles, textbooks and answers, and most of that text is written confidently. So it writes confidently by default, whether the content underneath is solid or invented.

A useful mental model: the AI is like a student who has read a lot, writes beautifully, and never leaves a blank in the exam. On topics they studied, the answers are excellent. On topics they skipped, they still write a full page, in the same polished style.

My contrarian take: hallucination is not a random bug that will be patched next month. It is a side effect of the same ability that makes these models useful. A system that can write a new poem, draft fresh code or suggest an original business name is, by design, producing text that did not exist before. The goal is not to remove creativity. It is to separate creative tasks from factual ones and add checks for the factual ones.

Modern models are better at this than early chatbots were. Many now say "I am not sure" more often, cite sources when they use web search, or refuse to invent references. But "better" is not "solved", and the confident tone is still there when they get it wrong.

How RAG and grounding reduce AI hallucination

RAG and grounding reduce hallucination by giving the model real source text to answer from, instead of relying only on what it memorised during training.

Grounding means tying an answer to specific, trusted information: a document, a database, a search result. RAG (retrieval-augmented generation) is the most common way to do it. The idea was described in a 2020 research paper from Facebook AI Research and university collaborators, and it is now standard in AI products.

How RAG works, simply:

1. You ask a question, for example "What is the attendance rule for my university?"
2. The system searches a trusted collection (the university handbook) for the most relevant passages.
3. Those passages are added to the prompt inside the model's context window.
4. The model is instructed to answer using only those passages, ideally with a citation to each one.

Now the model is closer to an open-book exam than a closed-book one. It can still make mistakes, but it has far less reason to invent things, and you can check its answer against the cited passage. Read our full guide to RAG for the details.

Other techniques that help:

• Web search and citations in chat tools, so you can click through to the source.
• Tool use, such as letting the model run a calculator or code for maths instead of doing arithmetic in its head.
• Asking for uncertainty, by instructing the model to say "not found in the document" when the answer is missing.
• Human review for anything high-stakes: legal, medical, financial or exam-critical.

Grounding is only as good as the sources. If the retrieved document is outdated or wrong, the answer will faithfully repeat that error.

How to prevent AI hallucination when you use ChatGPT, Claude or Gemini

You cannot fully prevent AI hallucination, but you can cut it sharply with how you prompt and what you give the model.

Prompting habits that work:

• Provide the source. Paste the chapter, the policy or the article and say "Answer only from the text below". Of all these habits, this one helps the most.
• Give permission to not know. Add "If you are not sure, say so instead of guessing." Models follow this instruction more often than you might expect.
• Ask for quotes. "Quote the exact sentence from the document that supports each point" makes invented claims easy to spot.
• Avoid leading questions. Ask "Did X happen?" before asking "Why did X happen?".
• Split big tasks. Ask for an outline, check it, then ask for each part. Errors are easier to catch in small pieces.
• Turn on web search for current facts. For news, prices, cut-offs and exam dates, use a mode that searches and cites, then click the citations.
• Use a tool for maths. Ask the AI to show working, or to write and run code if your tool supports it.

Good prompt engineering reduces hallucinations, but it does not replace checking. I tell learners to think of prompts as lowering the error rate and verification as catching what slips through. You need both.

How to check if an AI answer is correct: a student checklist

To check whether an AI answer is correct, verify every specific claim (names, numbers, dates, quotes and references) against an independent, trusted source before you use it.

Use this checklist for assignments, projects and exam prep:

1. Highlight the checkable claims. Underline every number, date, name, formula, quote and citation in the answer.
2. Open every reference. Search the exact paper title on Google Scholar, the case name on a court database, or the book on a library catalogue. If you cannot find it, treat it as fake.
3. Cross-check with your syllabus source. For school and university topics, compare with NCERT books, your prescribed textbook or official lecture notes.
4. Use official sites for official facts. Exam dates, eligibility rules and government schemes should come from the official website or notification, never from a chatbot's memory.
5. Run the code. If the AI suggested a library or function, check that it exists in the official documentation before installing anything.
6. Ask a second time, differently. Rephrase the question or ask another model. If answers disagree, that is a signal to dig deeper.
7. Ask the AI to critique itself. "List any claims in your answer that might be wrong or need checking" often surfaces weak spots.
8. Be extra strict for high stakes. Medical, legal, financial and safety advice needs a qualified human, not just a second chatbot.

Five minutes of checking can save you a failed submission or an embarrassing viva. I would rather a student use AI heavily and verify carefully than avoid AI completely. The verification habit is the real skill.

Can AI hallucination be fixed completely?

No, AI hallucination cannot be fixed completely with today's methods, but it can be reduced a lot, and the reduction is improving with each generation of models.

Where things stand:

• Newer models hallucinate less on many common factual questions than early chatbots did, but rates vary widely by model, task and topic. Claims about exact hallucination rates depend heavily on how they were measured, so treat any single number with caution.
• Grounded systems that answer from retrieved documents, with citations, are much more reliable for factual work than a model answering from memory.
• Reasoning models that think before answering can catch some of their own errors, though they can also produce long, confident reasoning that ends in a wrong answer.
• Better training incentives, such as rewarding a model for admitting uncertainty, are an active research area.

What this means for you: hallucination is a permanent part of how you should use AI, at least for now. Build habits that assume it can happen. Use AI for drafting, explaining, brainstorming and summarising sources you provide. Rely on real sources for facts that matter.

If you are preparing for AI roles, expect hallucination questions in interviews, such as "how would you reduce hallucinations in a customer support bot?". A good answer mentions grounding, RAG, evaluation, refusal instructions and human review. Our AI interview prep section covers questions like this.

Key takeaways

• AI hallucination is when an AI model presents false or made-up information as if it were true, often with full confidence.
• It happens because language models predict likely-sounding text instead of checking facts, and training can reward guessing over saying 'I don't know'.
• Common types are factual errors, fake citations, made-up code packages, reasoning slips and summaries that contradict the source.
• The 2023 Mata v. Avianca case, where lawyers filed ChatGPT-invented court decisions, shows why AI output must be verified.
• RAG, grounding, clear prompts and a verification checklist reduce hallucinations, but no current method removes them completely.