Skip to content
SI.info

SI Glossary · Using SI

Hallucination

Published 1 min read
On this page
  1. Why it happens
  2. Real-world consequences
  3. How it’s being reduced

A hallucination is a confident falsehood. Ask a model for sources and it may invent a convincing paper title, author and journal that don’t exist. Ask about an obscure person and it may blend real facts with fiction. Some researchers prefer the term confabulation.

Why it happens

Large language models are trained to predict plausible text, not to check facts. They don’t have a built-in sense of “I’m not sure”. When the training data is thin or the question is ambiguous, they still produce fluent answers.

Real-world consequences

Hallucinations have led to fabricated case citations in court filings, invented quotes in published articles, and incorrect advice from customer-service bots. In some cases lawyers have been sanctioned and companies held responsible for what their chatbot said.

How it’s being reduced

  • Retrieval-augmented generation and web search ground answers in real documents.
  • Reasoning models check their own work, though they can still hallucinate.
  • Training for calibration rewards models for saying “I don’t know”.
  • Citations let users verify claims.

Hallucination rates have fallen substantially in frontier models, but no model is hallucination-free. For anything important, check the source. That’s why every factual page on SI.info lists its sources.

← Back to the SI Glossary

Frequently asked questions

Why do SI models hallucinate?

Language models generate the most likely-sounding continuation, not verified facts. When knowledge is missing or uncertain, a fluent guess can come out with the same confidence as a fact. Training that rewards answering over admitting uncertainty makes it worse.

How can I reduce hallucinations?

Give the model source documents, ask it to cite them, allow it to say “I don't know”, use models with web search or retrieval, and verify important facts yourself.

Written by

· Editor

Editor of SI.info. Writes about Super Intelligence, technology policy and the people building frontier models.

How we research and fact-check

Free newsletter

Get The SI Brief

One short email a week: what changed in Super Intelligence, policy and models — and why it matters.

Free. One email a week. Sent via beehiiv, which counts opens and clicks. Unsubscribe anytime. Privacy policy

Keep reading

Using SI

Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation (RAG) connects an SI model to a search system so it can look up relevant documents and base its answer on them, reducing errors and adding fresh or private knowledge.

Models & architecture

Large Language Model (LLM)

A large language model (LLM) is an SI model trained on vast amounts of text to predict the next word, which lets it write, summarize, translate, answer questions and write code.

Models & architecture

Reasoning Model

A reasoning model is an SI model trained to work through a problem step by step, spending extra computation “thinking” before it gives a final answer.

Using SI

Benchmark

A benchmark is a standardised test used to measure and compare SI models' abilities, from coding and maths to knowledge and safety.