Why ChatGPT Hallucinates Financial Numbers, and How to Catch It

A general chatbot does not look up a company’s revenue when you ask for it. It predicts the most plausible next words, and a plausible-looking number is not the same thing as a verified one.

That single gap explains most of what people call “AI hallucination” in finance. The model is not lying and it is not broken. It is doing exactly what it was built to do, which is to continue text convincingly. When the honest continuation would be “I was never given that figure,” the model instead fills the blank with something that fits the shape of the answer.

A language model predicts, it does not retrieve

Underneath a chatbot is a system trained to guess the next token, roughly the next fragment of a word, given everything before it. Ask it to complete “the capital of France is” and it reliably produces “Paris,” because that sequence appears so often in its training that the pattern is locked in. Useful, and it feels like knowledge.

Now ask for a specific company’s net profit in a specific quarter. There is no single dominant sequence to fall back on. The model has seen millions of sentences that look like “net profit for the quarter was X crore,” so it generates a sentence in that exact form, with a number that sits in a believable range. The grammar is perfect. The figure is a guess dressed as a fact.

This is the core point, and it is easy to miss: the model has no separate step where it checks a number against a source. Text generation and fact retrieval are different operations. A plain chatbot only does the first one. Unless it has been explicitly handed the real data inside the conversation, it is improvising, and it improvises with total fluency.

The danger is not that the model is usually wrong. It is that it is usually plausible, and plausible is what disarms your judgement.

Why finance punishes this harder than most fields

Ask a chatbot for a dinner recipe and a slightly-off answer costs you a pinch of salt. Financial numbers are unforgiving in ways that turn small model errors into real mistakes.

Numbers must be exact. “Roughly right” is wrong. A margin of 18.4 percent is a different fact from 18.9 percent, and a model that lands near the answer has still landed on a false one.

The period must match. The same company has a different revenue for the quarter, the trailing twelve months, and the full year, and different again year on year. A figure can be genuinely correct for one period and completely wrong for the one you asked about. Chatbots blur these constantly.

The entity must match. Standalone versus consolidated, parent versus group, the operating company versus the listed holding company. These are different sets of books. A confident answer that quietly mixes them is a trap.

The numbers change. Companies restate results. Accounting standards shift. A figure that was accurate when the model’s training data was collected may since have been revised. The model has no built-in sense of “as reported on this date versus corrected later.”

Wrong-by-a-little is the worst failure mode. A hallucination that is obviously absurd gets caught immediately. One that is off by a rounding, or drawn from the adjacent quarter, sails straight past a busy reader. In finance, the plausible near-miss is more dangerous than the wild miss, because nothing about it triggers suspicion.

Put together, finance is close to the worst possible arena for a system that generates confident text without checking it. The answers need to be exact, correctly scoped, current, and precise, and the failure that matters most is the one designed to look right.

What a hallucinated number tends to look like

There is no perfect tell, but hallucinated figures share a family resemblance once you know what to watch for.

Signal	What you tend to see	What it suggests
Suspiciously round	”About 5,000 crore,” “roughly 20 percent”	An estimate shaped like a fact, not a figure lifted from a document
No source, no date	A number with no filing, no period, no “as of”	Nothing to verify against; treat as unconfirmed
Period is vague	”Recent,” “last year,” “the latest quarter”	The model may not know which period it is even answering for
Total confidence, zero hedging	Crisp figures for obscure or very recent data	Fluency substituting for retrieval
Consistent story, shifting numbers	Ask twice, get two different values	A strong sign the figure is being generated, not recalled

That last one is the cheapest test you have. Ask the same question again in a fresh session, phrased a little differently. A grounded figure stays put. A hallucinated one often drifts, because there was never a fixed fact underneath it, only a fresh roll of plausible text.

Why grounding is the real fix, not a smarter chatbot

The instinct is to assume bigger, newer models simply hallucinate less. They hallucinate differently, and often more convincingly, but the underlying gap does not close on its own. A model that predicts text will keep predicting text. Making it more fluent can make a wrong number harder to spot, not easier.

The reliable path is not a cleverer guesser. It is to stop guessing. That means grounding every answer in real, source-linked data: the model does not recall the number, it is handed the actual figure from an actual document, tied to a specific period and entity, and it reports that. When the figure carries a source and a date you can open, the machine is no longer the authority. The filing is. The model becomes a way to read and explain the data, not the origin of it.

Two properties matter most. Point-in-time means the answer reflects what was known and reported as of a given date, so restatements and revisions are handled honestly rather than silently overwritten. Source-linked means every number traces back to where it came from, so you can check it in seconds instead of trusting it on faith. This is the discipline serious research runs on, and it is the approach Altys takes for Indian equities: answers anchored to the underlying filings, with the source visible rather than assumed.

None of this requires you to trust a vendor’s word either. The whole point of source-linking is that you should not have to. A number you can verify is worth more than a number you are told to believe, no matter how advanced the model that produced it.

How to catch it

You do not need to be a data scientist to defend yourself. A short checklist handles most cases.

• Ask for the source and the date. Any figure worth using can name where it came from and what period it covers. “I do not have that” is a better answer than a confident invention, and a chatbot that cannot cite is telling you something.
• Open the actual filing. For anything that will inform a decision, match the exact line, the exact period, and the exact entity (standalone or consolidated) against the real document. This is the step that catches the plausible near-miss.
• Be suspicious of clean round numbers. Real financials are messy. Numbers that arrive suspiciously tidy are often estimates wearing the costume of facts.
• Pin down the period and entity yourself. Do not accept “recent” or “the latest.” Name the quarter or year and the reporting basis you mean, and make the answer conform to it.
• Ask twice. Re-run the question in a fresh session. A number that shifts was never anchored to anything.
• Prefer tools that ground and cite. When a system links each figure to a point-in-time source you can inspect, verification takes seconds instead of trust. That is the whole game.

A chatbot is a superb reader and a poor witness. Let it explain, summarize, and reason over data you can verify. Do not let it be the source of the numbers themselves. The moment a figure has a document and a date behind it, hallucination stops being a risk you carry and becomes a claim you can check.