Data Quality Beats Model Quality: A Year Reading Indian Filings

After a year building a system that reads Indian company filings and turns them into structured financials, the clearest lesson is simple: data quality beats model quality almost every time. The model turned out to be the last ten percent of the work. Making the underlying data trustworthy was the other ninety.

The best model cannot fix a units error

Early on our numbers were subtly wrong. Not nonsense, just off by roughly a hundred times in scattered rows. The cause was not the model. Some filings report in absolute rupees and some in crore, and the model faithfully extracted exactly what was printed on the page. The model was right and the answer was still wrong, because the data contract was broken.

No prompt fixes this. A deterministic unit normalization step does. That was the first time the pattern showed up, and it kept repeating: when the model is correct but the output is wrong, the bug is upstream of the model.

As filed versus restated will poison everything downstream

Pull a company’s numbers for a past year today and you might get a version that was restated much later. Build anything on top of that and you have quietly mixed in information that did not exist at the time. In testing this shows up as results that are too good, an effect known as lookahead bias.

The fix is not a smarter model. It is discipline. Store every number with the exact filing and date it came from, and never let a later revision overwrite what was known on an earlier date.

Extraction accuracy is a pipeline problem

We benchmarked frontier models on real filings. On clean pages they were all fine. The failures clustered on the messy parts: inconsistent XBRL tags, a profit and loss line that means different things across two filers, and the treatment of associates and minority interest. The gap between models on these cases was smaller than the gap you get from simply handling the edge case explicitly in code.

That reframes where effort should go. Chasing a marginally better model buys you less than writing careful rules for the ten situations that actually break.

The gains came from unglamorous infrastructure

Three things moved the needle far more than any model upgrade.

• Unit normalization, so every value is in a consistent scale.
• Point in time storage, so every number carries the date it was first known.
• Source linking, so every extracted value ties back to the exact line in the exact filing and can be audited.

None of it is exciting. All of it mattered more than swapping models ever did.

The takeaway

For anyone building AI over financial data, the model is close to a commodity and the proprietary data discipline is the moat. The model is the last ten percent. Getting the data trustworthy is the ninety percent that nobody talks about, and it is exactly where the defensibility lives.

This is the foundation we build on at Altys Labs. Every figure is normalized, stamped with the date it was known, and linked back to its source filing, so anything built on top of it starts from data you can actually trust.