Finance intelligence research
What makes a financial document hard for a language model, and what a desk has to add before an answer is worth acting on.
A financial answer is not a sentence. It is a number, the window it covers, the document it came from, and the date that document was read. Drop any one of those four and the sentence survives — which is exactly the problem. This page collects what the published work says about that gap, and what we build against it.
#Why filings resist a language model
Three properties of financial documents make them harder than they look, and each one is measured in the literature rather than asserted here.
- The answer is rarely in one place. TAT-QA was built specifically around hybrid context — a table and the prose around it — because in real reports the figure sits in the table and the qualification sits in the paragraph 3.
- Getting the number out is not the task; doing arithmetic on it is. FinQA annotates a reasoning program for every question so that the chain, not just the result, can be checked 2.
- Domain knowledge gates the calculation. BizBench separates reading comprehension from knowing which formula applies, and reports that the bottleneck sits in business understanding rather than in raw computation 4.
The scale of the gap is the part worth sitting with. FinanceBench evaluated sixteen model configurations on questions its authors describe as clear-cut, and found that a strong model paired with a retrieval system answered incorrectly or declined on 81% of the sampled cases 1. That result is from 2023 and the models have moved; the structural point has not. A retrieval system that returns the wrong page produces a fluent answer about the wrong page.
#Point-in-time truth
A figure is only true inside a window. Revenue for FY2025 is not revenue for the trailing twelve months, and a restated figure is not the figure that was published. A desk that answers without carrying the window is not wrong once — it is wrong in a way nobody can detect afterwards.
The consequence for a product is uncomfortable: when the data cannot answer the question at the date asked, the honest output is a refusal, not an approximation. In Finexia that refusal is a first-class state, not an error path.
#Facts, assumptions and calculations are three different things
A model that produces a number blends the three by default. A spreadsheet built by an analyst separates them, because that is what makes it auditable a year later. Finexia takes the analyst’s side: the workbook contract in the core keeps inputs, model, output, sources and checks on separate sheets, and a fact without a declared source is refused before the file is written 5.
EvidenceThat contract is written and covered end to end by tests — the file is composed, written, reopened by a path other than the one that wrote it, and its formulas found again. Version 0.0.23 extends it to a deck and a memo rendered from the same dossier, so the three cannot diverge. None of it has yet been exercised by a real agent run, and the core says so in those words 6.
#Why a specialist beats a generalist here
The argument for named roles is not that a specialist model is smarter. It is that a mandate is a written limit, and a limit is what makes a wrong answer visible. A role that may read filings and may not reach the open web cannot quietly substitute a blog post for a 10-K. A role that owns the workbook format is the only one that can produce one, so a deck request cannot silently return a spreadsheet.
This is a capability argument, not a benchmark claim. We have not run a head-to-head evaluation of one general assistant against a roster of mandates, and until we do, this page will not pretend otherwise.
#What this means for Finexia
- Every figure travels with its source and its retrieval date, or it does not travel.
- A question the data cannot answer at that date comes back blocked, not approximated.
- What an indicator could not measure is left blank and counted, never interpolated.
- A role declares what it may reach; no role holds both user data and arbitrary web access.
- A deliverable is validated by reopening it with different code than the code that wrote it.
LimitNothing on this page is a measured claim about Finexia’s own accuracy. We have not run any of the benchmarks below on our own stack. The evaluation page states the protocol we intend to follow and says plainly that it has not been run.
#Sources
- FinanceBench: A New Benchmark for Financial Question AnsweringPranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, Bertie Vidgenhttps://arxiv.org/abs/2311.11944
- FinQA: A Dataset of Numerical Reasoning over Financial DataZhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, William Yang Wanghttps://aclanthology.org/2021.emnlp-main.300/
- TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in FinanceFengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, Tat-Seng Chuahttps://aclanthology.org/2021.acl-long.254/
- BizBench: A Quantitative Reasoning Benchmark for Business and FinanceRik Koncel-Kedziorski, Michael Krumdick, Viet Lai, Varshini Reddy, Charles Lovering, Chris Tannerhttps://arxiv.org/abs/2311.06602
- Finexia OS — handover report 0.0.22FinexiaInternal document, not published
- Finexia OS — product state at 0.0.23FinexiaInternal document, not published
Read next
- MethodologyHow a Finexia run is set up so that its result can be argued with: scope, as-of date, admissible sources, missing data, provenance, checks, and what counts as evidence.
- Evaluations and benchmarksFour published benchmarks for financial reasoning, documented from their primary sources — and a plain statement that Finexia has not run any of them.
More in Research