The methodology, not the marketing
The accuracy benchmarkpage explains what the test proves and why it's different from a confidence score. This page is the layer underneath it — the specific mechanisms, in the order they actually run, with no step glossed over as “proprietary AI magic.” Every check here is arithmetic, and arithmetic doesn't need a black box.
None of this is a promise about a fixed test set. It's a description of code that runs, in the same shape, on whatever document you upload — which means the methodology is checkable, not just claimed.
It's also short by design. A methodology that needs pages of caveats before it can be trusted is usually hiding complexity that should have been simplified rather than documented — the two checks below, the deterministic score they feed, and the escalation rule that follows a failure are the entire mechanism, start to finish.
Why a methodology gets written down at all
Most extraction tools describe their accuracy in outcome terms — a percentage, a badge, a chart — without describing the mechanism that produced it. That's a reasonable choice if the mechanism is genuinely proprietary and the outcome is independently verifiable some other way. Here, neither is true: the mechanism is simple enough to write down in full, and writing it down is exactly what makes the outcome verifiable at all.
There's a second, quieter reason too. A team that writes down exactly how its own accuracy claim is computed is committing to being wrong in a specific, discoverable way if the method is flawed — rather than wrong in a vague, unfalsifiable way that no one outside the team could ever pin down. That commitment is itself a form of evidence, separate from whatever the method actually reports.
The total check
Take the opening balance the statement prints. Add the sum of every transaction extracted. Compare the result to the closing balance the same statement prints. If the two figures match to within a cent of rounding tolerance, the total check passes — the extraction accounts for every unit of money the statement claims moved.
| Input | Source |
|---|---|
| Opening balance | Printed on the statement, as the extraction reads it |
| Sum of transactions | Every extracted transaction's amount, signed |
| Closing balance | Printed on the statement, as the extraction reads it |
| Tolerance | Two cents, to absorb genuine rounding — not a margin for real errors |
This check runs on the header figures as they were finally settled, not on numbers derived from the rows and then checked against themselves — which would prove nothing, since a number always agrees with itself.
That distinction matters more than it sounds. An earlier version of a related check computed its own “expected closing balance” from the extracted rows and then compared that derived figure to itself under a different name — a check that can only ever pass, because both sides of the comparison came from the same source. The version that actually proves something has to anchor one side of the equation to a number the extraction never touched: the balance as printed on the page.
The row-level check
Where a statement prints a running balance on individual rows, a second, finer-grained check runs alongside the total: each row's own printed balance must equal the previous printed balance plus the amounts of every transaction in between. On a statement that prints a balance after every single row, that's simply the previous row's balance plus this row's amount.
Many statements don't print a balance on every row — a common layout prints one balance per day, with several transactions sharing it. In that case the check accumulates every transaction's amount between two printed balances before comparing, rather than checking only the last row against the previous printed one and discarding what happened in between.
Where the total check answers “is the whole document complete,” the row-level check answers “which specific row is the problem” — the difference between knowing something is wrong and knowing exactly where to look.
Why both checks run, not just one
The total check alone would catch a genuine gap in the money but couldn't say which row caused it on a long statement — genuinely useful for gating a pipeline, less useful for a reviewer trying to fix a specific mistake. The row-level check alone is more precise but structurally can't run at all on a statement that only prints a balance once per day, or once per week, since most rows in between simply have nothing to check against individually.
| Check | Answers | Blind spot alone |
|---|---|---|
| Total check | Is the document complete overall | Can't localise a failure to a specific row |
| Row-level check | Which specific row breaks the pattern | Needs balances printed reasonably often to work at all |
| Both together | Completeness, and where a failure sits when it exists | Neither's blind spot survives the other running alongside it |
Running both, and letting each cover the other's gap, is a small design choice with an outsized effect on how trustworthy the resulting score actually is.
How the 0-100 score is actually computed
The Quality Score starts at 100. Every check on the document — the balance checks above, plus structural checks like every transaction having a date and an amount — either passes silently or deducts a fixed amount: a larger deduction for a confirmed error, a smaller one for a warning that's worth a look but not yet proven wrong. The final number is clamped between 0 and 100 and reported alongside exactly which checks failed.
Nothing about this formula is tuned per document to produce a flattering number — the same deduction applies the same way whether the document is a test upload or a paying customer's statement, which is what makes the number mean the same thing every time it's shown.
Why the model's own confidence was removed
Earlier versions of the QA report led with the extraction model's own self-reported confidence — the average of scores the model assigned to its own output. In practice, that number clustered in a narrow high band regardless of whether the document actually reconciled, including on statements that reconciled to the penny. A four-point gap between a perfect document and a mediocre one is not a measurement doing its job.
So the number shown first is the one that's computed, not reported: the deterministic Quality Score above. The model's self-reported confidence still exists internally and still informs retry decisions — it's simply not presented as a fact about the document, because it isn't reliably one.
What happens when a page fails
A failed balance check on a page isn't simply recorded and forgotten — it can trigger a second, more careful pass specifically on that page, using a slower, more layout-aware reading approach than the first pass attempted. The goal is to see whether the failure was a fixable misread rather than a genuine gap in the source document.
First pass extracts the page
The standard reading approach, fast and usually sufficient.
The balance gate checks the result
Total and row-level checks run against the printed balances.
A failure can trigger a heavier re-read
A layout-aware pass specifically targeting the page that failed.
The gate runs again on the new result
If it now passes, the heavier read replaces the first; if not, the document ships flagged.
This is also why the same document can, in principle, be read twice and reported differently the second time — not because the arithmetic changed, but because the second read genuinely fixed what the first one missed. That's a feature of the escalation, not an inconsistency in the score: the number reported is always the result of the best reading the budget allowed, not a snapshot frozen at the first attempt.
The cost budget behind escalation, stated plainly
A heavier re-read costs more to run than the first pass, and that cost is bounded by a real budget rather than unlimited — which means it's honest to say plainly that not every failed page can afford an escalation. Where the budget doesn't stretch to cover it, the document ships with the failure flagged rather than silently, and the score reflects exactly that.
Being specific about this limit is part of the same commitment the rest of this page makes — a methodology that claimed unlimited retries at no cost would be describing a system that doesn't exist.
In practice the budget rarely binds on an ordinary document — the cases where it matters are the genuinely hard ones, where the first pass has already struggled and a heavier pass is expensive precisely because the document itself is expensive to read correctly. Those are also, not coincidentally, the documents most worth flagging honestly rather than shipping with a quietly optimistic score.
Grouping by account before scoring
A single PDF can carry several accounts in sequence, each restarting its own running balance. Checking the rows as one continuous chain would make the first row of every account after the first look like a broken transaction — a false failure produced by the check's own assumption, not by anything wrong with the extraction.
Rows are grouped by account label before either check runs, so a correct multi-account statement is verified account by account. The reported result states how many accounts were found and verified, so the number is never silently averaging together sequences that were never really one continuous chain.
The equivalent check for invoices
An invoice doesn't print an opening and closing balance the way a statement does, but it carries its own comparable proof: the line items should sum to the invoice's own printed total. The same principle applies — check the extraction against arithmetic the document itself supplies, rather than trusting that every line item was read correctly just because none of them looks obviously wrong.
When the method can't apply
A document with no running-balance column — many card statements, some exports — genuinely cannot run either check, because there's no printed opening-and-closing arithmetic to test the extraction against. The methodology's answer in that case is “not verifiable,” a distinct status from “passed,” shown as such rather than defaulted to a green result.
Reproducing this yourself
None of the arithmetic here depends on anything proprietary. Export your own extracted transactions from any tool, including FlowParse, and check them by hand: sum the amounts, add the printed opening balance, compare to the printed closing balance. If a tool's output doesn't reconcile and the tool never told you, that silence is itself informative about what its own accuracy claims are actually built on.
What changed, and why
The row-level check used to compare only a row against the single row immediately before it, which worked fine on statements that print a balance on every row and misread ordinary day-block statements — one printed balance per day, several transactions sharing it — as broken, because the amounts of the transactions in between were simply discarded rather than added up.
That was fixed by walking every transaction in document order and accumulating the amounts between two printed balances before comparing, rather than dropping everything but the last one. On real stored statements affected by the old version, the fix moved documents from a small fraction of rows reconciling to nearly all of them — the same underlying data, read the same way, simply checked correctly this time.
Documenting a change like this rather than quietly shipping it is part of the same principle as the rest of this page — a methodology is only trustworthy if its mistakes are as visible as its results.
It's worth being specific about what didn't change alongside what did: the fix touched only how the check accumulates amounts between two printed balances, not what counts as a pass. A document that genuinely reconciles still needs to land on the printed closing balance to the cent — the correction made the check see more of the real arithmetic, not made it more lenient about what passes.
What this methodology doesn't claim
Doesn't guarantee every field matches a human's judgment
It proves the total accounts for every unit of money. Whether a description or category is exactly how you'd have written it is a separate question.
Doesn't run on documents with no printed running balance
Those are marked not verifiable, not silently passed by a fallback assumption.
Doesn't offer unlimited free retries on a failed page
Escalation is bounded by a real, stated cost budget.
Doesn't eliminate the small chance of a false pass
Two equal and opposite errors could in principle cancel out — rare in practice, and part of why the row-level check runs alongside the total, not instead of it.
These limits share one thread: a methodology that claimed more than this would be describing a system that doesn't exist. Stating them is what keeps the parts it does prove worth trusting.