Failure case study August 2026 16 min read

The errors that pass every check

We argue a lot for one check: opening balance plus every transaction equals the printed closing balance. It is the strongest thing a statement gives you. So here is the other half of that argument, which we would rather write ourselves than have someone discover — the identity catches everything that changes a sum, and is perfectly blind to everything that does not.

FlowParse
flowparse.io

The limit of the check we like most

There is a temptation, when you have a genuinely good check, to let people believe it covers more than it does. The balance identity is a real proof, it is cheap, and it catches an entire class of failure decisively. It would be easy to leave the impression that a document which passes it is simply correct.

It is not. The identity examines a sum, so it has an opinion about exactly one thing: whether the amounts, taken together, move the balance from where the bank said it started to where the bank said it ended. Every field that is not an amount is outside its view entirely.

That includes who was paid, what the payment referenced, when it happened, and which account it went to. All of those can be wrong on a statement that reconciles to the cent — and some of them cause more damage than a wrong amount would, because a wrong amount gets caught and these do not.

What arithmetic genuinely does catch

Worth stating precisely, because the point of this article is the boundary rather than a retreat. Within its scope the identity is decisive.

Dropped rows. A transaction that never made it into the extraction leaves the sum short by exactly its value. This is the most common serious extraction failure and the identity catches it every time.

Invented rows. A subtotal or a carried-forward figure imported as a transaction inflates the sum. Same mechanism, opposite direction.

Misread amounts. A digit read wrongly changes the total, and the size of the gap often identifies the row — a difference divisible by nine points at a transposition, as covered in why reconciliations fail.

Lost signs. A minus that did not survive extraction changes the sum by twice the amount. Loud and unmistakable.

Glued amounts. Two adjacent values merged into one number — the failure that looks most plausible to a human reader and is obvious to the arithmetic.

FlowParse
flowparse.io

The rule that decides everything

One sentence covers the entire boundary, and it is worth memorising because it answers every case: if the error changes a sum, arithmetic finds it; if it does not, arithmetic cannot see it.

Apply it to anything. A payee misread — does that change the sum? No. Invisible. A reference number wrong — sum unchanged. Invisible. A date shifted — sum unchanged. Invisible. An amount wrong — sum changes. Caught.

The rule also explains the one asymmetry people find surprising: the fields that matter most for accounting are largely the ones arithmetic ignores. Amounts are what the maths protects, and amounts are only part of what makes a bookkeeping entry right.

Why this is not a reason to skip the check

Dropped rows are both the most common serious failure and the hardest to notice by eye, and the identity catches them completely. Knowing its boundary tells you what to put in front of the rest — not that the check is weak.

Payees and descriptions

The description is where most of the accounting meaning lives, and it has no arithmetic protection whatsoever.

A payee read with one character wrong is still a plausible string. Nothing about it looks broken, and the amount beside it is correct. But that string decides which expense account the cost lands in, how it is treated for tax, whether it matches a supplier record, and whether any automated categorisation rule recognises it.

So a single wrong character can silently move a cost into the wrong category, break a rule that has worked for a year, or create what looks like a brand-new counterparty appearing once. The money is entirely right and the bookkeeping is wrong.

What catches it is not arithmetic but memory. A payee that has appeared on every statement for two years and shows up once with a different spelling is almost certainly a misread, not a new supplier. Comparing extracted payees against the set you have seen before turns an invisible error into an obvious one.

FlowParse
flowparse.io

References, and the chain they break

Reference numbers are among the highest-consequence fields on a statement and among the least protected: long strings of characters, often in small print, with no relationship to any other value on the page.

Get one wrong and the payment stops matching the invoice it settles. The invoice then appears unpaid, which produces chasing, an unallocated cash entry, possibly a second payment — and a supplier relationship briefly strained over something that was paid correctly and recorded incorrectly.

Note the chain: a misread reference is a plausible origin for a duplicate payment, which is the subject of duplicate payments. An extraction error becomes a cash error two steps later, and by then nobody is looking at the extraction.

Format validation is the first defence — a reference that does not fit the expected shape for that counterparty is suspect before anyone tries to use it. Matching is the second: where references normally match, one that matches nothing is itself a signal worth surfacing rather than a dead end.

FlowParse
flowparse.io

Dates, which people assume are covered

This is the one that surprises people most. The arithmetic is entirely indifferent to when transactions happened — it sums amounts, and a sum does not care about order or timing. A statement with every date wrong reconciles exactly as well as one with every date right.

The consequences are real. A date shifted across a period boundary moves a transaction into another month, and possibly another quarter, reporting period or tax year. Every individual record looks perfect in both the period it left and the one it arrived in.

Two distinct causes produce it. Character confusion is one — a digit misread shifts a date by days. Convention ambiguity is the other, and it is more systematic: a whole file read under the wrong day-first or month-first convention displaces a third of its transactions, which is the subject of dates and numbers that lie.

What catches them is cheap: every date must fall within the period the statement claims to cover, dates on a statement should be broadly ordered, and a jump of several months mid-document is implausible. None of that requires arithmetic and all of it can run automatically.

FlowParse
flowparse.io

Account numbers and bank details

Long digit strings, high consequence, and no relationship to any other value on the document. If a bank detail is misread from a document and used to pay someone, the failure is expensive and immediate.

Some account identifier formats carry check digits, and where they do, validating them is genuinely valuable — an error that breaks the check digit is caught before anything happens. But not every format offers one, and a check digit confirms the shape of the number rather than that it belongs to the intended party.

The stronger control is not technical. Bank details for a supplier should come from an established record, verified out of band, rather than from whatever a document says today — a discipline that also protects against a class of payment fraud that has nothing to do with extraction at all.

The general principle: for the highest-consequence fields, do not rely on any reading at all. Take the value from a record you control and use the document only to confirm it.

FlowParse
flowparse.io

Misreads that cancel

The pure case: one amount read too high and another too low by exactly the same value. The sum is untouched, the identity holds, and two figures are wrong.

This is rare, because it requires a coincidence — two independent errors of identical magnitude in opposite directions. It is worth naming anyway, because it is the one failure that is genuinely undetectable by any check that examines totals, however carefully you run it.

It becomes slightly less rare in one situation: where a document has a repeated structural quirk that produces systematic misreads. Then errors are not independent, and correlated errors can offset each other more often than chance would suggest.

The only thing that surfaces it is examination of individual rows against the source — sampling. Which is the honest answer, and also the reason sampling never fully goes away no matter how good the automation gets.

LayerCoversBlind to
Totals identityCompleteness and amounts in aggregateEverything that is not an amount
Balance chainA misread amount on a row with a balanceA row dropped entirely
Shape validationDates, references, identifiersValues that are well-formed and wrong
Cross-document consistencyPayees and recurring patternsA genuinely new counterparty
SamplingAnything, in principleEverything not sampled

When there is no identity to run

Everything above assumes the check can run at all. Sometimes it cannot, and knowing when is as important as knowing what it covers.

No printed balances. Some documents carry no opening and closing figures — a transaction listing rather than a statement. There is nothing to test against, and no arithmetic backstop exists.

Several accounts under one header. One opening and closing pair cannot span several running-balance chains. Summing everything and comparing produces a number with no meaning.

Damaged totals. If the printed balances themselves were misread, the check is testing against a wrong target — and may fail on a correct extraction or, worse, pass on an incorrect one.

In each case the honest report is that the check could not be evaluated. A tool that shows a pass when it did not actually run is manufacturing confidence, which is worse than reporting nothing — this is why our own totals check stands down on multi-account documents rather than displaying a result.

FlowParse
flowparse.io

The repair trap

This one is a design hazard rather than a reading error, and it is the most dangerous thing in this article because it turns a caught failure back into a silent one.

If a system, on finding that the totals do not balance, adjusts values until they do, it has not fixed anything. It has removed the evidence. The output now reconciles perfectly and contains whatever adjustment made the arithmetic work, which is a strictly worse position than an honest failure.

There is a narrow version that is legitimate, and the distinction is precise. A repair is safe only when it is proved by the document: if removing a row that was probably a subtotal makes the totals hold exactly, against the bank's own printed figures, the document has confirmed the hypothesis. If it merely reduces the gap, it is tuning, and it must be rejected.

The test is whether the correction is validated by evidence that existed before the correction was attempted. Exact, or nothing — anything in between is a system persuading itself.

What to ask a vendor

Ask what happens when the totals do not balance. "We correct it automatically" deserves a follow-up: corrected how, and proved by what? A gap that is reported is a gap you can investigate. A gap that was quietly closed is one you never hear about.

The confusions that actually matter

Character recognition errors are not uniformly distributed — they cluster among shapes that resemble each other, and which ones bite depends on which field they land in.

Digit confusions among similarly shaped numerals are the classic case: several of the round and closed digits resemble one another, and the tall narrow ones do too. In an amount these are caught by the identity. In a date, a reference or an account number, the same confusion is completely unprotected.

Separators are their own hazard. A decimal point can be lost to a poor scan, or invented from a speck of dust, and either changes a value dramatically. A minus sign can vanish or be confused with other short strokes. Both of these do move sums, so both are caught — which is a genuine relief.

And in descriptions, letter-digit confusions produce text that reads oddly to a person and matches nothing to a machine. Those are the ones worth surfacing for review, because a human resolves them instantly and no automated check will.

Where the confusion landsChanges a sum?Caught by the identity?Consequence
An amountYesYesCaught before it reaches the ledger
A decimal separatorYes, dramaticallyYesCaught
A minus signYes, by twice the amountYesCaught
A dateNoNoTransaction lands in the wrong period
A payee or descriptionNoNoWrong category, broken rules, wrong tax treatment
A referenceNoNoPayment stops matching; chasing and duplicates
An account numberNoNoSevere if used to pay someone

What catches each one instead

For every gap the arithmetic leaves there is a cheap check that closes most of it. None of these is clever; all of them are things nobody gets around to.

ErrorThe check that finds itCost
Misread payeeCompare against payees seen before on this accountAutomatic once the history exists
Broken referenceFormat validation, then match failure as a signalA rule per counterparty format
Date outside the periodRange check against the statement's stated periodTrivial, and it never false-positives
Date out of sequenceOrder check down the statementTrivial
Wrong day-first / month-first readingInfer the convention from the whole columnOne pass over the dates
Account numberCheck digits where the format has them; use held recordsPolicy more than technology
Compensating misreadsSampling of individual rows against the sourceA handful of rows per document
No identity availableReport it as not evaluated, and sample insteadHonesty
FlowParse
flowparse.io

Layers, not a single check

The conclusion is not that validation is hopeless. It is that one check cannot cover fields it does not examine, and the answer is several cheap checks with different blind spots rather than one clever one.

Arithmetic covers completeness and amounts, decisively, and nothing else. Shape validation covers dates, references and identifiers by testing whether values look like what they claim to be. Consistency across documents covers payees and recurring patterns by comparing against what came before. Sampling covers whatever the other three cannot, which is a small residue but never zero.

The four together are strong. Any one alone leaves a hole you could lose a quarter through, and the most dangerous configuration is one strong check that everybody believes is comprehensive.

Notice also that the two arithmetic checks already have different blind spots between them — the row-to-row balance chain misses a row dropped entirely because the chain closes over the gap, while the totals identity catches exactly that. Even within one layer, redundancy is doing work.

Accuracy, honestly

We will not promise you 99 % — we will show you which rows to check

Every converter in this market advertises a number: 99 %, 99.5 %, 99.8 %. None of them publishes how it was measured or on which documents — and none of them can tell you which rows fall in the remainder. That is the part you find out later, when two amounts have been glued into one, a column has slid one place to the left, and a reconciliation will not close.

Our extraction is strong. It is also not magic — and neither is anyone else’s.

Reading a PDF is not a solved problem. A layout nobody has seen before, a faded thermal receipt, a bank that marks credits in its own way — each of those can produce a row that looks perfectly ordinary and is wrong. We build hard against that, and we still refuse to sell you a number, because the number is not the thing that protects you. Knowing exactly where to look is.

What actually goes wrong when a PDF is read

These are not hypotheticals. Every one of them is a defect we have found in real documents, reproduced, and built a check for — which is precisely why we can now point at them instead of averaging them into a percentage.

The defectWhat you see in the exportWhat it costs
Two amounts glued into oneOne plausible-looking figure instead of two rowsA total that is short by a whole transaction
A column slides one placeDates in the description, amounts in the balanceEvery row after it is wrong, and none looks wrong
A reference number read as the amount“Payment 910015” booked as 831.00A five-figure hole on a long statement
A credit sign droppedAn expense recorded as incomeThe error is twice the amount, in the wrong direction
A summary box counted as bookings“Previous balance / New balance” added as rowsTotals inflated by exactly the closing balance, twice
A page silently skippedA month that is simply shorterNothing to see — that is what makes it the worst one

So we built the layer that catches them

A second engine, deterministic — ours, and it runs on every document

After the AI reads the document, a separate layer re-does the document’s own arithmetic. Opening balance plus every transaction must equal the closing balance the bank printed. Each row’s running balance must follow from the one above it. Line items must sum to the invoice total. No AI, no confidence score, no guessing — these are proofs, and a document that fails one is provably misread.

It names the rows, not a percentage

When a check fails you do not get a lower score. You get row 48, row 133, row 1 902 — highlighted in place, red where a check proved the reading wrong and amber where it could not confirm it, with the column the check named brighter still. Open the Rows or JSON view, fix those, export. That is the whole loop, and it usually takes under a minute.

The document is the judge, not us

A bank statement carries its own proof: it can catch an error with no human, no reference data and no opinion from us. That is why we lead with it. Where a document genuinely cannot check itself — no running balance, no printed total — we say so, plainly, instead of letting silence imply that everything is fine.

Why the percentage is the wrong number to buy on

Statement sizeWhat “99 % accurate” quietly allowsWhat that means for you
300 rows3 wrong rowsAn afternoon — if you find them
1 200 rows12 wrong rowsA reconciliation that will not close
3 400 rows34 wrong rowsA six-figure error, in cases we have seen

Money work rewards being picky

Please read the flagged rows before you export. That is not a disclaimer — it is the one step that turns a good extraction into a correct one, and we have spent our engineering effort on making it short and precisely targeted rather than on rounding a number up. Anyone can print 99 %. Telling you exactly where the other 1 % is, is the harder promise, and it is the one we are willing to make.

Designing review that people actually finish

All of this ends with a person looking at something, and the design question is how to make that list short enough to be read properly rather than skimmed.

Order by consequence multiplied by probability. Rows that a check flagged come first, because something concrete is wrong. Unmatched references second, because the consequences cascade. New payees third, because they are cheap to confirm and frequently misreads. Dates at the period boundary fourth, because that is where displacement costs most.

Then stop. A review list of eight items gets read; a list of two hundred gets scrolled past, and a reviewer who has learned that the list is mostly noise stops reading the part that is not. Fewer, better-chosen items beat completeness in a queue that a human has to work through.

And show the source alongside. The single largest determinant of whether review works is whether the reviewer can see the original document region next to the extracted value without hunting for it.

FlowParse
flowparse.io

Where we stand on this

We could reasonably have not written this article. The balance identity is our strongest argument, and enumerating what it does not cover weakens a good sales point.

We would rather state the boundary than have someone find it. A customer who believes a passing check means the document is entirely correct will eventually be wrong about a date or a payee, and will be much more annoyed to discover the limit afterwards than to have been told it at the start.

So concretely: we run the identity on every statement and report the gap and the implicated rows when it fails. We report that it could not be evaluated rather than showing a pass when a document makes it impossible. We never invent a total that was not printed in the source. And a repair is kept only when it makes the document's own arithmetic hold exactly — never when it merely narrows a gap.

None of that makes an extraction infallible. It makes the failures visible and bounded, which is the most any tool in this category can honestly offer — and considerably more than a percentage.

Key takeaways

The balance identity catches anything that changes a sum — dropped rows, invented rows, misread amounts, lost signs, glued values — and is completely blind to anything that does not. That single rule tells you, for any error, whether arithmetic is your defence.

What survives it: payees, references, dates, account numbers, and the rare pair of misreads that cancel. Several of those matter more for the accounting than the amount does, and each has a cheap non-arithmetic check that finds it — range checks, format validation, comparison against payees seen before.

Beware any system that closes a totals gap by adjusting values. A repair is only legitimate when the document's own printed figures then hold exactly; anything less is a system removing the evidence of its own error.

Frequently asked questions

A check that tells you what it checked

Convert a statement and see the identity result, the rows any check implicated, and an explicit note when the check could not be evaluated at all. Then review the short list — with the source beside it.

FlowParse
flowparse.io

Related reading