How to choose a document parsing API

Define the job before you look at vendors

Write down six things first, because each of them eliminates vendors before any benchmark runs, and eliminating early is what keeps an evaluation to two days rather than three weeks.

Document types.Not "financial documents" but the actual list: supplier invoices, bank statements, receipts, remittance advice, purchase orders, pay stubs. Each has different structure and different failure modes, and coverage claims are usually strongest on invoices because that is where the market is.

Volume and shape. Documents per month, pages per document, the ratio of one-page receipts to twenty-page statements. This single distribution decides which pricing model favours you, and almost nobody writes it down before comparing prices.

Inputs. What proportion arrives as digital PDFs, scans, photographs, or email attachments of unknown provenance. A pipeline that is ninety per cent digital PDFs is a different procurement from one that is half phone photographs taken in a warehouse.

Destination. Whether the output feeds a database, a ledger, a spreadsheet or a model. A ledger destination raises the bar on completeness and on export formats; an analytics destination may tolerate gaps that a ledger cannot.

Constraints. Data residency, retention rules, whether documents may leave your infrastructure at all, and any certification your customers demand. These are hard filters — apply them on the first call rather than after the benchmark.

General platform or financial specialist

The market splits into two families that are genuinely different products, and choosing the wrong family is the most expensive mistake available in this category.

General document platforms and parsing primitives handle anything: contracts, forms, medical records, logistics documents. They give you flexibility — often a schema you define, sometimes markdown or chunks for a model — and they hand you the work of turning that output into the specific numbers your process needs. If your document set is diverse or unusual, this family is the right answer.

Financial specialists cover a narrow set of document families and know their conventions: debits and credits, running balances, tax breakdowns, line items that cross pages, decimal commas, day-first dates. They are less flexible by construction and typically require no schema authoring at all. If your documents are statements and invoices, this family does more of your job.

The tell is what the API returns. If it hands back text or markdown and expects a second step — often another model call — to produce numbers, you are in the first family and you own that step, including its errors. Our comparison against document-to-markdown parsers covers that distinction in detail; it is the difference between output for a model and output for a ledger.

flowparse.io

Why the accuracy percentage is the wrong metric

Every vendor publishes one and none of them are comparable. The figure is measured on the vendor's own test set, using the vendor's own definition of a field, at the vendor's own choice of granularity — character, field or document. A move from "99.5% character accuracy" to "95% document accuracy" is a change in wording, not in quality.

More importantly, published accuracy almost always describes captured fields: of the values the system extracted, how many were right. That says nothing about the values it never extracted. A system that reads eighty per cent of a statement's rows perfectly can report exceptional accuracy while quietly losing a fifth of your transactions.

So replace the question. Instead of "how accurate is it", ask "how would I know when it is wrong". That question has an answer you can test, and it is the one that determines how much manual checking your team does forever. Our own view of what accuracy figures can and cannot mean is on extraction accuracy.

Completeness, and the one way to prove it

Accuracy and completeness are separate properties. Accuracy asks whether the values captured are right; completeness asks whether anything is missing. Only the second is provable without ground truth, and only the second fails silently — a misread character often looks odd, while a missing row looks like nothing at all.

Financial documents carry their own audit, which is what makes this testable. A bank statement prints an opening balance, a closing balance and a series of transactions; if opening plus the transactions does not equal closing, something is missing or misread, no matter how confident the output looks. An invoice offers a weaker version: line items summing to the net, net plus tax equal to the gross.

Build that check into your benchmark and run it on every vendor's output. It costs a dozen lines of code, needs no labelled data, and it is the single most informative number in the evaluation — the share of documents that reconcile. Ask each vendor whether they run it themselves and whether they expose the result; the answers are revealing. The reasoning in full is on bank statement validation.

flowparse.io

Build a thirty-document benchmark

Thirty documents is the sweet spot: enough to separate vendors, small enough that you will finish. What matters is composition, not count — a benchmark of thirty clean invoices tells you nothing you did not already assume.

Freeze the set and version it. You will want to re-run it in six months when a vendor claims an improvement, when your document mix changes, or when someone proposes switching. Store the raw API responses alongside the documents, not just your scored summary — the responses are what let you re-score against a metric you had not thought of yet.

Run every vendor through the same harness, with the same code path, on the same day. Differences in how you call each API are a real source of noise, and a vendor that scores badly because your integration was rushed has been evaluated unfairly.

Scoring without labelled data

Hand-labelling thirty documents is a day of tedious work and most teams skip it, so design a scoring scheme that does not need it. Four measures do the job.

Reconciliation rate. The share of statements whose arithmetic checks out, and of invoices whose lines sum to the total. Objective, automatic, and the single number most predictive of production pain.

Corrections per document. Open each result next to the original and count the fields you would have to fix. Ten minutes per document, thirty documents, and it is the most honest measure of what your operators will experience.

Failure honesty. For each document that did not work, record whether the API told you. An explicit error is a good outcome. Empty fields with a success status is a bad one, and silent partial output is the worst — it is the case your pipeline will not catch.

Time and cost per document. Wall-clock latency and the billed amount, recorded per document rather than averaged, so you can see what the awkward ones actually cost.

flowparse.io

Decoding the pricing model

There are four models in this market and they are not comparable without your document mix. Take the distribution you wrote down at the start, put it in a spreadsheet, and compute a real monthly bill for each vendor. It takes twenty minutes and it routinely reorders the shortlist.

Credit systems deserve particular attention because the multipliers are where the real price lives. It is entirely normal for a credit-based vendor to charge one credit for an invoice, two for line-item extraction, and several per page for a bank statement — which means a month of statements can cost many times what the headline suggests. That is not dishonest, but it is only visible if you read the multiplier table and apply it to your mix.

Watch for the definition of a page as well. Some vendors count a document of up to a certain number of pages as a single unit; others count every page including blank separator sheets from a scanner. On a hundred-statement month those two definitions produce very different invoices.

The pricing traps

Billing for documents that failed.Ask explicitly whether a document the API could not read is charged. Charging for a failure is charging you for the vendor's limitation, and it also removes their incentive to fail fast rather than slowly.

Retries. If your client library retries automatically — and most do — every retry is usually a new billable call. A default retry policy of four attempts turns one transient error into four charges, and nobody notices until the invoice.

Re-extraction. If you discover in three months that you needed line items as well as totals, do you pay again? Almost always yes. That is an argument for storing the full response the first time rather than the fields you currently use.

Minimums and terms. An annual commitment with a monthly minimum is a different risk profile from pay-as-you-go, particularly for a first integration where your volume estimate is a guess. Price the exit as carefully as the entry.

The costs that never appear in the comparison

Unit price is usually the smallest number in the total. Three others are larger and none of them are on a pricing page.

Exception handling.If one document in twenty needs a human, and a human takes four minutes, then a thousand documents a month is over three hours of somebody's time. A vendor with half the exception rate is worth a materially higher unit price, and your benchmark is where that ratio becomes visible.

Integration and maintenance. A poorly documented API with an unstable schema costs engineering weeks that never appear in the business case. So does an API whose errors are ambiguous, because every ambiguous failure becomes a support conversation.

The errors you do not catch. The most expensive category and the hardest to price: a wrong number that reaches a ledger, a report or a credit decision. This is why completeness beats unit price in any sensible weighting — the cheapest vendor with silent failures is not cheap.

Latency, timeouts and the async question

Measure latency per document type rather than as an average. A one-page invoice and a twenty-page scanned statement are different workloads, and an average across them describes neither. What you need is the worst case for each type you will actually send.

Then find the provider's own timeout and write it down, because your client timeout must sit above it. A client that gives up at thirty seconds on a call the provider will spend ninety seconds completing produces the worst outcome available: you abandon the connection, the work completes, and you are billed for a result you never received.

Synchronous APIs are simpler and fine when documents finish quickly. Asynchronous APIs with a callback are necessary when they do not. What matters more than the model is whether your architecture matches it — if the API is synchronous, the queue and the retry logic are yours to build, which is exactly what our webhook automation guide covers.

If you are calling from an automation platform rather than your own code, its limits are usually tighter than the API's. Power Automate stops a synchronous HTTP call at two minutes and Airtable stops a script at thirty seconds — the practical consequences are on the Power Automate and Airtable pages.

API ergonomics: the details that decide the integration

Six things determine how much code you write, and you can check all of them in an hour with the documentation and a terminal.

Two of those rows deserve emphasis. Values returned as strings mean every consumer parses numbers and dates itself, and one of them will get a locale wrong. And a missing idempotency mechanism means duplicate protection is your problem — worth knowing before a retry storm rather than after.

An official SDK matters much less than people expect. A document API is normally one POST with a file, so a client library saves a handful of lines and adds a dependency with its own release cadence. Judge the HTTP interface; if it is clean, no SDK is needed, and our own Python integration example is fifteen lines for exactly that reason.

Schema stability, and what happens on a breaking change

Ask how the response schema is versioned and how changes are announced. A vendor that adds fields freely and never removes them is easy to live with; one that renames things in a minor release will break your parsing on a Tuesday morning with no warning.

Protect yourself regardless of the answer. Parse defensively — treat every field as optional, never assume a nested object exists — and keep a contract test in your suite that runs against a saved response and a live call, so a schema drift is a failed build rather than a production incident.

Store the raw response. It costs almost nothing, it lets you re-derive fields you did not originally map without paying to extract again, and it is the only way to reconstruct what a vendor returned on a document that later turned out to be wrong.

How it fails is more important than how it succeeds

Every vendor performs well on a good document. The differences show up on bad ones, and the axis that matters is whether failure is loud or quiet.

Loud failure is an explicit error with a reason: unreadable scan, unsupported type, nothing extractable. It is easy to route, easy to explain to a user, and it is not a data-quality risk. Quiet failure is partial output with no signal — some rows, some fields, a confident-looking response — and it is how bad numbers enter systems.

Test this deliberately with the deliberately-bad documents in your benchmark. Send a blank page, a photograph of something irrelevant, a password-protected PDF, and a document of a type the vendor does not support. Record exactly what comes back for each, and prefer the vendor whose answers are the most boring.

flowparse.io

Outputs beyond JSON

If the destination is a database or a model, JSON is all you need. If the destination is a person or a ledger, ask what else the API produces — a formatted workbook, a CSV shaped for a specific accounting package, a bank-feed file such as QBO or OFX. Producing those yourself is not hard, but it is a week of fiddly work per format and a maintenance obligation afterwards.

Be specific about which formats matter to you. "Exports to accounting software" can mean a generic CSV that needs mapping on import, or a file the software ingests directly. The difference is felt by whoever does the import every month, so ask them.

Check the multi-document case too. A month of statements is often more useful as one consolidated workbook than as thirty JSON responses, and whether the API can consolidate — with column alignment across banks — decides whether you build that yourself. Our take on that is on merging documents into one workbook.

The data-protection questions to ask in writing

Five questions, and the answers belong in the contract rather than in a sales call. Where is processing performed? How long are original documents retained? Are customer documents used to train models? Who are the sub-processors? Is a data processing agreement available?

For regulated work, add certifications and audit evidence, and ask what happens to your data at the end of the relationship. For work under strict internal policy, ask whether any self-hosted, on-premise or private-cloud option exists — and treat a vague answer as a no, because deployment options are the kind of thing a vendor states plainly when they have one.

Be honest internally about which of these are genuine requirements and which are preferences. "Documents may never leave our network" eliminates every hosted API on the market, and if it is real then building it yourself is the honest path — the trade-offs are laid out on parsing statements in Python.

flowparse.io

Human review: whose job is it?

Some vendors sell a review interface, some sell only an API, and a few sell managed human verification. Decide where review belongs before comparing them, because a review UI you cannot embed is only valuable if your operators are willing to work in a second application.

If review lives in your own product, what you need from the API is signals rather than screens: per-field confidence, validation results, explicit failure reasons. Those let you build a queue that shows a person only what needs a decision, which is the design that keeps review costs proportional to volume.

If you are considering managed verification — humans in the vendor's loop — price it against your own exception cost and ask about turnaround time and where those humans are. It can be excellent value at high volume and it changes the data-protection conversation entirely, because now people outside your organisation read your documents.

flowparse.io

Exit, portability and lock-in

Lock-in here is rarely contractual; it is architectural. It accumulates in mapping code, in downstream assumptions about field names, and in an exception queue built around one vendor's confidence flags. After a year, switching feels like a rewrite because in effect it is one.

The defence is cheap if you do it on day one: define your own internal document model and write a thin adapter from the vendor's response into it. Everything downstream depends on your model, so a swap means rewriting one file. This also makes running two vendors in parallel — for a bake-off or a fallback — a configuration change rather than a project.

Keep your benchmark and the stored raw responses. Together they let you re-run the comparison in a year with real evidence, which is the only way a renewal conversation becomes a decision rather than an inertia.

The scorecard

Weight the criteria before you see any results — it is the only way to avoid rationalising a preference afterwards. The weights below suit a finance-document pipeline feeding a ledger; adjust them if your destination is analytics, where completeness matters less and throughput matters more.

Score each shortlisted vendor out of ten per row and keep the working. The point is not the arithmetic — it is that a decision recorded with evidence survives the next reorganisation, and the person who inherits the integration can see why it was chosen.

A worked example: a two-day evaluation

A four-person engineering team at a bookkeeping business needed to automate document intake: roughly two thousand documents a month, seventy per cent supplier invoices, twenty per cent bank statements averaging six pages, ten per cent receipts. Around a quarter of everything arrives as a scan, because their clients photograph things.

Morning one. They wrote down the job and the constraints, which eliminated two vendors immediately on data residency. They assembled thirty documents from the last quarter — five of them files that had caused a support ticket — and wrote a fifty-line harness that called an API, saved the raw response and ran the arithmetic checks.

Afternoon one and morning two. Three vendors, half a day each. The reconciliation rates came out clearly separated, and one vendor that looked strongest on paper returned partial statements with a success status on two of the six-page files — the quiet failure they had specifically designed the benchmark to catch.

Afternoon two. They modelled cost against their real mix. The per-page vendor was cheapest on receipts and most expensive overall because statements dominate their page count; the per-document vendor reversed it. The gap between best and worst was roughly double, which nobody had guessed from the pricing pages.

They chose the second-cheapest option, on the grounds that its exception rate was less than half the cheapest and every exception costs a person four minutes. The evidence — thirty documents, the harness, the scorecard — took two days to produce and has been re-run once since, when a vendor announced an improvement. That re-run took an afternoon because the benchmark already existed.

Our bias, stated

So here is where we would lose. We do not handle arbitrary document types — contracts, clinical records, logistics paperwork — and a general platform will beat us on anything outside the financial families. We are not open source and cannot be self-hosted: there is no on-premise or air-gapped deployment, so a strict never-leaves-our-network policy rules us out. We do not do handwriting.

We also do not publish an SDK, do not offer managed human verification, do not provide a rules or template editor to configure, and do not push data anywhere — the API is synchronous with no callback, so the queue is yours. Whether those are drawbacks depends entirely on the job you defined at the start.

What we would put forward is narrow and testable: on statements and invoices we return normalised, ledger-ready values rather than text for a second model, and every statement is checked against its own balances so completeness is proven rather than assumed. Run your own thirty documents through our API and the documentation, and score us with the same sheet you use on everyone else.

Common mistakes

• Evaluating on the vendor's sample documents instead of your own worst thirty.
• Comparing published accuracy percentages that measure different things on different data.
• Comparing headline unit prices without modelling your real document mix.
• Forgetting that retries and failed documents may both be billable.
• Setting a client timeout below the provider's, so you pay for results you abandon.
• Mapping only the fields you need today and paying to re-extract for the rest later.
• Ignoring how the API fails, then discovering silent partial output in production.
• Skipping the internal document model, so switching vendors becomes a rewrite.

Frequently asked questions

Keep reading