Every operations team in India has a document backlog somewhere. Delivery challans photographed on a warehouse floor. Vendor invoices arriving as scanned PDFs in twenty different layouts. Site inspection photos that someone has to look at and grade. Handwritten gate registers. For twenty years the answer was either a rules-based OCR pipeline that broke whenever a template changed, or a person.
Multimodal models have genuinely shifted this. A model that reads an image and its text together does not need a template, because it understands that the number in the lower right of a table under a column headed “Amount” is a total, in the same way a person does.
What Changed Technically
Classical OCR gives you characters and coordinates. Everything after that, deciding which characters form the invoice number, which block is the shipping address, which row belongs to which line item, was custom code tuned per document type. That code was the expensive part and it was brittle in exactly the way business documents demand flexibility.
Modern vision-language models collapse that layer. Give one a scanned purchase order and a schema describing the fields you want, and it returns structured output directly, handling rotation, mixed English and Hindi text, stamps overlapping printed fields, and tables that continue across pages. Accuracy on clean printed documents is high enough to be useful without any template work at all.
The realistic performance picture, based on what teams report in production:
- Clean digital PDFs. Field-level accuracy in the high nineties. Effectively solved.
- Good quality scans. Low to mid nineties, with errors concentrated in dense numeric tables and low contrast stamps.
- Phone photographs at odd angles. Noticeably lower, and highly sensitive to capture quality. Fixing the capture step is usually cheaper than fixing the model.
- Handwriting. Still the weak point. Usable for triage and routing, not for unattended posting of financial values.
- Photographs judged against criteria, such as damage assessment or site compliance, are strong at categorisation and weak at fine measurement.
Design the Pipeline Around Doubt
The temptation is to treat extraction as solved and post the results straight into the ledger. The failure mode there is quiet and expensive, because a wrong digit in an amount field looks exactly like a right one.
The pattern that works: extract, then verify against something the model did not produce. Invoice totals must equal the sum of the line items. Extracted GSTIN values must pass a checksum and match a vendor master. Dates must fall in a plausible window. Quantities on a challan must reconcile with the open purchase order. Anything that fails a check routes to a human with the original image and the extracted field side by side, so the correction takes ten seconds rather than a full re-key.
Track two numbers over time: the share of documents that clear all checks automatically, and the error rate among those that cleared. The second number is the one that matters. A pipeline auto-clearing 60 percent with near-zero escapes beats one auto-clearing 90 percent with a one percent silent error rate, particularly once DPDP obligations mean you must be able to show what was done with personal data on those documents and why.
Keep the source image linked to every extracted record, and retain the model version used. When a discrepancy surfaces eight months later, that link is the difference between an explanation and an argument.
At Invexa, we build document pipelines with the verification layer designed first. Extraction quality varies with the input you can realistically expect, so the useful engineering is in catching what went wrong and routing it well, not in assuming it did not.