When I researched which operational pain points hit small and mid-sized businesses hardest, one came back ahead of everything else: document processing. Not strategy, not model selection, not chatbots. The boring, high-volume work of turning invoices, receipts, contracts, and statements into structured data that the rest of the business can use.
The numbers explain why. Processing a single invoice by hand costs roughly $12 to $16, while an automated pipeline brings that down to $2 to $4, a 70 to 80 percent reduction. At volume the gap is brutal: automating around 1,000 invoices a month saves on the order of $120,000 a year. Manual data entry also carries a 1 to 3 percent error rate per document, and once a wrong number lands in your accounting system it spreads to every downstream report that reads from it.
So the business case is settled. You point a vision-capable model at the document, it reads the fields, you store them. I built exactly that. It worked in the demo and fell over the moment real usage arrived. The model was never the hard part. The ingestion layer around it was, and that is the part almost nobody writes about.
The naive version works in the demo
Here is the core of the first build: a FastAPI backend, a Next.js frontend, Postgres for storage, and Claude Sonnet 4 doing vision-based extraction. A document comes in, gets base64-encoded, and goes to the model with a prompt that defines a strict JSON contract.
# claude_service.py: the extraction callEXTRACTION_PROMPT = """Analyze this document and extract structured data.Return a JSON object with: "doc_type": one of "invoice", "receipt", "contract", "statement", "other" "confidence": overall confidence in the extraction (0.0 to 1.0) "fields": array of {name, value, type, confidence}For currency values, use plain numbers (e.g. "1250.00").For dates, use ISO format YYYY-MM-DD.Return ONLY the JSON object, no markdown."""def extract_document(file_path: str, mime_type: str) -> dict: client = anthropic.Anthropic(api_key=settings.anthropic_api_key) b64 = base64.standard_b64encode(Path(file_path).read_bytes()).decode() if mime_type == "application/pdf": block = {"type": "document", "source": {"type": "base64", "media_type": "application/pdf", "data": b64}} else: block = {"type": "image", "source": {"type": "base64", "media_type": mime_type, "data": b64}} response = client.messages.create( model="claude-sonnet-4-20250514", max_tokens=4096, messages=[{"role": "user", "content": [block, {"type": "text", "text": EXTRACTION_PROMPT}]}], ) return json.loads(response.content[0].text)
That is genuinely most of the intelligence. Claude classifies the document, pulls vendor, invoice number, dates, totals, line items, and tax, and returns a confidence score for the document and for each field. On a clean PDF it is close to magic.
The mistake is thinking that function is the product. It is one call. The product is everything that has to happen reliably around it, thousands of times a day, when the inputs are messy and the network is not.
Where it broke under load
The first version ran extraction inside the request process, using FastAPI’s BackgroundTasks:
# documents.py: the upload endpoint, naive version@router.post("/upload")def upload_document(file: UploadFile, background_tasks: BackgroundTasks, ...): doc = save_document_record(file, user) background_tasks.add_task(run_extraction, str(doc.id)) # runs in the API process return {"id": doc.id, "status": "pending"}
This is fine for one user clicking one button. It is a liability under real conditions. Five problems showed up fast, and every one of them lives in the ingestion layer, not the model.
First, the extraction ran in the same Uvicorn process serving the API. A vision call against a multi-page PDF takes several seconds and is memory-hungry. A handful of concurrent uploads competed for the same thread pool and the entire API became unresponsive. The health check that should take 5 milliseconds was timing out behind a queue of base64 blobs.
Second, there was no retry path. The Anthropic API, like any API, returns 429s and the occasional 5xx. In the naive version a single rate-limit response marked the document failed permanently. The user’s only recovery was to delete and re-upload.
Third, duplicate and anomaly detection were O(n) Python loops. After extraction, the system checked the new document against the user’s history to catch duplicate charges and price spikes. The first implementation pulled every prior document and its fields into memory and compared in a loop. At a few dozen documents you do not notice. At a few thousand it is a wall.
Fourth, files lived on the local filesystem. That ties every file to one server and makes horizontal scaling impossible, because a worker on another container cannot read a path that only exists on the API box.
Fifth, there was no concurrency control and no fairness. One user uploading a 500-document backlog could consume the entire Claude rate limit and starve every other user’s single-page upload.
None of this is a model problem. The extraction quality was fine throughout. The system was failing at ingestion: how work enters, gets queued, retried, stored, and fanned out.
The ingestion layer that fixed it
The rebuild kept the extraction call almost untouched and rearchitected everything around it. The diagram below is the before and after.
The four changes that mattered:
Move processing out of the request path. Extraction, post-processing, and usage tracking became Celery tasks behind a Redis broker. The API now does one thing on upload: persist the record and dispatch a job. It returns in milliseconds regardless of how heavy the document is. The workers scale independently of the web tier.
# tasks/extraction.py: extraction as a retryable Celery task@celery_app.task( bind=True, autoretry_for=(RateLimitError, APITimeoutError, APIConnectionError), retry_backoff=2, # 2s, then 8s, then 32s retry_backoff_max=32, max_retries=3, retry_jitter=True,)def extract_document_task(self, document_id: str): doc = load_document(document_id) file_bytes = storage.download(doc.storage_key) # from object storage, not local disk result = extract_document(file_bytes, doc.mime_type) persist_extraction(doc, result) run_post_processing.delay(document_id) # chain, do not block
The distinction that matters here is between retryable and non-retryable failure. A 429 or a timeout should back off and try again, because the next attempt will probably succeed. A malformed model response or an unreadable file should fail immediately and loudly, because retrying it just wastes money and tokens. Collapsing those two categories into one generic except is how you get either silent data loss or infinite retry loops.
Abstract storage behind an interface. Files moved off the local disk to S3-compatible object storage (MinIO in development, any S3 provider in production) behind a small protocol so the workers and the API read the same bytes from anywhere.
class StorageBackend(Protocol): def upload(self, content: bytes, key: str) -> str: ... def download(self, key: str) -> bytes: ... def get_url(self, key: str) -> str: ... # presigned, short-lived
That one boundary is also what makes data residency a configuration choice instead of a rewrite, which matters the moment a regulated client asks where their documents physically live.
Replace O(n) scans with indexed queries. Cross-document checks moved from Python loops into indexed SQL. Duplicate detection and the vendor-history lookups that feed anomaly detection now hit indexes on user_id, status, and vendor_name instead of materializing the whole history per upload. The work the model does is per-document; the work the pipeline does is per-relationship, and relationships are what blow up at scale if you do not push them into the database.
Add fairness and a cost ceiling. Worker concurrency caps the number of simultaneous Claude calls, and a per-user rate limit (5 documents per minute by default) means one bulk upload can no longer starve everyone else. Token counts from each call are recorded so cost is tracked per user and per document instead of disappearing into a log line.
Why this is a data-readiness story, not a scaling story
It would be easy to file this under performance work. It is not. It is about what “ready for AI” actually means once you move past a demo.
The earlier post for business owners said the thing that has to be true before AI helps you is that your data is in a shape a tool can read. That is correct, and incomplete. Clean inputs are necessary but they do not survive contact with a 1 to 3 percent error rate and they do not give you cross-document intelligence. Two design decisions close that gap, and both live in the ingestion layer.
The first is confidence as a control surface, not a decoration. Every field comes back with a model confidence score, stored alongside the value, with a flag that records whether a human later edited it.
class ExtractedField(Base): field_name: Mapped[str] field_value: Mapped[str] field_type: Mapped[str | None] confidence: Mapped[float | None] # routes low-confidence fields to review edited: Mapped[bool] = mapped_column(default=False) # human-in-the-loop audit trail
That single confidence column is what lets you route the uncertain 2 percent to a human instead of letting it flow silently into the books. The edited flag gives you an audit trail of exactly which machine outputs a person corrected, which is both a quality signal and, later, training data.
The second is cross-document validation, the part a clean spreadsheet alone will never give you. Extraction tells you what a single invoice says. Ingestion is where you catch that this invoice is 4x the vendor’s historical average, or a duplicate of one from three days ago, or that a monthly vendor has gone silent for 50 days. That validation layer is the difference between a pipeline that reads documents and one that catches the expensive mistakes hiding in them.
This is the same conclusion the broader field keeps reaching. One industry analysis of document processing puts simple LLM wrappers at 66 to 77 percent accuracy on complex extraction, while validated, agentic pipelines reach 93 to 98 percent. The model in both cases can be identical. The accuracy gap is the validation, retry, and reconciliation logic wrapped around it, and a healthcare deployment cited by Automation Anywhere reached 98 percent classification accuracy precisely by adding that agentic layer rather than swapping models.
The compliance and security layer you cannot bolt on later
Documents are the most sensitive data most businesses hold: bank statements, contracts, anything with names and account numbers. The ingestion design decides whether that is handled well or handled badly, and the decisions are cheap up front and expensive in arrears.
Storage carries the personally identifiable information, so it has to be encrypted at rest and reached through short-lived presigned URLs rather than public paths, with the storage-residency boundary mentioned above making jurisdiction a setting. Access has to be scoped to the owning user on every read, every list, and every export, with no shared global queries that quietly cross tenant boundaries. The edited flag does double duty as an audit trail, recording which machine outputs a human touched, which is the kind of provenance an auditor asks for. And capturing token cost and processing outcome per document is the difference between an answerable “what did this client’s processing cost and what happened to it” and a shrug. I have sat through NIST 800-171 and CMMC assessments, and the pattern holds without exception: the teams that designed these controls into the data path passed cleanly, and the teams that planned to add them later paid for it twice.
What to build before you call an extraction pipeline “done”
If you are putting a vision model in front of real documents, the model is the part most likely to already work. Pressure-test the ingestion layer instead.
- Get processing out of the request path. A queue with workers, not in-process background tasks. The API should dispatch and return.
- Separate retryable from fatal failures. Back off and retry on 429s, timeouts, and 5xx. Fail fast and loud on malformed output or bad input. Never collapse the two.
- Decouple storage from compute. Object storage behind an interface so any worker reads any file, and data residency is a config value.
- Push cross-document work into the database. Indexed queries, not Python loops over history. Per-document cost is linear; per-relationship cost is not.
- Treat confidence as routing, not decoration. Store it, threshold on it, and route the uncertain tail to a human with an audit trail.
- Add fairness and a cost ceiling on day one. Per-user limits and bounded concurrency so one upload cannot starve the system or your budget.
The demo proves the model can read a document. The ingestion layer is what proves you can run that ten thousand times a day without losing data, starving a tenant, or quietly booking a wrong number. That is where production AI is actually won, and it is built from the same unglamorous infrastructure discipline that has always separated systems that work from systems that demo.
The next post takes apart what “AI-powered” actually means at the architecture level, including how to tell a real model doing real work from a keyword search with a chat box on top. Connect with me on LinkedIn or see the production side at coreagenticgroup.com.
