DEV Community: jaume boguñá

How to Replace Expensive LLM Calls in Financial Document Processing

jaume boguñá — Mon, 12 Oct 2026 13:49:20 +0000

Quick answer: Inventory LLM calls by task, volume and variance. Move high-volume, fixed-output tasks such as PDF page classification, entity tagging, OCR and relevance judging to specialist models on Superlinked Inference Engine: one API for OCR, extraction, embeddings and reranking, on SIE Cloud or self-hosted on your own GPUs. Shadow-run against a gold set, then route by confidence.

Open the usage breakdown for a pipeline that ingests PDF annual reports and client decks, and one line stands out. It is not the analyst chat, and it is not the summary of the chairman's letter. It is a page classifier, asking a frontier model "is this the income statement?" on each page of each PDF that comes in. Each call is cheap and nearly identical to the last. There are a great many of them. The model answering has read most of the public internet, and it is being paid to recognise a table heading it could find with its eyes closed.

For US filers on EDGAR, that call should not exist at all. Each XBRL filing ships a FilingSummary.xml and a set of rendered R files that name every statement, and the presentation linkbase orders the line items within each one. The statement map is free. The classifier earns its keep only where no such metadata exists: foreign annual reports published as PDFs, investor decks, data-room uploads and scanned exhibits.

That is the typical shape of an expensive document pipeline: a general model doing high-volume, low-variance work because it was the fastest thing to wire up during the prototype. The fix is to move that work onto specialist models on SIE, one task at a time, with evidence that the replacement holds up on your documents.

Step 0: Take the cheap discounts before migrating anything

Before you move a single task, check two things with your current provider.

Prompt caching. Classification and extraction prompts are mostly fixed text: the instructions, the label set, the output schema and a few examples, followed by a short page. Put the fixed part first and keep it byte-identical across calls, and the major hosted providers bill the cached prefix at a reduced rate. On a prompt that is 90% boilerplate, record that discounted rate, then inventory what still belongs on specialist models.

Batch pricing. Most of this work is not interactive. A quarter's worth of PDFs does not need answers within a second. OpenAI's Batch API and its equivalents at other providers charge about half the synchronous price in exchange for a completion window of up to a day.

Also check the obvious: whether you are sending pages that never needed a model, such as EDGAR statements already named in the R files, or cover pages and blank separators a simple rule can drop. Measure the bill after these changes. Everything below is worth doing only on what is left.

Step 1: Inventory the calls by task, volume and variance

You cannot choose what to move until you can see what you are calling. Tag every LLM request with a task name, then pull a month of logs and build a table. The volumes below are invented, to show the shape rather than any real pipeline:

Task	Monthly calls (illustrative)	Output shape	Input variance
PDF page classification (which statement is this?)	2,000,000	one label from a fixed set	low
Entity tagging (company, auditor, dates)	600,000	spans with labels	low
Table field extraction (revenue, net income, EPS)	400,000	fixed JSON schema	medium
LLM relevance judging on retrieved chunks	300,000	a score or yes/no	low
Analyst Q&A across several filings	20,000	free text	high
Summaries of management discussion	15,000	free text	high

The pattern usually holds even when the numbers differ: the highest-volume tasks tend to have the smallest output space, which is the work specialist models on SIE take.

Record variance honestly. A task with low variance sees inputs that look alike and produces outputs from a small, fixed space. A task with high variance sees new layouts, open questions or reasoning across several tables and several filings.

Step 2: Pick candidates with fixed outputs you can check

A task is a good candidate when all three hold:

fixed output: the answer comes from a closed label set or a fixed schema
high volume: it runs many times a month on similar inputs
mechanically checkable: you can score the output against a label without a person reading prose

PDF page classification, entity tagging, relevance judging and table field extraction are the tasks to move onto specialist models on SIE.

Map each candidate to the kind of specialist model that fits it: a small generative model or a trained classifier for page types, a zero-shot entity model such as GLiNER2 or NuNER_Zero for entity tagging, a cross-encoder reranker in place of an LLM asked "is this chunk relevant?", and a document model plus a structured-output model for table fields. The reranker swap is often the easiest win. A cross-encoder returns a calibrated score per query-chunk pair, which is what the LLM judge was being prompted to imitate.

Start with the task that has the highest volume and the simplest output. In a PDF-heavy pipeline, that is usually page classification.

Step 3: Build the evaluation set first

Do not move a task without a gold set for it. For field extraction, the companion guide in this series on evaluating extraction accuracy covers the method in detail: seed labels from XBRL facts where filings have them, add hand-labelled PDF pages, and score value, scale, unit, sign, period and entity separately. For classification and tagging, a few hundred labelled pages or passages per class is a sensible start. For relevance judging, take a sample of real queries and have a person grade the retrieved chunks.

Include the pages that currently cause trouble: statements split across two pages, restated comparatives, a segment note that looks like a primary statement. A replacement has to hold on those pages, not only on the clean ones.

Step 4: Shadow-run the specialist

Run the specialist model on SIE in parallel with the current model, without using the specialist output yet. Log both answers per request. After a week or two, you have three comparisons:

both models against the gold set
the specialist against the current model on live traffic, where no label exists
latency and failure rates for both

Disagreements on live traffic are the most valuable output of the shadow run. Sample them, label them, and add them to the gold set. They show you where to tighten the schema or swap the model ID on the same SIE client.

Step 5: Route by confidence

Few cutovers need to be all or nothing. Most specialist models give you a confidence signal: a classifier score, a reranker score, an entity model's span confidence, or a structured-output model returning null for a missing field. Use it:

high confidence: take the specialist's answer
low confidence or null: route to review
log every review, with the input, so those cases become the next gold-set rows

Tune the threshold on the gold set to hit the accuracy you need, and watch what fraction of traffic goes to review. If that rate is high, narrow the task or try the next catalog model on the same SIE client.

Step 6: Cut over and keep monitoring

When the specialist meets its accuracy threshold and the fallback rate is stable, make it the default path. Then keep watching:

☐ weekly accuracy on a rotating sample of production traffic, labelled by a person

☐ fallback rate per task, with an alert when it moves

☐ a rerun of the gold set on any model, prompt or parser change

☐ a check after each reporting season, since new layouts arrive in batches

Budget for the migration costs too. If the move also changes how you retrieve, for example a new embedding model alongside the new reranker, that means re-embedding and reindexing the corpus. Any new model means a new evaluation cycle. These are one-off costs, but they are real, and they belong in the plan.

Where SIE fits

Superlinked homepage, featuring the SIE inference engine.

Once you have carved tasks off the frontier model, they need somewhere to run. Moving four tasks to four vendors trades one bill for four integrations. SIE, from Superlinked, serves them behind one API: reranking, entity extraction, OCR, document-to-Markdown and structured output, alongside embeddings. You can use it as a managed service on SIE Cloud, self-host it in your own VPC, or run it locally. On SIE Cloud, per the Cloud pricing page (verified 2026-09-30), GLiNER2 Large is listed at $0.186 per million input tokens for entity extraction, Qwen3-Reranker-0.6B at $6.63 per million pairs (a top-50 rerank is 50 pairs, so about $0.0003 per query), LightOnOCR-2-1B at $2 per 1,000 pages, and Qwen3.8-27B (FP8), which backs doc field extraction, at $0.72 per million tokens in and out. Treat those as the unit prices in your inventory.

For tasks that already run through the OpenAI SDK, the Cloud page shows a migration pattern that only changes the base URL. For a priced Cloud shadow run, use Qwen3.8-27B (FP8) through the native SDK's chat call. Once its labels agree with your current model, a smaller catalog model such as GLiNER2's classification task is a strong production path on the same API. A shadow page classifier looks like this:

from sie_sdk import SIEClient

client = SIEClient(api_key="sk-sie-…", base_url="https://api.superlinked.com")
shadow = client.chat_completions(
    "Qwen/Qwen3.8-27B-FP8",
    page_classification_messages,
    max_completion_tokens=16,
)
label = shadow["choices"][0]["message"]["content"]

Entity tagging uses the native SDK's extract call with a label list, and relevance judging uses score, as the SIE docs show. The engine is open source under Apache 2.0 (GitHub), so a task you move to Cloud can move to a self-hosted cluster later with the same model IDs.

Honest take: SIE is the destination for the tasks you move off a general model: OCR, extraction, embeddings and reranking behind one API. On SIE Cloud, Superlinked runs the GPUs and you pay per token, pair or page, with GLiNER2, the Qwen3 rerankers, LightOnOCR-2-1B and Qwen3.8-27B (FP8) on the price list. Self-host the same Apache 2.0 engine on your own GPUs, and a task you shadowed on Cloud moves to your cluster with the same model IDs. The weights are open and pinned. Page classification, entity tagging and relevance judging share that client with document conversion and field extraction, so four tasks stay on one vendor.

FAQs

Which call should we move first?

The one with the highest volume and the simplest output that free metadata does not already answer. For EDGAR filings, check first whether the R files and presentation linkbase make page classification unnecessary. For PDF annual reports and decks, page classification is usually first: a small, fixed label set, high volume, and errors that break something visibly downstream. Score table field extraction on a field-level gold set, then run that extractor on the same SIE client.

Why not just turn on caching and batch and stop there?

Run Step 0 so the inventory reflects discounted rates. Page classification, entity tagging and relevance judging are still high-volume, fixed-output work, and SIE serves those specialist models behind one API, on Cloud or on your own GPUs. The per-call prices on SIE Cloud are what you set next to the current bill.

Does this just move the operational burden onto us?

On SIE Cloud, Superlinked runs the GPUs. Self-hosting the same Apache 2.0 engine on your own Kubernetes cluster, as the deployment guide describes, keeps documents and model IDs in your environment. Running locally suits development. Evaluation and routing stay in your code either way.

Pay the frontier price only for frontier work

The expensive part of most document pipelines is easy work done at volume. Use EDGAR metadata where it exists, inventory what is left, build the gold set, shadow-run, and route by confidence. SIE is where those specialist tasks run: one API, open weights, Cloud or your own GPUs. SIE Cloud prices each one per token, pair or page, so you can put those figures next to your current bill.

Fivetran vs Estuary for Real-Time ETL

jaume boguñá — Wed, 07 Oct 2026 17:20:10 +0000

Quick answer: Estuary is the better tool for real-time ETL when a consumer needs database changes in seconds: it streams log-based CDC continuously, sub-100 ms in streaming mode per its figure, with a batch schedule you set per destination. Fivetran's managed product is scheduled ELT with a one-minute floor on its two top plans. Pick Fivetran when every consumer can wait for the next sync and your sources are long-tail SaaS.

The phrase "real-time" means something different at every tier of every pricing page, and the two tools in this comparison define it about as differently as two vendors can.

Fivetran's real-time is a sync frequency: the fastest cadence its managed product publishes is one minute, available on two plans, excluded for its Lite connectors and a long list of advertising sources. Estuary's real-time is a mode: change events flow from a database log to a destination as they happen, and "batch" is a schedule you attach to a destination that does not need them that fast.

A team comparing the two usually has one question underneath the search query: whether its slowest acceptable latency is minutes or seconds, and whether it wants to pay for the difference by the row or by the gigabyte.

What Fivetran does better is conceded early, because the catalog argument is not close, and the rest of the words go to the axis the title names. The sibling piece on Airbyte covers the open-core comparison; the decision guide on batch versus real-time explains how to sort your own consumers before you read either.

Estuary

estuary_landing

Estuary is a managed data-movement platform that unifies log-based change data capture, streaming, and batch in one system. A capture reads a source and writes documents into a collection, which lives as JSON files in your own S3, GCS, or Azure bucket once you configure a storage mapping. A materialization reads that collection and writes it to a destination at whatever latency you set: as fast as possible, or on a schedule. The pricing model is per gigabyte plus per connector instance, with a free tier of ten gigabytes a month and two connector instances, and a Cloud tier at fifty cents per gigabyte plus one hundred dollars per connector instance per month for the first six and fifty for each after that. The CDC connectors for PostgreSQL, MySQL, SQL Server, Oracle, and MongoDB are log-based and, per the documentation, in-house and purpose-built; MySQL captures are watermarkless by default, so the connector needs no write access to the source.

Two facts frame the comparison. Estuary's latency claim is sub-100 milliseconds in streaming mode, the vendor's own figure with no published methodology, and the same pipeline delivers on a batch schedule to a destination that prefers it, at no extra charge per the pricing page. And Estuary is not open source: the runtime is under the Business Source License 1.1 with an Apache 2.0 change date in 2029, while its connectors are Apache 2.0 or MIT. The vendor's own word is open-core.

Honest take: The case for Estuary here is narrow and real. When a fraud model, a support tool, or an agent needs a database change within seconds, Fivetran's one-minute floor is a ceiling, and a continuous capture with a per-destination schedule is what you want.

Fivetran

fivetran_landing

Fivetran's managed platform is commercial and proprietary: you operate nothing, and the bill is measured in Monthly Active Rows, the number of distinct rows synced per connection in a calendar month, counted once however often a row changes. A Free plan covers 500,000 MAR; Standard, Enterprise, and Business Critical are priced through an estimator, and every paid-plan connection between one and one million MAR carries a five-dollar base charge, per Fivetran's pricing guide. Fivetran also documents two ways to bring it inside your network: a Hybrid Deployment agent on Enterprise and Business Critical, and HVR 6, which its documentation describes as a self-hosted version of Fivetran.

The connector count is where Fivetran's site disagrees with itself, with 700-plus on the pricing page and 750-plus on the connectors page, and either number dwarfs the field. Since the merger with dbt Labs closed on 1 June 2026, transformations and the dbt ecosystem sit under the same roof, and activations, the reverse-ETL product, are on the same bill. For databases, Fivetran reads logs: PostgreSQL logical replication, the MySQL binary log, SQL Server CDC, MongoDB change streams, and on-premises HVA agents for Oracle, SQL Server, Db2 for i, and SAP ERP. What the managed product does not offer is streaming delivery. Syncs run on a schedule from one minute to twenty-four hours, and the one-minute option needs an Enterprise or Business Critical plan. The exception is HVR 6, whose documentation describes a capture process that continuously acquires changes at the source; it is licensed software you install and operate, which is a real path to low latency and also the reason it sits outside a comparison of managed services. The managed product is built for the warehouse-first team that measures freshness in minutes and counts its sources in the hundreds.

What "real-time" costs on each

	Fivetran	Estuary
Fastest delivery	1-minute scheduled sync, Enterprise and Business Critical only	Sub-100 ms in streaming mode, per Estuary
Default database cadence	Every 15 minutes	Capture is continuous; warehouse materializations default to a 30-minute sync you can set to zero or any schedule
Pricing unit	Monthly Active Rows per connection, plus a $5 base per paid-plan connection under 1M MAR	Gigabytes sourced, transformed, and delivered, plus $100 per connector instance
Free tier	500,000 MAR	10 GB per month, 2 connector instances
Durable change log	None; changes are processed in transit and land in your destination	Your own cloud bucket, once a storage mapping is configured
Deployment inside your network	Hybrid agent (Enterprise+), HVR 6 self-hosted	Private or BYOC data plane (Enterprise, annual contract)

The table hides a subtlety about speed on both sides. Fivetran's own documentation for its Oracle agent says that choosing a one-minute sync frequency does not guarantee that the sync completes within one minute; a one-minute schedule is an interval between starts, and a large table or a slow API can stretch the gap. Estuary's streaming mode has no interval: the connector tails the log continuously, and a materialization set to sync as fast as possible commits transactions as they arrive. But its warehouse materializations default to a 30-minute sync schedule, per the documentation, so a team that never changes that default has bought a slower warehouse than Fivetran's fifteen minutes. The difference between the two, for a consumer that needs seconds, is that one can be set to zero and the other cannot.

Where the bill diverges

Fivetran counts a row once per month no matter how many times it changes. Estuary meters the bytes that move. Those two rules point in opposite directions depending on the shape of your data. A table of a million customer records, each updated dozens of times a day, is a million MAR on Fivetran and many gigabytes of change events on Estuary; here the per-row model favours Fivetran. A wide event table with hundreds of millions of new rows a month is hundreds of millions of MAR on Fivetran and, at fifty cents a gigabyte, a bill you can compute from the row size on Estuary; here the per-gigabyte model favours Estuary. Estuary's compare pages describe its cost as two to five times lower than the alternatives, and that figure is the vendor's own, with no methodology published, so treat it as a claim to check against your own tables rather than a fact.

The per-connector-instance fee cuts the other way. Twenty small SaaS sources on Estuary cost twenty instances a month before a byte moves, at one hundred dollars each for the first six and fifty thereafter. Twenty small SaaS sources on Fivetran are twenty five-dollar base charges plus their MAR. For a long tail of low-volume connectors, Fivetran is cheaper, and the honest recommendation is to run those there.

Which tool fits which consumer

Sort the things that read your data by how long they can wait. Nightly reports, the finance close, and model training sets tolerate hours; a scheduled sync serves them well on either tool, and Fivetran's catalog breadth and dbt integration make it the default for a warehouse-first team. A support console that shows an order the moment it is placed, a fraud check, a search index, or an agent that acts on a customer record cannot wait for the next sync, and those consumers are the reason to look at Estuary at all. Teams with both kinds of consumer tend to want one pipeline, capturing each change once into their own bucket and delivering it fast to the consumers that need speed and on a schedule to the ones that do not. Estuary's deployment options matter here too: a Private or BYOC data plane keeps that bucket and the processing inside your cloud, though both are Enterprise, annual-contract features.

Frequently asked questions

As a head of data on Fivetran Standard, what does moving to Estuary actually change?
Standard syncs every fifteen minutes. Moving buys you continuous capture from your databases, a schedule you can set per destination rather than per plan, and a change log that lives in your own bucket so a new destination backfills from it without re-reading the source. It costs you Fivetran's long-tail catalog, its dbt-native transformations, and the once-per-month row counting that makes high-churn tables cheap, plus a migration: a fresh snapshot of each moved table, a second schema in the warehouse while both run, and dbt sources repointed. Estuary's catalog covers the mainstream SaaS sources, Salesforce, HubSpot, and NetSuite among them; the gap is the long tail, and the practical path many teams take is to move the database sources that feed operational consumers and leave the rest where they are.

As the analyst who owns the data bill, is MAR or per-gigabyte cheaper for us?
It depends on the shape of your largest tables, not on the vendors. Count the distinct rows that change per month and multiply by the bytes each change carries. Tables with many updates to few rows favour Fivetran's model; tables with many new rows favour Estuary's. Add Estuary's per-instance fee for every connector you would run and Fivetran's five-dollar base charge per paid-plan connection. Fivetran publishes a pricing estimator and worked list-price examples rather than a per-MAR rate; Estuary publishes its per-gigabyte rate and a calculator on its pricing page. Volume discounts on either side are a sales conversation, so run the arithmetic on your own numbers.

As a data engineer, can we keep Fivetran for SaaS sources and add Estuary for CDC?
Yes, and it is a common arrangement. Fivetran keeps the connectors Estuary does not have and the transformation layer your analysts already use. Estuary takes the databases whose changes have operational consumers, delivers them in streaming mode where needed, and materializes to the same warehouse on a schedule for analytics. The two tools do not conflict at the destination; they land in different schemas. What you should avoid is capturing the same database twice, once per tool, because that means two replication slots, twice the decoding load on the primary, and a bill on both sides.

Which one should you pick

Pick Estuary when at least one consumer needs a database change in seconds, when you want the change log in your own bucket, or when the same capture must feed a fast consumer and a scheduled one without two pipelines; its product overview is the place to start. If you have both kinds of consumer, that last case is yours, and it is the one in which Estuary is the better tool.