Quick answer: Inventory LLM calls by task, volume and variance. Move high-volume, fixed-output tasks such as PDF page classification, entity tagging, OCR and relevance judging to specialist models on Superlinked Inference Engine: one API for OCR, extraction, embeddings and reranking, on SIE Cloud or self-hosted on your own GPUs. Shadow-run against a gold set, then route by confidence.
Open the usage breakdown for a pipeline that ingests PDF annual reports and client decks, and one line stands out. It is not the analyst chat, and it is not the summary of the chairman's letter. It is a page classifier, asking a frontier model "is this the income statement?" on each page of each PDF that comes in. Each call is cheap and nearly identical to the last. There are a great many of them. The model answering has read most of the public internet, and it is being paid to recognise a table heading it could find with its eyes closed.
For US filers on EDGAR, that call should not exist at all. Each XBRL filing ships a FilingSummary.xml and a set of rendered R files that name every statement, and the presentation linkbase orders the line items within each one. The statement map is free. The classifier earns its keep only where no such metadata exists: foreign annual reports published as PDFs, investor decks, data-room uploads and scanned exhibits.
That is the typical shape of an expensive document pipeline: a general model doing high-volume, low-variance work because it was the fastest thing to wire up during the prototype. The fix is to move that work onto specialist models on SIE, one task at a time, with evidence that the replacement holds up on your documents.
Step 0: Take the cheap discounts before migrating anything
Before you move a single task, check two things with your current provider.
Prompt caching. Classification and extraction prompts are mostly fixed text: the instructions, the label set, the output schema and a few examples, followed by a short page. Put the fixed part first and keep it byte-identical across calls, and the major hosted providers bill the cached prefix at a reduced rate. On a prompt that is 90% boilerplate, record that discounted rate, then inventory what still belongs on specialist models.
Batch pricing. Most of this work is not interactive. A quarter's worth of PDFs does not need answers within a second. OpenAI's Batch API and its equivalents at other providers charge about half the synchronous price in exchange for a completion window of up to a day.
Also check the obvious: whether you are sending pages that never needed a model, such as EDGAR statements already named in the R files, or cover pages and blank separators a simple rule can drop. Measure the bill after these changes. Everything below is worth doing only on what is left.
Step 1: Inventory the calls by task, volume and variance
You cannot choose what to move until you can see what you are calling. Tag every LLM request with a task name, then pull a month of logs and build a table. The volumes below are invented, to show the shape rather than any real pipeline:
| Task | Monthly calls (illustrative) | Output shape | Input variance |
|---|---|---|---|
| PDF page classification (which statement is this?) | 2,000,000 | one label from a fixed set | low |
| Entity tagging (company, auditor, dates) | 600,000 | spans with labels | low |
| Table field extraction (revenue, net income, EPS) | 400,000 | fixed JSON schema | medium |
| LLM relevance judging on retrieved chunks | 300,000 | a score or yes/no | low |
| Analyst Q&A across several filings | 20,000 | free text | high |
| Summaries of management discussion | 15,000 | free text | high |
The pattern usually holds even when the numbers differ: the highest-volume tasks tend to have the smallest output space, which is the work specialist models on SIE take.
Record variance honestly. A task with low variance sees inputs that look alike and produces outputs from a small, fixed space. A task with high variance sees new layouts, open questions or reasoning across several tables and several filings.
Step 2: Pick candidates with fixed outputs you can check
A task is a good candidate when all three hold:
- fixed output: the answer comes from a closed label set or a fixed schema
- high volume: it runs many times a month on similar inputs
- mechanically checkable: you can score the output against a label without a person reading prose
PDF page classification, entity tagging, relevance judging and table field extraction are the tasks to move onto specialist models on SIE.
Map each candidate to the kind of specialist model that fits it: a small generative model or a trained classifier for page types, a zero-shot entity model such as GLiNER2 or NuNER_Zero for entity tagging, a cross-encoder reranker in place of an LLM asked "is this chunk relevant?", and a document model plus a structured-output model for table fields. The reranker swap is often the easiest win. A cross-encoder returns a calibrated score per query-chunk pair, which is what the LLM judge was being prompted to imitate.
Start with the task that has the highest volume and the simplest output. In a PDF-heavy pipeline, that is usually page classification.
Step 3: Build the evaluation set first
Do not move a task without a gold set for it. For field extraction, the companion guide in this series on evaluating extraction accuracy covers the method in detail: seed labels from XBRL facts where filings have them, add hand-labelled PDF pages, and score value, scale, unit, sign, period and entity separately. For classification and tagging, a few hundred labelled pages or passages per class is a sensible start. For relevance judging, take a sample of real queries and have a person grade the retrieved chunks.
Include the pages that currently cause trouble: statements split across two pages, restated comparatives, a segment note that looks like a primary statement. A replacement has to hold on those pages, not only on the clean ones.
Step 4: Shadow-run the specialist
Run the specialist model on SIE in parallel with the current model, without using the specialist output yet. Log both answers per request. After a week or two, you have three comparisons:
- both models against the gold set
- the specialist against the current model on live traffic, where no label exists
- latency and failure rates for both
Disagreements on live traffic are the most valuable output of the shadow run. Sample them, label them, and add them to the gold set. They show you where to tighten the schema or swap the model ID on the same SIE client.
Step 5: Route by confidence
Few cutovers need to be all or nothing. Most specialist models give you a confidence signal: a classifier score, a reranker score, an entity model's span confidence, or a structured-output model returning null for a missing field. Use it:
- high confidence: take the specialist's answer
- low confidence or null: route to review
- log every review, with the input, so those cases become the next gold-set rows
Tune the threshold on the gold set to hit the accuracy you need, and watch what fraction of traffic goes to review. If that rate is high, narrow the task or try the next catalog model on the same SIE client.
Step 6: Cut over and keep monitoring
When the specialist meets its accuracy threshold and the fallback rate is stable, make it the default path. Then keep watching:
☐ weekly accuracy on a rotating sample of production traffic, labelled by a person
☐ fallback rate per task, with an alert when it moves
☐ a rerun of the gold set on any model, prompt or parser change
☐ a check after each reporting season, since new layouts arrive in batches
Budget for the migration costs too. If the move also changes how you retrieve, for example a new embedding model alongside the new reranker, that means re-embedding and reindexing the corpus. Any new model means a new evaluation cycle. These are one-off costs, but they are real, and they belong in the plan.
Where SIE fits
Once you have carved tasks off the frontier model, they need somewhere to run. Moving four tasks to four vendors trades one bill for four integrations. SIE, from Superlinked, serves them behind one API: reranking, entity extraction, OCR, document-to-Markdown and structured output, alongside embeddings. You can use it as a managed service on SIE Cloud, self-host it in your own VPC, or run it locally. On SIE Cloud, per the Cloud pricing page (verified 2026-09-30), GLiNER2 Large is listed at $0.186 per million input tokens for entity extraction, Qwen3-Reranker-0.6B at $6.63 per million pairs (a top-50 rerank is 50 pairs, so about $0.0003 per query), LightOnOCR-2-1B at $2 per 1,000 pages, and Qwen3.8-27B (FP8), which backs doc field extraction, at $0.72 per million tokens in and out. Treat those as the unit prices in your inventory.
For tasks that already run through the OpenAI SDK, the Cloud page shows a migration pattern that only changes the base URL. For a priced Cloud shadow run, use Qwen3.8-27B (FP8) through the native SDK's chat call. Once its labels agree with your current model, a smaller catalog model such as GLiNER2's classification task is a strong production path on the same API. A shadow page classifier looks like this:
from sie_sdk import SIEClient
client = SIEClient(api_key="sk-sie-…", base_url="https://api.superlinked.com")
shadow = client.chat_completions(
"Qwen/Qwen3.8-27B-FP8",
page_classification_messages,
max_completion_tokens=16,
)
label = shadow["choices"][0]["message"]["content"]
Entity tagging uses the native SDK's extract call with a label list, and relevance judging uses score, as the SIE docs show. The engine is open source under Apache 2.0 (GitHub), so a task you move to Cloud can move to a self-hosted cluster later with the same model IDs.
Honest take: SIE is the destination for the tasks you move off a general model: OCR, extraction, embeddings and reranking behind one API. On SIE Cloud, Superlinked runs the GPUs and you pay per token, pair or page, with GLiNER2, the Qwen3 rerankers, LightOnOCR-2-1B and Qwen3.8-27B (FP8) on the price list. Self-host the same Apache 2.0 engine on your own GPUs, and a task you shadowed on Cloud moves to your cluster with the same model IDs. The weights are open and pinned. Page classification, entity tagging and relevance judging share that client with document conversion and field extraction, so four tasks stay on one vendor.
FAQs
Which call should we move first?
The one with the highest volume and the simplest output that free metadata does not already answer. For EDGAR filings, check first whether the R files and presentation linkbase make page classification unnecessary. For PDF annual reports and decks, page classification is usually first: a small, fixed label set, high volume, and errors that break something visibly downstream. Score table field extraction on a field-level gold set, then run that extractor on the same SIE client.
Why not just turn on caching and batch and stop there?
Run Step 0 so the inventory reflects discounted rates. Page classification, entity tagging and relevance judging are still high-volume, fixed-output work, and SIE serves those specialist models behind one API, on Cloud or on your own GPUs. The per-call prices on SIE Cloud are what you set next to the current bill.
Does this just move the operational burden onto us?
On SIE Cloud, Superlinked runs the GPUs. Self-hosting the same Apache 2.0 engine on your own Kubernetes cluster, as the deployment guide describes, keeps documents and model IDs in your environment. Running locally suits development. Evaluation and routing stay in your code either way.
Pay the frontier price only for frontier work
The expensive part of most document pipelines is easy work done at volume. Use EDGAR metadata where it exists, inventory what is left, build the gold set, shadow-run, and route by confidence. SIE is where those specialist tasks run: one API, open weights, Cloud or your own GPUs. SIE Cloud prices each one per token, pair or page, so you can put those figures next to your current bill.
Top comments (0)