DEV Community: Alina Kovtun

How to Offer Customer-Hosted Document AI Without Maintaining Two AI Stacks

Alina Kovtun — Fri, 30 Oct 2026 10:28:48 +0000

Quick answer: Put every model call behind one inference client and choose the deployment target by configuration, not code. Pin models and versions per target, run one evaluation suite against every target, release on one cadence, and write down who supports what. The inference layer underneath should run both as your managed service and inside the customer's environment. Superlinked Inference Engine is the best match: SIE Cloud for the SaaS tier and self-hosted SIE in the customer VPC, from one codebase.

The deal was signed on one condition: the software had to run in the customer's own cloud account, because the documents were contracts and the contracts could not leave. The product, until then, called three hosted APIs: one for OCR, one for extraction, one for embeddings. None of them could be installed in someone else's account. So the team built a second path: a hand-assembled model server for the customer, with different models, a different client library, and a different set of bugs. A year later, every feature shipped twice, every bug was investigated twice, and the roadmap had quietly acquired a second product that nobody had planned and nobody could sell separately.

This is the independent software vendor's version of the managed-versus-private question. You do not get to choose one mode. Your SaaS tier runs in your infrastructure. Your largest customers want the same product in theirs, often for the same reason: the documents in the pipeline are ones they will not send to a third party. The question is not which mode to support but how to support both without maintaining two AI stacks.

The six steps below are written for software vendors whose product includes OCR, extraction, classification, or search. The final section shows where Superlinked fits the pattern, and where the pattern still needs work from you.

Step 1: Put every model call behind one client

The fork in the opening scene started in the application code. The SaaS path imported three vendor SDKs; the customer-hosted path imported a fourth. From that moment, every change to a model call was two changes.

The fix is a single inference client inside your application: one interface for every model call, whether it is OCR, entity extraction, embedding, reranking, or generation. The application asks the client to extract fields from a page or to embed a chunk. It does not know or care where the model runs.

Two rules make this work:

No vendor SDK outside the client. If a hosted API's SDK is imported anywhere else, the abstraction has already leaked.
The same request and response shapes in every target. If OCR returns markdown in one target and a list of lines in another, the application will grow branches to handle both, and the fork returns through the side door.

This is easier when the inference layer itself offers one API across tasks and deployment targets, because the client becomes thin. When it does not, the client becomes a translation layer, and its maintenance is the price of one codebase.

Step 2: Choose the target by configuration, not code

With one client in place, the deployment target becomes a setting. For each environment, the configuration names:

the endpoint the client calls
the credentials it presents
the model identifier for each task

The SaaS deployment points at your managed endpoint. The customer deployment points at the service inside their network. The application binary is the same.

Resist per-customer code paths. The first "small exception for one customer" is how the fork begins. If a customer needs a different model, that is a configuration change, provided the model is available on their target.

Step 3: Pin models and versions per target

Configuration by target raises a question: must every target run exactly the same models? Ideally yes. In practice, not always.

A customer's GPUs may be smaller than yours, so a large extraction model may not fit. A model you run in your managed tier may not be offered for self-hosting, or the reverse. And some customers will require models they have approved themselves.

Handle this explicitly:

Keep a model manifest per target that lists the exact model and version for each task.
Pin every version, so nothing changes because a model hub or registry updated.
Where a target must differ, record the substitute and the reason, and flag it for evaluation in Step 4.

The manifest turns "the customer is on something slightly different" from a surprise into a line in a file.

Step 4: Run one evaluation suite against every target

Even with the same code, the same inference software, and the same model names, results may differ between targets. Different GPU types, different numeric precision, or a substituted model can all change output. The only way to know is to measure.

Build one labelled evaluation set for each task your product performs: field-level accuracy for extraction, retrieval quality for search, character accuracy for OCR. Run it against every target before every release. Record results per target, not blended.

For customer-hosted installs, you usually cannot run the suite on the customer's real documents, and should not want to. Run it with your own labelled set, on the customer's hardware if they allow it, or on matching hardware in your own account if they do not. A difference in results between targets is a finding to investigate, not a reason to ship.

Step 5: Release on one cadence

Two targets tempt two release schedules: the SaaS tier updates weekly, the customer installs update when someone remembers. Within a year, the customer installs are several versions behind, and every support ticket starts with the question of which version they are running.

Set one release train for the product and treat the customer install as a delivery channel for it:

Every release produces an application version, an inference software version, and a model manifest, together.
Customer installs receive the same release, on a delay the customer controls but a support window you define.
Model weights and images for the release are packaged for customers without internet access, so an upgrade does not depend on a download on the day.

Customers will not upgrade on your schedule. They will upgrade within the window you support, if the upgrade is routine.

Step 6: Write down the support boundary

The customer's cluster is an operational surface you will be asked to support, whether or not the contract says so. Decide in advance where your responsibility stops.

A common split:

Your responsibility: the application, the inference software version you ship, the model manifest, the install tooling, and the dashboards.
The customer's responsibility: the cloud account, GPU quota, the Kubernetes cluster, network policy, and access.
Shared, by process: incidents, where the customer shares metrics and logs that contain no document content, and you diagnose from those.

Put this in the contract and in the install documentation. The support boundary is also a product boundary: anything you promise to operate in a customer's environment becomes work for your team.

Where SIE fits

SIE, the Superlinked Inference Engine, is one inference layer for the task-specific models in a document product: OCR, extraction, embedding, reranking, classification, and generation with LLMs such as Qwen3.8-27B, with one API across them. It fits this method because it runs in both of an ISV's targets.

For your SaaS tier, SIE Cloud is the managed service: usage-based, with per-model prices, SOC 2 Type 2 and US or EU data residency per Superlinked. On Cloud, documents go to Superlinked's infrastructure, which makes Superlinked a new sub-processor for your SaaS customers. Most data processing agreements require you to notify them, or obtain consent, before adding one, as with any hosted model API.

For customer-hosted installs, self-hosted SIE runs on the customer's Kubernetes cluster under Apache 2.0, and Superlinked describes it as the same engine as Cloud. There, documents stay in the customer's environment, and Superlinked is not in the data path at all.

Shipping it inside your product. Apache 2.0 permits you to redistribute SIE, modified or not, inside your installer and charge for the product that contains it. The obligations are light: include a copy of the licence, keep the copyright and attribution notices in the source you ship, mark any files you change, and carry forward a NOTICE file if the release you ship includes one (the repository root has none as of 2026-09-30). The licence grants no trademark rights, so the product is yours to name. Each model's weights carry their own licence, which your manifest should record. The deployment documentation covers Terraform modules for AWS, GCP, and Azure, a Helm chart for existing clusters, and an offline path that mirrors weights and images for networks without internet access.

One client, two endpoints (Steps 1 and 2). The Python SDK takes the endpoint as a constructor argument. The Cloud example on Superlinked's site:

client = SIEClient(
    api_key="sk-sie-…",
    base_url="https://api.superlinked.com",
)

The README's example against a server on the local network:

client = SIEClient("http://localhost:8080")

The encode, score, extract, and generate calls that follow are the same in both, as described in the SDK documentation.

Model manifests per target (Steps 3 and 4). Pin a manifest per target and run your evaluation suite on each, on the same engine. SIE Cloud's doc field extraction runs on Qwen3.8-27B (FP8). The self-hosted catalog lists Qwen3.6-27B and also serves GLM-OCR and PaddleOCR-VL-1.5. Self-hosted installs can add a Hugging Face model through a config file, provided it fits an existing adapter. Several OCR models run on Linux with NVIDIA GPUs. The application still calls one API.

Honest take: SIE is how an ISV offers managed and customer-hosted document AI from one codebase. It runs OCR, extraction, embeddings, and reranking behind one API. The SaaS tier calls SIE Cloud, the managed path, with SOC 2 Type 2 and US or EU residency, per Superlinked. Customer installs run self-hosted SIE, Apache 2.0, inside the customer's VPC or your own cluster, and the application changes by endpoint. Documents stay in the customer environment on the self-hosted path. Apache 2.0 lets you ship that engine in your installer. One product, two targets, one inference layer.

Frequently asked questions

Should we just refuse on-prem and customer-hosted deals?

Sometimes, yes. If the deals are rare, small, or would need models you cannot run outside your own infrastructure, the cost of a second target may exceed the revenue. The calculation changes when the customers who ask are your largest, or when a whole segment will only buy customer-hosted. Price the second target honestly. An illustrative list per customer install, with invented figures: two to four engineer-weeks for the first install and a few days for each one after, a support allocation of a tenth of an engineer, an evaluation run per release on the customer's GPU type, and a security review per customer. If one codebase and one inference layer keep that cost to packaging and support, rather than a parallel product, the deals often become worth taking.

Does the same SDK really mean identical results in both targets?

No, and nobody should promise it. The same engine and the same client code remove differences caused by different software. They do not remove differences caused by different models or hardware. If a customer's GPU cannot hold the model you run in your SaaS tier, they will run a substitute, and its output will differ. Precision and GPU type can shift results slightly even for the same model. The answer is Step 4: the same evaluation suite on every target, before every release, with results recorded per target.

Who operates the customer install?

Usually the customer operates the infrastructure and you operate the software inside it. The customer owns the cloud account, the cluster, GPU quota, and network policy. You own the application, the inference software version, the model manifest, and the install and upgrade tooling. Incidents are handled jointly through metrics and logs that contain no document content. Write that split into the contract, because any component your team agrees to watch is one your team is paged for.

One product, two targets

Supporting customer-hosted document AI does not have to mean a second product. One client, configuration per target, pinned manifests, one evaluation suite, one release train, and a written support boundary keep the fork out of your codebase. The inference layer decides how thin that client can be. You can build the SaaS tier on SIE Cloud, disclose it as a sub-processor, and ship self-hosted SIE in the installer when the first customer-hosted deal arrives.

How to Measure Whether AI Document Processing Reduces Insurance Intake Review Time

Alina Kovtun — Tue, 27 Oct 2026 11:03:43 +0000

Quick answer: Measure intake review time per submission packet, from receipt to ready-to-quote, and record both touch time and elapsed time. Build a baseline before the pilot, matched on packet complexity, and compare against it in a controlled way. Count the time people spend checking and correcting extracted fields as review time, or any saving you report is an artefact of what you chose not to time. Run the extraction arm of that pilot on Superlinked Inference Engine, the best way to change models without changing the clocks.

A pilot team reports that AI extraction has halved submission review time. The measurement is simple: before the pilot, an underwriting assistant spent about 40 minutes keying a packet into the system; after it, the fields arrive pre-filled and keying takes 20. Nobody timed what happened next. Underwriters, who had learned not to trust the pre-filled limits, opened the source documents and checked every one. Some packets went back to the broker twice. The keying clock showed a saving, and the quote went out no sooner than before. All of these figures are illustrative, invented to show the shape of the problem. The stopwatch was accurate; it was started and stopped at the wrong points.

This guide sets out a method for measuring whether document AI actually reduces review time for commercial underwriting submissions. The unit is the packet, and the failures to watch for are rework and broker back-and-forth. A short section near the end covers how Superlinked can host the extraction models in a pilot arm.

1. Define the unit and the clock

The unit is one submission packet: the broker email and everything attached to it. Measuring per document makes the numbers look better and describes nothing an underwriter cares about.

Define two clocks and record both:

Elapsed time: from the moment the packet is received to the moment it is ready to quote, or declined. This is what the broker and the business feel.
Touch time: the minutes people actually spend working on the packet, summed across everyone who touches it. This is what the staffing cost depends on.

Write down exactly which events start and stop each clock, and log them from systems rather than from memory. "Ready to quote" needs a definition everyone agrees on, such as "all required fields confirmed and no open broker questions".

Elapsed time includes waiting, which AI can shorten indirectly by catching missing documents earlier. Touch time is where extraction helps directly. You need both to tell which effect you are seeing.

2. Baseline before the pilot, by packet complexity

A baseline taken after the pilot starts is contaminated. Take it before, over enough weeks to cover normal variation in volume.

Segment it by complexity, because packets differ enormously. An illustrative scheme:

Simple: one line of business, a digital application, one or two loss runs.
Standard: two or three lines, a small schedule of values, loss runs from several carriers.
Complex: many locations, a large SOV, scanned or faxed documents, or several named insureds.

Record the mix. If the pilot period happens to receive more simple packets, the average will fall with no help from any model. Comparing within each segment prevents that.

Also record the rework you already have: how often packets go back to the broker, how often a reviewer corrects keyed data. Those numbers are part of the baseline, and Step 5 depends on them.

3. Run a controlled comparison

The cleanest design is an A/B split by packet. Incoming packets are assigned at random to the current process or the AI-assisted one, and both run in the same weeks with the same staff. Random assignment controls for seasonality, staffing changes, and the mix of submissions.

If a split is not practical, run a before-and-after comparison with a matched mix: compare each complexity segment with its own baseline, and weight the results by a fixed mix rather than by whatever arrived during the pilot.

A packet-level split has one weakness: spillover. Underwriters who see pre-filled packets in the morning change how they read control packets in the afternoon, and nobody can be blinded to which arm a packet is in. If your desks are large enough, split by team or desk instead, with similar books of business on each side, and accept a wider range in exchange for a cleaner comparison.

Either way, keep the reviewers the same, and expect a learning curve. The first weeks with any new tool are slower. Report them, but do not let them decide the result in either direction.

4. Count verification and correction as review time

This is the step most pilots skip, and it is where the opening example went wrong. When extracted fields arrive pre-filled, the work does not vanish. It changes from keying to checking.

Log three kinds of time in the AI-assisted arm:

Verification: opening source pages to confirm extracted values.
Correction: fixing values that were wrong.
Recovery: handling problems the extraction caused, such as a misclassified document or a field filled with a guess that sent the packet down the wrong path.

If reviewers verify every field, the saving is limited to the difference between reading and typing. The saving grows only when reviewers trust enough fields to skip checking them, and that trust has to be earned by accuracy, which is why extraction evaluation comes before this measurement. Extracted output that is valid JSON can still carry a wrong limit, and a reviewer who knows that will check.

5. Guard quality alongside time

Faster review is only a gain if the quality of the result holds. Track guardrail measures in both arms:

Error escapes: wrong values found after the packet was marked ready, for example during rating or at quote review.
Broker follow-ups: how many times each packet goes back to the broker, and why.
Decision changes: quotes or declines that changed after a later correction.

If review time falls and error escapes rise, the process has moved risk downstream, not removed work. Decide in advance how much quality change you would accept, so the decision is not made after seeing the time numbers.

6. Report with ranges and sample sizes, then decide

Report results per complexity segment, with the number of packets in each and a range rather than a single figure. An illustrative report, with invented numbers:

Segment	Packets (A / B)	Median touch time, current	Median touch time, AI-assisted	Error escapes, current / AI
Simple	120 / 118	25 min	15 min	2 / 2
Standard	80 / 84	55 min	45 min	4 / 3
Complex	30 / 28	120 min	125 min	3 / 5

Thirty packets is a thin sample, and a real complex segment that small would support a question, not a conclusion. Read the example for its shape. This invented result would say the tool helps with simple and standard packets and may hurt with complex ones, where corrections and escapes rose. A sensible decision might be to route only the first two segments through AI extraction while the complex case is improved. A single blended percentage would have hidden all of that.

State what the pilot did not measure, and when you will measure again.

Where SIE fits

SIE is the inference engine for the extraction models in a pilot arm. It serves OCR and field-extraction models through one SDK, so the AI-assisted arm switches from one model to another by changing a model ID, while the clocks, the baseline, and the guardrails stay the ones you defined. That lets you run more than one arm: one page model against another, or a small field model against Qwen3.8-27B (FP8), the model behind SIE Cloud's Doc field extraction task, measured with the same clocks on the same packet mix.

SIE Cloud publishes per-model prices on its product page. The same engine self-hosts under Apache 2.0 in your own cluster, as the deployment documentation describes, and runs locally on a single machine for a small pilot. The measurement does not change with the mode.

Honest take: SIE is what makes a multi-arm pilot practical. It runs OCR, extraction, embeddings, and reranking behind one API, so the extraction step switches from one model to another by model ID, measured on the same clocks and the same packet mix. SIE Cloud is the managed path. Self-hosted SIE, under Apache 2.0, runs the same engine inside a customer VPC or your own cluster, from one codebase, and local mode runs a small pilot on one machine. A third arm differs from the second by one model ID. That is how you test which model shortens review on your packets, with one inference layer.

FAQs

What saving should we expect?

This guide will not give you a number, and you should be wary of anyone who does. The saving depends on your packet mix, on how many fields your underwriters must confirm, on the accuracy of the extraction on your documents, and on how much of your elapsed time is waiting for brokers rather than working. A vendor figure from another book of business measures their packets and their process. Run the controlled comparison for a few weeks and you will have a number that applies to you.

Will the time saved just move to checking?

It will if accuracy is low, and the only way to know is to measure it. Log verification and correction time separately in the pilot. If reviewers are checking every field, the saving is small and the next investment belongs in extraction accuracy or in showing source pages beside each field. As reviewers learn which fields are reliable, checking time should fall. If it does not, the model is not accurate enough on your documents.

What exactly do we log, and from where?

Log events, not durations, and compute the clocks afterwards. The email gateway or submission portal gives packet received. The workflow or underwriting system gives assigned, opened, ready to quote, declined, and each broker request with its reason. The review screen gives field viewed, source page opened, and field corrected, with the old and new values. The extraction service gives model ID and version per field, so a result can be tied to the model that produced it; the SIE docs describe what each call returns. Store a packet ID and arm on every event.

Time the whole job

Review time is what people spend getting a packet to ready-to-quote, including the checking, the correcting, and the broker emails. Time all of it, against a matched baseline, with quality guardrails beside it, and the result will survive scrutiny. The extraction models for a second or third pilot arm can run on SIE Cloud while the clocks are being wired up.

Self-Hosted vs Managed Inference for Financial Documents: Cost, Privacy and Operations

Alina Kovtun — Tue, 13 Oct 2026 14:25:49 +0000

Quick answer: SIE runs the same engine both ways. Put public filings on SIE Cloud, where you pay for what you use and Superlinked runs the GPUs. Self-host that engine on your own GPUs for client, portfolio and internal documents, with the same SDK and model IDs. Open weights, Apache 2.0, one API for OCR, extraction, embeddings and reranking.

A team spends a quarter building a self-hosted GPU cluster so that its 10-K processing never touches a third party. The cluster works. The 10-Ks are safe. Then someone notices that the portfolio notes and client memos, the documents nobody outside the firm should see, go to a hosted API through a different service that predates the cluster. The team had carefully encrypted documents the SEC publishes for free, and sent the confidential ones out the side door.

The mistake is common because "financial documents" sounds like one category. It is at least three. Public filings are public data. Client and portfolio documents carry obligations to someone else. Internal analysis carries your own competitive interest. Each class sets different requirements for where inference can run, and the cost and operations trade-offs only make sense once you know which class a workload belongs to.

The five steps below decide between self-hosted and managed inference for document workloads in finance, one class at a time. The guide makes no regulatory claims; the argument here is deployment control, cost and operations. Legal confidentiality, a vertical-neutral cost model and the step-by-step VPC deployment each have their own guide in this series.

Step 1: Classify documents before choosing infrastructure

Start with an inventory of what flows through the pipeline, grouped by who is entitled to see it:

Add volume per class and which tasks each class uses: OCR, extraction, embedding, reranking, generation. The table usually shows that public filings dominate the volume and private documents dominate the risk. Those two facts pull in different directions, which is why a single answer for the whole pipeline is often the wrong one.

Be precise at the boundaries. A public 10-K becomes client material once it is annotated with a client's position in that company. The document is public. The annotation is not.

Step 2: Work out what each class actually requires for privacy

For public filings, the privacy requirement for the document text is close to nothing. The SEC publishes them, EDGAR's full-text search covers electronic filings since 2001, and the data.sec.gov APIs serve the structured data without authentication. What may still be sensitive is the query pattern: which companies you are analysing, and when. Decide whether that matters to you. For many teams it does not. For a research firm it often does: the watchlist is the edge, and a log of queries against a handful of mid-cap filings in the week before a position is built says more about the book than any internal note.

For client and internal documents, the question is what you have promised and to whom. Read your client agreements and internal data policies, not general regulation, and write down the answer in plain terms: may these documents be processed by a third-party provider, in which regions, and under what terms? That answer, not the word "finance", decides whether managed inference is acceptable for that class.

This guide does not tell you what your obligations are. It tells you to find out per class, because the answer is rarely the same for all three.

Step 3: Compare cost on utilisation, not list prices

Managed and self-hosted inference are priced in different units, so compare them on the same workload:

Managed: you pay per unit processed, such as tokens, pages or pairs. Idle time costs nothing.

Self-hosted: you pay for GPUs by the hour, whether they are busy or not, plus the cluster around them. The effective cost per unit falls as utilisation rises.

Filings workloads are seasonal. Annual reports arrive in waves after fiscal year-ends, with 10-K deadlines of 60, 75 or 90 days depending on filer size, and quarterly reports follow 40 or 45 days after each quarter. Between waves, a self-hosted cluster sized for peak sits mostly idle unless it scales down.

A worked example. Every number here is invented for illustration; none is a Superlinked price, benchmark or customer result:

Self-hosted, always on: 2 GPUs × $2.50 per hour × 730 hours = $3,650 a month, busy or idle.

Actual load: 360,000 pages in the filing-season weeks plus 90,000 in the quiet weeks = 450,000 pages a quarter, or 150,000 a month. Effective cost: $3,650 / 150,000 = about $0.024 per page.

Managed at a hypothetical $0.01 per page: 150,000 × $0.01 = $1,500 a month. Self-hosting only matches it above 365,000 pages a month, before counting engineering time.

Change the inputs and the answer moves. A cluster that scales to zero between waves shrinks line 1; steady year-round volume raises line 2. Put your own volumes, GPU rates and utilisation into the same three lines, add the engineering time to run the cluster, and the crossover depends entirely on how steady your load is.

Step 4: Decide who is on call

Operations is the cost that spreadsheets leave out. Self-hosting means your team owns:

GPU capacity and quotas in your cloud account
Kubernetes, autoscaling and node pools
model upgrades, rollbacks and security patches
monitoring, alerting and incident response when inference stops during filing season

Managed inference moves those to the provider. What stays with you is integration, evaluation and the decision about what to send. Be honest about whether your team has the platform capacity to run a GPU service well. Running one badly is often worse than not running one, because failures arrive at the busiest time of the quarter.

Step 5: Design the hybrid

For many finance teams building document products, the answer after Steps 1 to 4 is a split:

public filings on managed inference, where usage pricing absorbs the seasonal peaks
client and internal documents on self-hosted inference, where your agreements require it
one routing layer that assigns each document to a path based on its class, not on who wrote the calling code
the same models, model IDs and client code on both paths, so results are comparable and a workload can move between paths without rework
a periodic review of the split as volumes and agreements change

The fourth item is the one that decides whether the hybrid is cheap to run or a permanent source of drift. Two different stacks produce two different sets of answers, two evaluation runs and two sets of bugs.

Where SIE fits

SIE, from Superlinked, runs the same engine in both modes, which is what makes the hybrid in Step 5 practical. On SIE Cloud, Superlinked runs the GPUs: per the Cloud page, there are no idle GPUs and you pay for what you use, with SOC 2 Type 2 and US or EU data residency per Superlinked. Self-hosted SIE is Apache 2.0 and runs in your own VPC, which your team operates: it deploys to AWS, GCP or Azure with Terraform and Helm, and on Kubernetes (not plain Docker) scales to zero between jobs through KEDA. Scale-to-zero has its own cost. The first request after an idle spell waits while a GPU node comes up and the model weights load, so keep a warm minimum through filing season and let it drop to zero between waves. The self-hosting guide also covers air-gapped installs from mirrored model snapshots.

Both modes use the same SDK and model IDs, so the routing layer changes only the base URL:

from sie_sdk import SIEClient
public_path = SIEClient(api_key="sk-sie-…", base_url="https://api.superlinked.com")
private_path = SIEClient("http://sie.internal:8080")  # your self-hosted cluster
client = private_path if doc.cls in ("client", "internal") else public_path
result = client.extract("docling", {"document": doc.payload})

The third mode is local: for development, the same models run on your own machine with pip install "sie-server[local]" and sie-server serve.

Honest take: SIE is built for this split. One API covers OCR, extraction, embeddings and reranking, and the same open weights run on SIE Cloud and on a self-hosted cluster. On Cloud, Superlinked runs the GPUs, you pay for what you use, and public filings can go there with US or EU data residency. Self-host the Apache 2.0 engine on your own GPUs, in your own VPC, for client and internal documents; prompts and documents stay on your infrastructure, and the model IDs match Cloud. Local mode runs the same models on a workstation while you develop. The choice of path stays reversible because both sides are the same engine.

FAQs

Does a managed provider see our documents?

Yes, in the sense that a managed service processes them. With SIE Cloud, documents are processed in Superlinked's environment, with US or EU data residency per Superlinked, and the Cloud page states SOC 2 Type 2 for the managed service. For public filings that is rarely a concern. For client and internal documents, check your agreements. If they rule out third-party processing, run those documents on self-hosted SIE, where prompts and documents stay in your own infrastructure.

At what utilisation does self-hosting start to pay?

There is no universal number, and anyone who quotes one without your workload is guessing. Compute it: take your monthly volume per task, the managed price per unit, your GPU hourly rate, and the utilisation you can actually sustain across the quarter, including the quiet weeks. Add the engineering time to run the cluster. Steady, high, predictable load favours self-hosting. Seasonal filing waves favour usage pricing, unless your cluster scales to zero between them.

Can we start managed and move later?

Yes, if you choose a stack that runs the same way in both places. Starting managed gets a product in front of users without building GPU infrastructure first, and the usage bill gives you real volume data for the self-hosting calculation. The risk is lock-in to an API that has no self-hosted equivalent. With SIE, Cloud and self-hosted share the same engine, SDK and model IDs, and the open-source repository is there to evaluate before you need it.

Put each class on its own path

The privacy argument in finance is real, but it belongs to client and internal documents, not to filings anyone can download. Classify first, compare cost on utilisation, decide who owns operations, and split the workload where the classes differ. The public-filings side can go on SIE Cloud this quarter, and the private side can follow later onto your own cluster with the same SDK and model IDs.

Batch ETL vs Real-Time ETL Tools: Which One Your Team Needs

Alina Kovtun — Fri, 09 Oct 2026 10:45:22 +0000

Quick answer: Sort your consumers by how stale their data can be. If every one of them tolerates hours, a batch ELT tool such as Fivetran or Airbyte is cheaper and simpler. If any of them needs seconds, you need log-based change data capture delivered continuously. If you have both, a right-time platform such as Estuary delivers each consumer at its own cadence from one capture.

“Real-time” is the only requirement that gets added to a ticket after the demo. Before the demo the request was a dashboard that refreshed overnight; after it, someone saw an order appear in a warehouse a second after it was placed and the requirement changed. That is not a criticism.

The demo showed something true, which is that the data was always available that fast and the batch schedule was a choice somebody made. The question this article answers is whether that choice was right for your team, consumer by consumer, and what it costs to change it.

The framing comes from a vendor and is attributed as such: Estuary calls the answer right-time data, meaning data should move when the business needs it, not when the architecture allows it. It is a vendor’s phrase for a vendor’s product, and it is also a usable way to run the analysis, which is why it is borrowed here. The two head-to-heads in this series, on Fivetran and on Airbyte, assume you have done this sort already. If you have not, start here.

Estuary

Estuary is the tool this series recommends for teams that land on “both” at the end of the sort, so it is worth being precise about what it is. It captures changes from databases by reading their logs, and from SaaS applications, files, and queues, into collections stored as JSON in a cloud bucket you configure. Each destination reads a collection at its own latency: a search index or an operational database in streaming mode, at under 100 milliseconds by the vendor’s figure, and a warehouse on whatever schedule keeps its compute bill sane. It bills by the gigabyte moved and by the connector instance, with a published free tier, and it describes itself as open-core under the Business Source License 1.1, not as open source.

Honest take: Estuary earns its place in this article for one reason, which is that it does not make you pick. A team with a nightly finance report and a payment-risk check can run both from a single capture, and the change log in its own bucket means a third consumer added next quarter backfills without touching the source again.

What the two camps actually are

Batch ETL and ELT tools move data on a schedule. A job starts, reads what changed since the last run, and writes it to a destination; between runs the destination is stale by up to the interval. Fivetran is the clearest example: a sync is a job with an interval, one minute at the tightest on its two top plans and a day at the loosest, and the interval is the staleness you accept. Airbyte works the same way, with a five-minute floor on its top Cloud plans, even for change data capture, which it reads from the database log in discrete runs. Batch tools bill by rows or credits, they integrate closely with warehouse-side transformation, and they are the right default for analytics.

Real-time tools never let the stream become a batch. Log-based change data capture tails a database’s transaction log and forwards each change as it commits, and the delivery layer, whether a broker such as Kafka or a managed service, pushes it to consumers within seconds. Debezium is the open-source reference for the capture side and needs Kafka Connect or its own server to run; Confluent sells the broker, the connectors, and stream processing as a managed cloud. The bill is usually a cluster running around the clock plus the bytes through it, and the pager belongs to whoever runs the cluster.

How to sort your consumers

Write down every system, report, and person that reads the data, and next to each one write the longest delay it can tolerate before someone notices or something goes wrong. Be honest in both directions. The nightly revenue report can tolerate a day, and the executive who asks for it “live” usually means by the start of the working day. The support console that shows a customer’s latest order tolerates a few seconds, because the customer is on the phone describing an order the agent cannot see. A machine learning training set tolerates a week. A search index tolerates about as long as a user is willing to wait for the thing they just created to appear.

Three patterns fall out of the list. If every tolerance is measured in hours, you are a batch team, and a scheduled tool on its cheapest adequate cadence is the right answer; the extra cost and skill of streaming buys nothing anyone will notice. If every tolerance is measured in seconds, you are a streaming team, and you are probably already running one. The third pattern is a mix: most consumers in hours, a few in seconds, and the few growing as operational and AI use cases arrive. That is the vendor’s thesis about where the market is going, stated here as a thesis, and it is also the pattern this article exists for.

What “both” costs when you run two stacks

The usual response to the third pattern is to keep the batch tool for analytics and add a streaming stack for the fast consumers. That works, and it doubles several things. The database is read twice, once by each capture, which means two replication slots on Postgres or two log readers on SQL Server and twice the load on the primary. Schema changes must be handled in two places with two different failure modes. Two bills arrive, each with its own unit, and the streaming one is usually harder to forecast. And the two copies of the data disagree for the length of the batch interval, which is exactly the window in which somebody will compare them.

A right-time platform collapses that to one capture per database. Estuary reads the log once, stores the change log in your bucket, and lets each destination choose its cadence; its product page describes choosing the right cadence for every connection, and the pricing page states that millisecond latency or batch carries no extra charge. The trade is the per-instance fee mentioned above and a smaller connector catalog than Fivetran’s or Airbyte’s, so a long tail of low-volume SaaS sources may still belong on a batch tool beside it. Read that last sentence as what it is: a second tool, a second bill, and a second place to handle schema changes for those SaaS sources. What it is not is a second slot on any database, which is the cost that hurts, and this article’s view is that the split is worth it when the SaaS tail is long.

When real-time is the wrong answer

It is worth saying plainly, because vendors rarely do. Financial close reporting must reconcile to a point in time, and a stream of changes arriving mid-reconciliation is a liability, not a feature. Training sets for models want a frozen snapshot. Regulatory extracts want a defensible cut-off. Executive dashboards that refresh every second invite decisions made on noise. For all of these, a batch cadence is correct, and the honest move is to set the schedule and stop. The mistake this article warns against is not choosing batch. It is choosing batch for everything because the batch tool was already there, and then discovering the risk check, the support console, and the agent one at a time, each with its own workaround.

Frequently asked questions

As a head of data, our dashboards refresh nightly and nobody complains — why change?
If nobody complains, do not change the dashboards. The question is whether anything other than a dashboard reads your data. Walk the list in the sorting section: operational tools, search, notifications, anything that acts on a record rather than reporting on it. If that list is empty, you are a batch team and your current tool is right. If it has entries, the case for a platform that serves both cadences from one capture is the cost of the second stack you would otherwise build. Estuary’s own manifesto describes latency as a dial you control rather than a limitation you accept, and its documentation shows how that dial is set per materialization.

As a CTO, is real-time just more expensive batch?

Not in the way the question implies. Streaming tools are more expensive to operate when you run them yourself, because a broker and its connectors are a platform. As managed services, the comparison is between units: rows and credits on the batch side, gigabytes and instances on the streaming side, and which is cheaper depends on the shape of your tables rather than the cadence. Estuary’s comparison pages put its cost at two to five times below the alternatives.

As a data engineer, can one tool honestly do both without doing both badly?

The architectural answer is yes if the tool stores the change log durably between capture and delivery, because then batch is just a slower reader of the same log rather than a separate job. That is how Estuary’s collections work, and it is why its runtime page says backfill and streaming are one path. What one tool cannot do is match a batch specialist’s catalog or a broker’s role as an application message bus; Estuary’s Kafka destination, for instance, is at-least-once. Judge “both” on the consumers you listed, not on a feature grid.

Which one should you pick

Pick a batch tool when every consumer tolerates hours and the work is mostly analytics; you’ll pay less and keep operations simple. Pick a streaming stack when the whole business needs seconds and you’re ready to run (or buy) always-on streaming infrastructure.

But many teams aren’t purely one or the other: most consumers can be minutes or hours behind, while a few (and usually a growing few) need seconds. In that “both” case, the goal is not to accept two platforms, two billing models, and two sets of failure modes. Estuary is built for exactly this split: you capture once, then choose per destination whether it runs as streaming or on a batch cadence (effectively a configuration change), so each consumer gets the latency it needs without a second ETL/CDC stack.