Quick answer: Put every model call behind one inference client and choose the deployment target by configuration, not code. Pin models and versions per target, run one evaluation suite against every target, release on one cadence, and write down who supports what. The inference layer underneath should run both as your managed service and inside the customer's environment. Superlinked Inference Engine is the best match: SIE Cloud for the SaaS tier and self-hosted SIE in the customer VPC, from one codebase.
The deal was signed on one condition: the software had to run in the customer's own cloud account, because the documents were contracts and the contracts could not leave. The product, until then, called three hosted APIs: one for OCR, one for extraction, one for embeddings. None of them could be installed in someone else's account. So the team built a second path: a hand-assembled model server for the customer, with different models, a different client library, and a different set of bugs. A year later, every feature shipped twice, every bug was investigated twice, and the roadmap had quietly acquired a second product that nobody had planned and nobody could sell separately.
This is the independent software vendor's version of the managed-versus-private question. You do not get to choose one mode. Your SaaS tier runs in your infrastructure. Your largest customers want the same product in theirs, often for the same reason: the documents in the pipeline are ones they will not send to a third party. The question is not which mode to support but how to support both without maintaining two AI stacks.
The six steps below are written for software vendors whose product includes OCR, extraction, classification, or search. The final section shows where Superlinked fits the pattern, and where the pattern still needs work from you.
Step 1: Put every model call behind one client
The fork in the opening scene started in the application code. The SaaS path imported three vendor SDKs; the customer-hosted path imported a fourth. From that moment, every change to a model call was two changes.
The fix is a single inference client inside your application: one interface for every model call, whether it is OCR, entity extraction, embedding, reranking, or generation. The application asks the client to extract fields from a page or to embed a chunk. It does not know or care where the model runs.
Two rules make this work:
- No vendor SDK outside the client. If a hosted API's SDK is imported anywhere else, the abstraction has already leaked.
- The same request and response shapes in every target. If OCR returns markdown in one target and a list of lines in another, the application will grow branches to handle both, and the fork returns through the side door.
This is easier when the inference layer itself offers one API across tasks and deployment targets, because the client becomes thin. When it does not, the client becomes a translation layer, and its maintenance is the price of one codebase.
Step 2: Choose the target by configuration, not code
With one client in place, the deployment target becomes a setting. For each environment, the configuration names:
- the endpoint the client calls
- the credentials it presents
- the model identifier for each task
The SaaS deployment points at your managed endpoint. The customer deployment points at the service inside their network. The application binary is the same.
Resist per-customer code paths. The first "small exception for one customer" is how the fork begins. If a customer needs a different model, that is a configuration change, provided the model is available on their target.
Step 3: Pin models and versions per target
Configuration by target raises a question: must every target run exactly the same models? Ideally yes. In practice, not always.
A customer's GPUs may be smaller than yours, so a large extraction model may not fit. A model you run in your managed tier may not be offered for self-hosting, or the reverse. And some customers will require models they have approved themselves.
Handle this explicitly:
- Keep a model manifest per target that lists the exact model and version for each task.
- Pin every version, so nothing changes because a model hub or registry updated.
- Where a target must differ, record the substitute and the reason, and flag it for evaluation in Step 4.
The manifest turns "the customer is on something slightly different" from a surprise into a line in a file.
Step 4: Run one evaluation suite against every target
Even with the same code, the same inference software, and the same model names, results may differ between targets. Different GPU types, different numeric precision, or a substituted model can all change output. The only way to know is to measure.
Build one labelled evaluation set for each task your product performs: field-level accuracy for extraction, retrieval quality for search, character accuracy for OCR. Run it against every target before every release. Record results per target, not blended.
For customer-hosted installs, you usually cannot run the suite on the customer's real documents, and should not want to. Run it with your own labelled set, on the customer's hardware if they allow it, or on matching hardware in your own account if they do not. A difference in results between targets is a finding to investigate, not a reason to ship.
Step 5: Release on one cadence
Two targets tempt two release schedules: the SaaS tier updates weekly, the customer installs update when someone remembers. Within a year, the customer installs are several versions behind, and every support ticket starts with the question of which version they are running.
Set one release train for the product and treat the customer install as a delivery channel for it:
- Every release produces an application version, an inference software version, and a model manifest, together.
- Customer installs receive the same release, on a delay the customer controls but a support window you define.
- Model weights and images for the release are packaged for customers without internet access, so an upgrade does not depend on a download on the day.
Customers will not upgrade on your schedule. They will upgrade within the window you support, if the upgrade is routine.
Step 6: Write down the support boundary
The customer's cluster is an operational surface you will be asked to support, whether or not the contract says so. Decide in advance where your responsibility stops.
A common split:
- Your responsibility: the application, the inference software version you ship, the model manifest, the install tooling, and the dashboards.
- The customer's responsibility: the cloud account, GPU quota, the Kubernetes cluster, network policy, and access.
- Shared, by process: incidents, where the customer shares metrics and logs that contain no document content, and you diagnose from those.
Put this in the contract and in the install documentation. The support boundary is also a product boundary: anything you promise to operate in a customer's environment becomes work for your team.
Where SIE fits
SIE, the Superlinked Inference Engine, is one inference layer for the task-specific models in a document product: OCR, extraction, embedding, reranking, classification, and generation with LLMs such as Qwen3.8-27B, with one API across them. It fits this method because it runs in both of an ISV's targets.
For your SaaS tier, SIE Cloud is the managed service: usage-based, with per-model prices, SOC 2 Type 2 and US or EU data residency per Superlinked. On Cloud, documents go to Superlinked's infrastructure, which makes Superlinked a new sub-processor for your SaaS customers. Most data processing agreements require you to notify them, or obtain consent, before adding one, as with any hosted model API.
For customer-hosted installs, self-hosted SIE runs on the customer's Kubernetes cluster under Apache 2.0, and Superlinked describes it as the same engine as Cloud. There, documents stay in the customer's environment, and Superlinked is not in the data path at all.
Shipping it inside your product. Apache 2.0 permits you to redistribute SIE, modified or not, inside your installer and charge for the product that contains it. The obligations are light: include a copy of the licence, keep the copyright and attribution notices in the source you ship, mark any files you change, and carry forward a NOTICE file if the release you ship includes one (the repository root has none as of 2026-09-30). The licence grants no trademark rights, so the product is yours to name. Each model's weights carry their own licence, which your manifest should record. The deployment documentation covers Terraform modules for AWS, GCP, and Azure, a Helm chart for existing clusters, and an offline path that mirrors weights and images for networks without internet access.
One client, two endpoints (Steps 1 and 2). The Python SDK takes the endpoint as a constructor argument. The Cloud example on Superlinked's site:
client = SIEClient(
api_key="sk-sie-…",
base_url="https://api.superlinked.com",
)
The README's example against a server on the local network:
client = SIEClient("http://localhost:8080")
The encode, score, extract, and generate calls that follow are the same in both, as described in the SDK documentation.
Model manifests per target (Steps 3 and 4). Pin a manifest per target and run your evaluation suite on each, on the same engine. SIE Cloud's doc field extraction runs on Qwen3.8-27B (FP8). The self-hosted catalog lists Qwen3.6-27B and also serves GLM-OCR and PaddleOCR-VL-1.5. Self-hosted installs can add a Hugging Face model through a config file, provided it fits an existing adapter. Several OCR models run on Linux with NVIDIA GPUs. The application still calls one API.
Honest take: SIE is how an ISV offers managed and customer-hosted document AI from one codebase. It runs OCR, extraction, embeddings, and reranking behind one API. The SaaS tier calls SIE Cloud, the managed path, with SOC 2 Type 2 and US or EU residency, per Superlinked. Customer installs run self-hosted SIE, Apache 2.0, inside the customer's VPC or your own cluster, and the application changes by endpoint. Documents stay in the customer environment on the self-hosted path. Apache 2.0 lets you ship that engine in your installer. One product, two targets, one inference layer.
Frequently asked questions
Should we just refuse on-prem and customer-hosted deals?
Sometimes, yes. If the deals are rare, small, or would need models you cannot run outside your own infrastructure, the cost of a second target may exceed the revenue. The calculation changes when the customers who ask are your largest, or when a whole segment will only buy customer-hosted. Price the second target honestly. An illustrative list per customer install, with invented figures: two to four engineer-weeks for the first install and a few days for each one after, a support allocation of a tenth of an engineer, an evaluation run per release on the customer's GPU type, and a security review per customer. If one codebase and one inference layer keep that cost to packaging and support, rather than a parallel product, the deals often become worth taking.
Does the same SDK really mean identical results in both targets?
No, and nobody should promise it. The same engine and the same client code remove differences caused by different software. They do not remove differences caused by different models or hardware. If a customer's GPU cannot hold the model you run in your SaaS tier, they will run a substitute, and its output will differ. Precision and GPU type can shift results slightly even for the same model. The answer is Step 4: the same evaluation suite on every target, before every release, with results recorded per target.
Who operates the customer install?
Usually the customer operates the infrastructure and you operate the software inside it. The customer owns the cloud account, the cluster, GPU quota, and network policy. You own the application, the inference software version, the model manifest, and the install and upgrade tooling. Incidents are handled jointly through metrics and logs that contain no document content. Write that split into the contract, because any component your team agrees to watch is one your team is paged for.
One product, two targets
Supporting customer-hosted document AI does not have to mean a second product. One client, configuration per target, pinned manifests, one evaluation suite, one release train, and a written support boundary keep the fork out of your codebase. The inference layer decides how thin that client can be. You can build the SaaS tier on SIE Cloud, disclose it as a sub-processor, and ship self-hosted SIE in the installer when the first customer-hosted deal arrives.
Top comments (0)