How to Deploy Document AI Inside a Customer’s VPC

Quick answer: Decide on GPU quota, drivers, cluster version, and network egress with the customer before writing any code. Use infrastructure as code to set up the environment, store model weights and container images locally so the system doesn't need to download anything at runtime, and share the review pack with the customer’s security team early. Keep endpoints private, support the system through metrics rather than documents, and treat every model change as a release. Self-hosted Superlinked Inference Engine is the best way to deploy this document stack inside the customer’s Virtual Private Cloud (VPC).

The deployment passed every test in the staging environment. The Helm installation went smoothly, the pods started, and health checks went green. But when the team installed it at the customer's site, the system kept crashing for two days. The problem was that the model server tried to download model weights from a public model hub when it started. The customer's network blocked outbound traffic, so the download failed and the system could not start. This problem had gone unnoticed because the staging environment allowed open egress. Nobody had tested what would happen if downloads were blocked.

Deploying document AI in a customer's environment is difficult because you cannot assume it works like your own. In your cloud account, you already have the GPU resources, Kubernetes version, and internet access you need. The customer's environment may have different settings and restrictions. Their security team also has good reasons for every restriction, especially when the system handles sensitive documents such as contracts, underwriting submissions, and client financials - the material that made them insist on hosting the system themselves.

This guide explains how to deploy document AI in a customer's restricted environment. It walks engineers through the implementation steps, whether they work for a software vendor installing the system in a customer's cloud or an internal platform team deploying into a locked-down account. The final section explains how to apply these steps to Superlinked's SIE based on its documentation. Unlike a hosted API, this approach keeps documents inside the customer's environment, regardless of which inference server you use.

Step 1: Run a pre-flight with the customer

Many installation failures can be traced back to issues that a thirty-minute call could have identified. Before writing any code, document the following requirements and confirm them with the customer.

• GPU quota. Confirm the available GPU types, in which region, and how many. Getting extra GPU capacity approved can take days or weeks. A pipeline built on vision-language page models need GPUs to run at a useful speed. However, some text extraction models, such as GLiNER2, can run on CPUs. Make sure the required quota is approved before the scheduled installation date rather than requested on the day of deployment.

• Cluster. Will you deploy onto an existing Kubernetes cluster or provision a new one? If the customer already has a cluster, identify its version, the ingress controller in use, and the team responsible for its administration. If a new cluster is needed, clarify who will own and manage it after the installation is complete.

• GPU software. Confirm which node image and NVIDIA driver version the cluster uses, along with the GPU device plugin or GPU Operator installed on the nodes. Also, identify the team responsible for upgrades. If the NVIDIA driver is incompatible with the CUDA version required by the container, the installation may stall even when the pods appear to be running normally.

• Network egress. Confirm whether the cluster can access the internet and, if so, which external hosts are permitted. Model hubs, container registries, and package mirrors are often blocked in restricted environments. Unless the customer confirms otherwise, assume that none of these resources are accessible and plan the deployment accordingly.

• Access. Confirm who on your team will need access to the customer's environment, how they will connect, and how long access will remain available. Many customers grant access only during a scheduled window, through their own bastion hosts or remote session tooling.

• Storage. Determine where model weights and container images will be stored within the customer's account, such as an object storage bucket or a private registry. Also, clarify who manages the credentials required to access and retrieve these assets.

Document all the answers in a one-page installation checklist. Keep it readily accessible so you can refer to it whenever the installation encounters an issue or stalls.

Step 2: Provision with infrastructure as code

A customer deployment is rarely a one-time task. You will need to repeat the process for new customers, redeploy the system after upgrades, and rebuild the environment when moving to a different region. Use infrastructure as code from the outset to make deployments consistent, repeatable, and easier to maintain.

Two deployment approaches cover most customer environments:

• New cluster. Use Terraform or the customer's preferred tool to provision the Kubernetes cluster, GPU node pools, and storage in one reviewed plan. This allows the customer's cloud team to review the plan before it applies, and address most questions upfront.

• Existing cluster. Use a Helm chart or an equivalent deployment package to install the workload into the customer's existing Kubernetes cluster. The customer's platform team retains ownership of the cluster; you own the workload inside it.

In both cases, the goal is to deliver a repeatable installation process that the customer has reviewed and approved. The deployment should not rely on anyone remembering or manually repeating a sequence of commands.

Step 3: Get model weights in without the internet

This is the step that caused the failure described at the beginning of the article. The solution is straightforward: mirror everything the system downloads at runtime into storage the customer controls.

1. Download the model weights on a machine that has internet access.

2. Copy them into a storage bucket inside the customer's account.

3. Copy the container images into the customer's private container registry.

4. Configure the deployment to read weights from the bucket and pull images from the registry.

5. Test the installation with egress blocked before you call it done.

This approach works in fully air-gapped networks as well as environments with restricted outbound internet access. It also ensures that installations remain reproducible because the deployment uses the exact model weights that were tested and stored in the customer's bucket, rather than relying on whatever version the model hub provides at the time of installation.

Plan for storage and transfer size. A pipeline with several OCR, extraction, and embedding models can carry many gigabytes of weights. Agree on a transfer method with the customer's security team, since many will want to scan the files before they are transferred into the environment.

Before the installation, prepare a security review package for the customer. This typically includes a Software Bill of Materials (SBOM) for each container image, image signatures or digests to verify and pin image versions, recent vulnerability scan results, a data-flow diagram, and the network policy. Also, document the source and format of each model's weights. (Prefer the Safetensors format over pickle, and avoid models that require remote code execution). Prepare this package once and reuse it for each customer, updating it whenever the images, models, or security requirements change.

Step 4: Keep the endpoints inside the VPC

A document inference service deployed within a customer's network should remain accessible only from inside that network, not from outside it. To achieve this, you need to make three key decisions.

• Ingress. Expose the service through an internal load balancer or internal ingress, not a public-facing one. If the Helm chart defaults to a public ingress, override that setting and document the reason for the change.

• TLS. Encrypt traffic between the application and the inference service, even when both run inside the VPC. If the customer has an internal certificate authority, use it to issue the required certificates. The customer's security team will likely ask about this during the review.

• Cross-account access. If the application that calls the models runs in a different network from the inference service, use the cloud provider's private connectivity options instead of routing traffic over the internet. For example, AWS PrivateLink allows one VPC to access services hosted in another VPC without requiring an internet gateway, NAT device, or public IP address. You can also create your own endpoint service for other AWS accounts to consume.

The test for this step is simple. From outside the customer's network, the service should not answer at all.

Step 5: Support the system without seeing the documents

Once the system is up and running, the customer will need support, and your team will need visibility into its operation. However, your team cannot access the documents being processed.

Build your support process around signals that provide useful insights without exposing document content:

• Metrics. Track request rates, latency, error rates, GPU utilization, and queue depth for each model.

• Logs. Use structured logs that capture request IDs and error codes. Configure logging to ensure that document inputs and model outputs are not recorded.

• Dashboards. Include dashboards with the installation so the customer's team has access to the same views and operational insights as your support team.

If troubleshooting requires a document to reproduce an issue, ask the customer to use a synthetic or redacted document instead. Agree on this process before the first incident occurs, rather than trying to define it during an incident.

Step 6: Treat upgrades and model changes as releases

In your own cloud environment, you can upgrade a model whenever needed. In a customer's environment, however, every change must follow a formal release process that includes a change ticket, a scheduled maintenance window, and a rollback plan.

• Pin versions. Specify the server version and each model version in the installation configuration. This ensures that updates to a model hub or container registry do not introduce unexpected changes.

• Stage the weights. Transfer the new model weights to the customer's bucket through the mirroring process described in Step 3 before the scheduled upgrade window.

• Test on the customer's hardware. A model that performs well on one GPU type may not deliver the same results on another. Run your evaluation tests on the target hardware before directing traffic to the new model.

• Keep the old version. Retain the previous weights and images in the mirror until the new version has run reliably without issues.

The customer's change process may seem slow, but it serves an important purpose. Since the system handles their most sensitive documents, following this process helps build and maintain their trust in your installation.

Where SIE fits

Superlinked Inference Engine (SIE) is an Apache 2.0-licensed inference server that supports task-specific models across a document processing pipeline, including OCR, extraction, embedding, reranking, and LLM-based generation using models such as Qwen3.8-27B. It brings these capabilities together behind a single API. When self-hosted within a customer's VPC, SIE can be configured to support the deployment steps outlined above.

• Provisioning (Step 2). The deployment documentation list Terraform modules for each major cloud platform: terraform-aws-sie for EKS, terraform-google-sie for GKE, and terraform-azure-sie for AKS. The Azure deployment guide outlines a Terraform and Helm-based approach. If the customer already has a Kubernetes cluster running version 1.29 or later, SIE can be installed using a Helm chart. The repository README also lists Helm charts, KEDA-based autoscaling with scale-to-zero support, and Grafana dashboards.

• Offline weights (Step 3). SIE's offline deployment guide follows the same mirroring approach. It explains how to copy model weights in Hugging Face cache format to private S3-compatible storage and mirror the images to a private registry. The guide also confirms that this approach works for clusters with restricted outbound internet access.

• Several models, one service (Steps 4 and 5). SIE can run multiple models on shared GPUs with on-demand loading and LRU eviction. This allows a pipeline's OCR, extraction, and embedding models to use a single internal endpoint instead of requiring a separate endpoint for each model. When multiple teams share a cluster, per-tenant pools help keep their workloads separate.

• Egress and the review pack (Steps 3 and 5). By default, the installation pulls container images from ghcr.io/superlinked and model weights from Hugging Face. The offline guide explains how to mirror both into the customer's own storage and registry. The Helm chart ships with remote telemetry export switched off, allowing outbound network access to be restricted. The software is licensed under Apache 2.0, and documents remain within the customer's VPC.

• How to size the install. For production deployments, SIE recommends Kubernetes and documents support for scale-to-zero, Helm chart deployment, and Grafana dashboards. For smaller workloads with a steady demand, SIE documents a single-server Docker installation as an option. On Azure, standard GPU node pools configured with the cluster autoscaler and a minimum node count of zero provide a way to scale to zero. You can find the source code in the SIE GitHub repository. The SIE documentation also explains how to use the SDK to integrate SIE with your application.

Honest take: SIE brings multiple document-processing models together in a single service that runs inside the customer's VPC. It handles OCR, extraction, embeddings, and reranking through one API. With self-hosted SIE, licensed under Apache 2.0, you can run this engine in the customer's VPC or your own cluster. The deployment uses one Helm chart, one offline mirror, one set of dashboards, and one internal endpoint, all from a single codebase. SIE Cloud offers a managed alternative, with documents processed on Superlinked's infrastructure. Terraform modules support deployments on AWS, GCP, and Azure. For offline deployments, you can mirror model weights and container images into private storage, allowing outbound network access to be restricted. Both deployment options use the same underlying engine. This makes SIE a suitable inference layer to deploy as part of the sequence outlined above.

Frequently asked questions

What if the customer bans internet egress entirely?

In this case, mirror all required resources before installation and verify that the deployment works without egress. Download the model weights and container images outside the customer's network, then transfer them to the customer's object storage and private registry using a method approved by the security team. Configure the deployment to use these internal locations. Before the scheduled installation, test the entire process on a cluster with outbound traffic blocked. SIE's offline documentation describes this approach for both air-gapped and restricted-egress clusters. The key is to identify and eliminate any runtime downloads before installation day.

What if we do not run GPUs on Kubernetes?

SIE uses the same underlying engine across different deployment options.

• For a small, steady workload, you can use the Docker deployment on a single GPU server.

• For larger or fluctuating workloads, Kubernetes with GPU node pools is the better option, as it supports scale-to-zero and multi-model scheduling.

• If processing documents on Superlinked's infrastructure is acceptable, SIE Cloud provides a managed alternative.

Choose the deployment model that best fits your workload and data residency requirements.

How can the vendor provide support without accessing our data?

Support can be designed around metrics and logs that contain no document content. The installation should include dashboards that track request rates, latency, errors, and GPU utilization. Logs should also be configured to exclude document inputs and outputs. During an incident, your team can share these metrics and logs with the vendor or grant time-limited access to them. If a document is needed to reproduce an issue, use a redacted or synthetic version instead. Before approving the deployment, ask the vendor to demonstrate the logging configuration and verify that a sample request does not leave any document text in the logs.

Starting the install

The order of deployment steps matters more than the tools you use. Start with the pre-flight checks, provision the infrastructure from code, mirror the model weights, keep the endpoints private, use metrics for support, and roll out changes in a controlled manner. Each step helps prevent a different type of failure that might otherwise surface only in the customer's environment. When you are ready to map these steps onto SIE's modules, Helm chart, and offline deployment process, begin with the SIE deployment guide.