Deploying Hugging Face models to NVIDIA Triton on Kubernetes — end-to-end walkthrough

I’ve been working on Triton Control, an open-source project that simplifies deploying models to NVIDIA Triton Inference Server.

This article walks through two complete workflows:

• Deploying an existing Triton model repository
• Converting a Hugging Face DistilBERT model to TorchScript and serving it with Triton

It covers S3-backed model repositories, Kubernetes deployment, GPU inference, and endpoint testing.

I’m particularly interested in feedback on the architecture and deployment workflow.

Article: https://medium.com/@owilken/from-model-to-inference-endpoint-with-nvidia-triton-and-triton-control-7636439a8f28

GitHub: https://github.com/ai-lab-tech/triton-control