Turned a notebook that had been "almost ready" for three weeks into an actual service this week. The model didn't change — the packaging did.
What made the difference:
- One
predict.pywith a pure function, no global state - A tiny test that loads the model and runs a fixed input
- Docker with a pinned wheel, not a
pip installat boot
The notebook is still useful for experiments. It just isn't the deployment artifact.
Next: I want to try compiling one of these small models to WebAssembly so it can run client-side. If you've done that with a PyTorch model, I'd love to hear what worked.
Top comments (1)
I'm currently working on AI agents with LangChain and FastAPI, and I'm learning how to move from experimentation to building reliable, production-ready AI applications.
Your approach of separating prediction logic into a pure function, testing the model with fixed inputs, and pinning dependencies in Docker really resonates with me. I believe these engineering practices are just as important as the model itself, especially when building AI services that need to be maintainable and reproducible.
I'm also interested in exploring how models can be deployed more efficiently beyond traditional server-based inference. Your idea of compiling PyTorch models to WebAssembly is particularly interesting.
I'd love to learn more about your experience with model deployment. Have you experimented with ONNX or other approaches to make PyTorch models run efficiently in WebAssembly?