From Jupyter Notebook to Production: 7 Hard Lessons I Learned Deploying an AI Model

My model worked perfectly in the notebook.

Accuracy? Great. F1 score? Even better. I could run the whole pipeline in a few cells, get predictions on a test set, and feel like I'd solved the problem.
Then we tried to put it in production.

Suddenly, the same model that felt so reliable became a source of latency spikes, weird errors at 3 a.m., and endless "but it worked in my notebook" conversations.

If you've ever moved an AI or machine learning model from Jupyter to a real system, you know this pain. Here are the lessons I learned the hard way, with concrete examples and patterns you can copy.

1. Notebooks Are for Exploration, Not Deployment This one hurts, but it's important: your notebook is not your codebase. In the notebook, I had: Cells run out of order. Global variables hiding important state. Hard-coded paths like /home/me/projects/ai-demo/data/train.csv. In production, none of that survives.

What I did instead:
Moved all logic into proper Python modules (src/training, src/inference, src/preprocessing).
Used the notebook only for:
Exploratory data analysis.
Quick experiments with models.
Visualizations and one-off checks.

A simple rule that helped:
If it needs to run more than once, it doesn't belong in a notebook.

1. Reproducibility Is Not Optional In the notebook, I could: Install a package manually. Upgrade a library when something broke.

Forget to pin versions.
In production, that chaos becomes incidents.

What fixed this:
A requirements.txt (or pyproject.toml) with pinned versions:
scikit-learn==1.4.2
pandas==2.2.1
numpy==1.26.4
fastapi==0.111.0
uvicorn==0.30.1
A Dockerfile that builds from a clean base image:
FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY src ./src
COPY models ./models

CMD ["uvicorn", "src.api:app", "--host", "0.0.0.0", "--port", "8000"]

Now, anyone (including future me) can rebuild the exact same environment months later.

1. Your Data Pipeline Will Betray You First In the notebook, my preprocessing looked innocent: notebook_cell_preprocess.py (simplified)

import pandas as pd
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder

df = pd.read_csv("data/train.csv")

num_cols = ["age", "income"]
cat_cols = ["city", "device_type"]

imputer = SimpleImputer(strategy="median")
df[num_cols] = imputer.fit_transform(df[num_cols])

encoder = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
encoded = encoder.fit_transform(df[cat_cols])
It worked beautifully—until production saw:
New categories in city.
A new column added by another team.
Missing values in fields that were previously complete.

Lessons:
Save and version your preprocessing objects (imputer, encoder, scaler) alongside the model.
Treat preprocessing as part of the model, not a separate step.
In my inference service, I ended up with something like:
src/inference/predict.py

import joblib
import pandas as pd

preprocessor = joblib.load("models/preprocessor.joblib")
model = joblib.load("models/model.joblib")

def predict(df: pd.DataFrame) -> pd.Series:

Ensure column order matches training

expected_cols = ["age", "income", "city", "device_type"]
df = df[expected_cols]

features = preprocessor.transform(df)
return model.predict(features)
No ad-hoc Pandas magic in the API layer. Just: load, transform, predict.

1. Latency Is Part of Your Model's "Accuracy" In the notebook, I didn't care if a prediction took 200 ms or 2 seconds. In production, users do. My first FastAPI endpoint looked like this: src/api.py (naive version)

from fastapi import FastAPI
from pydantic import BaseModel
import pandas as pd
from src.inference.predict import predict

app = FastAPI()

class UserInput(BaseModel):
age: float
income: float
city: str
device_type: str

@app.post("/predict")
def predict_endpoint(user: UserInput):
df = pd.DataFrame([user.dict()])
pred = predict(df)
return {"prediction": pred[0]}
It worked, but:
Cold starts were slow.
Loading the model on every request was a disaster (I tried that once, don't).
No caching, no timeouts, no metrics.
Improvements that mattered:
Load model and preprocessor once at startup:
src/api.py (better version)

from fastapi import FastAPI
from pydantic import BaseModel
import pandas as pd
import joblib

app = FastAPI()

preprocessor = None
model = None

@app.on_event("startup")
def load_artifacts():
global preprocessor, model
preprocessor = joblib.load("models/preprocessor.joblib")
model = joblib.load("models/model.joblib")

class UserInput(BaseModel):
age: float
income: float
city: str
device_type: str

@app.post("/predict")
def predict_endpoint(user: UserInput):
df = pd.DataFrame([user.dict()])
features = preprocessor.transform(df)
pred = model.predict(features)
return {"prediction": pred[0]}
Added timeouts and circuit breakers when calling external services (feature stores, vector DBs, etc.).
Monitored p50/p95 latency, not just average.
In production, a slightly less accurate model that's 5× faster is often the better product decision.

1. Monitoring Is Not Just "Is the Service Up?"
   In the notebook, I checked performance once on a static test set. In production, the world keeps moving.
   Issues I faced:
   Data drift: Input distributions changed over time (e.g., new user segments).
   Concept drift: The relationship between features and target shifted (e.g., behavior after a product change).
   Silent failures: service up, no errors, but predictions slowly became useless.
   What helped:
   Logging input statistics (means, distributions) for key features.
   Tracking prediction distributions (are we suddenly predicting "high risk" for 90% of users?).
   Simple dashboards for:
   Request volume.
   Latency.
   Error rates.
   Prediction stats.
   You don't need a fancy ML monitoring platform on day one. Start with:
   Structured logs.
   A basic time-series dashboard.
   Alerts when something clearly weird happens.

2. Version Everything: Models, Data, Configs
   I lost count of how many times I asked:
   "Which model is running in production right now?"
   If you can't answer that instantly, you'll have a bad time.

My setup now:
Model artifacts stored with version tags:
models/risk-score/v1.3.0/model.joblib
Configs (thresholds, feature lists) in version-controlled files.
A simple manifest or metadata file per deployment:
{
"model_name": "risk_score",
"version": "1.3.0",
"trained_on": "2026-06-10",
"training_data_hash": "abc123...",
"preprocessor_version": "1.1.0"
}
This makes rollbacks boring instead of terrifying.

1. Design for Humans, Not Just for Metrics In the notebook, I optimized for AUC, F1, RMSE. In production, I had to optimize for: Can a support engineer understand why this prediction happened? Can a product manager explain this to a user? Can we safely turn this off if it misbehaves? That meant:

Adding simple explanations (e.g., top contributing features) to responses when possible.
Exposing a "safe mode" (e.g., fallback to rules-based logic if the model is uncertain).
Writing documentation that non-ML engineers can read.

An AI model in production isn't just a function; it's a feature that affects real people.

Key Takeaways
Treat notebooks as sandboxes, not deployment artifacts.
Enforce reproducibility: pin versions, use containers, automate builds.
Make preprocessing part of the model; don't reinvent it in the API layer.

Care about latency, reliability, and cost, not just accuracy.
Monitor data, predictions, and system health—not just uptime.
Version models, data schemas, and configs like code.

Design for humans: explainability, safety, and clear ownership matter.
What's Your Worst "Notebook vs Production" Story?
If you've deployed an AI or machine learning model, what surprised you the most when moving from notebook to production? Was it data drift, latency, or something completely unexpected?

Share your war stories in the comments. I'm especially interested in patterns that worked for you—maybe they'll make it into my next deployment.