Part of a series on building production-grade AI systems. This post covers the setup that happened before a single line of RAG logic was written — the tooling, structure, and engineering discipline that everything else in this series builds on.
It's tempting to think production-readiness is something you bolt on at the end — add monitoring, write some tests, containerize it, ship it. In practice, most of what makes a system production-grade or not is decided before the first feature is built. Reproducible environments, enforced typing, centralized configuration, and a project structure with real boundaries either exist from commit one, or they get retrofitted under pressure later, badly.
This post is about the decisions we made before writing any application code — and just as importantly, the tools we deliberately didn't add yet.
Reproducibility starts at dependency management
The first decision was uv over pip, and pyproject.toml as the single source of truth instead of requirements.txt. This isn't a style preference — it's the difference between "I can reinstall this exact environment on any machine" and "it works on my laptop." uv.lock pins the full dependency graph, not just top-level packages, which means dev, staging, and whoever picks this project up in six months are running identical resolved versions, not whatever pip happened to resolve on the day they ran install.
For an AI system specifically, this matters more than it does for a typical web app: embedding models, vector clients, and LLM SDKs change behavior across minor versions constantly. An unpinned environment doesn't just risk "it broke" — it risks silently different embeddings or API behavior between two runs that were supposed to be identical.
Code quality enforced before there's code to review
Three tools, doing three distinct jobs, all wired in from the start:
- Ruff — linting and formatting in one tool, replacing what used to be a black + isort + flake8 stack. One config, one fast pass, no tool-conflict edge cases.
- BasedPyright — strict static type checking. This is the one that pays for itself specifically in AI pipelines: a misconfigured type flowing through a chunking → embedding → storage pipeline doesn't fail loudly at the point of the mistake, it fails silently three layers downstream, or doesn't fail at all and just produces subtly wrong vectors. Strict typing catches the shape mismatch at the function boundary, before it ever runs.
- pre-commit — moves the quality gate to commit time, not CI time. The feedback loop for "this violates our standards" is measured in seconds locally, not minutes in a CI queue after the fact.
The principle underneath all three: the cost of an error is proportional to how far it travels before it's caught. A type error caught by BasedPyright before commit costs nothing. The same error caught by CI costs a round trip. The same error surfacing as a production bug in an ingestion job costs a debugging session and possibly bad data already written to a vector store. Every one of these tools exists to catch problems as close to their origin as possible.
Configuration as a typed, validated object — not scattered os.getenv calls
The default way most Python projects handle config is load_dotenv() followed by os.getenv("SOME_KEY") calls scattered through the codebase, each one a string, each one silently returning None if you misspell the key or forget to set it — and you find out only when that specific code path runs, possibly in production, possibly under load.
Instead, configuration is a single pydantic-settings BaseSettings object:
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_file=".env")
qdrant_url: str
qdrant_api_key: str
gemini_api_key: str
embedding_model: str
# ...
This gets instantiated once, cached, and imported everywhere it's needed. The difference that actually matters in production: a missing or malformed environment variable now fails at startup, with a clear validation error naming exactly which field is wrong — not at 2am when a request happens to hit the one code path that reads that particular env var. Fail fast, fail loud, fail at the boundary where it's cheap to fix.
.env.example documents every variable a new environment needs without ever containing real secrets — onboarding a new environment (or a new developer) means copying one file and filling in real values, not archaeology through the codebase to find every os.getenv call.
A project structure that encodes responsibility before the code does
app/
├── agents/
├── api/
├── config/
├── ingestion/
├── models/
├── retrieval/
├── services/
├── utils/
└── main.py
This is the same single-responsibility principle that shows up throughout the ingestion pipeline in Part 2 — one folder, one concern — just applied one level up, at the repository level instead of the class level. ingestion/ doesn't reach into retrieval/'s internals; api/ doesn't contain business logic that belongs in services/. The boundary existing in the folder structure before any code is written means the question "where does this new piece of logic go" has an answer on day one, instead of becoming an architectural debate after fifty files already violate whatever boundary you wish you'd set.
The observability and reliability stack
Beyond the environment tooling above, a specific set of tools makes up the actual production-observability and reliability layer for this system:
-
Logfire — structured tracing for the application itself. Every
logfire.span()and structured log call throughout Part 2's ingestion pipeline — load, split, embed, store, and the manifest's change-detection logic — comes from this. It's what makes each step of a run individually inspectable instead of a black box that either produced the right output or didn't. - LangSmith — observability specifically for LLM calls: prompts, completions, token usage, latency, and reasoning traces. This is distinct from Logfire's general application tracing — it's built for the failure modes specific to LLM-driven systems: a chain that silently changed its prompt, a completion that quietly degraded in quality, a retrieval step that returned nothing useful and nobody noticed.
- Guardrails — output validation for LLM responses against a schema or safety policy, so a malformed or unsafe completion is caught before it reaches a user.
- Portkey — an LLM gateway: unified retries, fallback across providers, and cost/latency observability when more than one model or provider is in play.
- RAGAS — a RAG-specific evaluation framework, for measuring retrieval and answer quality against real queries rather than eyeballing outputs.
Together these cover the two things a production AI system actually needs to be observable and reliable: visibility into what the application is doing (Logfire), and visibility + guardrails specifically around what the LLM is doing (LangSmith, Guardrails, Portkey, RAGAS). Neither category is optional in production — an AI system without them can fail silently, with no trace of what went wrong or why an answer was bad.
Why front-loading this matters
None of this — uv, Ruff, BasedPyright, pre-commit, typed settings, folder boundaries — is specific to AI systems. It's the baseline discipline of any production software project. That's the point. An AI system doesn't become production-ready by adding a vector database and an LLM API key to an otherwise undisciplined codebase. It becomes production-ready when the discipline exists underneath the AI-specific parts — so that when ingestion, retrieval, and agent logic get built on top (as they were, in Part 2 and onward), they're being added to a foundation that already fails loudly, catches errors early, and keeps its boundaries clean, rather than a foundation that has to be hardened retroactively once something's already broken in production.
Reproducible environment, enforced typing, centralized config, clear structure, and an observability stack that covers both the application and the LLM calls inside it — that's what "production-ready" looks like before there's a single feature to demo.
Next
With the environment and tooling in place, Part 2 covers the first real piece of application logic built on top of it: the RAG ingestion pipeline.
Top comments (0)