Productionizing an MCP-Based AI Agent with Docker, Kubernetes, CI/CD, and Observability

Building an AI agent locally is an exciting first step. Running that same agent reliably in production is a different challenge.

Once real users and external services are involved, the application needs more than working code. It needs repeatable deployments, secure configuration, health checks, monitoring, controlled updates, and a clear recovery process.

This article is part of my MCP series. If you are new to the topic, start with my first article: Model Context Protocol (MCP) Servers Explained: A Complete Beginner’s Guide.

In this article, I will outline a practical architecture for taking a Model Context Protocol, or MCP-based, AI agent from a local development environment to Kubernetes.

This is a production architecture blueprint. The exact implementation will depend on the AI provider, MCP servers, cloud platform, and security requirements used by the application.

What Is an MCP-Based AI Agent?

The Model Context Protocol provides a standardized way for AI applications to connect with external tools, services, and data sources.

An MCP-based agent may interact with:

• Internal APIs
• Databases
• File systems
• Search services
• Monitoring platforms
• Business applications
• Custom automation tools

A basic implementation might work well on a developer's machine. In production, however, every dependency introduces operational questions:

• How will the application be deployed?
• Where will credentials be stored?
• How will failed requests be detected?
• Can the service handle additional traffic?
• How can a broken release be rolled back?
• What happens when an MCP server becomes unavailable?

These are familiar DevOps and Site Reliability Engineering problems applied to a new type of workload.

Target Architecture

A practical delivery flow could look like this:

Developer
    ↓
GitHub Repository
    ↓
GitHub Actions
    ↓
Container Registry
    ↓
Kubernetes Cluster
    ↓
MCP Servers and External Services
    ↓
Logs, Metrics, Traces, and Alerts

Each component has a clear responsibility:

1. GitHub stores the application code and deployment configuration.
2. GitHub Actions tests the application and builds the container image.
3. The container registry stores versioned images.
4. Kubernetes runs and scales the agent.
5. Secrets management protects API keys and credentials.
6. Observability tools provide visibility into reliability and performance.

Step 1: Containerize the Agent

Containerization gives the application a consistent runtime across development, testing, and production environments.

A simple Python-based agent could use the following Dockerfile:

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

RUN useradd --create-home appuser
USER appuser

EXPOSE 8000

CMD ["python", "app.py"]

This example follows several useful practices:

• Uses a lightweight base image
• Installs dependencies before copying the source code
• Runs the application as a non-root user
• Exposes only the required application port
• Keeps the runtime configuration outside the image

The container image should not contain API keys, access tokens, or environment-specific credentials.

Step 2: Deploy the Agent to Kubernetes

Kubernetes provides a consistent way to deploy, restart, scale, and update the service.

A simplified deployment might look like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mcp-agent
spec:
  replicas: 2
  selector:
    matchLabels:
      app: mcp-agent
  template:
    metadata:
      labels:
        app: mcp-agent
    spec:
      containers:
        - name: mcp-agent
          image: registry.example.com/mcp-agent:1.0.0
          ports:
            - containerPort: 8000
          envFrom:
            - secretRef:
                name: mcp-agent-secrets
          readinessProbe:
            httpGet:
              path: /ready
              port: 8000
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"

This configuration introduces several production controls:

• Multiple replicas improve availability
• Readiness probes prevent traffic from reaching an unprepared container
• Liveness probes allow Kubernetes to restart an unhealthy container
• Resource requests support reliable scheduling
• Resource limits reduce the risk of one workload consuming excessive cluster capacity

The values should be adjusted after observing the application's real resource usage.

Step 3: Manage Secrets Securely

An AI agent may require credentials for model providers, MCP servers, databases, or external APIs.

These values should never be committed to Git or embedded in a container image.

Kubernetes Secrets provide a basic separation between application code and sensitive configuration. For stronger production security, the cluster can integrate with a dedicated secrets platform such as:

• Azure Key Vault
• AWS Secrets Manager
• Google Cloud Secret Manager
• HashiCorp Vault

Access should follow the principle of least privilege. The agent should receive only the permissions it needs, and credentials should have a defined rotation process.

Step 4: Build a CI/CD Pipeline

A reliable CI/CD pipeline should verify the application before deploying it.

A typical pipeline could include:

1. Code quality checks
2. Unit and integration tests
3. Dependency and container security scans
4. Container image creation
5. Image publication with a unique version
6. Deployment to a non-production environment
7. Health and smoke tests
8. Production deployment with approval controls
9. Automated rollback when validation fails

A simplified GitHub Actions workflow could begin like this:

name: Build and Deploy

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Run tests
        run: |
          pip install -r requirements.txt
          pytest

      - name: Build container image
        run: |
          docker build -t mcp-agent:${{ github.sha }} .

Production pipelines should use pinned action versions, protected environments, secure authentication, and immutable image tags.

Using the Git commit SHA as an image tag also makes it easier to identify exactly which code version is running.

Step 5: Add Observability

Traditional infrastructure metrics are important, but they are not enough for an AI agent.

A useful observability strategy should cover both the platform and the application.

Platform metrics

Monitor:

• CPU and memory utilization
• Pod restarts
• Replica availability
• Request rate
• Error rate
• Response latency
• Network failures

AI and MCP metrics

Monitor:

• Model request latency
• Token consumption
• MCP tool execution time
• Tool success and failure rates
• External API availability
• Timeouts and retries
• Requests rejected by rate limits
• Estimated cost per request

Logs

Structured logs should include fields such as:

• Request or correlation ID
• MCP server name
• Tool name
• Execution duration
• Response status
• Retry count
• Error category

Sensitive prompts, credentials, personal information, and full model responses should not be written to logs without appropriate controls.

Traces

Distributed tracing can help follow a request across:

User Request → Agent → Model Provider → MCP Server → External Service

This becomes especially valuable when the total response time depends on several external systems.

Step 6: Design for Failure

An MCP server or external API will eventually become slow, unavailable, or rate limited. The agent should handle these situations without causing a wider service failure.

Useful reliability controls include:

• Request timeouts
• Limited retries with exponential backoff
• Circuit breakers
• Graceful fallback responses
• Rate limiting
• Queue-based processing for long-running tasks
• Pod disruption budgets
• Controlled rollouts
• Tested rollback procedures

Retries should be used carefully. Repeating an unsafe or non-idempotent action could create duplicate records or trigger the same operation multiple times.

Step 7: Scale Based on Meaningful Signals

Kubernetes can scale replicas horizontally, but CPU usage may not always reflect the true load of an AI application.

Depending on the architecture, scaling decisions could consider:

• Concurrent requests
• Queue length
• Request latency
• Active MCP sessions
• Number of tool executions
• Model provider rate limits

Scaling the agent does not automatically scale its dependencies. A larger number of agent replicas can place additional pressure on databases, MCP servers, and third-party APIs.

Capacity planning should therefore consider the complete request path.

Security Considerations

Production AI systems introduce risks beyond normal application security.

Important controls include:

• Authenticate requests to the agent
• Authorize every MCP tool operation
• Validate tool inputs
• Restrict network access between services
• Use separate identities for separate workloads
• Scan application dependencies and container images
• Record security-relevant actions for auditing
• Prevent untrusted input from bypassing tool permissions
• Avoid exposing secrets through prompts, logs, or error messages

An agent should not receive broad infrastructure or business-system access simply because it can use an MCP tool. Every action should still pass through clear authentication and authorization controls.

A Practical Production Checklist

Before releasing an MCP-based agent, confirm that:

• The application is packaged as a reproducible container image
• Images are scanned and versioned
• Credentials are stored outside the codebase
• Health and readiness endpoints are available
• Resource requests and limits are configured
• Logs are structured and searchable
• Metrics and alerts cover both platform and MCP behavior
• External calls have timeouts
• Retries are limited and safe
• Access follows least privilege
• Rollback procedures have been tested
• Operational documentation is available

Final Thoughts

Building an AI agent demonstrates application functionality. Productionizing it demonstrates engineering maturity.

Docker provides a consistent runtime. Kubernetes manages availability and scaling. CI/CD enables controlled releases. Observability shows how the system behaves. Security and reliability controls determine whether the service can be trusted in a real environment.

MCP may introduce a new integration model, but the production principles remain familiar: automate delivery, reduce unnecessary access, observe every dependency, expect failures, and make recovery part of the design.

How would you approach scaling and monitoring an MCP-based agent in your environment?

Comments

[Yuhai Xia]

Solid writeup. One observability gap worth adding from running ours in production: watch for the event loop itself going silent.

We had a scheduler calling a synchronous embedding client directly on the async loop. Health checks stopped responding, the container still looked "up", and our external watchdog killed and restarted it — every six hours, for a week, before we caught the pattern. Nothing in normal metrics showed it: no crash, no OOM, no error rate spike. The process was alive and simply not doing anything.

Two things that fixed it for good: moving every blocking call off the loop, and adding a side thread that dumps all-thread stacks when the loop goes quiet for >10s — so the next hang documents itself instead of leaving us guessing.

Also worth isolating failures per tenant: one user's expired provider key returned 401 forever, and our adaptive concurrency controller read that as a global error rate and throttled everyone.