DEV Community: Jasdeep Singh Bhalla

What is a Vector Database?

Jasdeep Singh Bhalla — Tue, 17 Mar 2026 16:00:00 +0000

𝐌𝐨𝐬𝐭 𝐝𝐚𝐭𝐚𝐛𝐚𝐬𝐞𝐬 𝐬𝐞𝐚𝐫𝐜𝐡 𝐟𝐨𝐫 𝐞𝐱𝐚𝐜𝐭 𝐦𝐚𝐭𝐜𝐡𝐞𝐬.

But AI doesn’t work like that.

💡 𝐓𝐡𝐚𝐭’𝐬 𝐰𝐡𝐞𝐫𝐞 𝐕𝐞𝐜𝐭𝐨𝐫 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞𝐬 𝐜𝐨𝐦𝐞 𝐢𝐧.

👉 In AI, text (like a sentence or document) is converted into numbers called 𝐞𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠𝐬.

These embeddings capture the meaning of the text. So instead of searching for exact words…

👉 𝐕𝐞𝐜𝐭𝐨𝐫 𝐝𝐚𝐭𝐚𝐛𝐚𝐬𝐞𝐬 𝐬𝐞𝐚𝐫𝐜𝐡 𝐟𝐨𝐫 𝐬𝐢𝐦𝐢𝐥𝐚𝐫 𝐦𝐞𝐚𝐧𝐢𝐧𝐠.

🧠 𝐒𝐢𝐦𝐩𝐥𝐞 𝐢𝐝𝐞𝐚:
Normal database → finds exact matches
Vector database → finds similar meaning

📌 𝐄𝐱𝐚𝐦𝐩𝐥𝐞:
Search: “𝘏𝘰𝘸 𝘵𝘰 𝘧𝘪𝘹 𝘭𝘰𝘨𝘪𝘯 𝘪𝘴𝘴𝘶𝘦”

Vector DB can return:
• “𝘢𝘶𝘵𝘩𝘦𝘯𝘵𝘪𝘤𝘢𝘵𝘪𝘰𝘯 𝘦𝘳𝘳𝘰𝘳 𝘴𝘰𝘭𝘶𝘵𝘪𝘰𝘯”
• “𝘵𝘳𝘰𝘶𝘣𝘭𝘦𝘴𝘩𝘰𝘰𝘵𝘪𝘯𝘨 𝘴𝘪𝘨𝘯-𝘪𝘯 𝘱𝘳𝘰𝘣𝘭𝘦𝘮𝘴”

Even if words don’t match exactly ✅

🚀 𝐖𝐡𝐲 𝐢𝐭 𝐦𝐚𝐭𝐭𝐞𝐫𝐬:
• Powers RAG (Retrieval-Augmented Generation)
• Used in AI search & chatbots
• Helps AI understand context
• Core building block for AI agents

🔧 𝐏𝐨𝐩𝐮𝐥𝐚𝐫 𝐕𝐞𝐜𝐭𝐨𝐫 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞𝐬:
• Pinecone
• Weaviate
• FAISS (by Meta)
• Milvus
• Qdrant
• Chroma

If you're building AI systems today, vector databases are not optional—they’re foundational.

What is RAG?

Jasdeep Singh Bhalla — Tue, 17 Mar 2026 08:59:20 +0000

𝐌𝐨𝐬𝐭 𝐀𝐈 𝐦𝐨𝐝𝐞𝐥𝐬 𝐝𝐨𝐧’𝐭 𝐚𝐜𝐭𝐮𝐚𝐥𝐥𝐲 “𝐤𝐧𝐨𝐰” 𝐲𝐨𝐮𝐫 𝐝𝐚𝐭𝐚.

They generate answers based on what they were trained on —

which means they can be:

• outdated
• incorrect
• or missing context

💡 𝐓𝐡𝐚𝐭’𝐬 𝐰𝐡𝐞𝐫𝐞 𝐑𝐀𝐆 (𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥-𝐀𝐮𝐠𝐦𝐞𝐧𝐭𝐞𝐝 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧) 𝐜𝐨𝐦𝐞𝐬 𝐢𝐧.

👉 Instead of answering directly, RAG works in 3 steps:
Search → Find relevant information (docs, PDFs, databases)
Retrieve → Pull the most useful pieces
Generate → Answer using that information

🧠 Simple idea:
Normal AI → guesses
RAG → looks up info first, then answers

📌 Example:
Ask: “𝘞𝘩𝘢𝘵’𝘴 𝘰𝘶𝘳 𝘤𝘰𝘮𝘱𝘢𝘯𝘺’𝘴 𝘭𝘦𝘢𝘷𝘦 𝘱𝘰𝘭𝘪𝘤𝘺?”

Without RAG → generic answer ❌
With RAG → pulls actual company document ✅

🚀 Why it matters:
• More accurate answers
• Uses real-time/private data
• Reduces hallucinations
• Powers AI agents & copilots

𝐈𝐟 𝐲𝐨𝐮’𝐫𝐞 𝐛𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐀𝐈 𝐭𝐨𝐝𝐚𝐲, 𝐜𝐡𝐚𝐧𝐜𝐞𝐬 𝐚𝐫𝐞 𝐲𝐨𝐮’𝐫𝐞 𝐮𝐬𝐢𝐧𝐠 𝐑𝐀𝐆 — 𝐞𝐯𝐞𝐧 𝐢𝐟 𝐲𝐨𝐮 𝐝𝐨𝐧’𝐭 𝐫𝐞𝐚𝐥𝐢𝐳𝐞 𝐢𝐭.

🐳 10 Important Docker Terms Every Developer Should Know

Jasdeep Singh Bhalla — Sun, 15 Mar 2026 09:18:32 +0000

Docker has become one of the most important tools in modern software
development. It allows developers to package applications with their
dependencies and run them consistently across environments.

If you're getting started with Docker, understanding a few key concepts
can make the learning curve much easier.

Here are 10 essential Docker terms every developer should know.

📦 1. Container

A container is a lightweight, portable environment that runs an
application along with everything it needs.

This includes: - application code - runtime - system libraries -
dependencies

Containers isolate applications from the host system so they run
consistently across environments.

Example:

docker run nginx

Learn more:

https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-a-container/

🧱 2. Docker Image

A Docker image is a blueprint used to create containers.

It contains: - application code - dependencies - runtime - environment
configuration

Images are read-only templates used to start containers.

Example:

docker pull nginx

Learn more:

https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-an-image/

📝 3. Dockerfile

A Dockerfile is a text file that defines how to build a Docker
image.

Example:

FROM node:20
WORKDIR /app
COPY . .
RUN npm install
CMD ["npm","start"]

Learn more:

https://docs.docker.com/build/concepts/dockerfile/

🌐 4. Docker Hub

Docker Hub is the most popular public registry for Docker images.

Developers use it to: - store container images - share images - download
official images

Examples of official images:

nginx
redis
postgres
node

https://hub.docker.com

🗄️ 5. Container Registry

A container registry stores Docker images so they can be pulled and
deployed.

Examples:

Docker Hub
https://hub.docker.com

Amazon ECR
https://aws.amazon.com/ecr/

Google Artifact Registry
https://cloud.google.com/artifact-registry

Azure Container Registry
https://azure.microsoft.com/products/container-registry

💾 6. Docker Volumes

Containers are ephemeral, meaning their data disappears when the
container stops.

Docker volumes provide persistent storage.

Common use cases:

databases
logs
uploads

Example:

docker run -v mydata:/data postgres

Learn more:

https://docs.docker.com/storage/volumes/

🌐 7. Docker Networking

Docker networking allows containers to communicate with each other.

Common network types:

bridge
host
overlay

Example:

docker network create mynetwork

Learn more:

https://docs.docker.com/network/

⚙️ 8. Docker Compose

Docker Compose lets you run multi-container applications using a
single YAML configuration.

Example:

version: "3"

services:
  web:
    image: nginx
  database:
    image: postgres

Run:

docker compose up

Learn more:

https://docs.docker.com/compose/

⚡ 9. Docker Layer Caching

Docker builds images using layers.

Each instruction in a Dockerfile creates a layer. Docker caches layers
to speed up future builds.

Example:

FROM node:20
COPY package.json .
RUN npm install
COPY . .

Learn more:

https://docs.docker.com/build/cache/

🧠 10. Container Runtime

A container runtime is responsible for running containers.

Examples include: - containerd - runc

They manage: - container lifecycle - process execution - resource
isolation

Learn more:

https://docs.docker.com/engine/

🚀 Final Thoughts

Docker has transformed how software is built and deployed by making
applications portable and reproducible.

Understanding these concepts gives developers a strong foundation for
working with containerized applications.

Official Docker resources:

https://www.docker.com
https://docs.docker.com

🤖 Docker in AI Production: Why It Matters (and What Breaks Without It)

Jasdeep Singh Bhalla — Fri, 06 Feb 2026 21:30:56 +0000

Docker has become one of the most important tools in modern AI engineering.

From model serving to agent execution, almost every AI platform today relies on containers:

LLM inference APIs
GPU-based training workloads
Retrieval-Augmented Generation (RAG) pipelines
Autonomous agents running tools
MCP server deployments
AI DevOps workflows

But here’s the key point:

Docker is not the problem — Docker is what makes AI production possible.

This article explains why Docker is so valuable, and what kinds of AI failures teams face without containerization, especially as MCP-powered agents become mainstream.

1. AI Systems Without Docker Are Hard to Reproduce

Without Docker, teams run into:

dependency mismatches
inconsistent Python environments
CUDA version conflicts
“works on my machine” model behavior

Example:

developer runs PyTorch 2.2
production server runs PyTorch 2.0
inference output changes subtly

Docker solves this by packaging the runtime environment.

2. Model Serving Without Containers Becomes Deployment Chaos

Deploying an LLM without Docker often means:

installing libraries manually on servers
configuring drivers by hand
repeating setup across environments

With Docker, serving becomes:

docker run --gpus all my-llm-server

Portable, repeatable, automated.

3. MCP Tool Servers Need Isolation

The Model Context Protocol (MCP) enables AI agents to call tools:

filesystem tools
cloud APIs
databases
CI/CD automation
internal governance systems

But MCP introduces a new requirement:

Tool execution must be sandboxed.

Running MCP servers without Docker means:

tools run directly on host machines
agents may access sensitive files
prompt injection can trigger real commands

Docker provides safe boundaries:

isolated filesystem
controlled networking
least-privilege execution

4. AI Agents Without Docker Become a Security Risk

Modern AI agents are not passive chatbots.

They can:

run shell commands
modify repositories
deploy infrastructure
call external APIs

Without Docker sandboxing, this creates risks:

credential leaks
unintended host access
tool poisoning attacks
container escape becomes host escape

Docker Sandboxes and hardened images are now critical for safe agent execution.

5. Scaling AI Workloads Without Docker Is Expensive and Slow

Without containers, scaling means:

configuring new servers manually
inconsistent runtime setups
slow onboarding of new nodes

With Docker + orchestration (Kubernetes/ECS):

replicas spin up predictably
environments stay consistent
scaling becomes automated

6. RAG Pipelines Without Docker Become Unmanageable

A real RAG system includes:

LLM server
embedding model
vector database
retriever service
MCP tool servers

Without Docker Compose, deployment becomes messy.

With Compose:

services:
  llm:
  vectordb:
  retriever:
  tools:

One command brings the stack up:

docker compose up

7. Observability Without Containers Gets Worse

AI systems require monitoring:

token throughput
hallucination rates
retrieval quality
agent tool calls

Docker provides consistent logging + metrics hooks that integrate with:

Prometheus
OpenTelemetry
Grafana
cloud observability

Without Docker, monitoring becomes inconsistent across machines.

8. Supply Chain Security Improves With Docker

AI workloads depend on massive open-source stacks.

Docker helps teams:

pin base images
scan for vulnerabilities
enforce hardened runtimes

Tools like:

Docker Scout
Docker Hardened Images
signed registries

are becoming mandatory for AI governance.

Conclusion: Docker Is the Foundation for Safe AI + MCP Deployment

AI production introduces complexity:

huge dependencies
GPU runtime requirements
agent tool execution
security threats like prompt injection

Docker is what makes these systems:

portable
reproducible
scalable
governable
secure

And as MCP-powered AI agents become standard, Docker-style sandboxing will be non-negotiable.

Docker didn’t create AI production problems.

Docker is what prevents AI production from collapsing into chaos.

🚢 Docker Explained: Everything You Need to Know (Beginner Pro)

Jasdeep Singh Bhalla — Fri, 06 Feb 2026 21:21:52 +0000

Docker has become one of the most important tools in modern software engineering.

Whether you're building microservices, deploying AI agents, running cloud workloads, or just trying to avoid the classic:

“It works on my machine…”

Docker solves a huge problem:

consistent environments
fast deployments
portable applications
scalable infrastructure

1. What is Docker?

Docker is a platform that allows you to package applications into containers.

A container includes:

application code
runtime (Python, Node, Java, etc.)
dependencies
system libraries
configuration

So the app runs the same everywhere:

laptop
server
cloud
CI/CD pipeline

2. Docker vs Virtual Machines

Feature	Virtual Machine	Docker Container
Includes OS?	Yes	No
Startup time	Minutes	Seconds
Resource usage	Heavy	Lightweight
Best for	Full OS virtualization	App packaging

3. Key Docker Concepts

📦 Image

An image is a blueprint/template.

🏗 Container

A running instance of an image.

Dockerfile

Defines how to build an image.

Example:

FROM python:3.11
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "main.py"]

4. Docker Compose

Docker Compose helps run multi-container apps.

Example:

version: "3.9"

services:
  web:
    build: .
    ports:
      - "5000:5000"

  redis:
    image: redis:latest

Run:

docker compose up

5. Docker Security Basics

Best practices:

use slim images
avoid running as root
scan images
manage secrets properly

Example:

USER nobody

6. Docker + AI Agents + MCP Servers

Docker is now powering AI agents safely:

Claude Code in sandboxes
MCP servers as containers
Secure tool execution

Conclusion

Docker enables:

portable apps
reproducible builds
scalable infra
safe AI execution

If you're learning cloud + AI + security, Docker is mandatory.

Docker Compose for AI Agents: From Local Prototype to Production in One Workflow

Jasdeep Singh Bhalla — Fri, 06 Feb 2026 07:33:06 +0000

AI agents are quickly becoming the next major platform shift in software engineering.

They are no longer limited to answering questions in a chat window. Today’s agentic applications can:

reason over tasks
call external tools
query APIs
orchestrate workflows
interact with cloud infrastructure
operate autonomously inside DevOps pipelines

But with this new power comes a familiar engineering challenge:

How do we build, run, and ship AI agents reliably—from laptop to production?

The answer is surprisingly simple:

Docker Compose.

Compose is evolving from a local developer convenience into the backbone of agentic application deployment.

AI Agents Are More Than Just Models

An AI agent is not just an LLM.

A production-grade agentic system typically includes:

the agent runtime (LangGraph, CrewAI, Semantic Kernel, etc.)
one or more LLM backends (OpenAI, local Llama, Amazon Bedrock)
tool integrations (MCP servers, APIs, databases)
memory/state stores (Redis, Postgres, vector DBs)
observability (logs, tracing, metrics)
security boundaries (network + identity controls)

In other words:

Agents are distributed systems.

And distributed systems need orchestration.

Why Docker Compose Fits Agents Perfectly

Docker Compose has always been good at one thing:

Defining multi-service applications with a single declarative file.

Agentic apps are inherently multi-service, which makes Compose a natural match.

Helpful references:

Compose gives you:

reproducible local environments
consistent dependency wiring
portable deployment artifacts
scalable service definitions
security controls through isolation

And most importantly:

No new workflow.

A Minimal `docker-compose.yml` for an AI Agent

version: "3.9"

services:
  agent:
    build: ./agent
    container_name: ai-agent
    ports:
      - "8080:8080"
    environment:
      - MODEL_PROVIDER=openai
      - MCP_SERVER_URL=http://mcp-server:9000
      - REDIS_HOST=redis
    depends_on:
      - redis
      - mcp-server

  redis:
    image: redis:7
    container_name: agent-memory
    ports:
      - "6379:6379"

  mcp-server:
    image: myorg/mcp-tool-server:latest
    container_name: mcp-gateway
    ports:
      - "9000:9000"

Run everything locally:

docker compose up --build

Adding Local Model Execution (Docker Model Runner)

Docker now supports running open-source models locally through Docker Model Runner.

Docker Model Runner Documentation

This makes it easy to test agentic workflows without sending data to external providers.

Tool Integration with MCP Servers

The Model Context Protocol (MCP) is emerging as a standard way for AI agents to connect to tools and services.

Model Context Protocol (MCP) Official Site

Docker is also building MCP-native infrastructure, including MCP Gateway and Hub MCP servers.

From Development to Production

Docker is pushing Compose into production workflows through:

Docker Offload (GPU-backed remote engines)
Docker Cloud deployments
Docker Cloud Documentation for deploying Compose apps

Now the same Compose file can support:

laptop development
staging deployment
production rollout

Production Considerations for AI Agents

Secrets Management

Never hardcode API keys.

Use:

Network Isolation

Agents should not have unrestricted outbound access.

Compose Networking Guide

Least Privilege Tool Access

Your MCP gateway should enforce scoped permissions and audit logging.

On AWS, follow:

IAM Security Best Practices

Observability Built In

Agents are long-running systems.

Add logging + tracing early:

Secure Supply Chain for Agent Containers

AI agents must run on trusted, secure base images.

Docker Hardened Images
Docker Scout for vulnerability scanning

Final Thought

AI agents are becoming core application infrastructure.

Docker Compose provides the missing workflow layer:

Compose. Build. Deploy. Agents—From Dev to Prod.

Using a Docker Sandbox for a Coding Agent

Jasdeep Singh Bhalla — Tue, 03 Feb 2026 23:57:09 +0000

This guide shows a concrete, end-to-end example of how Docker Sandboxes can be created and used to safely run autonomous coding agents that can install packages, modify files, and even run Docker — without touching your host machine.

Scenario

You want to let a coding agent:

Modify a real codebase
Install system dependencies
Build and run containers
Run unattended with permissive flags

…but without risking your laptop or credentials.

Docker Sandboxes solve this by running the agent inside a disposable microVM with only your project workspace mounted.

Step 1: Create a Sandbox

Create a new sandbox using your local project directory as the workspace:

docker sandbox create   --name agent-sandbox   --workspace ./my-project

What this does:

Creates a dedicated microVM
Mounts only ./my-project into the sandbox
Keeps your OS, home directory, and secrets isolated

Step 2: Enter the Sandbox

Start an interactive shell inside the sandbox:

docker sandbox exec agent-sandbox bash

You are now inside the sandbox, not your host machine.

From this point on:

Any package installs
Any config changes
Any Docker commands

…are fully isolated.

Step 3: Run a Coding Agent (Unattended)

Inside the sandbox, run your coding agent in permissive mode:

claude-code run   --dangerously-skip-permissions   --project /workspace

Why this is safe:

The agent runs inside a microVM
Only the project directory is writable
No access to your host OS, SSH keys, or credentials

This is the intended workflow for Docker Sandboxes.

Step 4: Let the Agent Install Dependencies

The agent can freely modify the environment:

apt-get update
apt-get install -y nodejs npm

No permission prompts.

No approval loops.

No risk to your machine.

Step 5: Let the Agent Use Docker

Inside the sandbox, the agent can build and run containers:

docker build -t my-app .
docker run -p 8080:8080 my-app

Important notes:

This does not use your host Docker daemon
Containers run entirely inside the sandbox microVM

This capability is what makes Docker Sandboxes fundamentally different from regular containers.

Step 6: Review the Results on the Host

Exit the sandbox and inspect the changes:

git diff

You’ll see only the intentional code changes made by the agent.

No stray system packages.

No modified OS files.

No lingering background processes.

Step 7: Delete the Sandbox

When you’re done, delete the sandbox:

docker sandbox delete agent-sandbox

The microVM is destroyed immediately.

The environment is wiped
Nothing persists except your code changes
You start clean the next time

Why This Pattern Works

Docker Sandboxes give agents:

A real operating system
Package managers and system tools
Docker access
Full autonomy

While giving you:

Strong isolation via microVMs
Disposable environments
Zero host contamination
Confidence to use permissive agent modes

When to Use Docker Sandboxes

This pattern is ideal when:

Running coding agents unattended
Using flags like --dangerously-skip-permissions
Allowing agents to install tools dynamically
Letting agents build and run containers
Experimenting aggressively without fear

Agents need freedom.

Your machine doesn’t.

Skip the Cloud, Not the Control: Running AI Models Locally with Docker Model Runner

Jasdeep Singh Bhalla — Tue, 03 Feb 2026 23:52:15 +0000

AI development is moving fast—but for many teams, the default workflow still means shipping data to the cloud, managing tokens, and worrying about privacy, latency, and cost. What if you could run powerful AI models locally, using the same Docker tools you already trust in production?

That’s exactly what Docker Model Runner enables.

In this post, we’ll walk through:

What Docker Model Runner is
Why running models locally matters
How to run AI models with a single Docker command
How it fits naturally into real production and CI/CD workflows

Why Local-First AI Matters

Cloud-based LLM APIs are convenient—but they come with tradeoffs:

💸 Token costs add up quickly
🔒 Sensitive data leaves your machine
🌐 Latency and rate limits slow iteration
⚙️ Limited control over model behavior

Running models locally flips that equation. You keep full ownership of your data, avoid per-request costs, and iterate faster—especially during development and testing.

Docker Model Runner is designed to make that local-first approach simple.

What Is Docker Model Runner?

Docker Model Runner lets you run AI models locally using familiar Docker CLI commands. Models are packaged and distributed as OCI artifacts, meaning they work seamlessly with existing Docker infrastructure like Docker Hub, Docker Compose, and CI pipelines.

It supports:

Any OCI-compliant registry
Popular open-source LLMs
OpenAI-compatible APIs for easy app integration
Native GPU acceleration for high-performance inference

All without reinventing your toolchain.

Running Your First Model

If you already use Docker, you’re 90% of the way there.

Running a model locally is as simple as:

docker model run <model-name>

That’s it.

Docker Model Runner pulls the model from an OCI registry, initializes it locally, and exposes an inference endpoint you can immediately start using.

No Python environments.

No custom scripts.

No fragile dependencies.

For a full walkthrough, see the Docker Model Runner Quick Start Guide.

Models Ready to Go

You can:

Explore a curated catalog of open-source AI models on Docker Hub
Pull models directly from Hugging Face using OCI-compatible workflows

Because models are OCI artifacts, they’re:

Versioned
Portable
Easy to share across teams

This makes collaboration and reproducibility dramatically simpler.

Easy Integration with Your Apps

Docker Model Runner supports OpenAI-compatible APIs, which means many existing apps work out of the box.

You can connect it to frameworks like:

Your app talks to a local endpoint—but behaves as if it’s using a hosted API.

This makes swapping between local development and production workflows painless.

GPU Acceleration Without the Headaches

For teams running on capable hardware, Docker Model Runner supports native GPU acceleration, unlocking fast, efficient inference on your local machine.

No manual CUDA setup.

No driver gymnastics.

Just Docker doing what it does best: abstracting complexity.

Learn more about GPU support in Docker Desktop.

Built for Real Production Workflows

Docker Model Runner isn’t just a dev toy—it’s designed to scale across teams:

Use Docker Compose for multi-service applications
Integrate with Testcontainers for AI-powered testing
Package and publish models securely to Docker Hub
Manage access and permissions for enterprise teams

Because it’s Docker-native, it fits naturally into CI/CD pipelines and existing governance models.

When Should You Use Docker Model Runner?

Docker Model Runner is ideal when you want to:

Prototype AI features without cloud costs
Keep sensitive data fully local
Test models before production deployment
Standardize AI workflows across teams
Avoid vendor lock-in

If you already trust Docker in production, this is the missing piece for AI.

Get Started Today

Local AI doesn’t have to be complicated.

With Docker Model Runner, you can:

Run LLMs locally
Keep control of your data
Cut costs
Use the Docker tools you already know

👉 Try Docker Model Runner and bring AI development into your local workflow.

Hassle-free local inference starts here 🚀

Running Claude Code with Docker

Jasdeep Singh Bhalla — Tue, 03 Feb 2026 23:42:27 +0000

A beginner-friendly setup for private, on-device coding agents

Modern AI coding tools are powerful—but many developers don’t want their source code leaving their machine. The good news is that Claude Code can run entirely locally, powered by Docker.

This guide shows how to connect Claude Code to Docker Model Runner so you can use a local language model for agentic coding, with minimal setup and full control over your data.

What This Setup Does

Instead of sending prompts to a hosted API, Claude Code talks to a local model running inside Docker.

High-level flow:

Claude Code (terminal)
        ↓
Docker Model Runner
        ↓
Local LLM container

Your repository stays local. No API keys required.

Prerequisites

Make sure you have:

Docker Desktop installed and running
Claude Code installed on your system
At least 16 GB of RAM (recommended)

Step 1: Install Claude Code

macOS / Linux

curl -fsSL https://claude.ai/install.sh | bash

Windows (PowerShell)

irm https://claude.ai/install.ps1 | iex

Verify installation:

claude --version

Step 2: Enable Docker Model Runner

Docker Model Runner lets Docker pull, run, and serve large language models locally.

If you’re using Docker Desktop, enable TCP access:

docker desktop enable model-runner --tcp

Once enabled, the local API will be available at:

http://localhost:12434

You only need to do this once.

Step 3: Pull a Local Model

Download a model using Docker’s model CLI. For example:

docker model pull gpt-oss

List available models:

docker model ls

Step 4: Increase the Context Window

For real-world repositories, a larger context window helps significantly.

Create a new model variant with a bigger context (example: 32K tokens):

docker model package   --from ai/gpt-oss   --context-size 32000   gpt-oss:32k

This creates a new tagged model without modifying the original.

Step 5: Connect Claude Code to Docker

Claude Code supports custom API endpoints via an environment variable.

Run Claude Code like this:

ANTHROPIC_BASE_URL=http://localhost:12434 claude --model gpt-oss:32k "Summarize this repository."

Claude Code now sends all requests to your local Docker model.

Step 6: Make the Setup Persistent

To avoid setting the environment variable every time, add it to your shell profile.

# ~/.bashrc or ~/.zshrc
export ANTHROPIC_BASE_URL=http://localhost:12434

Now you can simply run:

claude --model gpt-oss:32k "Explain the main service logic."

Optional: Inspect Claude Code Requests

You can inspect the raw requests sent by Claude Code:

docker model requests --model gpt-oss:32k | jq .

This is useful for debugging, learning prompt structure, and understanding how agentic coding tools work.

Why Run Claude Code Locally?

Full control over your source code
No API keys or usage-based billing
Offline-friendly development
Reproducible setup with Docker
Custom models and context sizes

Next Steps

Try different local coding models
Tune context size for large repositories
Pair with Docker sandboxes for safe execution
Use this setup as a foundation for custom coding agents

Happy hacking 🚀

DEV Community: Jasdeep Singh Bhalla

What is a Vector Database?

What is RAG?

🐳 10 Important Docker Terms Every Developer Should Know

📦 1. Container

🧱 2. Docker Image

📝 3. Dockerfile

🌐 4. Docker Hub

🗄️ 5. Container Registry

💾 6. Docker Volumes

🌐 7. Docker Networking

⚙️ 8. Docker Compose

⚡ 9. Docker Layer Caching

🧠 10. Container Runtime

🚀 Final Thoughts

🤖 Docker in AI Production: Why It Matters (and What Breaks Without It)

1. AI Systems Without Docker Are Hard to Reproduce

2. Model Serving Without Containers Becomes Deployment Chaos

3. MCP Tool Servers Need Isolation

4. AI Agents Without Docker Become a Security Risk

5. Scaling AI Workloads Without Docker Is Expensive and Slow

6. RAG Pipelines Without Docker Become Unmanageable

7. Observability Without Containers Gets Worse

8. Supply Chain Security Improves With Docker

Conclusion: Docker Is the Foundation for Safe AI + MCP Deployment

🚢 Docker Explained: Everything You Need to Know (Beginner Pro)

1. What is Docker?

2. Docker vs Virtual Machines

3. Key Docker Concepts

📦 Image

🏗 Container

Dockerfile

4. Docker Compose

5. Docker Security Basics

6. Docker + AI Agents + MCP Servers

Conclusion

Docker Compose for AI Agents: From Local Prototype to Production in One Workflow

AI Agents Are More Than Just Models

Why Docker Compose Fits Agents Perfectly

A Minimal docker-compose.yml for an AI Agent

Adding Local Model Execution (Docker Model Runner)

Tool Integration with MCP Servers

From Development to Production

Production Considerations for AI Agents

Secrets Management

Network Isolation

Least Privilege Tool Access

Observability Built In

Secure Supply Chain for Agent Containers

Final Thought

Using a Docker Sandbox for a Coding Agent

Scenario

Step 1: Create a Sandbox

Step 2: Enter the Sandbox

Step 3: Run a Coding Agent (Unattended)

Step 4: Let the Agent Install Dependencies

Step 5: Let the Agent Use Docker

Step 6: Review the Results on the Host

Step 7: Delete the Sandbox

Why This Pattern Works

When to Use Docker Sandboxes

Skip the Cloud, Not the Control: Running AI Models Locally with Docker Model Runner

Why Local-First AI Matters

What Is Docker Model Runner?

Running Your First Model

Models Ready to Go

Easy Integration with Your Apps

GPU Acceleration Without the Headaches

Built for Real Production Workflows

When Should You Use Docker Model Runner?

Get Started Today

Running Claude Code with Docker

What This Setup Does

Prerequisites

Step 1: Install Claude Code

macOS / Linux

Windows (PowerShell)

Step 2: Enable Docker Model Runner

Step 3: Pull a Local Model

Step 4: Increase the Context Window

A Minimal `docker-compose.yml` for an AI Agent