DEV Community: Ajeet Singh Raina

Building an Autonomous A/B Testing Agent on Docker Cloud Sandboxes

Ajeet Singh Raina — Mon, 12 Oct 2026 06:04:38 +0000

Have you ever wondered if an AI agent could run an A/B test all by itself?

In this post, we will build one together. We will go slowly, one file at a time, so you can follow along even if you have never used Docker Cloud Sandboxes before.

Full code: github.com/ajeetraina/sbx-ab-agent

What are we building?

Think of a small online shop. Some people say it is hard to find things in it. We want to fix that, but we are not sure which fix is best.

So we do this:

Put the shop online in a safe cloud box (a sandbox).
Send in some pretend shoppers (AI agents) and ask them how easy the shop was to use.
Ask an analyst (another AI agent) to suggest a few fixes.
Give each fix to a builder (another AI agent) who writes the code, each in its own sandbox.
Send the same pretend shoppers to test each new version.
Pick the version with the best score and copy its code back to your laptop.

Here is the same idea as a picture:

                ┌──────────── Docker Cloud Sandboxes ────────────┐
npm start ──►   │ A  original shop  → shoppers test → analyst    │
                │ B  new sandbox → builder adds fix B → re-test  │
                │ C  new sandbox → builder adds fix C → re-test  │
                └────────────────────────────────────────────────┘
                                   │
                                   ▼
                    winner + report + winning code

"A" is the original shop. "B" and "C" are the new versions. That is all A/B testing really means: try a few versions and see which one works better.

Why use a sandbox at all?

Our builder agent will write and run code that nobody has checked. You do not want that running on your laptop.

A Docker Cloud Sandbox is a small, throwaway computer in the cloud (a microVM). It gives us four things we need:

Safety. If the agent does something silly, only the sandbox is affected.
Separation. Fix B and fix C each get their own sandbox, so they never mix.
A real link. Each sandbox can share a public HTTPS URL, so you can open every version in your browser.
Safe keys. Your OpenAI key is stored as a secret. The sandbox uses it for outgoing calls, but the key never sits inside the sandbox.

We control all of this from a small TypeScript program using the @docker/sandboxes SDK.

What you need

Node.js 20 or newer
A Docker account with access to Docker Cloud Sandboxes
An OpenAI API key (the agents use OpenAI's Codex)

Step 1: Get the code and look around

git clone https://github.com/ajeetraina/sbx-ab-agent
cd sbx-ab-agent
npm install

The project is small. Here is a map:

app/                 the demo shop
  server.mjs         a tiny web server
  index.html         the page
  app.js             shows the products
  styles.css         the look
  products.json      the product list
src/
  sbx.ts             helpers that talk to Docker Cloud Sandboxes
  prompts.ts         the instructions for each AI agent
  ab.ts              the main program that runs every step
  dashboard.ts       a live web page to watch the run
  cleanup.ts         deletes the sandboxes when you are done

A good reading order is: app/ first, then prompts.ts, then sbx.ts, then ab.ts. That is the order we will follow.

Step 2: Meet the shop (`app/`)

The shop shows a long list of products. That is it. There is no search, no filter and no sort. This is on purpose. It is the problem we want our agents to find and fix.

Here is the code that draws the products, from app/app.js:

async function main() {
  const products = await (await fetch('/products.json')).json();
  render(products);
}

It loads every product and puts them all on the page. Nothing more.

Now look at the server in app/server.mjs. The key line is this one:

const body = await readFile(file);

The server reads the file from disk every time someone asks for it. It never keeps an old copy. This matters later: when the builder agent changes app.js, the live site changes right away. No restart needed.

Try the shop on your own machine:

npm run shop
# open http://localhost:3000

Step 3: Meet the three AI helpers (`src/prompts.ts`)

We have three roles. They all use the same AI (Codex). The only difference is the instructions (the prompt) we give each one.

Helper 1: the pretend shopper

We have three shoppers, each with a simple goal:

Shopper	Who they are	What they want
`budget-student`	a student with about $20	a gift under $20 for a developer friend
`hoodie-hunter`	a busy engineer	all hoodies in size L, then the cheapest one
`sticker-collector`	a sticker fan	see every sticker and compare prices

In code, a shopper is just a small object:

{
  id: 'hoodie-hunter',
  who: 'a busy engineer who already knows they want a hoodie in size L',
  task: 'find all hoodies available in size L and pick the cheapest one',
}

The shopper is told: "You are a real person, not a developer. Do not edit any files." It looks at the shop using curl and imagines the clicks and scrolling a real person would need.

Most important: the shopper must answer in JSON, not in a paragraph. That way our program can read the answer easily.

{
  "persona": "hoodie-hunter",
  "completed": true,
  "steps": 14,
  "score": 4,
  "friction": ["had to scroll the whole list; no size filter"],
  "quote": "I found it, but I had to look at everything."
}

score goes from 1 to 10. A 10 means the task was very easy.

Helper 2: the analyst

The analyst reads all the shopper reports and suggests a few fixes. The rules we give it:

Each fix must be small.
Only change index.html, app.js and styles.css.
No new libraries, no backend changes.
Each fix should be a different idea.

It also answers in JSON:

{
  "summary": "Shoppers scroll the whole list. There is no way to narrow it down.",
  "variants": [
    { "id": "B", "title": "Search box and category filters", "hypothesis": "...", "spec": "..." },
    { "id": "C", "title": "Sort by price", "hypothesis": "...", "spec": "..." }
  ]
}

Helper 3: the builder

The builder gets one fix (for example "Sort by price") and writes the code. It has strict rules too:

Only edit index.html, app.js and styles.css.
Plain HTML, CSS and JavaScript. No frameworks.
Do not restart the server.
When done, check the page still loads with curl.

These rules keep every version fair. Each version is the same shop plus one small change.

Step 4: Connect to Docker and store your key (`src/sbx.ts`)

Now we start talking to Docker Cloud Sandboxes. Everything in this file is a small helper we will call later.

Signing in

const auth = oauth({
  store: fileOAuthCredentialStore({ path: '.docker-sandboxes/oauth.json' }),
  onVerification: ({ verificationUriComplete, verificationUri, userCode }) => {
    console.log(`Sign in to Docker: ${verificationUriComplete ?? verificationUri}`);
    console.log(`Code: ${userCode}`);
  },
});
await auth.getAccessToken();
return new Sandboxes({ auth });

In plain words:

The first time, it prints a link and a code. You open the link and sign in.
It saves your sign-in to a local file, so next time it does not ask again.
new Sandboxes({ auth }) gives us a client. We use this client for everything else.

If you prefer no browser step, set DOCKER_USERNAME and DOCKER_PAT (a personal access token). The code then uses pat() instead.

Storing the OpenAI key as a secret

const secret = await client.secrets.create({
  displayName: 'sbx-ab-agent-openai',
  serviceType: 'openai',
  token: { value: key },
});

This saves your OpenAI key in Docker, not inside any sandbox. Later we attach the secret to each sandbox. When Codex calls OpenAI, the sandbox adds the key to the request on the way out. Codex never sees the key itself.

One small thing to remember: the SDK does not attach your OpenAI secret for you. You must pass it in when you create a sandbox. We will do that in the next step.

Step 5: Start a sandbox and put the shop inside (`launchShop`)

This one function does five small jobs. Let us take them one by one.

1. Create the sandbox

const sandbox = await client.kits.launchAndWait('codex', {
  displayName,
  resources: 'small',
  storage: { secrets: [secret] },
});

'codex' is a kit: a ready-made sandbox that already has Codex installed. launchAndWait creates it and waits until it is ready. Notice we pass our secret here.

2. Copy the shop files in

await sandbox.files.mkdir(APP_DIR, { mode: 0o755, parents: true });
for (const f of await readdir(localAppDir)) {
  await sandbox.files.write(`${APP_DIR}/${f}`, await readFile(join(localAppDir, f), 'utf8'));
}

Make a folder in the sandbox, then copy each file from your app/ folder into it.

3. Start the web server

await sandbox.processes.start({
  args: ['node', 'server.mjs'],
  workingDir: APP_DIR,
  env: { PORT: '3000' },
  label: 'shop-server',
});

processes.start runs a command in the background. The server keeps running while we do other things.

4. Wait until the server answers

The waitForHttp helper uses sandbox.run(...) to try the server once a second, up to 30 times. When it gets a good answer, we move on.

5. Get a public link

const port = await sandbox.ports.create({ number: 3000, protocol: 'tcp' });
return { sandbox, url: port.url };

This opens port 3000 to the internet and gives back a real HTTPS URL. You can open it in your browser.

That is it. One function, and we have a live shop running in the cloud.

Step 6: Ask Codex to do a job (`runAgent`)

Every AI helper (shopper, analyst, builder) runs through this one function. It runs Codex inside the sandbox with no chat window, just a prompt in and an answer out.

const script =
  `printf '%s' "$1" | codex exec -C ${APP_DIR} --skip-git-repo-check --ephemeral ` +
  `--dangerously-bypass-approvals-and-sandbox -o ${outFile} -`;

const r = await sandbox.processes.run(
  { args: ['bash', '-lc', script, 'bash', prompt], workingDir: APP_DIR },
  { timeoutMs },
);

This looks scary, so let us break it down:

codex exec runs Codex once, without asking us questions.
printf ... | ... - sends the prompt into Codex. The - at the end means "read the prompt from input". If you pass the prompt another way, Codex waits and then fails.
--dangerously-bypass-approvals-and-sandbox turns off Codex's own safety box. That is fine here, because we are already inside a throwaway sandbox.
-o ${outFile} saves only Codex's final answer to a file.
Each call uses a new random file name, so many shoppers can run at the same time without mixing up answers.

After it finishes, we read the answer back:

return await sandbox.files.read(outFile, { encoding: 'utf8' });

Pulling out the JSON

AI answers are not always clean. Sometimes the JSON comes wrapped in a code block, or with a sentence before it. The extractJson helper finds the first { and the last } and reads what is between them. Simple, and it works well.

Step 7: Put it all together (`src/ab.ts`)

Now the main program. It just calls the helpers above in order. Read main() from top to bottom and you will see these steps:

Step A: Start the original shop

const base = await launchShop(client, 'ab-A-baseline', secret, 'app');

Step B: Send the shoppers in

const baseSessions = await simulate(base.sandbox, 'A', PERSONAS);

simulate runs all three shoppers at the same time with Promise.all. Each one returns its JSON report.

Step C: Ask the analyst for fixes

const plan = extractJson(
  await runAgent(base.sandbox, analystPrompt(GOAL, baseSessions, N_VARIANTS)),
);

Step D: Build and test every fix, all at once

const settled = await Promise.allSettled(
  variants.map((v) => runVariant(client, secret, v)),
);

For each fix, runVariant does three things:

Starts a new sandbox with a fresh copy of the shop.
Asks the builder to add the fix.
Sends the same three shoppers in to test it.

We use Promise.allSettled instead of Promise.all. The difference: if one fix fails, the others still finish. One bad version does not ruin the whole run.

Step E: Pick the winner

const ranked = [...arms].sort((a, b) =>
  b.avgScore - a.avgScore || b.completion - a.completion || a.avgSteps - b.avgSteps);

In plain words: highest average score wins. If two are tied, the one where more shoppers finished wins. If still tied, fewer steps wins.

Step F: Bring the winning code home

const body = await winner.sandbox.files.read(`${APP_DIR}/${f}`, { encoding: 'utf8' });
await writeFile(`results/winner/${f}`, body);

The winning files are copied from the sandbox to results/winner/ on your laptop. The program also writes results/report.md and results/report.json with all the scores and shopper quotes.

Step 8: Run it

Set up your key:

cp .env.example .env
# open .env and add: OPENAI_API_KEY=sk-...
set -a; source .env; set +a

Then start the run:

npm start

The first time, you will see a Docker sign-in link. Open it, sign in, and the run continues.

A normal run (2 fixes, 3 shoppers) uses 3 small sandboxes and about 12 Codex calls. It takes around 5 to 10 minutes.

Some handy options:

npm start -- --variants 3 --personas 2
npm start -- --goal "Users can't tell which items are on sale"
npm start -- --cleanup          # delete the sandboxes at the end
npm start -- --no-dashboard     # skip the live web page

Step 9: Watch it live and read the results

While it runs, open http://localhost:4000. This is the live dashboard (src/dashboard.ts). You will see each step light up, and a box for A, B and C showing scores as they come in.

The terminal also prints a public link for each version. Open A, B and C in three tabs. You can watch B and C change while the builder is still working.

When it finishes, you get a table like this:

Arm	Variant	Ease	Done	Steps	Lift
A	Baseline	4.0	67%	14.0	-
B (winner)	Search and filters	8.3	100%	4.0	+108%
C	Sort by price	6.0	100%	8.0	+50%

(Your numbers will be different. Every run is a little different.)

"Lift" means how much better the score is compared to A.

To see exactly what the AI changed, compare the files:

diff -u app/app.js results/winner/app.js

That diff is basically your pull request.

A quick note on ports, since there are a few:

URL	Where	What
`http://localhost:4000`	your laptop	the live dashboard
`https://3000-....sbx.sandboxes-cloud.docker.com`	the cloud	each version of the shop
`http://localhost:3000`	your laptop	only when you run npm run shop

Step 10: Clean up

By default, the sandboxes stay running so you can click around each version. When you are done:

npm run cleanup

This deletes every sandbox whose name starts with ab-. It uses client.allSandboxes() to list them and delete({ force: true }) to remove each one.

Cheat sheet: every SDK call we used

What we wanted	SDK call
Sign in	`oauth()` or `pat()`
Save the OpenAI key	`client.secrets.create(...)`
Create a sandbox	`client.kits.launchAndWait('codex', ...)`
Copy files in and out	`sandbox.files.mkdir` / `files.write` / `files.read`
Start a server in the background	`sandbox.processes.start(...)`
Run a quick command	`sandbox.run([...])`
Run a command and wait for it	`sandbox.processes.run(...)`
Get a public URL	`sandbox.ports.create({ number: 3000 })`
Clean up	`client.allSandboxes()`, `sandbox.delete(...)`

That is the whole SDK surface for this project. About ten calls.

Common problems

"No OpenAI credential": you did not set OPENAI_API_KEY. Check your .env and run set -a; source .env; set +a again.
Codex exits with code 1: the error message is at the end of the output. Read the last few lines.
A version shows "failed": that is fine. The other versions still finish because of Promise.allSettled.
Too many sandboxes left over: run npm run cleanup.

Things to keep in mind

The shoppers read the page with curl, not a real browser. They can tell when search or filters are missing, but they cannot judge how the page looks.
Scores come from AI judgement on only a few shoppers. Treat them as a hint, not hard proof.
The Sandboxes SDK is still experimental. This project uses @docker/sandboxes 0.53.0, so some names may change later.

What you learned

You now know how to:

Sign in and create a client with the Docker Sandboxes SDK.
Store an API key safely as a secret.
Start a sandbox, copy files in, run a server and get a public URL.
Run an AI agent inside a sandbox and read its answer.
Run many sandboxes at the same time and compare the results.

The shop is just an example. The same pattern works for many things: testing prompts, tuning settings, trying different copy. Anywhere you can pretend to be a user and give a score, you can use this loop.

The big idea is simple. When a safe, throwaway computer is just one function call away, letting an agent "try it and see" stops being risky.

Give it a try, and let me know what you build with it!

AIGov Meetup Singapore: Agentic AI Governance

Ajeet Singh Raina — Fri, 09 Oct 2026 06:45:59 +0000

AI agents are no longer just chat assistants. A year ago, most of us used AI to answer questions or suggest a few lines of code. Today, agents write whole features, run shell commands, install packages, call APIs and push changes to real systems. Many of them do this with very little human input.

That is a big step forward. It is also a big shift in risk. An agent that can run rm -rf can also delete the wrong folder. An agent that can read your .env file can also send your secrets somewhere you never planned. An agent that can call a tool can also call the wrong one, many times, very fast.

So every team that uses AI agents now has to answer two hard questions:

How do we keep these agents safe?
How do we know the same rules apply to every agent, on every laptop, across the whole company?

This is what agentic AI governance is about. And this is what we will talk about at the AIGov Meetup Singapore on October 29th.

What the evening is about

This is an evening of talks, live demos and good conversation with practitioners from Docker, Thoughtworks and Cloudera. Each speaker looks at a different layer of the problem:

Sandboxing and policy: how to run coding agents in an isolated space, with clear rules on what they can reach.
Verification: how to check what an agent did, instead of just trusting it.
Data and platform governance: how to keep control of the data and systems that agents touch at scale.

Put together, these talks give you a full picture. You will leave with ideas you can try with your own team the next morning.

Who should come

This meetup is for you if you are:

A developer using Claude Code, Cursor, Codex or other coding agents every day
A platform engineer who has to support these tools across many teams
A security lead who wants to know what agents are doing on company machines
Or just curious about where agentic AI is heading

No deep background is needed. Bring your questions.

Event details


Date	Thursday, October 29, 2026
Time	6:00 PM to 9:00 PM SST
Venue	Thoughtworks, 18 Cross St, #12-01 18 Cross, Singapore 048423
Hosted by	Docker Singapore Meetup
Entry	Registration required

Agenda

Time (SST)	Session
6:00 PM to 7:00 PM	Doors open, food and networking
7:00 PM to 7:30 PM	Define Once, Enforce Everywhere: Governing AI Coding Agents with Docker Sandboxes
7:30 PM to 8:00 PM	Trust the Agent, But Verify: AI Governance with Agent/works
8:00 PM to 8:30 PM	Talk by Cloudera (details coming soon)
8:30 PM to 9:00 PM	Networking

Define Once, Enforce Everywhere: Governing AI Coding Agents with Docker Sandboxes

Ajeet Singh Raina, Developer Advocate, Docker

Most teams today have no single place to set rules for their coding agents. Each developer sets things up their own way, on their own laptop. That makes it very hard to know what any agent can actually do.

This talk shows a different approach. With Docker Sandboxes, each agent runs inside an isolated sandbox. Platform and security teams write the policy once: which folders the agent can see, which network hosts it can reach, and which secrets it can use. That same policy then applies to every agent, whether it is Claude Code, Codex, Gemini CLI or another tool.

Expect a live demo of an agent hitting a policy wall, and a look at how teams can roll this out without slowing developers down.

Trust the Agent, But Verify: AI Governance with Agent/works

Shen Ting Ang, Thoughtworks

Guardrails are only half the story. Once an agent has done its work, you still need to check what it did and why. This talk looks at how Thoughtworks approaches AI governance with Agent/works, and how teams can build trust in agents through verification rather than blind faith.

Talk by Cloudera

Speaker and topic coming soon

Agents are only as safe as the data they can reach. Cloudera will share their view on data and platform governance in the age of AI agents. We will update this post once the details are confirmed.

See you there

AI agents are moving fast, and the rules around them are still being written. The best way to get this right is to learn from each other: what worked, what broke, and what we would do differently.

Come for the talks, stay for the conversations. Food and drinks are on us, and there is plenty of time to meet the speakers and other practitioners.

Seats are limited, so please register early.

Register here

See you in Singapore on October 29th!

The $250 Agent Challenge: Build Something Real With Docker Cloud Sandboxes

Ajeet Singh Raina — Wed, 07 Oct 2026 01:21:22 +0000

What would you build if someone handed you $250 of cloud compute for AI agents?

That's not a hypothetical. Docker is giving every new Docker Agentic Platform signup a $250 Cloud Sandbox credit. That's enough to run 4 agents, 7 hours a day, for a whole month.

So we're turning it into a hackathon. 🐳

The $250 Agent Challenge is an open, online hackathon run by the Collabnix community. Claim the credit, build something real with it, and show us what an agent can do while you sleep. Winners get announced on stage at our Meetup in Bangalore on 21 November.

Why this is worth your weekend

AI agents are moving off our laptops. AWS, Microsoft and Docker now all run agents the same way:

🧱 every agent gets its own microVM
🔑 the agent never holds your real API keys
🔌 tools come in through a gateway (MCP)
🧯 outbound network is locked down by default
⏰ agents run on a timer while nobody is watching

Docker Cloud Sandboxes give you all five, with the same sbx CLI on your laptop and in the cloud. This hackathon is your chance to try that pattern hands-on, without an enterprise contract.

Pick a track

Track	Build an agent that...
🌙 Night Shift	does real work while your laptop is closed
🔐 No Keys Allowed	uses real APIs but never sees your tokens
🔌 Tool Master	connects MCP servers to do something useful
👥 The Crew	runs several agents in parallel, each in its own sandbox
🧰 Kit Maker	ships as a kit others can run with one command

Solo or teams of up to 3.

Special awards: 💰 Best Bang for the Buck (best build for the least credit spent) · ✍️ Best Build Log · 🙌 People's Choice

Get started in four commands

# 1. Install (Windows: winget install -h Docker.sbx)
brew install docker/tap/sbx

# 2. Sign in
sbx login

# 3. Store your model key for the cloud (the agent only ever sees a placeholder)
sbx --cloud secret set anthropic

# 4. Run an agent in the cloud
sbx --cloud run claude --name my-hack

That's the whole loop. Claude Code, Codex, Copilot and Gemini all work out of the box.

Tip: set an auto-stop timer on every sandbox so nothing runs up your credit while you sleep.

No credit card? You can still compete. The same sbx CLI runs locally, and local entries are fully welcome. 💰 Best Bang for the Buck is the only award judged on cloud spend.

What you submit

Four things, by the deadline:

A build log (a blog post or a repo README) that lets anyone repeat your build.
A 2-minute demo video. Record it and share the link; no live demo needed.
A screenshot of your spend, with private details blanked out.
The submission form.

Then the community votes on the recorded demos.

The timeline

When	What
Sat, Oct 10	Online kickoff: walkthrough and Q&A
Oct 10 to Oct 31	Build
Sat, Oct 31, 11:59 PM IST	Submissions close
Nov 1 to Nov 14	Judging and People's Choice voting on the recorded demos
Sat, Nov 21	🎉 Winners announced at our Meetup in Bangalore, with Docker swag and goodies

⏰ Claim the credit early. Docker's offer ends Oct 31, 2026, 11:59 PM PT. It's one credit per account, and a credit card is needed at signup.

Everything lives in the repo

Rules, FAQ, the getting-started guide and the build log template:

ajeetraina / 250-dollar-agent-challenge

💰 The $250 Agent Challenge — an open hackathon: claim $250 in Docker Cloud Sandbox credit and build an AI agent that works while you sleep.

💰 The $250 Agent Challenge

An open hackathon where you claim $250 in Docker Cloud Sandbox credit, build something real with it, and show the community what an AI agent can do while you sleep.

The deal

Docker is giving new Docker Agentic Platform signups a one-time $250 compute credit (10× the standard $25) for its Cloud Sandboxes: microVMs that each run their own kernel in the cloud, so your agents keep working after you close your laptop.

What $250 actually buys you (Docker's own example, priced at the Medium tier of 4 vCPU, 8 GB, $0.28/hr):

🧑‍🤝‍🧑 4 agents in parallel, 7 hours a day, for 30 days, or
⚡ ~3,500 fifteen-minute automated runs

⏰ The offer runs through October 31, 2026, 11:59 PM PT. A credit card is required at signup, and it's one credit per user / account.

👉 Claim it: …

View on GitHub

Ready?

⭐ Star and read the repo: github.com/ajeetraina/250-dollar-agent-challenge
💳 Claim your $250: docker.com/c/sbx-promo
📝 Submit when you're done: the submission form

Come build in the open with us. Show the community what a month of agents can really do, and we'll see the winners on stage in Bangalore on 21 November.

Drop your idea in the comments 👇 Teams often form right here.

Getting Started with Docker Cloud Sandboxes

Ajeet Singh Raina — Sat, 03 Oct 2026 03:43:16 +0000

AI coding agents now run for a long time. A single task can take hours: a refactor, a full test run, a migration. The question is no longer whether the model can finish the work, but where that work should run.

Your laptop is fine for the first few minutes, when you want to watch the agent and step in if it goes wrong. It's a poor place to leave a job running for hours, and a worse one for running many agents at once.

Docker Sandboxes answer both ends of that spectrum with the same isolation model and a single command to move between them. In this post we start an agent locally, run real code in it, and then - without changing how it runs - promote that exact sandbox into the cloud. Start on your laptop. Finish in the cloud.

Everything below was run end-to-end on sbx v0.45.1 (macOS, Oct 2026). The terminal output is copied verbatim from a real session - including the one gotcha that bites everybody the first time (keep reading to Part 2).

What you'll need

sbx CLI, v0.45.0 or later

A Docker Agentic Platform subscription for the cloud half (new accounts get free credit to start)

An agent credential - we'll use Anthropic for Claude Code

Where this fits: the Docker Agentic Platform

Cloud Sandboxes aren't a standalone gadget - they're the runtime layer of the Docker Agentic Platform, Docker's platform for configuring, connecting, running, and controlling AI agents in one place. The pieces that matter here:

Component	Role
Docker Sandboxes (`sbx`)	Isolated microVM environments where agents actually run - local on your laptop
Cloud Sandboxes	The same sandboxes, running on Docker-managed cloud infrastructure
MCP Catalog & Enterprise Gateway	Connect MCP tools (Jira, Linear, Grafana, …) once; expose them to agents, with governance
Docker Model Runner	Local-first LLM inference
Gordon	Docker's built-in AI agent
AI Governance	Policy and control over agents across teams

The throughline is that sandboxes are the execution substrate for the whole platform, and the same isolation model runs whether a sandbox lives on your laptop or in Docker's cloud. That's the bit to keep in mind as we go: moving to the cloud doesn't swap you onto a different, less-safe runtime - it's the identical sandbox, hosted.

What a sandbox actually is

A Docker Sandbox is an isolated microVM that an AI agent runs inside. It has its own Linux kernel, its own filesystem, its own Docker daemon, and all of its network traffic flows through a host-managed proxy. The agent gets sudo inside the box - but the box can't see your host, your processes, or your credentials.

Layer	What it isolates
Hypervisor	Separate Linux kernel per sandbox; the agent's processes are invisible to the host
Network proxy	All HTTP/HTTPS flows through a proxy; no raw TCP/UDP/ICMP; DNS is mediated
Docker Engine	An isolated Docker daemon per sandbox - no access to your host Docker
Credentials	The proxy injects auth headers per request; raw API keys never enter the VM

Cloud Sandboxes are the same four layers on Docker-managed remote infrastructure, with their own credentials, network policies, and lifecycle controls (TTL, hibernation, persistent volumes) - independent of your laptop.

Part 1 - Start local

Step 1: Install and check the version

brew install docker/tap/sbx     # macOS - Docker Desktop NOT required
# winget install -h Docker.sbx  # Windows

Confirm you're on 0.45.0+ (real output):

$ sbx version
sbx version: v0.45.1 9d79d90ee4c5d297fb3d36b75384e8cea7a4fbcb

📸 Screenshot 1 - terminal showing sbx version.

Step 2: Sign in

sbx login

This opens your browser for Docker OAuth. On first login you also pick a default network policy:

Policy	Behavior
Open	All outbound traffic allowed
Balanced (recommended)	Deny-by-default + common dev sites pre-allowed
Locked Down	Everything blocked unless you explicitly allow it

📸 Screenshot 2 - the Docker OAuth browser page and/or the policy-selection prompt.

Step 3: Store your agent credential

The proxy injects this into outbound requests - the raw value never lands inside the sandbox:

sbx secret set -g anthropic     # paste your Anthropic API key when prompted

-g makes it global (available to every sandbox). Confirm it's stored without ever revealing the value:

$ sbx secret ls
SCOPE        TYPE      NAME        SECRET
(global)     service   anthropic   (stored)

💡 Gotcha: global secrets are read at sandbox creation time. Change one, and you need to recreate the sandbox for it to take effect.

Step 4: Create a sandbox and run a real, long-ish job in it

The demo for this post is the kind of task you'd want to walk away from - a job that churns through a batch of work and takes a while. Here's worker.py: pass --full and it's the "overnight" run; otherwise it's a quick smoke test. It writes progress and a result file as it goes:

$ cat worker.py
#!/usr/bin/env python3
"""A deliberately long-running job: the kind of task you start on your laptop
and would rather not babysit."""
import sys, time
from datetime import datetime

FULL = "--full" in sys.argv
N = 20 if FULL else 3          # --full is the "overnight" run; default is a quick smoke test
STEP = 1.5 if FULL else 0.3

RESULT = "/home/agent/result.txt"
start = time.time()
with open(RESULT, "w") as f:
    for i in range(1, N + 1):
        time.sleep(STEP)
        line = f"[{datetime.now():%H:%M:%S}] processed item {i}/{N}"
        print(line, flush=True)
        f.write(line + "\n"); f.flush()
    print(f"DONE: {N} items in {time.time() - start:.1f}s", flush=True)
open("/home/agent/DONE", "w").close()   # sentinel so a watcher knows we finished

Create a sandbox with Claude Code, mounting that project (real output - note the microVM gets its own CPU/memory allocation and pulls the agent image):

$ sbx create claude ~/laptop-to-cloud-demo --name laptop-to-cloud-demo
── RESOLVE SETUP
   resolving configuration…
     sandbox    laptop-to-cloud-demo
     agent      claude
     workspace  /Users/you/laptop-to-cloud-demo (rw)
     skills     …/agent-skills → /home/agent/.claude/skills · 5 folders · (ro)
     image      docker/sandbox-templates:claude-code-docker
     cpu        18
     memory     18 GiB
   ✓ configuration resolved
── PREPARE IMAGE
   → pull docker/sandbox-templates:claude-code-docker
   ✓ image ready
── CREATE SANDBOX
   ✓ Created sandbox laptop-to-cloud-demo

To connect to this sandbox, run:
  sbx run --name laptop-to-cloud-demo

💡 run vs create: sbx run claude (no path) mounts your current directory and attaches you interactively in one step. sbx create runs in the background and does not default to the cwd - pass the workspace path explicitly, as above.

Confirm it's live (real output):

$ sbx ls
SANDBOX                AGENT     STATUS    PORTS   WORKSPACE
laptop-to-cloud-demo   claude    running           /Users/you/laptop-to-cloud-demo

Now develop on the laptop the way you normally would: run a quick smoke test of the job inside the microVM (real output):

$ sbx exec laptop-to-cloud-demo bash -lc 'python3 worker.py'
[02:33:08] processed item 1/3
[02:33:08] processed item 2/3
[02:33:09] processed item 3/3
DONE: 3 items in 0.9s

⚠️ Mind the path. sbx exec drops you into the workspace as the working directory, so a relative worker.py just works. But inside the sandbox your project is bind-mounted at its host absolute path (/Users/you/laptop-to-cloud-demo), not under the agent's home - ~ is /home/agent. So ~/laptop-to-cloud-demo/worker.py will fail with "No such file"; use worker.py (relative) or the full host path.

It works. The full --full run would take far longer - exactly the kind of thing you don't want to babysit on your laptop. Before we hand it off, stage the job into the sandbox's image filesystem so it survives the move (the next section explains why the mounted workspace alone won't):

$ sbx exec laptop-to-cloud-demo bash -lc 'cp worker.py /home/agent/worker.py && ls -l /home/agent/worker.py'
-rw-r--r-- 1 agent agent 862 Oct  3 02:33 /home/agent/worker.py

📸 Screenshot 3 - split view: Claude Code running in the sandbox on the left, sbx ls showing it running on the right. Your "local" money shot.

This is what the laptop is good at: watching, steering, a fast iteration loop. Now that the long run is ready, we move it up and let it finish without us.

Part 2 - Finish in the cloud

Step 0: Activate Docker Agentic Platform / cloud access

Follow the Signup and billing guide to activate your Docker Agentic Platform subscription and review compute charges. New accounts get free credit to start. Pay-as-you-go, billed per second, by sandbox size:

Size	vCPUs	Memory	Per hour
Micro	1	2 GiB	$0.07
Small (default)	2	4 GiB	$0.14
Medium	4	8 GiB	$0.28
Large	8	16 GiB	$0.56
XL	16	32 GiB	$1.12

📸 Screenshot 4 - the Docker Agentic Platform billing/usage dashboard.

The pivot: move the running sandbox to the cloud

The one-liner the whole post builds toward. Here is the real, unedited output of moving our sandbox up - read the warnings, they matter:

$ sbx move laptop-to-cloud-demo --to cloud --name laptop-to-cloud-demo --ttl 30m --force
warning: Secrets managed by sbx are not copied. The destination may use its own secrets; you may need to sign in again.
warning: Credentials saved in copied files can still travel with the sandbox.
warning: Local network policies are not copied. The destination uses cloud network policies.
warning: sandbox "laptop-to-cloud-demo" uses files outside its container filesystem. These files won't be copied to the cloud:
  "/Users/you/laptop-to-cloud-demo" (workspace, host bind mount)
→ move sandbox laptop-to-cloud-demo to cloud
Using cloud shape small (2 vCPU / 4096 MiB).
Pushing sandbox laptop-to-cloud-demo to cloud ...
Upload complete.
✓ move sandbox laptop-to-cloud-demo to cloud (41s)
✓ Moved to cloud sandbox laptop-to-cloud-demo (sbx_001m3zsrjd2pxfxt5fh5k9z0vrs)
  Attach with:  sbx --cloud run sbx_001m3zsrjd2pxfxt5fh5k9z0vrs
  Created cloud template tmpl_001m3zsqt39yxnwtwr8ksx3qxka to back it; remove it with
  `sbx --cloud template rm tmpl_001m3zsqt39yxnwtwr8ksx3qxka` once the sandbox is deleted.
  Expires in 29m; the sandbox will then be stopped (it can be started again later).

What just happened: sbx move captured the sandbox's filesystem as a container image and started a fresh cloud sandbox from it. --ttl 30m capped its life at 30 minutes; when the TTL lapses it stops in place (hibernates) so you can resume later, rather than being deleted. Neither side is destroyed - the move just stops the local source; restart it any time with sbx run --name laptop-to-cloud-demo.

📸 Screenshot 5 - the sbx move … --to cloud run with these warnings + the "Moved to cloud sandbox" line. This is the hero shot of the post.

⚠️ The gotcha everybody hits: the workspace does NOT travel

Look again at that warning:

warning: sandbox uses files outside its container filesystem. These files won't be copied to the cloud:
  "/Users/you/laptop-to-cloud-demo" (workspace, host bind mount)

Your mounted workspace is a host bind mount, and bind mounts don't travel. What travels is the sandbox's own image filesystem (e.g. anything under /home/agent) - which is exactly why we staged worker.py there in Step 4. Here's the proof, live, in the cloud sandbox after the move:

$ sbx --cloud exec laptop-to-cloud-demo bash -lc \
    'echo "staged job (image fs):"; ls -l /home/agent/worker.py; \
     echo "workspace (bind mount):"; ls -la /Users/you/laptop-to-cloud-demo'
staged job (image fs):
-rw-r--r--+ 1 agent agent 862 Oct  3 02:33 /home/agent/worker.py    # ← TRAVELED
workspace (bind mount):
total 8
drwxr-xr-x+  2 root  root   27 Oct  3 02:33 .                       # ← arrived EMPTY
drwxr-xr-x+  4 agent agent 110 Oct  3 02:33 ..

The staged job came across; the mounted workspace arrived empty. So the practical rule for "finish in the cloud":

Stage what the cloud run needs into the image filesystem (e.g. /home/agent) before you move - as we did with worker.py.
Or commit and push from inside the sandbox (its injected GitHub token works), which is the cleanest way to get work off any sandbox.
Or pull files back with sbx cp when you move the sandbox back down.

This is the single most important thing to understand before you trust an overnight run.

Confirm it's running in the cloud

The --cloud global flag dispatches a supported verb to the hosted service. Real output:

$ sbx --cloud ls
SANDBOX                       AGENT    ID                                STATUS    PORTS   WORKSPACE
claude/laptop-to-cloud-demo   claude   sbx_001m3zsrjd2pxfxt5fh5k9z0vrs   running           -

📸 Screenshot 6 - sbx ls (local, source now stopped) next to sbx --cloud ls (running in the cloud). Side by side, this proves the same workload moved.

The payoff: kick off the long run, then close the laptop

This is the whole point of the post, made visible. Launch the --full job in the background in the cloud (one caveat the move output hinted at: running processes don't travel - so you start the long job after you land in the cloud, not before):

$ sbx --cloud exec laptop-to-cloud-demo bash -lc \
    'nohup python3 /home/agent/worker.py --full > /home/agent/run.log 2>&1 & echo "pid $!"'
pid 1941

Peek once while the lid's still open - it's genuinely running, with nothing attached:

$ sbx --cloud exec laptop-to-cloud-demo bash -lc 'cat /home/agent/run.log'
[02:34:20] processed item 1/20
[02:34:21] processed item 2/20
[02:34:23] processed item 3/20

Now close your laptop. Come back later, reconnect, and the work finished on its own in the cloud:

$ sbx --cloud exec laptop-to-cloud-demo bash -lc 'tail -1 /home/agent/result.txt'
DONE: 20 items in 30.0s

Started on the laptop, finished in the cloud - no VM to provision, no lid to keep open.

📸 Screenshot 7 - the three steps stacked: launch (pid 1941), the mid-flight run.log, and the final DONE: 20 items in 30.0s. This is the hero shot.

Working with a cloud sandbox

Everything has a --cloud form:

sbx --cloud run sbx_001m3zsrjd2pxfxt5fh5k9z0vrs      # attach (starts it if stopped)
sbx --cloud exec laptop-to-cloud-demo bash -lc '...' # run a command inside it
sbx --cloud ttl laptop-to-cloud-demo                 # check / extend remaining lifetime
sbx cp laptop-to-cloud-demo:/home/agent/result.txt . # copy results back down
sbx move laptop-to-cloud-demo --to local             # bring the whole sandbox back

Clean up (stop billing)

Billing is per-second, so tear down when done. Real output from this session's cleanup:

$ sbx --cloud rm laptop-to-cloud-demo -f
   ✓ remove sandbox laptop-to-cloud-demo (1s)
$ sbx --cloud template rm tmpl_001m3zsqt39yxnwtwr8ksx3qxka --force
Removed: moved-laptop-to-cloud-demo-7513a8b6 (tmpl_001m3zsqt39yxnwtwr8ksx3qxka)

💡 A move leaves behind a backing cloud template (shown in the move output). Remove it after you delete the sandbox, or it lingers.

Or: start fresh in the cloud

You don't have to start local. For the "run 100 agents in parallel" case, launch straight into the cloud:

sbx --cloud run claude
sbx --cloud run codex

Cloud mode supports a growing set of verbs - run, create, attach, exec, cp, ls, move, stop, rm, ttl, plus cloud-only persistent volumes (sbx volume). Run sbx --cloud --help for the current list.

Part 3 - Go programmatic with the Sandboxes API & SDK

The CLI is the fast path, but everything above is also in the Docker Sandboxes API and SDK - which is where the "100 parallel agents" story gets real. The API lets you:

Create sandboxes and run commands inside them
Transfer files in and out
Manage images, snapshots, volumes, and secrets as first-class resources

That makes cloud sandboxes a natural fit for CI and fan-out jobs: spin up N sandboxes from a known image, dispatch a task to each, collect results, tear them down - no VM provisioning in sight. See the API docs for the current SDK surface.

Local vs Cloud, at a glance

	Local Sandbox	Cloud Sandbox
Runs on	Your laptop (microVM)	Docker-managed infra (microVM)
Isolation model	Hypervisor + proxy + isolated Docker + credential injection	Identical
Best for	Interactive iteration, steering, quick jobs	Long-horizon work, overnight runs, scale
Credentials	Host keychain, injected by proxy	Its own - sign in on the destination
Network policy	Local default (Open / Balanced / Locked Down)	Its own cloud policy
Workspace	Host bind mount (live)	Does not travel on move - use image fs / `sbx cp` / git
Lifecycle	Runs while your machine is on	TTL, hibernation, persistent volumes
Needs local virtualization?	Yes	No
Cost	Your hardware	Pay-as-you-go by size, per second

Wrapping up

It isn't "local sandboxes" or "cloud sandboxes." It's the same sandbox, part of the Docker Agentic Platform, and you choose where the hours run with one command. Iterate on your laptop where watching matters; sbx move … --to cloud the moment the work gets long or wide. Same isolation, same safety model, no VMs to provision - just mind the workspace bind-mount rule before you walk away.

brew install docker/tap/sbx
sbx login
sbx run claude                     # start here
sbx move <name> --to cloud --ttl 8h  # finish here

Learn more

Docker Agentic Platform: https://www.docker.com/products/docker-agentic-platform/
Cloud Sandboxes: https://docs.docker.com/ai/sandboxes/cloud/
Sandboxes overview: https://docs.docker.com/ai/sandboxes/
Sandboxes API: https://docs.docker.com/ai/sandboxes-api/
Announcement blog: https://www.docker.com/blog/introducing-cloud-sandboxes-start-on-your-laptop-finish-in-the-cloud/

Running DeepAgents in a Docker Sandbox, with no cloud keys

Ajeet Singh Raina — Fri, 02 Oct 2026 16:57:21 +0000

Agent frameworks are easy to pip install and surprisingly hard to run responsibly. The moment you give an agent a filesystem, a shell, and a network, you have handed arbitrary generated code the same reach your laptop has. You also inherit a second problem that has nothing to do with safety: reproducing the exact environment, the exact package versions, and the exact model wiring on someone else's machine.

This post walks through a small, self contained answer to both problems: a Docker Sandbox Kit that drops the deepagents harness into an isolated sandbox, pre wired to a local Docker Model Runner so it runs with no cloud credentials at all. The whole kit is four files, and it is published on Docker Hub so you can run it in one command.

What we are packaging

DeepAgents is an opinionated agent harness built on LangGraph. Out of the box it gives an agent a planning tool, a virtual filesystem, sub agent delegation, and a detailed system prompt. You construct an agent in a few lines:

from deepagents import create_deep_agent

agent = create_deep_agent(model=my_model, system_prompt="...")

Two things make it interesting to package. First, it is a library rather than a turnkey CLI, so the useful unit to ship is "deepagents, installed and wired to a model, ready to import." Second, it defaults to a cloud model (Anthropic), which means a naive setup needs an API key and open egress to a provider. We want neither.

What a Docker Sandbox Kit is

Docker Sandboxes (the sbx CLI) run agents inside isolated microVM style environments with a credential proxy and an enforced network policy. A kit is the unit of composition: a single OCI image whose manifest carries a descriptor describing what the kit offers, what it needs from the host as typed capability requests, and what it needs from other kits.

Kits come in two kinds:

A workload owns the environment the agent runs in. Its layers are a root filesystem, and it sets the entrypoint, user, and working directory.
A mixin adds a tool, a credential, or a policy on top of somebody else's workload. Its layers are an overlay that lands on a base it has never seen.

DeepAgents is a capability you add to an environment, not an environment of its own, so it is a natural mixin. It composes onto any workload that carries a Python runtime, for example the stock docker/sbx-kit-shell image.

The architecture

          your host
  ┌──────────────────────────────────────────────────────────┐
  │                                                            │
  │   Docker Model Runner  ──  :12434  (OpenAI-compatible)     │
  │            ▲                                               │
  │            │  egress allowed only to                       │
  │            │  host.docker.internal:12434                   │
  │   ┌────────┴──────────────  sandbox  ──────────────────┐   │
  │   │  shell workload  +  deepagents mixin               │   │
  │   │    • deepagents + langchain-openai (pip, create)   │   │
  │   │    • OPENAI_BASE_URL / OPENAI_API_KEY pre-wired     │   │
  │   │    • ~/deepagents_quickstart.py                     │   │
  │   └─────────────────────────────────────────────────────┘   │
  └──────────────────────────────────────────────────────────┘

The sandbox reaches exactly one host at runtime, the Model Runner on port 12434, and nothing else. The model speaks the OpenAI wire format, so deepagents talks to it through a standard ChatOpenAI client. No traffic leaves your machine.

Four files

The kit is authored as a companion pair plus its guidance and docs:

deepagents.yaml          the v3 descriptor
deepagents.dockerfile    a FROM scratch overlay that carries runtime ENV
deepagents-context.md    agent guidance, staged into the sandbox
README.md

The descriptor

The descriptor declares the kit's identity, its version, and the capabilities it requests. Here are the parts that carry the design.

Phase scoped network policy. Egress is granted per phase, and an absent phase grants nothing. The install phase may reach PyPI; the running agent may reach only the Model Runner. The PyPI grant closes before the agent ever starts.

capabilities:
  - type: com.docker.sandbox/network-policy@1
    config:
      install:
        allow: [pypi.org, files.pythonhosted.org]
      runtime:
        allow: [host.docker.internal:12434]

Create time install, not a baked layer. A mixin overlay lands on a base whose Python version and site-packages path you cannot know in advance, so copying a prebuilt package tree into the image would not resolve. Instead the kit installs into the composed base's Python at sandbox create time, via lifecycle hooks. The hooks also do two things worth calling out: they fail early with a clear message if the base Python is older than 3.11, and after installing they re read the installed version and fail on mismatch. A pinned version claim is only honest if the build enforces it.

  - type: com.docker.sandbox/lifecycle@1
    config:
      install:
        - command: "python3 -c \"import sys; sys.exit(0 if sys.version_info >= (3, 11) else 1)\" || { echo 'deepagents needs Python >= 3.11' >&2; exit 1; }"
          user: "1000"
        - command: "pip install --break-system-packages 'deepagents==0.7.21' langchain-openai"
          user: "1000"
          env: [HTTP_PROXY, HTTPS_PROXY]
        - command: "python3 -c \"import importlib.metadata as m, sys; sys.exit(0 if m.version('deepagents') == '0.7.21' else 1)\""
          user: "1000"

One subtlety: hook environments are deny by default, so HTTP_PROXY and HTTPS_PROXY are declared explicitly. pip reads them to fetch through the sandbox's forced proxy; without the declaration the install would hang.

No hard requires. It is tempting to declare requires: ["deb/python3"], but requires is a closed set check: a name nothing in the composition provides makes the kit refuse to compose everywhere, and a deb/ name rules out every Alpine or Wolfi base that would otherwise have worked. The Python 3.11 guard hook above is the better tool: it degrades gracefully with an actionable error rather than refusing up front.

No credential. Because the model is the local Model Runner, the API key is the sentinel string "dmr" rather than a real secret, so the kit declares no credential@1 capability and asks for nothing from the credential proxy.

The overlay

The content recipe is a FROM scratch overlay that installs nothing. Its only job is to carry static environment onto the composed image. A mixin's ENV is an additive image config field that merges at assembly, so it reaches both the composed image and the agent process.

FROM scratch

ENV OPENAI_BASE_URL="http://host.docker.internal:12434/engines/v1" \
    OPENAI_API_KEY="dmr" \
    DEEPAGENTS_MODEL="ai/qwen3" \
    LANGSMITH_TRACING="false"

LANGSMITH_TRACING=false keeps LangChain from attempting hosted tracing, which the runtime policy would block anyway, but switching it off avoids the noise. NO_PROXY is deliberately not set here, because the shell workload already defines it and two kits setting the same variable to different values is a hard composition conflict.

The quickstart

A lifecycle files entry stages a runnable example into the sandbox, marked so it is never overwritten if you have edited it. It passes an explicit ChatOpenAI instance to override the Anthropic default:

import os
from langchain_openai import ChatOpenAI
from deepagents import create_deep_agent

model = ChatOpenAI(
    model=os.environ.get("DEEPAGENTS_MODEL", "ai/qwen3"),
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ.get("OPENAI_API_KEY", "dmr"),
    temperature=0,
)

agent = create_deep_agent(model=model, system_prompt="You are a concise assistant. Plan before you act.")

if __name__ == "__main__":
    result = agent.invoke({"messages": [{"role": "user", "content": "In two sentences, what is a sandbox?"}]})
    print(result["messages"][-1].content)

Build, verify, run

A kit is just an OCI artifact built by docker buildx, with the descriptor validated before any content is built. A fast first check is to validate the descriptor without exporting anything:

docker buildx build . -f deepagents.yaml --output type=cacheonly

Then export an OCI layout and run the conformance suite. The kit passes all 18 checks on both linux/amd64 and linux/arm64, including the overlay ownership checks that catch a mixin accidentally taking over the agent's home directory:

docker buildx build . -f deepagents.yaml -t deepagents:0.7.21 \
  --output type=oci,dest=/tmp/deepagents-layout,tar=false
kit-tck validate --layout /tmp/deepagents-layout 0.7.21

A build proves the recipe ran, not that the overlay works, so the check that counts is a real composition. Point --kit at the kit directory and let sbx assemble it onto a shell workload:

sbx run docker/sbx-kit-shell:1.0.0 \
  --kit "$(pwd)" --name deepagents-demo /path/to/workspace

On create, sbx assembles the two kits, runs the three install hooks, and writes the quickstart file. Inside the sandbox the result is exactly what the descriptor promised:

$ sbx exec deepagents-demo python3 -c "import deepagents; print(deepagents.__version__)"
0.7.21
$ sbx exec deepagents-demo sh -lc 'echo $OPENAI_BASE_URL'
http://host.docker.internal:12434/engines/v1

Publishing and using it

Publishing a kit is just a build with --push. Both platforms go in one invocation so the index that consumers resolve through is written once:

docker buildx build . -f deepagents.yaml --platform linux/amd64,linux/arm64 --push \
  -t docker.io/ajeetraina777/sbx-kit-deepagents:0.7.21 \
  -t docker.io/ajeetraina777/sbx-kit-deepagents:latest

Once it is on Docker Hub, anyone can run it without cloning the repo. First make sure the Model Runner is serving a tool calling model:

docker model pull ai/qwen3

sbx run docker/sbx-kit-shell:1.0.0 \
  --kit docker.io/ajeetraina777/sbx-kit-deepagents:0.7.21 --name deepagents-demo .

Why this shape is worth copying

The pattern generalizes well beyond deepagents:

Package the capability, not the environment. A mixin composes onto whatever workload a team already uses, so you are not forcing a base image on anyone.
Wire to a local model when you can. Pointing an OpenAI compatible client at Docker Model Runner removes the API key, the egress, and the per token cost from the loop. The sandbox can then run with essentially no network.
Make the network policy phase scoped. Letting install reach PyPI while the running agent reaches only the model is a meaningful boundary, and it is declared in a few lines.
Pin, and make the build enforce the pin. A version claim that the content can quietly violate is worse than no claim.

The full kit is on GitHub at ajeetraina/docker-sbx-deepagents and on Docker Hub at ajeetraina777/sbx-kit-deepagents. It is small enough to read in one sitting and a good starting point for packaging your own agent tooling the same way.

Running Docker Sandboxes on Windows Platform

Ajeet Singh Raina — Tue, 30 Jun 2026 07:36:52 +0000

Prerequisite

Enable WSL2 or Hyper V

In this guide, we tested it with Hyper V enablement.

Enable-WindowsOptionalFeature -Online -FeatureName HypervisorPlatform -All
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux -All
Enable-WindowsOptionalFeature -Online -FeatureName VirtualMachinePlatform -All

Note: sbx's microVM backend rests on the Windows hypervisor + VirtualMachinePlatform regardless of whether you went the WSL2 or Hyper-V route. All three features you enabled with Enable-WindowsOptionalFeature modify boot-time kernel components, so none of them are fully live until a restart — the API returns success, but the capability isn't actually present in the running kernel yet.
Hence, reboot the system once before proceeding to the next step.

Step 1. Download the latest version of sbx

winget install -h Docker.sbx

If you want to use nightly build, download the release directly from this link: https://github.com/docker/sbx-releases/releases/tag/nightly

Download the .msi file below and double-click to install or
use the command line:

msiexec /i DockerSandboxes.msi /quiet

Step 2. Verify if you've the latest version of sbx

sbx version
sbx version: v0.34.0-rc1-216-g0be67a90 0be67a900e2a642c494353de466135d18d53f338

Step 3. Login to sbx

sbx login

The CLI prints a one-time device confirmation code and a URL.

Step 4. Choose your agent and provider

This lab works with any mainstream coding agent. Pick the provider whose API key you have - the rest of the lab will adapt.

Pick a provider for the rest of this lab. Each option below assumes you've chosen one of these:

- Use OpenAI + Codex
- Use Anthropic + Claude
- Use Google + Gemini

OpenAI configuration:

You'll run Codex inside the sandbox, authenticated to OpenAI.

First, export your API key in your host terminal** (don't paste it into this page):

If you're using Windows:

$env:OPENAI_API_KEY = "sk-proj-..."

Then store it as a global sbx secret. The command below reads $OPENAI_API_KEY from your shell - your key never leaves the terminal and is not displayed anywhere:

Anthropic configuration:

You'll run Claude Code inside the sandbox, authenticated to Anthropic.

First, export your API key in your host terminal (don't paste it into this page):

$env:ANTHROPIC_API_KEY = "sk-ant-..."

Then store it as a global sbx secret. The command below reads `$ANTHROPIC_API_KEY` from your shell - your key never leaves the terminal and is not displayed anywhere:

Gemini configuration

You'll run Gemini CLI inside the sandbox, authenticated to Google.

First, export your API key in your host terminal (don't paste it into this page):

$env:GEMINI_API_KEY = "AIza..."

Then store it as a global sbx secret.

The command below is just an example that shows how it reads keys from your shell - your key never leaves the terminal and is not displayed anywhere:

echo "$GOOGLE_API_KEY" | sbx secret set -g google
echo "$ANTHROPIC_API_KEY" | sbx secret set -g claude
echo "$OPENAI_API_KEY" | sbx secret set -g openai

Step 5. Clone the lab repository

The exercises use DevBoard - a full-stack FastAPI + Next.js issue tracker with intentional bugs. It's pre-configured for this lab.

git clone https://github.com/dockersamples/sbx-quickstart ~/sbx-lab
cd ~/sbx-lab

Step 6. Create your sandbox

cd ~/sbx-lab
sbx create --name=sbxlab <agent> .

First run: The agent image will pull (1–2 minutes) and the sandbox will be created with the Balanced network policy you selected at login.

sbx ls

You should see sbxlab in the list with status stopped:

PS C:\Users\ajeet> sbx ls
SANDBOX AGENT STATUS PORTS WORKSPACE
shell-sbx-kits-box-main shell stopped C:\Users\ajeet\Downloads\sbx-kits-box-main\sbx-kits-box-main
PS C:\Users\ajeet>

Once you run it (Step 6), the status changes to running.

Step 7. Run your sandbox

sbx run shell-sbx-kits-box-main

Select 1. Yes, continue to launch the agent.

Why this prompt exists: sbx mounts only your project directory into the microVM. The trust check ensures you're aware of what the agent can see and act on.

sbx ls
SANDBOX AGENT STATUS PORTS WORKSPACE
shell-sbx-kits-box-main shell running C:\Users\ajeet\Downloads\sbx-kits-box-main\sbx-kits-box-main
PS C:\Users\ajeet>

Set up the Network Policy

PS C:\Users\ajeet> sbx policy reset
Found 1 running sandbox(es):

shell-sbx-kits-box-main
The daemon will be stopped to apply the policy reset.
Running sandboxes will be terminated.

Are you sure you want to continue? (y/N): y
Stopping daemon at \.\pipe\docker_kaname_sandboxd...
✓ Daemon stopped successfully
✓ Policies reset.

Daemon started (PID: 28492, socket: \.\pipe\docker_kaname_sandboxd)
Logs: C:\Users\ajeet\AppData\Local\DockerSandboxes\sandboxes\state\sandboxd\daemon.log

Verify that you're in a Sandbox environment:

agent@shell-sbx-kits-box-main:sbx-kits-box-main$ cat /etc/os-release
PRETTY_NAME="Ubuntu 26.04 LTS"
NAME="Ubuntu"
VERSION_ID="26.04"
VERSION="26.04 (Resolute Raccoon)"
VERSION_CODENAME=resolute
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=resolute
LOGO=ubuntu-logo
agent@shell-sbx-kits-box-main:sbx-kits-box-main$ sbx policy reset
bash: sbx: command not found
agent@shell-sbx-kits-box-main:sbx-kits-box-main$

Claude Code Won't Start on Windows 11? Fixing the Bun Segfault

Ajeet Singh Raina — Tue, 30 Jun 2026 07:21:37 +0000

If you just installed Claude Code on Windows 11 and it crashes the moment you launch it, you're not doing anything wrong. This is a known issue with the runtime Claude Code now ships with, and the fix takes about two minutes.

Here's exactly what I hit, why it happens, and the step-by-step fix that got me running.

The symptom

You install Claude Code, type claude, and instead of the welcome screen you get a wall of text like this:

Bun v1.4.0 (...) Windows x64 (baseline) Windows v10.26100
...
panic(main thread): Segmentation fault at address 0x113
oh no: Bun has crashed. This indicates a bug in Bun, not your code.

Annoyingly, claude --version and claude doctor often work fine — it's only the interactive session that dies. That makes it look like the install succeeded when it didn't.

Why it happens

Claude Code used to run on Node.js. As of v2.1.113 (April 2026), the default build switched to a Bun-compiled native binary. Bun has a segfault bug that triggers on certain Windows builds during interactive startup, so the new binary crashes on launch.

The catch: even installing via npm doesn't save you anymore. The npm package now pulls in that same Bun binary as a platform dependency, and there's no Node.js fallback left in recent versions. So npm i -g @anthropic-ai/claude-code installs the broken binary too.

The fix is to drop back to the last version that still ran on Node.js (2.1.112) and stop the auto-updater from dragging you forward again.

The fix (step by step)

1. Check your Node version

You need Node 18+ for the JS build, but 18 has its own crash, use Node 20 or newer.

node -v

If you're below 20, upgrade before continuing.

2. Pin to the last pre-Bun release

npm i -g @anthropic-ai/claude-code@2.1.112

You should see npm remove a package during this, that's the win32-x64 Bun binary being dropped and replaced with the Node-run build. That's the whole point. (If 2.1.112 misbehaves, @2.1.110 is another confirmed-good version.)

3. Disable the auto-updater

This step is the one people skip — and then their install silently "upgrades" back to the broken binary overnight. There's no settings file toggle to fully turn updates off; the supported way is an environment variable:

[Environment]::SetEnvironmentVariable("DISABLE_AUTOUPDATER","1","User")

(DISABLE_AUTOUPDATER stops background checks. If you want to block every update path including manual ones, use DISABLE_UPDATES instead.)

4. Open a fresh terminal and verify

The env var only reaches new windows, so close the current terminal and open a new one.

claude doctor

You want to see 2.1.112, install method npm-global, and the point of the whole exercis, no segfault. Then start it:

claude

First launch hands you the auth screen (browser login for Pro/Max, or an API key for Console billing) and a trust prompt for your folder. You're in.

Two things to know afterwards

Ignore the "switch to native installer" banner. Claude Code will nudge you to run claude install. Don't ! that's exactly the path back to the Bun binary and the crash. Leave it alone.

You're now frozen at 2.1.112. That means you'll miss newer features that shipped later. For real work it's perfectly functional, but if you want the latest, the right move isn't to un-pin on Windows — it's WSL2. The Bun crash is Windows-specific; the Linux build runs fine. Install Node inside your distro (make sure which node returns a /usr/... path, not /mnt/c/...), keep your projects on the Linux filesystem, and install Claude Code there to get the current version with no crash.

TL;DR

npm i -g @anthropic-ai/claude-code@2.1.112
[Environment]::SetEnvironmentVariable("DISABLE_AUTOUPDATER","1","User")
# open a new terminal
claude

Pin to the last Node.js build, freeze updates, done.

References

Claude Code - Troubleshoot installation: https://code.claude.com/docs/en/troubleshoot-install
Claude Code - Advanced setup (auto-updates, version pinning): https://code.claude.com/docs/en/setup
Tracking issue (Bun segfault on Windows): https://github.com/anthropics/claude-code/issues/25630

You just can’t miss this…

Ajeet Singh Raina — Tue, 26 May 2026 05:43:22 +0000

Harsh Manvar

May 26

Docker with AI: A Practical Guide to Running LLMs, Agents and MCP

#ai #docker #machinelearning #llm

7 min read

Announcing the NVIDIA Nemotron 3 Super Build Contest

Ajeet Singh Raina — Fri, 01 May 2026 08:45:20 +0000

Have you ever wanted to use AI to make a real, tangible difference in your community? The NVIDIA Nemotron 3 Super Build Contest, organized by Docker Bangalore, Collabnix, and NVIDIA, is your perfect opportunity to do just that. With a deadline of May 25, 2026, developers are invited to build impactful solutions and compete for exciting prizes

Click here for details

The Challenge: 4 Tracks for Public Good

This contest isn't about building just another chatbot; every project must solve a real public problem. Participants must choose one of the following four tracks

Track 1: Community Benefits Navigator - Many people struggle to access the public services they qualify for. This track challenges you to build an assistant that digests complex policy PDFs, eligibility rules, and checklists to help users apply for services like scholarships, legal aid, or health schemes in plain language
Track 2: Open-Source Maintainer Copilot for Small Community Projects - Help prevent burnout among maintainers of public-interest repositories. Build a GitHub helper tailored for nonprofits and civic-tech teams that can automatically triage issues, draft release notes, and assist with newcomer onboarding
Track 3: Local Governance / Public Meeting Explainer - Local decisions are often opaque to the public. Use the model's multi-document reasoning skills to translate city council minutes, budgets, and policy updates into plain-language explanations that tell residents exactly what changed and what actions they can take
Track 4: Crisis Information and Resource Routing Assistant - During chaotic events like floods or disease outbreaks, people need fast, localized guidance. Build a community response assistant that synthesizes rapidly changing context—like shelter lists, school closures, and verified guidance—into one easy-to-use conversational workflow

Prizes and Judging

Projects will be rigorously evaluated across six core dimensions: community value, grounded accuracy, actionability, safety, equity/accessibility, and operational efficiency. A panel of expert judges will also look at your innovation, code quality, and presentation.

If you build one of the top projects, you could win big:

Top 3 Winners: Amazon vouchers + NVIDIA swag
2 Runners-up: NVIDIA swag

How to Submit Your Project

Ready to get started? Make sure your submission meets all the official requirements before the May 25, 2026 deadline. To successfully submit, you must:

Host your code on a public GitHub repository and include an architecture diagram in your README
Record a YouTube demo video showing your project in action
Write a blog post (just like this one!) about your project and share it on social media
Use the right tags: Your social posts must include #NVIDIA and #Nemotron, and you must explicitly call out the Bengaluru meetup
Submit officially by opening a Submission issue on the GitHub repository

Head over to the collabnix/nvidia-nemotron-contest GitHub repository to open your submission issue and get your project on the public wall. If you need help along the way, you can reach out via the Collabnix Slack or open a question issue.

Happy building!

Announcing Operational AI with Docker Book by Ajeet Singh Raina & Harsh Manvar

Ajeet Singh Raina — Wed, 29 Apr 2026 07:29:29 +0000

Get the Book

📘 Packt: Operational AI with Docker
📦 Amazon: Available now (paperback + Kindle)
🔖 ISBN: 9781807301095

For the better part of two years, I had a problem I couldn't solve in 280 characters or a blog post.

Every time I ran a Collabnix Meetup and we were running them often, with three or four hundred DevOps engineers and developers showing up each time, mostly new faces - someone would walk up to me afterwards and ask the same question:

"Ajeet, where do I start with Docker and AI?"

And honestly? My answer used to be a bit lame.

"Follow my blogs."

I was writing one almost every day. The landscape was shifting that fast. We were moving from what I think of as GenAI 1.0 to 2.0 - from chatbots to automated AI workflows. MCP had just emerged. Docker Model Runner was new. Sandboxes hadn't even shipped yet. There was massive confusion about where to even begin.

Then came December 2024. MCP was introduced, and within a few weeks the ecosystem had over 3,000 MCP servers. That's when it really hit me: blog posts aren't enough anymore. People need a single, structured place to learn this end to end.

So Harsh Manvar and I decided to write the book.

Today, when someone walks up to me at a meetup and asks where to start with Docker and AI, I can finally say with confidence: go grab this book. You can learn it from scratch.

It's called Operational AI with Docker, it's published by Packt, and as of today — it's live.

Order Now

Why "Operational AI"?

I keep going back to 2013 to explain what we're trying to do with this book.

When Docker was introduced, the whole idea of build, ship, and run software became wildly popular. Suddenly, moving from monoliths to microservices wasn't a giant migration project - it was something a developer could actually do on a Tuesday afternoon. The way I describe it: it was like ordering an iPhone from Amazon. You unbox it, switch it on, and it just works. No "CPU unsupported" errors. No dependency hell. No "works on my machine."

Docker solved that exact problem for software workloads.

AI today has the exact same packaging problem, just with heavier baggage. Models, weights, tokenizers, GPU drivers, fragile Python environments, agents that need tools, tools that need secrets, secrets that need policies.

So Docker is doing for AI what it did for software a decade ago. The same three-word story, extended:

Build AI agents using the MCP Toolkit, the MCP Catalog, and Docker Model Runner.
Ship them with Agentic Compose and Docker Agents.
Run them inside Docker Sandboxes.

The philosophy hasn't changed. Build, ship, run, extended to AI workloads. That's the entire thesis of this book.

Docker isn't trying to be an AI framework. It's the runtime and packaging layer underneath, so the AI parts can actually be portable, reproducible, and shippable. That's where Docker fits. That's what this book teaches you to do.

The Real Problems This Book Solves

When teams come to me asking about Docker and AI, they usually think their problem is technical. Most of the time, it isn't. Here are the four challenges I see over and over, and the book tackles each of them head-on:

1. Choosing the right model

Hugging Face today has more than two million AI models on it. Two million. So when a team starts an AI project, the first thing they hit isn't a Docker problem, it's a decision problem.

In Chapter 6, I break this down with a diagram, splitting models into three buckets:

Small Language Models (SLMs) — roughly 0 to 7 billion parameters
Medium Language Models — the middle range
Large Language Models — the 70-billion-plus class

Most teams don't need an LLM. They need an SLM that runs cheaply, fast, and close to the user. But figuring that out takes effort, and very few people do it before they start coding.

2. The "AI = cloud-only" perception trap

Most developers, when they hear "AI model," immediately think OpenAI, Gemini, Claude - something that lives in the cloud, behind an API key, costing money per token.

That assumption shapes the entire architecture before anyone has written a line of code. The book breaks that assumption open and shows you what local-first AI actually looks like in practice.

3. GPU fear

The moment you say "run a model locally," people panic about hardware. "Do I need a beefy GPU? Will my MacBook even handle this?"

There are great open-source tools out there — llmfit literally scans your hardware and tells you which models will run well on your machine, and it supports Docker Model Runner as a runtime. But most developers don't know these tools exist, so they default to the cloud out of fear, not necessity. The book walks through this so you can make the call with data, not anxiety.

4. "Should I even run my model inside a container?"

This is the question I get more than any other. And there's a real gap in understanding here.

With Docker Model Runner, you don't actually have to wrap the model in a container at all. DMR runs the model natively on the host, uses the GPU directly, and exposes it through an OpenAI-compatible endpoint — but it gives you Docker's packaging, versioning, and pull experience on top of that. You get the best of both worlds.

That nuance is missed by a lot of teams, and it's exactly the kind of thing the book unpacks early on.

What You'll Learn

The book is structured around the real lifecycle of an AI workload — from running a model on your laptop, to building agents, to securing and scaling them in production:

✅ Run LLMs locally with Docker Model Runner — pull a model the same way you pull a container image, with OpenAI- and Anthropic-compatible APIs.
✅ Build and secure AI agents with Docker MCP Gateway — the piece I think is most underrated in the current AI ecosystem. Dynamic tool discovery, policy enforcement, secrets isolation, audit logs.
✅ Orchestrate multi-agent workflows declaratively — defining agents, sub-agents, and tools in YAML using Docker Agent. Versioned, reproducible, reviewable.
✅ Isolate agent execution with Docker Sandboxes — running untrusted, agent-generated code inside microVMs so a hallucinated rm -rf never reaches your host.
✅ Build, run, and share multi-agent systems using Docker Agents — orchestrator-worker patterns, agent-to-agent communication, shared state, and when not to reach for multi-agent.
✅ Deploy and scale GenAI services on Kubernetes — taking everything from your laptop to a cluster, with the observability and cost-routing patterns production demands.

Built for developers. Grounded in real tools. No fluff.

Who Should Read It

Back in 2023, Harsh and I co-wrote a Docker blog called *"LLM Everywhere: Docker for Local and Hugging Face Hosting."* It went deep into model quantization formats — GPTQ, GGML — and how to actually run these models locally in containers. That blog took off. From an SEO standpoint, it became one of Docker's biggest hits in the AI space.

But what really shaped the audience for this book was the questions that came pouring in afterwards. People wanted to understand model training, model quantization, the trade-offs between formats. And one question came up over and over: "Should I even run a model inside a container?"

That tells you exactly where people are. They're not AI researchers. They're not pure ML engineers. They're developers, DevOps folks, and platform teams who suddenly need to make AI work in their existing world.

So that's who we wrote for:

If you're a developer who built an agent demo and is now being asked to "productionize it" — this book is for you.
If you're a platform engineer whose team is shipping LLM-powered services and you're trying to figure out the runtime story — this book is for you.
If you're an architect mapping out an agentic AI strategy and need a concrete reference for what the operational layer actually looks like — this book is for you.

We didn't assume a deep ML background. If you know containers and you're curious about AI, you're in the right place. And if you know AI but containers feel like a black box, the early chapters meet you there.

Thank You

To Harsh Manvar — Docker Captain, Google Developer Expert, CNCF Ambassador, and the steadiest co-author I could have asked for. Our collaboration goes back years of Docker community blogs, and this book is the natural culmination of that.

To Apramit Bhattacharya, Preet Ahuja and the Packt editorial team — for pushing back on chapters I thought were finished, and for making the book sharper at every pass.

To the Collabnix community — 17,000+ of you across Slack and Discord — who have been the sounding board for every idea in this book, often without realizing it. Every meetup question, every late-night DM, every "Ajeet, but how does this actually work in production?" — those shaped this book more than anything else.

And to my family, who put up with a year of evenings and weekends.

If you pick it up, I'd genuinely love to hear what you think. Tag me on LinkedIn or X (@ajeetsraina), drop into the Collabnix Slack, or just reach out directly. The companion code is open source, and we'll be maintaining it as the Docker AI stack continues to evolve.

Operational AI is not a finished problem. This book is a snapshot of where the practice stands in 2026 — and a foundation you can build on as it keeps changing.

Let's keep figuring it out, together.

— Ajeet

How Autonomous AI Agents Become Secure by Design With Docker Sandboxes

Ajeet Singh Raina — Wed, 15 Apr 2026 03:52:42 +0000

I've been running AI coding agents for a while now. Claude Code on my MacBook, pointed at a project directory, autonomously editing files, running tests, pushing commits. It's genuinely useful — the kind of useful that makes you wonder how you shipped code without it.

But a few months ago I started asking myself a question I'd been quietly avoiding: what exactly can this agent reach while it's running?

The answer, once I actually looked, was uncomfortable. Everything. It could reach everything I could reach ~ my SSH keys, my AWS credentials, my .env files, my Git tokens. Not because it was malicious. Just because it was running on my laptop, as me, with my permissions.

The risk isn't that your agent is malicious. It's that agents are increasingly reading external content — READMEs, web pages, GitHub issues, pull request descriptions. Any of that content could contain a prompt injection that redirects the agent's behavior. You don't need a sophisticated attack. You just need an agent that's trying to do its job.

That's when Docker Sandboxes (sbx) started making a lot more sense to me. In the full post I walk through how a single architectural change collapses the blast radius of an AI agent — without slowing it down.

👉 Continue reading on ajeetraina.com

Interested to learn more about AI Coding Agent and Docker Sandboxing ? Don't miss out my upcoming session this Saturday 18th April at "Docker for AI" Show-n-Tell event at FAI Office, Indiranagar, Bengaluru.

Running NVIDIA Nemotron on a Mac with Docker Model Runner: What You Need to Know

Ajeet Singh Raina — Sun, 08 Mar 2026 08:50:51 +0000

Docker Model Runner just got a major upgrade for Mac users. With the introduction of vllm-metal - a new backend that brings vLLM inference to macOS via Apple Silicon's Metal GPU - you can now run MLX models through the same OpenAI-compatible API, the same Claude Code-compatible API, and the same Docker workflow you already know.

I put this to the test by running NVIDIA Nemotron models on my Mac. Here's what the experience looks like, what works today, and what's coming very soon.

What is vllm-metal?

vllm-metal is a plugin for vLLM that brings high-performance LLM inference to Apple Silicon. Developed by Docker engineers and now contributed to the open-source vLLM community, it unifies MLX (Apple's machine learning framework) and PyTorch under a single compute pathway — plugging directly into vLLM's existing engine, scheduler, and OpenAI-compatible API server.

The architecture is elegant:

+-------------------------------------------------------------+
|                          vLLM Core                          |
|        Engine | Scheduler | API | Tokenizers                |
+-------------------------------------------------------------+
                             |
+-------------------------------------------------------------+
|                   vllm_metal Plugin Layer                   |
|   | Platform  |  | Worker    |  | ModelRunner            |  |
+-------------------------------------------------------------+
                             |
+-------------------------------------------------------------+
|                   Unified Compute Backend                   |
|   | MLX (Primary inference) | PyTorch (model loading)  |   |
+-------------------------------------------------------------+
                             |
+-------------------------------------------------------------+
|              Metal GPU / Apple Silicon Unified Memory       |
+-------------------------------------------------------------+

Source ~ https://www.docker.com/blog/docker-model-runner-vllm-metal-macos/

What makes this particularly powerful on Apple Silicon is unified memory. Unlike discrete GPUs where data must be copied between CPU and GPU memory, Apple Silicon shares a single memory pool. vllm-metal exploits this with zero-copy tensor operations — combined with paged attention for KV cache management and Grouped-Query Attention support.

Getting Started

Update to Docker Desktop 4.62 or later for Mac, then install the backend:

docker model install-runner --backend vllm-metal

That's it. Docker Model Runner automatically routes MLX models to vllm-metal when the backend is installed.

vLLM Now Runs Everywhere with Docker Model Runner

This release completes vLLM support across all three major platforms:

Platform	Backend	GPU
Linux	vllm	NVIDIA (CUDA)
Windows (WSL2)	vllm	NVIDIA (CUDA)
macOS	vllm-metal	Apple Silicon (Metal)

The same docker model commands work regardless of platform. Docker Model Runner picks the right backend automatically.

Which Models Work with vllm-metal?

vllm-metal works with safetensors models in MLX format. The mlx-community on Hugging Face maintains a large collection of quantized models optimized for Apple Silicon. Some great starting points:

# Lightweight and fast
docker model run hf.co/mlx-community/Llama-3.2-1B-Instruct-4bit

# Strong 7B
docker model run hf.co/mlx-community/Mistral-7B-Instruct-v0.3-4bit

# Latest coding model
docker model run hf.co/mlx-community/Qwen3-Coder-Next-4bit

Running NVIDIA Nemotron on Mac

With vllm-metal installed, let's look at what the Nemotron lineup offers on Mac today.

Llama-3.1-Nemotron-Nano-8B (Recommended)

This is a standard transformer model fine-tuned by NVIDIA for instruction following and reasoning. It runs well on Mac via vllm-metal:

docker model run hf.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1

On a 32GB or 64GB Mac, this model runs comfortably. On a 16GB Mac, free up unified memory first by reducing Docker Desktop's VM memory limit (Settings → Resources → Advanced → set to 4–5 GB).

Nemotron-3-Nano-30B: The Mamba2 Frontier

The 30B model uses a hybrid SSM (State Space Model) architecture — combining Mamba2 layers with attention layers. This is architecturally novel and represents the next frontier of efficient inference: better long-context performance, lower memory footprint at runtime, and compelling throughput characteristics.

Mamba2 support in mlx-lm is actively maturing, and as it does, models like Nemotron-3-Nano-30B will run natively through vllm-metal. This is an exciting space to watch — especially on higher-memory Macs where the 30B model size becomes practical.

The $599 AI Development Rig

One of the most compelling stories in this release: a base Mac Mini M4 at $599 is now a viable vLLM development environment. Because Apple Silicon uses unified memory, the 16GB (or upgraded 32GB/64GB) RAM is directly accessible by the GPU, enabling you to:

Develop and test locally using the same OpenAI-compatible API as production
Mirror production — the same API surface as an H100 cluster, on your desk
Run efficiently — a fraction of the power consumption and heat of a discrete GPU rig

This democratizes access to vLLM development in a way that wasn't possible before. Previously, getting started with high-throughput vLLM required an RTX 4090 ($1,700+) or enterprise-grade GPU cards. Now, the barrier to entry is a Mac Mini.

vllm-metal vs llama.cpp: Benchmark Context

Docker benchmarked both backends on Llama 3.2 1B Instruct with 4-bit quantization:

max_tokens	llama.cpp (tok/s)	vLLM-Metal (tok/s)
128	333.3	251.5
512	345.1	279.0
1024	338.5	275.4
2048	339.1	279.5

llama.cpp shows ~1.2–1.3x higher raw throughput on this benchmark. However, vllm-metal brings things that raw token speed doesn't capture: the full vLLM engine, including its scheduler, paged attention, batching, and production-grade OpenAI-compatible API. For developers building real applications, that consistency and API compatibility often matters more than peak throughput on a single request.

Hardware Recommendations

Mac Configuration	What to Run
M-series, 16GB	`mlx-community/Llama-3.2-1B-Instruct-4bit`, `ai/phi4-mini`
M-series, 32GB	Nemotron 8B, `mlx-community/Mistral-7B-Instruct-v0.3-4bit`
M-series Max/Ultra, 64GB+	Nemotron 8B comfortably, 30B models as SSM support matures

I'm upgrading to a MacBook Pro Max this month — full benchmarks and multi-model Nemotron demos coming soon.

Open Source at the Core

Docker contributed vllm-metal to the vLLM open-source community — it now lives under the vLLM GitHub organization. This means every developer in the ecosystem can benefit from and contribute to high-performance inference on Apple Silicon. The project has also had significant contributions from Lik Xun Yuan, Ricky Chen, and Ranran Haoran Zhang.

Quick Start

# Install vllm-metal backend (Docker Desktop 4.62+ required)
docker model install-runner --backend vllm-metal

# Run a Nemotron model
docker model run hf.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1

# Or try an mlx-community model
docker model run hf.co/mlx-community/Mistral-7B-Instruct-v0.3-4bit

# List all downloaded models
docker model ls

Learn More

Want to learn more? Come, join me at Collabnix Community Meet and NVIDIA GTC 2026 Watch Party this March 21st virtually. Don't forget to register here