DEV Community: Ajeet Singh Raina

Getting Started with Docker Cloud Sandboxes

Ajeet Singh Raina — Sat, 03 Oct 2026 03:43:16 +0000

AI coding agents now run for a long time. A single task can take hours: a refactor, a full test run, a migration. The question is no longer whether the model can finish the work, but where that work should run.

Your laptop is fine for the first few minutes, when you want to watch the agent and step in if it goes wrong. It's a poor place to leave a job running for hours, and a worse one for running many agents at once.

Docker Sandboxes answer both ends of that spectrum with the same isolation model and a single command to move between them. In this post we start an agent locally, run real code in it, and then - without changing how it runs - promote that exact sandbox into the cloud. Start on your laptop. Finish in the cloud.

Everything below was run end-to-end on sbx v0.45.1 (macOS, Oct 2026). The terminal output is copied verbatim from a real session - including the one gotcha that bites everybody the first time (keep reading to Part 2).

What you'll need

sbx CLI, v0.45.0 or later

A Docker Agentic Platform subscription for the cloud half (new accounts get free credit to start)

An agent credential - we'll use Anthropic for Claude Code

Where this fits: the Docker Agentic Platform

Cloud Sandboxes aren't a standalone gadget - they're the runtime layer of the Docker Agentic Platform, Docker's platform for configuring, connecting, running, and controlling AI agents in one place. The pieces that matter here:

Component	Role
Docker Sandboxes (`sbx`)	Isolated microVM environments where agents actually run - local on your laptop
Cloud Sandboxes	The same sandboxes, running on Docker-managed cloud infrastructure
MCP Catalog & Enterprise Gateway	Connect MCP tools (Jira, Linear, Grafana, …) once; expose them to agents, with governance
Docker Model Runner	Local-first LLM inference
Gordon	Docker's built-in AI agent
AI Governance	Policy and control over agents across teams

The throughline is that sandboxes are the execution substrate for the whole platform, and the same isolation model runs whether a sandbox lives on your laptop or in Docker's cloud. That's the bit to keep in mind as we go: moving to the cloud doesn't swap you onto a different, less-safe runtime - it's the identical sandbox, hosted.

What a sandbox actually is

A Docker Sandbox is an isolated microVM that an AI agent runs inside. It has its own Linux kernel, its own filesystem, its own Docker daemon, and all of its network traffic flows through a host-managed proxy. The agent gets sudo inside the box - but the box can't see your host, your processes, or your credentials.

Layer	What it isolates
Hypervisor	Separate Linux kernel per sandbox; the agent's processes are invisible to the host
Network proxy	All HTTP/HTTPS flows through a proxy; no raw TCP/UDP/ICMP; DNS is mediated
Docker Engine	An isolated Docker daemon per sandbox - no access to your host Docker
Credentials	The proxy injects auth headers per request; raw API keys never enter the VM

Cloud Sandboxes are the same four layers on Docker-managed remote infrastructure, with their own credentials, network policies, and lifecycle controls (TTL, hibernation, persistent volumes) - independent of your laptop.

Part 1 - Start local

Step 1: Install and check the version

brew install docker/tap/sbx     # macOS - Docker Desktop NOT required
# winget install -h Docker.sbx  # Windows

Confirm you're on 0.45.0+ (real output):

$ sbx version
sbx version: v0.45.1 9d79d90ee4c5d297fb3d36b75384e8cea7a4fbcb

📸 Screenshot 1 - terminal showing sbx version.

Step 2: Sign in

sbx login

This opens your browser for Docker OAuth. On first login you also pick a default network policy:

Policy	Behavior
Open	All outbound traffic allowed
Balanced (recommended)	Deny-by-default + common dev sites pre-allowed
Locked Down	Everything blocked unless you explicitly allow it

📸 Screenshot 2 - the Docker OAuth browser page and/or the policy-selection prompt.

Step 3: Store your agent credential

The proxy injects this into outbound requests - the raw value never lands inside the sandbox:

sbx secret set -g anthropic     # paste your Anthropic API key when prompted

-g makes it global (available to every sandbox). Confirm it's stored without ever revealing the value:

$ sbx secret ls
SCOPE        TYPE      NAME        SECRET
(global)     service   anthropic   (stored)

💡 Gotcha: global secrets are read at sandbox creation time. Change one, and you need to recreate the sandbox for it to take effect.

Step 4: Create a sandbox and run a real, long-ish job in it

The demo for this post is the kind of task you'd want to walk away from - a job that churns through a batch of work and takes a while. Here's worker.py: pass --full and it's the "overnight" run; otherwise it's a quick smoke test. It writes progress and a result file as it goes:

$ cat worker.py
#!/usr/bin/env python3
"""A deliberately long-running job: the kind of task you start on your laptop
and would rather not babysit."""
import sys, time
from datetime import datetime

FULL = "--full" in sys.argv
N = 20 if FULL else 3          # --full is the "overnight" run; default is a quick smoke test
STEP = 1.5 if FULL else 0.3

RESULT = "/home/agent/result.txt"
start = time.time()
with open(RESULT, "w") as f:
    for i in range(1, N + 1):
        time.sleep(STEP)
        line = f"[{datetime.now():%H:%M:%S}] processed item {i}/{N}"
        print(line, flush=True)
        f.write(line + "\n"); f.flush()
    print(f"DONE: {N} items in {time.time() - start:.1f}s", flush=True)
open("/home/agent/DONE", "w").close()   # sentinel so a watcher knows we finished

Create a sandbox with Claude Code, mounting that project (real output - note the microVM gets its own CPU/memory allocation and pulls the agent image):

$ sbx create claude ~/laptop-to-cloud-demo --name laptop-to-cloud-demo
── RESOLVE SETUP
   resolving configuration…
     sandbox    laptop-to-cloud-demo
     agent      claude
     workspace  /Users/you/laptop-to-cloud-demo (rw)
     skills     …/agent-skills → /home/agent/.claude/skills · 5 folders · (ro)
     image      docker/sandbox-templates:claude-code-docker
     cpu        18
     memory     18 GiB
   ✓ configuration resolved
── PREPARE IMAGE
   → pull docker/sandbox-templates:claude-code-docker
   ✓ image ready
── CREATE SANDBOX
   ✓ Created sandbox laptop-to-cloud-demo

To connect to this sandbox, run:
  sbx run --name laptop-to-cloud-demo

💡 run vs create: sbx run claude (no path) mounts your current directory and attaches you interactively in one step. sbx create runs in the background and does not default to the cwd - pass the workspace path explicitly, as above.

Confirm it's live (real output):

$ sbx ls
SANDBOX                AGENT     STATUS    PORTS   WORKSPACE
laptop-to-cloud-demo   claude    running           /Users/you/laptop-to-cloud-demo

Now develop on the laptop the way you normally would: run a quick smoke test of the job inside the microVM (real output):

$ sbx exec laptop-to-cloud-demo bash -lc 'python3 worker.py'
[02:33:08] processed item 1/3
[02:33:08] processed item 2/3
[02:33:09] processed item 3/3
DONE: 3 items in 0.9s

⚠️ Mind the path. sbx exec drops you into the workspace as the working directory, so a relative worker.py just works. But inside the sandbox your project is bind-mounted at its host absolute path (/Users/you/laptop-to-cloud-demo), not under the agent's home - ~ is /home/agent. So ~/laptop-to-cloud-demo/worker.py will fail with "No such file"; use worker.py (relative) or the full host path.

It works. The full --full run would take far longer - exactly the kind of thing you don't want to babysit on your laptop. Before we hand it off, stage the job into the sandbox's image filesystem so it survives the move (the next section explains why the mounted workspace alone won't):

$ sbx exec laptop-to-cloud-demo bash -lc 'cp worker.py /home/agent/worker.py && ls -l /home/agent/worker.py'
-rw-r--r-- 1 agent agent 862 Oct  3 02:33 /home/agent/worker.py

📸 Screenshot 3 - split view: Claude Code running in the sandbox on the left, sbx ls showing it running on the right. Your "local" money shot.

This is what the laptop is good at: watching, steering, a fast iteration loop. Now that the long run is ready, we move it up and let it finish without us.

Part 2 - Finish in the cloud

Step 0: Activate Docker Agentic Platform / cloud access

Follow the Signup and billing guide to activate your Docker Agentic Platform subscription and review compute charges. New accounts get free credit to start. Pay-as-you-go, billed per second, by sandbox size:

Size	vCPUs	Memory	Per hour
Micro	1	2 GiB	$0.07
Small (default)	2	4 GiB	$0.14
Medium	4	8 GiB	$0.28
Large	8	16 GiB	$0.56
XL	16	32 GiB	$1.12

📸 Screenshot 4 - the Docker Agentic Platform billing/usage dashboard.

The pivot: move the running sandbox to the cloud

The one-liner the whole post builds toward. Here is the real, unedited output of moving our sandbox up - read the warnings, they matter:

$ sbx move laptop-to-cloud-demo --to cloud --name laptop-to-cloud-demo --ttl 30m --force
warning: Secrets managed by sbx are not copied. The destination may use its own secrets; you may need to sign in again.
warning: Credentials saved in copied files can still travel with the sandbox.
warning: Local network policies are not copied. The destination uses cloud network policies.
warning: sandbox "laptop-to-cloud-demo" uses files outside its container filesystem. These files won't be copied to the cloud:
  "/Users/you/laptop-to-cloud-demo" (workspace, host bind mount)
→ move sandbox laptop-to-cloud-demo to cloud
Using cloud shape small (2 vCPU / 4096 MiB).
Pushing sandbox laptop-to-cloud-demo to cloud ...
Upload complete.
✓ move sandbox laptop-to-cloud-demo to cloud (41s)
✓ Moved to cloud sandbox laptop-to-cloud-demo (sbx_001m3zsrjd2pxfxt5fh5k9z0vrs)
  Attach with:  sbx --cloud run sbx_001m3zsrjd2pxfxt5fh5k9z0vrs
  Created cloud template tmpl_001m3zsqt39yxnwtwr8ksx3qxka to back it; remove it with
  `sbx --cloud template rm tmpl_001m3zsqt39yxnwtwr8ksx3qxka` once the sandbox is deleted.
  Expires in 29m; the sandbox will then be stopped (it can be started again later).

What just happened: sbx move captured the sandbox's filesystem as a container image and started a fresh cloud sandbox from it. --ttl 30m capped its life at 30 minutes; when the TTL lapses it stops in place (hibernates) so you can resume later, rather than being deleted. Neither side is destroyed - the move just stops the local source; restart it any time with sbx run --name laptop-to-cloud-demo.

📸 Screenshot 5 - the sbx move … --to cloud run with these warnings + the "Moved to cloud sandbox" line. This is the hero shot of the post.

⚠️ The gotcha everybody hits: the workspace does NOT travel

Look again at that warning:

warning: sandbox uses files outside its container filesystem. These files won't be copied to the cloud:
  "/Users/you/laptop-to-cloud-demo" (workspace, host bind mount)

Your mounted workspace is a host bind mount, and bind mounts don't travel. What travels is the sandbox's own image filesystem (e.g. anything under /home/agent) - which is exactly why we staged worker.py there in Step 4. Here's the proof, live, in the cloud sandbox after the move:

$ sbx --cloud exec laptop-to-cloud-demo bash -lc \
    'echo "staged job (image fs):"; ls -l /home/agent/worker.py; \
     echo "workspace (bind mount):"; ls -la /Users/you/laptop-to-cloud-demo'
staged job (image fs):
-rw-r--r--+ 1 agent agent 862 Oct  3 02:33 /home/agent/worker.py    # ← TRAVELED
workspace (bind mount):
total 8
drwxr-xr-x+  2 root  root   27 Oct  3 02:33 .                       # ← arrived EMPTY
drwxr-xr-x+  4 agent agent 110 Oct  3 02:33 ..

The staged job came across; the mounted workspace arrived empty. So the practical rule for "finish in the cloud":

Stage what the cloud run needs into the image filesystem (e.g. /home/agent) before you move - as we did with worker.py.
Or commit and push from inside the sandbox (its injected GitHub token works), which is the cleanest way to get work off any sandbox.
Or pull files back with sbx cp when you move the sandbox back down.

This is the single most important thing to understand before you trust an overnight run.

Confirm it's running in the cloud

The --cloud global flag dispatches a supported verb to the hosted service. Real output:

$ sbx --cloud ls
SANDBOX                       AGENT    ID                                STATUS    PORTS   WORKSPACE
claude/laptop-to-cloud-demo   claude   sbx_001m3zsrjd2pxfxt5fh5k9z0vrs   running           -

📸 Screenshot 6 - sbx ls (local, source now stopped) next to sbx --cloud ls (running in the cloud). Side by side, this proves the same workload moved.

The payoff: kick off the long run, then close the laptop

This is the whole point of the post, made visible. Launch the --full job in the background in the cloud (one caveat the move output hinted at: running processes don't travel - so you start the long job after you land in the cloud, not before):

$ sbx --cloud exec laptop-to-cloud-demo bash -lc \
    'nohup python3 /home/agent/worker.py --full > /home/agent/run.log 2>&1 & echo "pid $!"'
pid 1941

Peek once while the lid's still open - it's genuinely running, with nothing attached:

$ sbx --cloud exec laptop-to-cloud-demo bash -lc 'cat /home/agent/run.log'
[02:34:20] processed item 1/20
[02:34:21] processed item 2/20
[02:34:23] processed item 3/20

Now close your laptop. Come back later, reconnect, and the work finished on its own in the cloud:

$ sbx --cloud exec laptop-to-cloud-demo bash -lc 'tail -1 /home/agent/result.txt'
DONE: 20 items in 30.0s

Started on the laptop, finished in the cloud - no VM to provision, no lid to keep open.

📸 Screenshot 7 - the three steps stacked: launch (pid 1941), the mid-flight run.log, and the final DONE: 20 items in 30.0s. This is the hero shot.

Working with a cloud sandbox

Everything has a --cloud form:

sbx --cloud run sbx_001m3zsrjd2pxfxt5fh5k9z0vrs      # attach (starts it if stopped)
sbx --cloud exec laptop-to-cloud-demo bash -lc '...' # run a command inside it
sbx --cloud ttl laptop-to-cloud-demo                 # check / extend remaining lifetime
sbx cp laptop-to-cloud-demo:/home/agent/result.txt . # copy results back down
sbx move laptop-to-cloud-demo --to local             # bring the whole sandbox back

Clean up (stop billing)

Billing is per-second, so tear down when done. Real output from this session's cleanup:

$ sbx --cloud rm laptop-to-cloud-demo -f
   ✓ remove sandbox laptop-to-cloud-demo (1s)
$ sbx --cloud template rm tmpl_001m3zsqt39yxnwtwr8ksx3qxka --force
Removed: moved-laptop-to-cloud-demo-7513a8b6 (tmpl_001m3zsqt39yxnwtwr8ksx3qxka)

💡 A move leaves behind a backing cloud template (shown in the move output). Remove it after you delete the sandbox, or it lingers.

Or: start fresh in the cloud

You don't have to start local. For the "run 100 agents in parallel" case, launch straight into the cloud:

sbx --cloud run claude
sbx --cloud run codex

Cloud mode supports a growing set of verbs - run, create, attach, exec, cp, ls, move, stop, rm, ttl, plus cloud-only persistent volumes (sbx volume). Run sbx --cloud --help for the current list.

Part 3 - Go programmatic with the Sandboxes API & SDK

The CLI is the fast path, but everything above is also in the Docker Sandboxes API and SDK - which is where the "100 parallel agents" story gets real. The API lets you:

Create sandboxes and run commands inside them
Transfer files in and out
Manage images, snapshots, volumes, and secrets as first-class resources

That makes cloud sandboxes a natural fit for CI and fan-out jobs: spin up N sandboxes from a known image, dispatch a task to each, collect results, tear them down - no VM provisioning in sight. See the API docs for the current SDK surface.

Local vs Cloud, at a glance

	Local Sandbox	Cloud Sandbox
Runs on	Your laptop (microVM)	Docker-managed infra (microVM)
Isolation model	Hypervisor + proxy + isolated Docker + credential injection	Identical
Best for	Interactive iteration, steering, quick jobs	Long-horizon work, overnight runs, scale
Credentials	Host keychain, injected by proxy	Its own - sign in on the destination
Network policy	Local default (Open / Balanced / Locked Down)	Its own cloud policy
Workspace	Host bind mount (live)	Does not travel on move - use image fs / `sbx cp` / git
Lifecycle	Runs while your machine is on	TTL, hibernation, persistent volumes
Needs local virtualization?	Yes	No
Cost	Your hardware	Pay-as-you-go by size, per second

Wrapping up

It isn't "local sandboxes" or "cloud sandboxes." It's the same sandbox, part of the Docker Agentic Platform, and you choose where the hours run with one command. Iterate on your laptop where watching matters; sbx move … --to cloud the moment the work gets long or wide. Same isolation, same safety model, no VMs to provision - just mind the workspace bind-mount rule before you walk away.

brew install docker/tap/sbx
sbx login
sbx run claude                     # start here
sbx move <name> --to cloud --ttl 8h  # finish here

Learn more

Docker Agentic Platform: https://www.docker.com/products/docker-agentic-platform/
Cloud Sandboxes: https://docs.docker.com/ai/sandboxes/cloud/
Sandboxes overview: https://docs.docker.com/ai/sandboxes/
Sandboxes API: https://docs.docker.com/ai/sandboxes-api/
Announcement blog: https://www.docker.com/blog/introducing-cloud-sandboxes-start-on-your-laptop-finish-in-the-cloud/

Running DeepAgents in a Docker Sandbox, with no cloud keys

Ajeet Singh Raina — Fri, 02 Oct 2026 16:57:21 +0000

Agent frameworks are easy to pip install and surprisingly hard to run responsibly. The moment you give an agent a filesystem, a shell, and a network, you have handed arbitrary generated code the same reach your laptop has. You also inherit a second problem that has nothing to do with safety: reproducing the exact environment, the exact package versions, and the exact model wiring on someone else's machine.

This post walks through a small, self contained answer to both problems: a Docker Sandbox Kit that drops the deepagents harness into an isolated sandbox, pre wired to a local Docker Model Runner so it runs with no cloud credentials at all. The whole kit is four files, and it is published on Docker Hub so you can run it in one command.

What we are packaging

DeepAgents is an opinionated agent harness built on LangGraph. Out of the box it gives an agent a planning tool, a virtual filesystem, sub agent delegation, and a detailed system prompt. You construct an agent in a few lines:

from deepagents import create_deep_agent

agent = create_deep_agent(model=my_model, system_prompt="...")

Two things make it interesting to package. First, it is a library rather than a turnkey CLI, so the useful unit to ship is "deepagents, installed and wired to a model, ready to import." Second, it defaults to a cloud model (Anthropic), which means a naive setup needs an API key and open egress to a provider. We want neither.

What a Docker Sandbox Kit is

Docker Sandboxes (the sbx CLI) run agents inside isolated microVM style environments with a credential proxy and an enforced network policy. A kit is the unit of composition: a single OCI image whose manifest carries a descriptor describing what the kit offers, what it needs from the host as typed capability requests, and what it needs from other kits.

Kits come in two kinds:

A workload owns the environment the agent runs in. Its layers are a root filesystem, and it sets the entrypoint, user, and working directory.
A mixin adds a tool, a credential, or a policy on top of somebody else's workload. Its layers are an overlay that lands on a base it has never seen.

DeepAgents is a capability you add to an environment, not an environment of its own, so it is a natural mixin. It composes onto any workload that carries a Python runtime, for example the stock docker/sbx-kit-shell image.

The architecture

          your host
  ┌──────────────────────────────────────────────────────────┐
  │                                                            │
  │   Docker Model Runner  ──  :12434  (OpenAI-compatible)     │
  │            ▲                                               │
  │            │  egress allowed only to                       │
  │            │  host.docker.internal:12434                   │
  │   ┌────────┴──────────────  sandbox  ──────────────────┐   │
  │   │  shell workload  +  deepagents mixin               │   │
  │   │    • deepagents + langchain-openai (pip, create)   │   │
  │   │    • OPENAI_BASE_URL / OPENAI_API_KEY pre-wired     │   │
  │   │    • ~/deepagents_quickstart.py                     │   │
  │   └─────────────────────────────────────────────────────┘   │
  └──────────────────────────────────────────────────────────┘

The sandbox reaches exactly one host at runtime, the Model Runner on port 12434, and nothing else. The model speaks the OpenAI wire format, so deepagents talks to it through a standard ChatOpenAI client. No traffic leaves your machine.

Four files

The kit is authored as a companion pair plus its guidance and docs:

deepagents.yaml          the v3 descriptor
deepagents.dockerfile    a FROM scratch overlay that carries runtime ENV
deepagents-context.md    agent guidance, staged into the sandbox
README.md

The descriptor

The descriptor declares the kit's identity, its version, and the capabilities it requests. Here are the parts that carry the design.

Phase scoped network policy. Egress is granted per phase, and an absent phase grants nothing. The install phase may reach PyPI; the running agent may reach only the Model Runner. The PyPI grant closes before the agent ever starts.

capabilities:
  - type: com.docker.sandbox/network-policy@1
    config:
      install:
        allow: [pypi.org, files.pythonhosted.org]
      runtime:
        allow: [host.docker.internal:12434]

Create time install, not a baked layer. A mixin overlay lands on a base whose Python version and site-packages path you cannot know in advance, so copying a prebuilt package tree into the image would not resolve. Instead the kit installs into the composed base's Python at sandbox create time, via lifecycle hooks. The hooks also do two things worth calling out: they fail early with a clear message if the base Python is older than 3.11, and after installing they re read the installed version and fail on mismatch. A pinned version claim is only honest if the build enforces it.

  - type: com.docker.sandbox/lifecycle@1
    config:
      install:
        - command: "python3 -c \"import sys; sys.exit(0 if sys.version_info >= (3, 11) else 1)\" || { echo 'deepagents needs Python >= 3.11' >&2; exit 1; }"
          user: "1000"
        - command: "pip install --break-system-packages 'deepagents==0.7.21' langchain-openai"
          user: "1000"
          env: [HTTP_PROXY, HTTPS_PROXY]
        - command: "python3 -c \"import importlib.metadata as m, sys; sys.exit(0 if m.version('deepagents') == '0.7.21' else 1)\""
          user: "1000"

One subtlety: hook environments are deny by default, so HTTP_PROXY and HTTPS_PROXY are declared explicitly. pip reads them to fetch through the sandbox's forced proxy; without the declaration the install would hang.

No hard requires. It is tempting to declare requires: ["deb/python3"], but requires is a closed set check: a name nothing in the composition provides makes the kit refuse to compose everywhere, and a deb/ name rules out every Alpine or Wolfi base that would otherwise have worked. The Python 3.11 guard hook above is the better tool: it degrades gracefully with an actionable error rather than refusing up front.

No credential. Because the model is the local Model Runner, the API key is the sentinel string "dmr" rather than a real secret, so the kit declares no credential@1 capability and asks for nothing from the credential proxy.

The overlay

The content recipe is a FROM scratch overlay that installs nothing. Its only job is to carry static environment onto the composed image. A mixin's ENV is an additive image config field that merges at assembly, so it reaches both the composed image and the agent process.

FROM scratch

ENV OPENAI_BASE_URL="http://host.docker.internal:12434/engines/v1" \
    OPENAI_API_KEY="dmr" \
    DEEPAGENTS_MODEL="ai/qwen3" \
    LANGSMITH_TRACING="false"

LANGSMITH_TRACING=false keeps LangChain from attempting hosted tracing, which the runtime policy would block anyway, but switching it off avoids the noise. NO_PROXY is deliberately not set here, because the shell workload already defines it and two kits setting the same variable to different values is a hard composition conflict.

The quickstart

A lifecycle files entry stages a runnable example into the sandbox, marked so it is never overwritten if you have edited it. It passes an explicit ChatOpenAI instance to override the Anthropic default:

import os
from langchain_openai import ChatOpenAI
from deepagents import create_deep_agent

model = ChatOpenAI(
    model=os.environ.get("DEEPAGENTS_MODEL", "ai/qwen3"),
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ.get("OPENAI_API_KEY", "dmr"),
    temperature=0,
)

agent = create_deep_agent(model=model, system_prompt="You are a concise assistant. Plan before you act.")

if __name__ == "__main__":
    result = agent.invoke({"messages": [{"role": "user", "content": "In two sentences, what is a sandbox?"}]})
    print(result["messages"][-1].content)

Build, verify, run

A kit is just an OCI artifact built by docker buildx, with the descriptor validated before any content is built. A fast first check is to validate the descriptor without exporting anything:

docker buildx build . -f deepagents.yaml --output type=cacheonly

Then export an OCI layout and run the conformance suite. The kit passes all 18 checks on both linux/amd64 and linux/arm64, including the overlay ownership checks that catch a mixin accidentally taking over the agent's home directory:

docker buildx build . -f deepagents.yaml -t deepagents:0.7.21 \
  --output type=oci,dest=/tmp/deepagents-layout,tar=false
kit-tck validate --layout /tmp/deepagents-layout 0.7.21

A build proves the recipe ran, not that the overlay works, so the check that counts is a real composition. Point --kit at the kit directory and let sbx assemble it onto a shell workload:

sbx run docker/sbx-kit-shell:1.0.0 \
  --kit "$(pwd)" --name deepagents-demo /path/to/workspace

On create, sbx assembles the two kits, runs the three install hooks, and writes the quickstart file. Inside the sandbox the result is exactly what the descriptor promised:

$ sbx exec deepagents-demo python3 -c "import deepagents; print(deepagents.__version__)"
0.7.21
$ sbx exec deepagents-demo sh -lc 'echo $OPENAI_BASE_URL'
http://host.docker.internal:12434/engines/v1

Publishing and using it

Publishing a kit is just a build with --push. Both platforms go in one invocation so the index that consumers resolve through is written once:

docker buildx build . -f deepagents.yaml --platform linux/amd64,linux/arm64 --push \
  -t docker.io/ajeetraina777/sbx-kit-deepagents:0.7.21 \
  -t docker.io/ajeetraina777/sbx-kit-deepagents:latest

Once it is on Docker Hub, anyone can run it without cloning the repo. First make sure the Model Runner is serving a tool calling model:

docker model pull ai/qwen3

sbx run docker/sbx-kit-shell:1.0.0 \
  --kit docker.io/ajeetraina777/sbx-kit-deepagents:0.7.21 --name deepagents-demo .

Why this shape is worth copying

The pattern generalizes well beyond deepagents:

Package the capability, not the environment. A mixin composes onto whatever workload a team already uses, so you are not forcing a base image on anyone.
Wire to a local model when you can. Pointing an OpenAI compatible client at Docker Model Runner removes the API key, the egress, and the per token cost from the loop. The sandbox can then run with essentially no network.
Make the network policy phase scoped. Letting install reach PyPI while the running agent reaches only the model is a meaningful boundary, and it is declared in a few lines.
Pin, and make the build enforce the pin. A version claim that the content can quietly violate is worse than no claim.

The full kit is on GitHub at ajeetraina/docker-sbx-deepagents and on Docker Hub at ajeetraina777/sbx-kit-deepagents. It is small enough to read in one sitting and a good starting point for packaging your own agent tooling the same way.

Running Docker Sandboxes on Windows Platform

Ajeet Singh Raina — Tue, 30 Jun 2026 07:36:52 +0000

Prerequisite

Enable WSL2 or Hyper V

In this guide, we tested it with Hyper V enablement.

Enable-WindowsOptionalFeature -Online -FeatureName HypervisorPlatform -All
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux -All
Enable-WindowsOptionalFeature -Online -FeatureName VirtualMachinePlatform -All

Note: sbx's microVM backend rests on the Windows hypervisor + VirtualMachinePlatform regardless of whether you went the WSL2 or Hyper-V route. All three features you enabled with Enable-WindowsOptionalFeature modify boot-time kernel components, so none of them are fully live until a restart — the API returns success, but the capability isn't actually present in the running kernel yet.
Hence, reboot the system once before proceeding to the next step.

Step 1. Download the latest version of sbx

winget install -h Docker.sbx

If you want to use nightly build, download the release directly from this link: https://github.com/docker/sbx-releases/releases/tag/nightly

Download the .msi file below and double-click to install or
use the command line:

msiexec /i DockerSandboxes.msi /quiet

Step 2. Verify if you've the latest version of sbx

sbx version
sbx version: v0.34.0-rc1-216-g0be67a90 0be67a900e2a642c494353de466135d18d53f338

Step 3. Login to sbx

sbx login

The CLI prints a one-time device confirmation code and a URL.

Step 4. Choose your agent and provider

This lab works with any mainstream coding agent. Pick the provider whose API key you have - the rest of the lab will adapt.

Pick a provider for the rest of this lab. Each option below assumes you've chosen one of these:

- Use OpenAI + Codex
- Use Anthropic + Claude
- Use Google + Gemini

OpenAI configuration:

You'll run Codex inside the sandbox, authenticated to OpenAI.

First, export your API key in your host terminal** (don't paste it into this page):

If you're using Windows:

$env:OPENAI_API_KEY = "sk-proj-..."

Then store it as a global sbx secret. The command below reads $OPENAI_API_KEY from your shell - your key never leaves the terminal and is not displayed anywhere:

Anthropic configuration:

You'll run Claude Code inside the sandbox, authenticated to Anthropic.

First, export your API key in your host terminal (don't paste it into this page):

$env:ANTHROPIC_API_KEY = "sk-ant-..."

Then store it as a global sbx secret. The command below reads `$ANTHROPIC_API_KEY` from your shell - your key never leaves the terminal and is not displayed anywhere:

Gemini configuration

You'll run Gemini CLI inside the sandbox, authenticated to Google.

First, export your API key in your host terminal (don't paste it into this page):

$env:GEMINI_API_KEY = "AIza..."

Then store it as a global sbx secret.

The command below is just an example that shows how it reads keys from your shell - your key never leaves the terminal and is not displayed anywhere:

echo "$GOOGLE_API_KEY" | sbx secret set -g google
echo "$ANTHROPIC_API_KEY" | sbx secret set -g claude
echo "$OPENAI_API_KEY" | sbx secret set -g openai

Step 5. Clone the lab repository

The exercises use DevBoard - a full-stack FastAPI + Next.js issue tracker with intentional bugs. It's pre-configured for this lab.

git clone https://github.com/dockersamples/sbx-quickstart ~/sbx-lab
cd ~/sbx-lab

Step 6. Create your sandbox

cd ~/sbx-lab
sbx create --name=sbxlab <agent> .

First run: The agent image will pull (1–2 minutes) and the sandbox will be created with the Balanced network policy you selected at login.

sbx ls

You should see sbxlab in the list with status stopped:

PS C:\Users\ajeet> sbx ls
SANDBOX AGENT STATUS PORTS WORKSPACE
shell-sbx-kits-box-main shell stopped C:\Users\ajeet\Downloads\sbx-kits-box-main\sbx-kits-box-main
PS C:\Users\ajeet>

Once you run it (Step 6), the status changes to running.

Step 7. Run your sandbox

sbx run shell-sbx-kits-box-main

Select 1. Yes, continue to launch the agent.

Why this prompt exists: sbx mounts only your project directory into the microVM. The trust check ensures you're aware of what the agent can see and act on.

sbx ls
SANDBOX AGENT STATUS PORTS WORKSPACE
shell-sbx-kits-box-main shell running C:\Users\ajeet\Downloads\sbx-kits-box-main\sbx-kits-box-main
PS C:\Users\ajeet>

Set up the Network Policy

PS C:\Users\ajeet> sbx policy reset
Found 1 running sandbox(es):

shell-sbx-kits-box-main
The daemon will be stopped to apply the policy reset.
Running sandboxes will be terminated.

Are you sure you want to continue? (y/N): y
Stopping daemon at \.\pipe\docker_kaname_sandboxd...
✓ Daemon stopped successfully
✓ Policies reset.

Daemon started (PID: 28492, socket: \.\pipe\docker_kaname_sandboxd)
Logs: C:\Users\ajeet\AppData\Local\DockerSandboxes\sandboxes\state\sandboxd\daemon.log

Verify that you're in a Sandbox environment:

agent@shell-sbx-kits-box-main:sbx-kits-box-main$ cat /etc/os-release
PRETTY_NAME="Ubuntu 26.04 LTS"
NAME="Ubuntu"
VERSION_ID="26.04"
VERSION="26.04 (Resolute Raccoon)"
VERSION_CODENAME=resolute
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=resolute
LOGO=ubuntu-logo
agent@shell-sbx-kits-box-main:sbx-kits-box-main$ sbx policy reset
bash: sbx: command not found
agent@shell-sbx-kits-box-main:sbx-kits-box-main$

Claude Code Won't Start on Windows 11? Fixing the Bun Segfault

Ajeet Singh Raina — Tue, 30 Jun 2026 07:21:37 +0000

If you just installed Claude Code on Windows 11 and it crashes the moment you launch it, you're not doing anything wrong. This is a known issue with the runtime Claude Code now ships with, and the fix takes about two minutes.

Here's exactly what I hit, why it happens, and the step-by-step fix that got me running.

The symptom

You install Claude Code, type claude, and instead of the welcome screen you get a wall of text like this:

Bun v1.4.0 (...) Windows x64 (baseline) Windows v10.26100
...
panic(main thread): Segmentation fault at address 0x113
oh no: Bun has crashed. This indicates a bug in Bun, not your code.

Annoyingly, claude --version and claude doctor often work fine — it's only the interactive session that dies. That makes it look like the install succeeded when it didn't.

Why it happens

Claude Code used to run on Node.js. As of v2.1.113 (April 2026), the default build switched to a Bun-compiled native binary. Bun has a segfault bug that triggers on certain Windows builds during interactive startup, so the new binary crashes on launch.

The catch: even installing via npm doesn't save you anymore. The npm package now pulls in that same Bun binary as a platform dependency, and there's no Node.js fallback left in recent versions. So npm i -g @anthropic-ai/claude-code installs the broken binary too.

The fix is to drop back to the last version that still ran on Node.js (2.1.112) and stop the auto-updater from dragging you forward again.

The fix (step by step)

1. Check your Node version

You need Node 18+ for the JS build, but 18 has its own crash, use Node 20 or newer.

node -v

If you're below 20, upgrade before continuing.

2. Pin to the last pre-Bun release

npm i -g @anthropic-ai/claude-code@2.1.112

You should see npm remove a package during this, that's the win32-x64 Bun binary being dropped and replaced with the Node-run build. That's the whole point. (If 2.1.112 misbehaves, @2.1.110 is another confirmed-good version.)

3. Disable the auto-updater

This step is the one people skip — and then their install silently "upgrades" back to the broken binary overnight. There's no settings file toggle to fully turn updates off; the supported way is an environment variable:

[Environment]::SetEnvironmentVariable("DISABLE_AUTOUPDATER","1","User")

(DISABLE_AUTOUPDATER stops background checks. If you want to block every update path including manual ones, use DISABLE_UPDATES instead.)

4. Open a fresh terminal and verify

The env var only reaches new windows, so close the current terminal and open a new one.

claude doctor

You want to see 2.1.112, install method npm-global, and the point of the whole exercis, no segfault. Then start it:

claude

First launch hands you the auth screen (browser login for Pro/Max, or an API key for Console billing) and a trust prompt for your folder. You're in.

Two things to know afterwards

Ignore the "switch to native installer" banner. Claude Code will nudge you to run claude install. Don't ! that's exactly the path back to the Bun binary and the crash. Leave it alone.

You're now frozen at 2.1.112. That means you'll miss newer features that shipped later. For real work it's perfectly functional, but if you want the latest, the right move isn't to un-pin on Windows — it's WSL2. The Bun crash is Windows-specific; the Linux build runs fine. Install Node inside your distro (make sure which node returns a /usr/... path, not /mnt/c/...), keep your projects on the Linux filesystem, and install Claude Code there to get the current version with no crash.

TL;DR

npm i -g @anthropic-ai/claude-code@2.1.112
[Environment]::SetEnvironmentVariable("DISABLE_AUTOUPDATER","1","User")
# open a new terminal
claude

Pin to the last Node.js build, freeze updates, done.

References

Claude Code - Troubleshoot installation: https://code.claude.com/docs/en/troubleshoot-install
Claude Code - Advanced setup (auto-updates, version pinning): https://code.claude.com/docs/en/setup
Tracking issue (Bun segfault on Windows): https://github.com/anthropics/claude-code/issues/25630

You just can’t miss this…

Ajeet Singh Raina — Tue, 26 May 2026 05:43:22 +0000

Harsh Manvar

May 26

Docker with AI: A Practical Guide to Running LLMs, Agents and MCP

#ai #docker #machinelearning #llm

7 min read

Announcing the NVIDIA Nemotron 3 Super Build Contest

Ajeet Singh Raina — Fri, 01 May 2026 08:45:20 +0000

Have you ever wanted to use AI to make a real, tangible difference in your community? The NVIDIA Nemotron 3 Super Build Contest, organized by Docker Bangalore, Collabnix, and NVIDIA, is your perfect opportunity to do just that. With a deadline of May 25, 2026, developers are invited to build impactful solutions and compete for exciting prizes

Click here for details

The Challenge: 4 Tracks for Public Good

This contest isn't about building just another chatbot; every project must solve a real public problem. Participants must choose one of the following four tracks

Track 1: Community Benefits Navigator - Many people struggle to access the public services they qualify for. This track challenges you to build an assistant that digests complex policy PDFs, eligibility rules, and checklists to help users apply for services like scholarships, legal aid, or health schemes in plain language
Track 2: Open-Source Maintainer Copilot for Small Community Projects - Help prevent burnout among maintainers of public-interest repositories. Build a GitHub helper tailored for nonprofits and civic-tech teams that can automatically triage issues, draft release notes, and assist with newcomer onboarding
Track 3: Local Governance / Public Meeting Explainer - Local decisions are often opaque to the public. Use the model's multi-document reasoning skills to translate city council minutes, budgets, and policy updates into plain-language explanations that tell residents exactly what changed and what actions they can take
Track 4: Crisis Information and Resource Routing Assistant - During chaotic events like floods or disease outbreaks, people need fast, localized guidance. Build a community response assistant that synthesizes rapidly changing context—like shelter lists, school closures, and verified guidance—into one easy-to-use conversational workflow

Prizes and Judging

Projects will be rigorously evaluated across six core dimensions: community value, grounded accuracy, actionability, safety, equity/accessibility, and operational efficiency. A panel of expert judges will also look at your innovation, code quality, and presentation.

If you build one of the top projects, you could win big:

Top 3 Winners: Amazon vouchers + NVIDIA swag
2 Runners-up: NVIDIA swag

How to Submit Your Project

Ready to get started? Make sure your submission meets all the official requirements before the May 25, 2026 deadline. To successfully submit, you must:

Host your code on a public GitHub repository and include an architecture diagram in your README
Record a YouTube demo video showing your project in action
Write a blog post (just like this one!) about your project and share it on social media
Use the right tags: Your social posts must include #NVIDIA and #Nemotron, and you must explicitly call out the Bengaluru meetup
Submit officially by opening a Submission issue on the GitHub repository

Head over to the collabnix/nvidia-nemotron-contest GitHub repository to open your submission issue and get your project on the public wall. If you need help along the way, you can reach out via the Collabnix Slack or open a question issue.

Happy building!

Announcing Operational AI with Docker Book by Ajeet Singh Raina & Harsh Manvar

Ajeet Singh Raina — Wed, 29 Apr 2026 07:29:29 +0000

Get the Book

📘 Packt: Operational AI with Docker
📦 Amazon: Available now (paperback + Kindle)
🔖 ISBN: 9781807301095

For the better part of two years, I had a problem I couldn't solve in 280 characters or a blog post.

Every time I ran a Collabnix Meetup and we were running them often, with three or four hundred DevOps engineers and developers showing up each time, mostly new faces - someone would walk up to me afterwards and ask the same question:

"Ajeet, where do I start with Docker and AI?"

And honestly? My answer used to be a bit lame.

"Follow my blogs."

I was writing one almost every day. The landscape was shifting that fast. We were moving from what I think of as GenAI 1.0 to 2.0 - from chatbots to automated AI workflows. MCP had just emerged. Docker Model Runner was new. Sandboxes hadn't even shipped yet. There was massive confusion about where to even begin.

Then came December 2024. MCP was introduced, and within a few weeks the ecosystem had over 3,000 MCP servers. That's when it really hit me: blog posts aren't enough anymore. People need a single, structured place to learn this end to end.

So Harsh Manvar and I decided to write the book.

Today, when someone walks up to me at a meetup and asks where to start with Docker and AI, I can finally say with confidence: go grab this book. You can learn it from scratch.

It's called Operational AI with Docker, it's published by Packt, and as of today — it's live.

Order Now

Why "Operational AI"?

I keep going back to 2013 to explain what we're trying to do with this book.

When Docker was introduced, the whole idea of build, ship, and run software became wildly popular. Suddenly, moving from monoliths to microservices wasn't a giant migration project - it was something a developer could actually do on a Tuesday afternoon. The way I describe it: it was like ordering an iPhone from Amazon. You unbox it, switch it on, and it just works. No "CPU unsupported" errors. No dependency hell. No "works on my machine."

Docker solved that exact problem for software workloads.

AI today has the exact same packaging problem, just with heavier baggage. Models, weights, tokenizers, GPU drivers, fragile Python environments, agents that need tools, tools that need secrets, secrets that need policies.

So Docker is doing for AI what it did for software a decade ago. The same three-word story, extended:

Build AI agents using the MCP Toolkit, the MCP Catalog, and Docker Model Runner.
Ship them with Agentic Compose and Docker Agents.
Run them inside Docker Sandboxes.

The philosophy hasn't changed. Build, ship, run, extended to AI workloads. That's the entire thesis of this book.

Docker isn't trying to be an AI framework. It's the runtime and packaging layer underneath, so the AI parts can actually be portable, reproducible, and shippable. That's where Docker fits. That's what this book teaches you to do.

The Real Problems This Book Solves

When teams come to me asking about Docker and AI, they usually think their problem is technical. Most of the time, it isn't. Here are the four challenges I see over and over, and the book tackles each of them head-on:

1. Choosing the right model

Hugging Face today has more than two million AI models on it. Two million. So when a team starts an AI project, the first thing they hit isn't a Docker problem, it's a decision problem.

In Chapter 6, I break this down with a diagram, splitting models into three buckets:

Small Language Models (SLMs) — roughly 0 to 7 billion parameters
Medium Language Models — the middle range
Large Language Models — the 70-billion-plus class

Most teams don't need an LLM. They need an SLM that runs cheaply, fast, and close to the user. But figuring that out takes effort, and very few people do it before they start coding.

2. The "AI = cloud-only" perception trap

Most developers, when they hear "AI model," immediately think OpenAI, Gemini, Claude - something that lives in the cloud, behind an API key, costing money per token.

That assumption shapes the entire architecture before anyone has written a line of code. The book breaks that assumption open and shows you what local-first AI actually looks like in practice.

3. GPU fear

The moment you say "run a model locally," people panic about hardware. "Do I need a beefy GPU? Will my MacBook even handle this?"

There are great open-source tools out there — llmfit literally scans your hardware and tells you which models will run well on your machine, and it supports Docker Model Runner as a runtime. But most developers don't know these tools exist, so they default to the cloud out of fear, not necessity. The book walks through this so you can make the call with data, not anxiety.

4. "Should I even run my model inside a container?"

This is the question I get more than any other. And there's a real gap in understanding here.

With Docker Model Runner, you don't actually have to wrap the model in a container at all. DMR runs the model natively on the host, uses the GPU directly, and exposes it through an OpenAI-compatible endpoint — but it gives you Docker's packaging, versioning, and pull experience on top of that. You get the best of both worlds.

That nuance is missed by a lot of teams, and it's exactly the kind of thing the book unpacks early on.

What You'll Learn

The book is structured around the real lifecycle of an AI workload — from running a model on your laptop, to building agents, to securing and scaling them in production:

✅ Run LLMs locally with Docker Model Runner — pull a model the same way you pull a container image, with OpenAI- and Anthropic-compatible APIs.
✅ Build and secure AI agents with Docker MCP Gateway — the piece I think is most underrated in the current AI ecosystem. Dynamic tool discovery, policy enforcement, secrets isolation, audit logs.
✅ Orchestrate multi-agent workflows declaratively — defining agents, sub-agents, and tools in YAML using Docker Agent. Versioned, reproducible, reviewable.
✅ Isolate agent execution with Docker Sandboxes — running untrusted, agent-generated code inside microVMs so a hallucinated rm -rf never reaches your host.
✅ Build, run, and share multi-agent systems using Docker Agents — orchestrator-worker patterns, agent-to-agent communication, shared state, and when not to reach for multi-agent.
✅ Deploy and scale GenAI services on Kubernetes — taking everything from your laptop to a cluster, with the observability and cost-routing patterns production demands.

Built for developers. Grounded in real tools. No fluff.

Who Should Read It

Back in 2023, Harsh and I co-wrote a Docker blog called *"LLM Everywhere: Docker for Local and Hugging Face Hosting."* It went deep into model quantization formats — GPTQ, GGML — and how to actually run these models locally in containers. That blog took off. From an SEO standpoint, it became one of Docker's biggest hits in the AI space.

But what really shaped the audience for this book was the questions that came pouring in afterwards. People wanted to understand model training, model quantization, the trade-offs between formats. And one question came up over and over: "Should I even run a model inside a container?"

That tells you exactly where people are. They're not AI researchers. They're not pure ML engineers. They're developers, DevOps folks, and platform teams who suddenly need to make AI work in their existing world.

So that's who we wrote for:

If you're a developer who built an agent demo and is now being asked to "productionize it" — this book is for you.
If you're a platform engineer whose team is shipping LLM-powered services and you're trying to figure out the runtime story — this book is for you.
If you're an architect mapping out an agentic AI strategy and need a concrete reference for what the operational layer actually looks like — this book is for you.

We didn't assume a deep ML background. If you know containers and you're curious about AI, you're in the right place. And if you know AI but containers feel like a black box, the early chapters meet you there.

Thank You

To Harsh Manvar — Docker Captain, Google Developer Expert, CNCF Ambassador, and the steadiest co-author I could have asked for. Our collaboration goes back years of Docker community blogs, and this book is the natural culmination of that.

To Apramit Bhattacharya, Preet Ahuja and the Packt editorial team — for pushing back on chapters I thought were finished, and for making the book sharper at every pass.

To the Collabnix community — 17,000+ of you across Slack and Discord — who have been the sounding board for every idea in this book, often without realizing it. Every meetup question, every late-night DM, every "Ajeet, but how does this actually work in production?" — those shaped this book more than anything else.

And to my family, who put up with a year of evenings and weekends.

If you pick it up, I'd genuinely love to hear what you think. Tag me on LinkedIn or X (@ajeetsraina), drop into the Collabnix Slack, or just reach out directly. The companion code is open source, and we'll be maintaining it as the Docker AI stack continues to evolve.

Operational AI is not a finished problem. This book is a snapshot of where the practice stands in 2026 — and a foundation you can build on as it keeps changing.

Let's keep figuring it out, together.

— Ajeet

How Autonomous AI Agents Become Secure by Design With Docker Sandboxes

Ajeet Singh Raina — Wed, 15 Apr 2026 03:52:42 +0000

I've been running AI coding agents for a while now. Claude Code on my MacBook, pointed at a project directory, autonomously editing files, running tests, pushing commits. It's genuinely useful — the kind of useful that makes you wonder how you shipped code without it.

But a few months ago I started asking myself a question I'd been quietly avoiding: what exactly can this agent reach while it's running?

The answer, once I actually looked, was uncomfortable. Everything. It could reach everything I could reach ~ my SSH keys, my AWS credentials, my .env files, my Git tokens. Not because it was malicious. Just because it was running on my laptop, as me, with my permissions.

The risk isn't that your agent is malicious. It's that agents are increasingly reading external content — READMEs, web pages, GitHub issues, pull request descriptions. Any of that content could contain a prompt injection that redirects the agent's behavior. You don't need a sophisticated attack. You just need an agent that's trying to do its job.

That's when Docker Sandboxes (sbx) started making a lot more sense to me. In the full post I walk through how a single architectural change collapses the blast radius of an AI agent — without slowing it down.

👉 Continue reading on ajeetraina.com

Interested to learn more about AI Coding Agent and Docker Sandboxing ? Don't miss out my upcoming session this Saturday 18th April at "Docker for AI" Show-n-Tell event at FAI Office, Indiranagar, Bengaluru.

Running NVIDIA Nemotron on a Mac with Docker Model Runner: What You Need to Know

Ajeet Singh Raina — Sun, 08 Mar 2026 08:50:51 +0000

Docker Model Runner just got a major upgrade for Mac users. With the introduction of vllm-metal - a new backend that brings vLLM inference to macOS via Apple Silicon's Metal GPU - you can now run MLX models through the same OpenAI-compatible API, the same Claude Code-compatible API, and the same Docker workflow you already know.

I put this to the test by running NVIDIA Nemotron models on my Mac. Here's what the experience looks like, what works today, and what's coming very soon.

What is vllm-metal?

vllm-metal is a plugin for vLLM that brings high-performance LLM inference to Apple Silicon. Developed by Docker engineers and now contributed to the open-source vLLM community, it unifies MLX (Apple's machine learning framework) and PyTorch under a single compute pathway — plugging directly into vLLM's existing engine, scheduler, and OpenAI-compatible API server.

The architecture is elegant:

+-------------------------------------------------------------+
|                          vLLM Core                          |
|        Engine | Scheduler | API | Tokenizers                |
+-------------------------------------------------------------+
                             |
+-------------------------------------------------------------+
|                   vllm_metal Plugin Layer                   |
|   | Platform  |  | Worker    |  | ModelRunner            |  |
+-------------------------------------------------------------+
                             |
+-------------------------------------------------------------+
|                   Unified Compute Backend                   |
|   | MLX (Primary inference) | PyTorch (model loading)  |   |
+-------------------------------------------------------------+
                             |
+-------------------------------------------------------------+
|              Metal GPU / Apple Silicon Unified Memory       |
+-------------------------------------------------------------+

Source ~ https://www.docker.com/blog/docker-model-runner-vllm-metal-macos/

What makes this particularly powerful on Apple Silicon is unified memory. Unlike discrete GPUs where data must be copied between CPU and GPU memory, Apple Silicon shares a single memory pool. vllm-metal exploits this with zero-copy tensor operations — combined with paged attention for KV cache management and Grouped-Query Attention support.

Getting Started

Update to Docker Desktop 4.62 or later for Mac, then install the backend:

docker model install-runner --backend vllm-metal

That's it. Docker Model Runner automatically routes MLX models to vllm-metal when the backend is installed.

vLLM Now Runs Everywhere with Docker Model Runner

This release completes vLLM support across all three major platforms:

Platform	Backend	GPU
Linux	vllm	NVIDIA (CUDA)
Windows (WSL2)	vllm	NVIDIA (CUDA)
macOS	vllm-metal	Apple Silicon (Metal)

The same docker model commands work regardless of platform. Docker Model Runner picks the right backend automatically.

Which Models Work with vllm-metal?

vllm-metal works with safetensors models in MLX format. The mlx-community on Hugging Face maintains a large collection of quantized models optimized for Apple Silicon. Some great starting points:

# Lightweight and fast
docker model run hf.co/mlx-community/Llama-3.2-1B-Instruct-4bit

# Strong 7B
docker model run hf.co/mlx-community/Mistral-7B-Instruct-v0.3-4bit

# Latest coding model
docker model run hf.co/mlx-community/Qwen3-Coder-Next-4bit

Running NVIDIA Nemotron on Mac

With vllm-metal installed, let's look at what the Nemotron lineup offers on Mac today.

Llama-3.1-Nemotron-Nano-8B (Recommended)

This is a standard transformer model fine-tuned by NVIDIA for instruction following and reasoning. It runs well on Mac via vllm-metal:

docker model run hf.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1

On a 32GB or 64GB Mac, this model runs comfortably. On a 16GB Mac, free up unified memory first by reducing Docker Desktop's VM memory limit (Settings → Resources → Advanced → set to 4–5 GB).

Nemotron-3-Nano-30B: The Mamba2 Frontier

The 30B model uses a hybrid SSM (State Space Model) architecture — combining Mamba2 layers with attention layers. This is architecturally novel and represents the next frontier of efficient inference: better long-context performance, lower memory footprint at runtime, and compelling throughput characteristics.

Mamba2 support in mlx-lm is actively maturing, and as it does, models like Nemotron-3-Nano-30B will run natively through vllm-metal. This is an exciting space to watch — especially on higher-memory Macs where the 30B model size becomes practical.

The $599 AI Development Rig

One of the most compelling stories in this release: a base Mac Mini M4 at $599 is now a viable vLLM development environment. Because Apple Silicon uses unified memory, the 16GB (or upgraded 32GB/64GB) RAM is directly accessible by the GPU, enabling you to:

Develop and test locally using the same OpenAI-compatible API as production
Mirror production — the same API surface as an H100 cluster, on your desk
Run efficiently — a fraction of the power consumption and heat of a discrete GPU rig

This democratizes access to vLLM development in a way that wasn't possible before. Previously, getting started with high-throughput vLLM required an RTX 4090 ($1,700+) or enterprise-grade GPU cards. Now, the barrier to entry is a Mac Mini.

vllm-metal vs llama.cpp: Benchmark Context

Docker benchmarked both backends on Llama 3.2 1B Instruct with 4-bit quantization:

max_tokens	llama.cpp (tok/s)	vLLM-Metal (tok/s)
128	333.3	251.5
512	345.1	279.0
1024	338.5	275.4
2048	339.1	279.5

llama.cpp shows ~1.2–1.3x higher raw throughput on this benchmark. However, vllm-metal brings things that raw token speed doesn't capture: the full vLLM engine, including its scheduler, paged attention, batching, and production-grade OpenAI-compatible API. For developers building real applications, that consistency and API compatibility often matters more than peak throughput on a single request.

Hardware Recommendations

Mac Configuration	What to Run
M-series, 16GB	`mlx-community/Llama-3.2-1B-Instruct-4bit`, `ai/phi4-mini`
M-series, 32GB	Nemotron 8B, `mlx-community/Mistral-7B-Instruct-v0.3-4bit`
M-series Max/Ultra, 64GB+	Nemotron 8B comfortably, 30B models as SSM support matures

I'm upgrading to a MacBook Pro Max this month — full benchmarks and multi-model Nemotron demos coming soon.

Open Source at the Core

Docker contributed vllm-metal to the vLLM open-source community — it now lives under the vLLM GitHub organization. This means every developer in the ecosystem can benefit from and contribute to high-performance inference on Apple Silicon. The project has also had significant contributions from Lik Xun Yuan, Ricky Chen, and Ranran Haoran Zhang.

Quick Start

# Install vllm-metal backend (Docker Desktop 4.62+ required)
docker model install-runner --backend vllm-metal

# Run a Nemotron model
docker model run hf.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1

# Or try an mlx-community model
docker model run hf.co/mlx-community/Mistral-7B-Instruct-v0.3-4bit

# List all downloaded models
docker model ls

Learn More

Want to learn more? Come, join me at Collabnix Community Meet and NVIDIA GTC 2026 Watch Party this March 21st virtually. Don't forget to register here

Arm Migration via Arm MCP Server, Docker MCP Toolkit, VS Code and Co-Pilot

Ajeet Singh Raina — Mon, 02 Mar 2026 06:17:12 +0000

This video detail the use of the Docker MCP Toolkit to automate the migration of software from x86 to Arm64 architecture. By integrating specialized Arm MCP Servers with AI assistants like GitHub Copilot, developers can significantly reduce the manual labor required for complex porting tasks.

The system automatically identifies x86-specific dependencies, converts legacy SIMD intrinsics to Arm NEON equivalents, and updates Dockerfiles to ensure hardware compatibility.

This workflow streamlines the modernization of legacy C++ applications by connecting natural language commands to containerized diagnostic and transformation tools. Ultimately, the guide demonstrates how to achieve substantial cost savings and performance gains on cloud platforms like AWS Graviton with minimal architectural expertise.

Click to read the blog

NanoClaw just moved from Apple Containers to Docker

Ajeet Singh Raina — Sat, 28 Feb 2026 06:57:32 +0000

NanoClaw just moved from Apple Containers to Docker and the reason behind it is a masterclass in open source stewardship.

Gavriel Cohen built NanoClaw AI as a personal project. He uses a Mac, loved Apple Containers for their lightweight Apple Silicon optimization, and shipped what worked for him. Simple, honest, personal.

Then thousands of developers started using it. Production workloads. Businesses building on top of it. A real community formed.

And that’s when he made a decision that not every creator has the humility to make:

“The defaults should serve the community, not just me.”

He switched to Docker ~ battle-tested, universally supported, runs everywhere because it was the right thing for the project, not his personal preference.

Apple Containers are still fully supported. Run /convert-to-apple-container and in ~30 seconds it merges changes into your codebase via git ~ minimal token usage, deterministic in most cases, with Claude resolving merge conflicts.

This is what great open source leadership looks like. Knowing when to get out of your own way.

Congrats to the NanoClaw community on this milestone 🐳

👉 https://x.com/Gavriel_Cohen/status/2026028681907339382

Running OpenClaw on NVIDIA Jetson Thor with Docker Model Runner: A Complete Guide

Ajeet Singh Raina — Mon, 23 Feb 2026 05:40:30 +0000

What if you could run your own AI-powered Discord bot completely local, no cloud APIs, no subscription fees on an NVIDIA Jetson Thor? That's exactly what we did. In this guide, I'll walk you through setting up OpenClaw, an open-source AI agent framework, powered by Docker Model Runner running Qwen3 8B locally on NVIDIA Jetson Thor.

The result? A fully functional Discord bot that responds to messages using a locally hosted LLM, with zero data leaving your network.

Prerequisites

Before we begin, make sure you have the following ready:

NVIDIA Jetson Thor with Docker Engine installed
Docker Model Runner plugin enabled
Node.js v22+ installed
A Discord account with server admin access
Basic familiarity with the terminal

Step 1: Install OpenClaw

OpenClaw provides a one-liner installer that detects your OS and sets everything up via npm:

curl -fsSL https://openclaw.ai/install.sh | bash

You should see output confirming the installation:

🦞 OpenClaw Installer
✓ Detected: linux
✓ Node.js v22.22.0 found
✓ npm configured for user installs
🦞 OpenClaw installed successfully (2026.2.21-2)!

Choose Custom Provider.

Please Note: The right URL would like https://localhost:12434/v1. We will change it later point of time

Step 2: Verify Docker Model Runner

Docker Model Runner lets you run LLMs locally as part of Docker's ecosystem. First, let's check what models are available:

docker model ls

MODEL NAME           PARAMETERS  QUANTIZATION     ARCHITECTURE  SIZE
llama3.2:3B-Q4_K_M   3.21 B      IQ2_XXS/Q4_K_M  llama         1.87 GiB
qwen3:8B-Q4_K_M      8.19 B      IQ2_XXS/Q4_K_M  qwen3         4.68 GiB
smollm2              361.82 M    IQ2_XXS/Q4_K_M  llama         256.35 MiB

We'll use Qwen3 8B as our primary model — it offers a solid balance of intelligence and performance for the Jetson Thor's capabilities.

Verify the API Endpoint

Docker Model Runner exposes an OpenAI-compatible API on port 12434:

curl -s http://localhost:12434/v1/models | jq .

{
  "object": "list",
  "data": [
    { "id": "ai/smollm2", "object": "model", "owned_by": "docker" },
    { "id": "ai/llama3.2:3B-Q4_K_M", "object": "model", "owned_by": "docker" },
    { "id": "ai/qwen3:8B-Q4_K_M", "object": "model", "owned_by": "docker" }
  ]
}

Test a Chat Completion

curl -s http://localhost:12434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3:8B-Q4_K_M",
    "messages": [{"role": "user", "content": "Hello, say hi in one sentence"}],
    "max_tokens": 500
  }' | jq '.choices[0].message.content'

"Hello! How can I assist you today?"

If you see a response, your model runner is working perfectly.

Important: Configure Context Size

By default, Docker Model Runner may use a 4096-token context window, which is too small for OpenClaw (minimum 16,000 tokens required). Bump it up:

docker model configure --context-size 32768 ai/qwen3:8B-Q4_K_M

Verify the configuration:

docker model configure show ai/qwen3:8B-Q4_K_M

[
  {
    "Backend": "llama.cpp",
    "Model": "ai/qwen3:8B-Q4_K_M",
    "Config": {
      "context-size": 32768
    }
  }
]

Step 3: A Note on Qwen3's Thinking Mode

Qwen3 has a built-in "thinking" mode that uses tokens for chain-of-thought reasoning before generating a visible response. If you set max_tokens too low (e.g., 50), you might get an empty content field because all tokens were consumed by reasoning_content.

The fix is simple: use a higher max_tokens value (500+), or disable thinking mode by adding /nothink as a system prompt. For OpenClaw usage with 32K+ context, this won't be an issue.

Step 4: Configure OpenClaw

Run the setup wizard:

openclaw setup

OpenClaw should auto-detect Docker Model Runner. The key configuration in ~/.openclaw/openclaw.json should look like this:

{
  "models": {
    "mode": "merge",
    "providers": {
      "dmr": {
        "baseUrl": "http://localhost:12434/v1",
        "apiKey": "dmr-local",
        "api": "openai-completions",
        "models": [
          {
            "id": "ai/qwen3:8B-Q4_K_M",
            "name": "Qwen3 8B (64K context)",
            "contextWindow": 65536,
            "maxTokens": 65536
          },
          {
            "id": "ai/llama3.2:3B-Q4_K_M",
            "name": "Llama 3.2 3B",
            "contextWindow": 32768,
            "maxTokens": 32768
          }
        ]
      }
    }
  },
  "agents": {
    "defaults": {
      "model": {
        "primary": "dmr/ai/qwen3:8B-Q4_K_M"
      }
    }
  }
}

Pro Tip: Make sure the baseUrl uses /v1 (not /engines/v1). The /engines/v1 endpoint may report incorrect context window sizes, causing OpenClaw to reject the model with a "context window too small" error.

Please Note: During the OpenClaw installer, if you chose Discord then it might ask for Discord Bot. Keep it ready before you proceed further.

Step 5: Create a Discord Bot

Now for the fun part — connecting OpenClaw to Discord.

Create the Application

Go to the Discord Developer Portal
Click New Application and name it (e.g., "OpenClaw Bot")
Click Create

Configure the Bot

Click Bot in the left sidebar
Scroll to Privileged Gateway Intents
Enable Message Content Intent — this is critical for the bot to read messages
Click Save Changes
Scroll up and click Reset Token to generate a bot token
Copy the token — you'll need it next

Fix the Install Link (Avoid "Code Grant" Errors)

This is a common gotcha. Go to Installation in the left sidebar and set the Install Link to None. This prevents the dreaded "Integration requires code grant" error when trying to invite the bot.

Generate the Invite URL

Go to OAuth2 → URL Generator
Under Scopes, check only bot
Under Bot Permissions, check: Send Messages, Read Message History, View Channels
Copy the generated URL at the bottom
Open it in your browser, select your server, and click Authorize

Or use this URL template directly (replace YOUR_CLIENT_ID):

https://discord.com/oauth2/authorize?client_id=YOUR_CLIENT_ID&permissions=68608&scope=bot

Step 6: Add Discord to OpenClaw

Add the Discord channel with your bot token:

openclaw channels add --channel discord --token YOUR_DISCORD_BOT_TOKEN

Or edit the config directly:

nano ~/.openclaw/openclaw.json

Add the Discord section under channels:

"channels": {
  "discord": {
    "enabled": true,
    "token": "YOUR_DISCORD_BOT_TOKEN",
    "groupPolicy": "open",
    "streamMode": "off"
  }
}

Step 7: Start the Gateway

openclaw gateway --verbose

You should see the bot come online:

[gateway] agent model: dmr/ai/qwen3:8B-Q4_K_M
[gateway] listening on ws://127.0.0.1:18789
[discord] [default] starting provider (@OpenClaw Bot)
[discord] logged in to discord as 1475353419764994181

Step 8: Pair Your Discord Account

OpenClaw uses a pairing system for DMs. Send a direct message to your bot on Discord (e.g., "Hello"). The bot will respond with a pairing code:

OpenClaw: access not configured.
Your Discord user id: 663426992733159434
Pairing code: 9TRNA3AL

Approve the pairing in your terminal:

openclaw pairing approve --channel discord 9TRNA3AL

Now send another message — and watch Qwen3 respond through your Discord bot, running entirely on your Jetson Thor!

The Architecture

Here's what's happening under the hood:

Discord (your messages)
        │
        ▼
┌─────────────────┐
│  OpenClaw       │
│  Gateway        │
│  (WebSocket)    │
│  Port 18789     │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Docker Model   │
│  Runner         │
│  Port 12434     │
│  (OpenAI API)   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Qwen3 8B       │
│  (llama.cpp)    │
│  NVIDIA GPU     │
└─────────────────┘

Everything runs locally on the Jetson Thor. Your messages go from Discord → OpenClaw Gateway → Docker Model Runner → Qwen3 on the GPU, and the response flows back the same way. No cloud, no API keys (except Discord's bot token), no per-token costs.

Troubleshooting

Here are some issues we encountered and how to fix them:

"Model context window too small (4096 tokens)"

This happens when Docker Model Runner defaults to 4096 context. Fix it with:

docker model configure --context-size 32768 ai/qwen3:8B-Q4_K_M

Also ensure your OpenClaw config uses baseUrl: "http://localhost:12434/v1" (not /engines/v1).

"Integration requires code grant" when inviting the bot

Go to the Discord Developer Portal → Installation → set Install Link to None. Then use a clean invite URL with only the bot scope.

Empty responses from Qwen3

Qwen3's thinking mode can consume all tokens before generating a visible response. Increase max_tokens to 500+ or use /nothink as a system prompt.

Bot not appearing in Discord server

Make sure you've authorized the bot using the OAuth2 URL with the bot scope. Check the gateway logs for [discord] logged in to discord as ... to confirm the bot is connected.

What's Next?

With OpenClaw running on Jetson Thor, you now have a foundation for building powerful local AI agents. Some ideas to explore:

Add more channels: Connect Telegram, WhatsApp, or Slack alongside Discord
Install skills: Extend your bot with OpenClaw skills for image generation, web browsing, and more
Run as a service: Use systemctl to keep the gateway running 24/7
Try different models: Swap between Qwen3 8B and Llama 3.2 3B depending on your speed vs. quality needs
Build custom skills: Create modular capability packages that teach your bot new tricks

The beauty of this setup is that it's entirely self-hosted. Your conversations stay on your hardware, your models run on your GPU, and you have complete control over the experience.

Conclusion

Running OpenClaw with Docker Model Runner on NVIDIA Jetson Thor demonstrates the power of edge AI. In under 30 minutes, we went from a bare Jetson Thor to a fully functional Discord bot powered by a locally running 8B parameter model. No cloud dependencies, no recurring API costs, and complete data privacy.

The combination of Docker's containerized model management, OpenClaw's multi-channel agent framework, and NVIDIA's GPU acceleration makes this setup both practical and powerful. Whether you're building a personal assistant, a community bot, or an edge AI prototype, this stack gives you everything you need.

Happy hacking! 🦞