DEV Community

Cover image for One Open Source Project a Day (No.52): Tank-OS - A Red Hat Engineer Baked an AI Agent Into a Bootable Linux Image Over a Weekend
WonderLab
WonderLab

Posted on

One Open Source Project a Day (No.52): Tank-OS - A Red Hat Engineer Baked an AI Agent Into a Bootable Linux Image Over a Weekend

#opensource #openclaw #linux #podman

Introduction

"When an AI Agent starts deleting emails, accessing databases, and calling external APIs, are you certain it can't go out of bounds?"

This is article No.52 in the "One Open Source Project a Day" series. Today's project is Tank-OS (GitHub).

In April 2026, TechCrunch reported on a project that one engineer built over a single weekend. The engineer is Sally O'Malley, Principal Software Engineer at Red Hat's Office of the CTO and a core maintainer of OpenClaw. The project answers a question that becomes more pressing as AI Agents get more capable: when you need to deploy a fleet of AI Agents across a company, how do you ensure every machine is isolated, secure, and consistently updatable?

Tank-OS's answer: pack the Agent, its runtime, the OS, Systemd units, and the upgrade mechanism into a single OCI container image, then boot entire machines directly from that image.

In cloud-native circles, this pattern (called bootc — Boot Container) isn't new. But applying it specifically to solve AI Agent enterprise deployment security? Tank-OS is the most concrete, complete open-source reference implementation available.

TechCrunch headline: "Red Hat's OpenClaw maintainer just made enterprise Claw deployments a lot safer."

What You'll Learn

  • What bootc is: why "OS as a container" is the next Linux deployment paradigm
  • rootless Podman Quadlet: how to run containers with no root privileges and no daemon
  • Tank-OS's layered architecture: how the immutable OS layer and mutable state layer stay separate
  • Transactional system updates: rolling back an entire machine's OS state with one command
  • Credential management: how API keys never touch the filesystem in plaintext
  • Tank-OS vs. NVIDIA NemoClaw: two enterprise AI Agent security approaches compared

Prerequisites

  • Basic understanding of Linux containers (images, OCI spec)
  • Familiarity with Podman or Docker
  • Basic Systemd service management knowledge
  • Familiarity with OpenClaw (AI Agent framework, 40,000+ GitHub Stars)

Project Background

What Is It?

The core problem Tank-OS addresses can be stated in one sentence: AI Agents are powerful enough to be dangerous, and most deployment approaches haven't taken that seriously.

Sally O'Malley's project documentation describes real incidents: OpenClaw Agents in production have deleted emails and accidentally modified user data. When an enterprise deploys dozens or hundreds of Agent-running machines, the traditional "apt install + config files" approach has four fundamental problems:

  1. Credential security: API Keys live in config files, readable by other processes or accidentally leaked
  2. Update consistency: Different OS versions and library versions across machines cause inconsistent Agent behavior
  3. Failure isolation: A crashed or compromised Agent process can affect the host OS
  4. Fleet management: No unified mechanism to push updates across the whole fleet

Tank-OS uses a bootc + rootless Podman Quadlet stack to systematically address all four.

About the Author

Sally Ann O'Malley (GitHub: sallyom; LobsterTrap is the account she created for this project)

  • Role: Red Hat Principal Software Engineer, Emerging Technologies, Office of the CTO
  • OpenClaw connection: Core maintainer of OpenClaw, collaborating with founder Peter Steinberger on enterprise use cases and Red Hat Linux ecosystem integration
  • Background: Long-time contributor to Red Hat container technologies; deep user and contributor to Podman, bootc, and related Red Hat open-source projects
  • Origin of Tank-OS: Built in a single weekend. O'Malley anticipated "what happens when OpenClaw enters the enterprise" and wanted a reference architecture ready

Red Hat's official blog also published a companion article: "Building a hardened, image-based foundation for AI agents" — signaling this is not just a personal project but Red Hat's formal exploration of AI Agent infrastructure.

Project Stats

  • ⭐ GitHub Stars: 104
  • 🍴 Forks: 11
  • 🔤 Language: Shell (81.7%), Dockerfile (18.3%)
  • 📄 License: MIT
  • 📦 Pre-built image: quay.io/sallyom/tank-os:latest (amd64 / arm64)
  • 📰 Press: TechCrunch (2026-04-28), Decrypt, Yahoo Tech
  • 📅 Released: April 2026

Key Features

The Core Architecture Shift

Tank-OS elevates "how to safely run an AI Agent" from a software configuration problem to an operating system architecture problem.

Traditional deployment:

Host OS (mutable)
  └── Userspace (mutable)
        └── OpenClaw process (can access host filesystem)
              └── API Keys (plaintext config file)
Enter fullscreen mode Exit fullscreen mode

Tank-OS deployment:

bootc image (immutable OS layer, read-only)
  └── openclaw user (no root privileges)
        └── rootless Podman Quadlet (container, no daemon)
              └── OpenClaw process (isolated from host)
                    └── API Keys (Podman secret store, encrypted)
  └── ~/.openclaw/ (mutable state layer, persisted but isolated from OS)
Enter fullscreen mode Exit fullscreen mode

Use Cases

1. Enterprise AI Agent fleet management
Dozens or hundreds of machines running the same OpenClaw version, kept consistent via unified image update — no "configuration drift" causing behavioral differences across machines.

2. Dev/prod environment parity
Developers run the exact same Tank-OS image locally in a VM, eliminating "works on my machine" problems.

3. Read-only OS security hardening
Even if an Agent process is compromised or has a bug, it cannot modify the host OS filesystem — the OS layer is immutable.

4. Cloud instances and edge devices
SSH key injection via cloud-init enables fast Agent instance startup on AWS EC2, GCP VMs, or Raspberry Pi devices.

5. Fast version rollback
One command to roll back to the previous OS + Agent version — no manual uninstall/reinstall of dependencies.

Quick Start

Option 1: Use the pre-built image (recommended)

No local compilation needed. Use Podman Desktop's BootC extension or bootc-image-builder to generate a virtual disk:

# Create output directory
mkdir -p out-tank-os

# (Optional) Prepare SSH key injection config
cat > config.json << 'EOF'
{
  "blueprint": {
    "customizations": {
      "user": [
        {
          "name": "openclaw",
          "groups": ["wheel"],
          "key": "ssh-ed25519 AAAA...your-public-key..."
        }
      ]
    }
  }
}
EOF

# Build QCOW2 disk image (requires rootful Podman)
sudo podman run \
  --rm \
  --privileged \
  --pull=newer \
  -v ./config.json:/config.json \
  -v ./out-tank-os:/output \
  -v /var/lib/containers/storage:/var/lib/containers/storage \
  quay.io/centos-bootc/bootc-image-builder:latest \
  --type qcow2 \
  --config /config.json \
  quay.io/sallyom/tank-os:latest

# Result: out-tank-os/qcow2/disk.qcow2
Enter fullscreen mode Exit fullscreen mode

SSH in and interact with OpenClaw:

# Log in as openclaw user (not root)
ssh openclaw@<vm-ip>

# Check Agent status
openclaw gateway status --deep

# Run health check
openclaw doctor

# Get Dashboard URL
openclaw dashboard --no-open

# List connected devices
openclaw devices list
Enter fullscreen mode Exit fullscreen mode

Update OS + Agent (transactional):

# First update: switch to latest image
sudo bootc switch --apply quay.io/sallyom/tank-os:latest

# Subsequent updates: pull new version and restart
sudo bootc upgrade --apply
Enter fullscreen mode Exit fullscreen mode

Core Features

  1. Immutable OS — Root filesystem mounted read-only; Agent processes and userspace programs cannot modify system files. Configuration drift is structurally impossible.

  2. rootless Podman Quadlet — OpenClaw runs in a Podman container with no root privileges, lifecycle managed by Systemd Quadlet units. Compromised container process cannot escalate to host OS.

  3. Transactional updates — bootc's atomic update mechanism: updates either succeed completely or roll back completely. No "half-updated broken machine" state.

  4. Secure credential management — API Keys stored in rootless Podman's secret store (encrypted), never appearing in plaintext config files or environment variables. SSH keys injected via cloud-init at first boot, not baked into the image.

  5. Multi-instance support — Single machine can run multiple OpenClaw instances (e.g., one for work, one for research), fully isolated with separate containers, ports, data directories, and secret namespaces.

  6. Multi-architecture — Pre-built images support both linux/amd64 and linux/arm64, covering x86 servers and Apple Silicon/ARM devices.

  7. cloud-init native integration — Standard cloud-init support for AWS, GCP, Azure instance initialization.

How It Compares

Dimension Tank-OS Bare metal OpenClaw Docker compose NVIDIA NemoClaw
OS immutability Complete (bootc read-only root) None None (host OS mutable) None
Container privileges rootless (no daemon) No containers Root Docker daemon K3s + Docker daemon
Transactional updates Atomic rollback Not supported Not supported Not supported
Credential storage Podman secret store Plaintext files Docker secrets or plaintext L7 proxy injection
Fleet management Suitable (unified image) Difficult Limited Suitable (more complex)
Deployment complexity Low (single image boot) Low Medium High (K3s cluster)
Security policy granularity Medium (OS-level isolation) Low Low High (multi-layer policy engine)

Deep Dive

Core Technology 1: bootc (Boot Container)

bootc is a Red Hat-led open-source project that implements a counterintuitive idea: manage the entire operating system as an OCI container image.

Traditional Linux update model:

System running
  → apt upgrade / dnf update (replacing files one by one)
  → partial success → system in intermediate state → rollback is hard
Enter fullscreen mode Exit fullscreen mode

bootc update model:

Currently running: image v1.0 (read-only mounted)
  → bootc switch quay.io/sallyom/tank-os:v1.1
  → Download new image layers, write to separate partition
  → Reboot → atomic switch to v1.1
  → Error → bootc rollback → back to v1.0
Enter fullscreen mode Exit fullscreen mode

This is analogous to the A/B partition mechanism in phone OTA updates: new version writes to standby partition, reboots to switch, keeps original partition if something fails.

Tank-OS's bootc/Containerfile builds from quay.io/fedora/fedora-bootc:latest, Fedora's official bootc base image:

FROM quay.io/fedora/fedora-bootc:latest

# Install required packages
RUN dnf install -y \
    podman \
    openssh-server \
    cloud-init \
    python3 \
    shadow-utils \
    sudo \
    vim

# Create openclaw user (UID/GID 1000)
# Configure subuid/subgid range (100000-165535) for rootless Podman
RUN useradd -m -u 1000 -g 1000 openclaw && \
    usermod -aG wheel openclaw && \
    echo "100000:65536" > /etc/subuid && \
    echo "100000:65536" > /etc/subgid

# Enable cloud-init service family
RUN systemctl enable \
    cloud-init-local.service \
    cloud-init-network.service \
    cloud-init.service \
    cloud-config.service \
    cloud-final.service \
    sshd.service

# Inject tank-os scripts and Quadlet units
COPY bootc/ /
Enter fullscreen mode Exit fullscreen mode

Core Technology 2: rootless Podman Quadlet

Why Podman instead of Docker?

Podman is Red Hat's daemonless container tool. The core security advantages:

  • No central daemon: Docker requires a root-privileged dockerd running continuously; Podman forks container processes directly, no daemon required
  • rootless mode: Regular users can run full containers; container "root" is an unprivileged user on the host
  • User namespace isolation: Via subuid/subgid mapping — container UID 1000 maps to host UID 101000

OpenClaw in Tank-OS runs as the openclaw user (UID 1000). Even if the OpenClaw process is exploited:

  1. It cannot access host OS system files (OS layer is read-only)
  2. It cannot escalate to root (rootless Podman namespace isolation)
  3. It cannot affect other users' data

What is Quadlet?

Quadlet (introduced in Podman 4.4+) lets you declare container runtime configuration using Systemd unit file syntax, with Systemd managing the container lifecycle directly:

# /etc/containers/systemd/users/1000/openclaw.container
[Unit]
Description=OpenClaw AI Agent Service
After=network-online.target

[Container]
Image=ghcr.io/openclaw/openclaw:latest
ContainerName=openclaw
UserNS=keep-id:uid=1000,gid=1000
Volume=%h/.openclaw:/home/openclaw/.openclaw:z
Secret=openclaw-api-key,type=env,target=OPENCLAW_API_KEY

[Service]
Restart=always

[Install]
WantedBy=default.target
Enter fullscreen mode Exit fullscreen mode

Systemd reads this file and automatically generates the corresponding .service unit — pulling the image, creating the container, managing restart policy. OpenClaw becomes a standard Systemd service: systemctl --user status openclaw for status, journalctl --user -u openclaw for logs.

State Layer Architecture

Tank-OS strictly separates immutable and mutable layers:

Immutable layer (OS image, read-only)
├── /usr/           ← System software
├── /etc/           ← System config (baked into image)
├── /opt/tank-os/   ← tank-os scripts and tools
└── Quadlet units   ← Container declaration files

Mutable layer (runtime state, persisted)
├── ~/.openclaw/            ← OpenClaw session state, plugins, history
├── ~/.config/containers/   ← Podman user config
└── Podman secret store     ← Encrypted API keys
Enter fullscreen mode Exit fullscreen mode

When you run bootc upgrade, only the immutable layer is replaced. OpenClaw's state, conversation history, and API keys in the mutable layer are fully preserved. Upgrading the Agent version while keeping all session history and configuration intact.

Credential Flow

The API key injection flow is a security highlight:

1. First boot: cloud-init executes
   ↓
2. Reads user-data (from cloud provider metadata or local config)
   ↓
3. Runs tank-os bootstrap script
   ↓
4. Writes API key into rootless Podman secret store (encrypted)
   ↓
5. Quadlet unit starts OpenClaw container
   ↓
6. Secret= directive injects secret as environment variable into container
   ↓
7. API key never appears in plaintext on the filesystem or in process environment
Enter fullscreen mode Exit fullscreen mode

Compared to a traditional .env file approach: Podman's secret store uses system keychain encryption; ordinary file reads cannot retrieve the contents.

Tank-OS vs. NVIDIA NemoClaw

The most common technical question after Tank-OS launched: "How does this differ from NVIDIA's NemoClaw reference architecture?"

Dimension Tank-OS NVIDIA NemoClaw
Core technology Fedora bootc + rootless Podman Docker + embedded K3s cluster
OS immutability Complete (bootc partition-level) None
Credential isolation Podman secret store L7 proxy injection (key never touches Agent filesystem)
Security policy OS-level isolation seccomp + Landlock + network namespace multi-layer policy
Update mechanism Atomic image replacement (rollback capable) Standard container update
Target scenario Enterprise fleet (standard IT ops toolchain) Research environments, fine-grained policy control
Deployment complexity Low (single image, standard VM tools) High (K3s cluster operations)

Conclusion: Tank-OS fits "treat AI Agent as standard IT infrastructure" enterprise scenarios. NemoClaw fits "need multi-layer fine-grained security policy" research or high-security-requirement environments.

Resources

Official

Press & Related

Summary

Key Takeaways

  1. Clear problem definition: Tank-OS targets real enterprise AI Agent deployment security problems — credential leakage, configuration drift, update consistency, failure isolation — not technology stack-stacking for its own sake
  2. Mature technology choices: bootc + rootless Podman + Systemd Quadlet are all Red Hat production-grade tools with full enterprise support and documentation. Not experimental.
  3. The layered design is the insight: Strict separation of immutable OS layer and mutable state layer is what lets Tank-OS simultaneously achieve "security isolation" and "state persistence." This design principle is worth borrowing for any stateful AI Agent deployment scenario.
  4. Transactional updates change the ops paradigm: A single bootc upgrade command updates the entire machine's OS + Agent, with atomic rollback on failure. For IT teams managing AI Agent fleets, this is a qualitative change.
  5. The weekend project lesson: Tank-OS uses ~500 lines of Shell scripts and Dockerfile to solve a real enterprise pain point. A reminder that many apparently complex infrastructure problems only need existing mature tools composed correctly.

Who Should Use This

  • Enterprise IT administrators: Deploying OpenClaw Agent fleets internally with unified management and security compliance requirements
  • DevOps/SRE engineers: Interested in bootc and immutable infrastructure; looking for an AI Agent deployment reference architecture
  • Red Hat/Fedora users: Want to integrate AI Agents seamlessly into existing RHEL infrastructure
  • AI security researchers: Studying AI Agent isolation, credential management, and enterprise security hardening

Where to Start

Start by understanding bootc's core concepts (Fedora bootc docs), then read Tank-OS's bootc/Containerfile and docs/build.md, following the documentation to run an instance in a local VM. The whole process takes under 2 hours, but you'll come away with a very concrete feel for the "OS as container" paradigm.

If you're already deploying OpenClaw in an enterprise environment, Tank-OS can be used almost directly as a production reference architecture. Sally O'Malley is OpenClaw's enterprise maintainer — every design decision in this project comes from real enterprise requirements.

A Question Worth Sitting With

Tank-OS's central premise is worth sitting with: when should AI Agent security be solved at the OS architecture level rather than the application configuration level?

Most current AI Agent deployments treat security as a software problem — use the right API key management library, set the right file permissions, configure the right network rules. Tank-OS argues that once an Agent is powerful enough to delete emails, modify databases, and call external APIs, software configuration isn't enough. The OS itself needs to be part of the security model.

That's a significant architectural claim. The fact that it came from a Red Hat engineer on a weekend — not a multi-year research project — suggests the underlying tools were already ready. Someone just needed to put them together.

Visit my personal site for more useful knowledge and interesting products

Top comments (0)