DEV Community

Kenneth Phang
Kenneth Phang

Posted on

I Reverse-Engineered My Own Rust CLI — Here is the Full Architecture of a 16-Step Cloud Deployer

#rust #architecture #devops #opensource

Most deployment tools are black boxes. You run a command, cross your fingers, and hope it works.

I wanted something different. I built clawmacdo — a Rust CLI that deploys a full AI assistant stack to DigitalOcean in one command. And today, I am going to crack it open and show you exactly how it works inside.

This is the full architecture, data flow, and design decisions behind a real-world Rust CLI that provisions cloud servers, SSHs into them, installs software, and streams live progress to a web UI.

The 30-Second Pitch

One command:

clawmacdo deploy --do-token=xxx --anthropic-key=xxx
Enter fullscreen mode Exit fullscreen mode

16 automated steps later, you have a fully configured AI assistant running on a DigitalOcean droplet with WhatsApp, Telegram, Claude, GPT, and Gemini — all wired up and ready to go.

But HOW does it actually work?

The Architecture: 5 Layers 

┌─────────────────────────────────────────┐
│  Layer 1: CLI + Web UI Orchestration    │
│  (Clap CLI + Axum web server + SSE)     │
├─────────────────────────────────────────┤
│  Layer 2: Infrastructure APIs           │
│  (DigitalOcean REST + SSH/SCP)          │
├─────────────────────────────────────────┤
│  Layer 3: Provisioning Pipeline         │
│  (6 ordered modules, each idempotent)   │
├─────────────────────────────────────────┤
│  Layer 4: Config + State Persistence    │
│  (DeployRecord JSON + .env + systemd)   │
├─────────────────────────────────────────┤
│  Layer 5: Progress + Error Surfaces     │
│  (Terminal UI + SSE streaming + typed    │
│   errors with actionable diagnostics)   │
└─────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Let me walk through each one.

Layer 1: Dual Interface — CLI and Web UI

clawmacdo works two ways:

CLI mode — for terminal lovers:

clawmacdo deploy --do-token=xxx
clawmacdo backup
clawmacdo status
clawmacdo destroy --name=my-droplet
clawmacdo migrate --source-ip=1.2.3.4
Enter fullscreen mode Exit fullscreen mode

Web UI mode — for everyone else:

clawmacdo serve
Enter fullscreen mode Exit fullscreen mode

This spins up an Axum web server with a full UI. You fill in a form, hit deploy, and watch real-time progress via Server-Sent Events (SSE).

Browser ──POST /api/deploy──▶ Axum
Browser ──GET /api/deploy/{id}/events──▶ SSE stream
                                         │
                              tokio::spawn(deploy task)
                                         │
                              SSH into droplet ──▶ provision
                                         │
                              emit progress events ──▶ Browser
Enter fullscreen mode Exit fullscreen mode

The web deploy runs in a tokio background task with a progress channel. The SSE endpoint reads from that channel and streams events to the browser in real-time. Terminal markers (DEPLOY_COMPLETE / DEPLOY_ERROR) signal the end state.

This pattern — async task + channel + SSE — is surprisingly clean in Rust with tokio + Axum.

Layer 2: Infrastructure as Code (Without the YAML)

No Terraform. No Pulumi. No CloudFormation. Just direct API calls.

DoClient wraps the DigitalOcean REST API via reqwest:

  • Create/delete droplets
  • Manage SSH keys
  • Poll for droplet status
  • Tag-based resource tracking

SSH utilities use ssh2 (libssh2 bindings):

  • Key generation (Ed25519)
  • Remote command execution
  • SCP file transfers
  • Readiness polling (wait for cloud-init + SSH)

Why no Terraform? Because clawmacdo needs to react to each step. It is not declaring desired state — it is orchestrating a sequence where each step depends on the previous one. SSH readiness polling, cloud-init sentinel files, health checks — these are imperative, not declarative.

Layer 3: The Provisioning Pipeline (The Heart)

This is where the magic happens. Six ordered modules, each handling one concern:

1. User Provisioning

  • Creates a dedicated openclaw system user
  • Sets up shell environment and sudo scope
  • Enables systemd lingering (so user services survive logout)
  • Migrates restored backup from root to the openclaw user

2. Firewall Hardening

  • Configures fail2ban for brute-force protection
  • Sets up unattended-upgrades for automatic security patches
  • Hardens UFW rules
  • Adds DOCKER-USER iptables isolation rules

3. Docker Setup

  • Writes optimized /etc/docker/daemon.json
  • Adds openclaw user to docker group
  • Restarts Docker daemon

4. Node.js + AI CLIs

  • Configures pnpm global directories
  • Installs Claude Code (Anthropic), Codex (OpenAI), Gemini CLI (Google)
  • Verifies and symlinks all binaries

5. OpenClaw Installation

  • Creates .openclaw directory structure
  • Writes .env with all API keys and messaging config
  • Installs OpenClaw globally via npm
  • Normalizes extension hardlinks

6. Tailscale (Optional)

  • Installs Tailscale for private networking
  • Runs tailscale up with auth key if provided

Each module is idempotent — you can re-run provisioning without breaking things. This matters when debugging partial failures.

Layer 4: State That Actually Makes Sense

Three locations, three purposes:

Local machine (~/.clawmacdo/):

~/.clawmacdo/
├── backups/     # Timestamped .tar.gz archives
├── keys/        # Generated SSH private keys
└── deploys/     # DeployRecord JSON files
Enter fullscreen mode Exit fullscreen mode

DeployRecord captures everything about a deployment:

{
  "id": "abc123",
  "droplet_id": 555085163,
  "hostname": "openclaw-b0c825bf",
  "ip_address": "178.128.222.239",
  "region": "sgp1",
  "size": "s-2vcpu-4gb",
  "ssh_key_path": "~/.clawmacdo/keys/abc123",
  "backup_restored": true,
  "timestamp": "2026-03-01T10:00:00Z"
}
Enter fullscreen mode Exit fullscreen mode

Remote droplet:

  • .env — runtime secrets
  • openclaw.json — channel/model/plugin config
  • systemd user service — gateway lifecycle

Layer 5: Errors That Help You Fix Things

This is where most deployment tools fail. Something breaks and you get Error: deployment failed.

clawmacdo takes a different approach:

  • Typed errors via a custom error enum — every failure mode has a specific type
  • No auto-destroy on failure — if step 12 fails, your droplet survives. You get SSH debug info so you can fix it manually
  • SSH/cloud-init timeouts include diagnostic output — not just "timed out" but actual logs
  • Gateway validation checks both systemd state AND the /health endpoint
  • Web UI streams progress incrementally — you see exactly which step failed

This is a deliberate design choice: a half-deployed server you can debug is better than a destroyed server you cannot.

The Full 16-Step Deploy Flow 

1.  Resolve params + defaults
2.  Generate SSH keypair locally
3.  Upload public key to DigitalOcean
4.  Create droplet with cloud-init
5.  Poll until droplet is active
6.  Wait for SSH + cloud-init sentinel
7.  SCP backup to droplet (if selected)
8.  Restore backup on droplet
9.  Create openclaw user + permissions
10. Harden firewall (UFW + fail2ban)
11. Configure Docker
12. Install Node.js + AI CLIs
13. Install + configure OpenClaw
14. Optional: Tailscale setup
15. Start gateway + health check
16. Configure model failover + save DeployRecord
Enter fullscreen mode Exit fullscreen mode

Each step reports progress. Each step can fail gracefully. Each step builds on the last.

Bonus: Post-Deploy Lifecycle

Deployment is not the end. clawmacdo also handles:

  • whatsapp-repair — fixes WhatsApp channel issues post-deploy (updates OpenClaw, normalizes config, restarts gateway, probes login capability)
  • migrate — SSHs into an existing droplet, backs up remotely, deploys a new one, restores
  • destroy — cleans up everything (droplet + SSH keys on DO + local keys)
  • status — lists all your openclaw-tagged droplets

What I Learned Building This

  1. Rust + tokio + Axum is a killer stack for CLI-with-web-UI tools. The type system catches so many deployment bugs at compile time.

  2. SSH operations are messy. Cloud-init takes variable time, SSH daemons need polling, and libssh2 has quirks. Build robust retry/wait logic.

  3. Never auto-destroy on failure. Leave the server running so people can debug. Print SSH commands they can copy-paste.

  4. Stream progress, do not batch it. Whether terminal or web, people want to see what is happening NOW.

  5. Idempotent provisioning modules save your sanity. When step 11 fails, you want to re-run from step 11, not step 1.

Try It Yourself

The entire codebase is open source:

github.com/kenken64/clawmacdo

Download a pre-built binary from Releases — available for Windows, Linux, and macOS (Apple Silicon).

If this kind of systems architecture content interests you, drop a star on the repo and follow for more deep dives.

The best deployment tool is not the one with the most features — it is the one that tells you exactly what went wrong when things break. 🦞🦀

Top comments (0)