DEV Community: pratikbin

S3 as a disk for agents: geesefs vs s3fs, measured

pratikbin — Thu, 04 Jun 2026 12:00:00 +0000

Most infra today is built for a human to click through. That's ending. The thing reading your docs, hitting your API, and filling your disk next year is an agent, not a person. And an agent doesn't have a laptop. It has a Linux box and a network.

So when you design for agents, two assumptions hold. They run on Linux. And the only storage layer that's actually everywhere — every cloud, every region, every cheap S3-compatible box in a closet — is S3. Not a managed filesystem, not a vendor volume. S3 won by being the lowest common denominator that the machine can reach from anywhere.

Which leads to the question we kept circling. What if S3 isn't an SDK call, but a disk? Mounted into the VM, inside our VPC, so the agent writes to /mnt/data and the bytes land in object storage with no code that knows S3 exists.

That matters because of how agents actually write code. They call open(), pandas.read_csv("/mnt/..."), they shell out to rsync and git. They do not reach for boto3 unless you force them. Mount the bucket and every POSIX tool speaks S3 for free. That's the point.

Two projects do this seriously over FUSE: s3fs-fuse and geesefs. We put both on the same VM, same bucket, same workload, and measured.

The setup

A 4-vCPU / 4 GB Linux VM, Ubuntu 24.04. Tigris as the S3 backend. Latest of each engine: geesefs v0.43.7, s3fs v1.97. apt still ships s3fs 1.93, so we built 1.97 from source. Same 128 MB sequential file, same 500 × 8 KiB small files, five runs, cold and warm. No on-disk cache on either side, so a remount is a true cold start.

The mount commands — the part everyone gets wrong first:

# s3fs
s3fs my-bucket /mnt/s -o url=https://t3.storage.dev \
  -o use_path_request_style -o passwd_file=/etc/passwd-s3fs

# geesefs  (--list-type 2 is mandatory on non-Yandex S3; forget it and listing breaks)
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... \
  geesefs --endpoint https://t3.storage.dev --list-type 2 my-bucket /mnt/g

Mount a sub-prefix with s3fs and it 404s unless that prefix already exists as a directory marker — you need -o compat_dir. geesefs mounts a prefix natively. First gotcha, before a single byte moves.

The numbers

Sequential throughput is a tie. ~45–50 MB/s on write and on cold read, both engines. That number is the wire to S3, not the filesystem. If your benchmark is dd if=/dev/zero of=/mnt/big, you'll conclude the two are identical, and you'll be wrong about everything that matters.

Everything an agent actually does is not close:

workload	s3fs	geesefs	gap
small-file write	2.6–7 files/s	563 files/s	80–217×
small-file warm read	5–15 files/s	1188 files/s	78–233×
sequential warm read	61 MB/s	883 MB/s	14×
mount	336 ms	139 ms	2×

The ranges are real: we ran two s3fs versions (1.93 from apt, 1.97 from source) and the gap held in both directions. Two mechanisms explain almost all of it.

Why

Small-file writes. s3fs does one synchronous PUT per file and waits for it. geesefs pipelines uploads in the background and returns. 500 tiny files is 500 serial round trips for s3fs; geesefs overlaps them. That's the 217×.

Directory listing. geesefs fills file size and mtime straight from the LIST response it already made, so ls -la stays cheap and consistent — single-digit milliseconds warm. s3fs is the variable one: depending on version and config it either reads attributes from the listing or does a getattr per file, and when it does the latter, an ls -la over a few hundred files becomes a serial HEAD storm. We watched the same listing swing from sub-second on one s3fs build to ~11 seconds on another. Either way this is the operation that bites agents, because agents stat everything: os.listdir + os.stat, git status, rsync's whole model. A names-only ls is cheap. The moment a tool wants metadata, s3fs is the one that might fall off a cliff — and you won't see it in a dd test.

The warm reads — 14× sequential, 233× small — are the kernel page cache. geesefs lets Linux cache file pages; s3fs disables that by default and re-fetches from S3 on every read. Be precise about what this is: it is not "geesefs uses less RAM." Its cached data lives in the page cache, which doesn't even show up in the process's RSS. geesefs spends memory to make re-reads free. s3fs spends almost none and pays the network every time. For an agent that reads the same dataset in a loop, free re-reads is the whole game.

The trade-off

geesefs is not a free win, and pretending otherwise is how you get paged at 3 AM.

Its cold directory listing is noisier — we saw a 0.3–5.7 s spread on the same ls. It buffers writes aggressively, so you sync -f before you tear the mount down or you lose the tail — bit us once, now it's reflex. It does not do concurrent multi-writer to one file — that's last-writer-wins with stale-read risk through each box's cache, so keep it to one writer or disjoint prefixes. And the async deletes that make it fast mean a rm -rf lingers in the bucket for a while after the call returns.

s3fs earns its place when you want a single synchronous writer with no surprises, or you're streaming a few large objects and never re-reading them. Boring, predictable, slow on metadata. Sometimes boring is the requirement.

The takeaway

If the consumer is an agent on Linux — many small files, repeated reads, metadata-heavy tools — mount with geesefs and give it page-cache headroom. The 80–230× isn't a marketing number; it's synchronous-per-file versus pipelined, and re-fetch versus page cache — measured across two s3fs versions. If you only ever stream big blobs once and want the simplest possible failure mode, s3fs is fine.

S3 is the storage layer that's actually everywhere, and the machine consuming it doesn't care about your SDK. So give it a disk. Just pick the FUSE layer that matches how the thing on top does I/O — and measure the workload it'll really run, not dd.

Migrating from Claude Code to Codex is not a search-replace

pratikbin — Tue, 19 May 2026 15:25:37 +0000

Migrating from Claude Code to Codex looks simple until you hit the stuff around the model.

CLAUDE.md is easy.

Hooks are not.
MCP servers are not.
Plugins are not.
Permissions are very not.
Sessions are where you find out whether you migrated the workflow or just copied a prompt.

I turned this into a migration skill because agent setups are becoming infra. They need boring checklists, not vibes.

The mistake

The naive migration is this:

CLAUDE.md -> AGENTS.md
.mcp.json -> config.toml
Done.

That only moves visible files.

A real Claude Code setup can include:

.claude/settings.json
.claude/commands/*.md
.claude/agents/*.md
.mcp.json
hooks inside settings
.claude-plugin/plugin.json
~/.claude/projects/* sessions

Codex has its own shape: AGENTS.md, .codex/config.toml, .codex/hooks.json, skills, plugins, MCP config, sessions.

The hard part is preserving behavior.

Inventory first

Before converting anything, print the map:

- Instructions: found
- Hooks: count by event
- CLI MCP servers: count by scope
- Desktop MCP servers: excluded
- Plugins: installed/local
- Commands: custom slash commands
- Subagents: project/user agents
- Permissions: allow/ask/deny rules
- Sessions: recent handoff candidates
- Secrets: names only, values not read

That last line matters.

Never read secret values as part of migration. Read names. Ask the user to reset values in the target tool.

Migration scripts that slurp env vars are how tokens land in git history.

Hooks are policy

Hooks usually encode the safety model.

In Claude Code, hooks can run on PreToolUse, PostToolUse, PermissionRequest, SessionStart, Stop, and more. Some hooks are commands. Some are HTTP. Some use MCP tools. Some are prompt or agent hooks.

Codex hooks are useful, but not identical.

So do not copy hook JSON and hope.

Start with a table:

PreToolUse Bash       -> Codex PreToolUse Bash
PreToolUse Edit/Write -> Edit|Write or apply_patch
MCP tool hooks        -> mcp__server__tool matcher
SessionStart          -> startup/resume/clear mapping
Stop                  -> check matcher semantics

Then flag anything that needs redesign: Notification, SessionEnd, PreCompact, prompt hooks, agent hooks, HTTP hooks.

Partial migration is okay. Silent migration is not.

MCP: not your whole desktop

Claude Desktop MCP and Claude Code CLI MCP are different migration targets.

For a CLI workflow, migrate CLI servers only:

claude mcp list
claude mcp get <name>

Plus project .mcp.json.

Do not accidentally drag every desktop connector into a coding agent. Different threat model. Different context cost.

A simple Codex stdio server usually becomes:

[mcp_servers.docs]
command = "npx"
args = ["-y", "@some/docs-mcp"]
env_vars = ["DOCS_TOKEN"]

Forward env vars when possible. Do not hardcode tokens.

Plugins are bundles

A Claude plugin is not just a skill.

It can include skills, commands, subagents, hooks, MCP servers, scripts, output styles, themes, monitors, even LSP config.

So the migration report should look like:

Plugin: infra-tools
- skills: direct
- commands: skill or AGENTS recipe
- agents: subagent or skill
- hooks: partial, review events
- MCP: plugin or project config
- scripts: keep
- styles/themes/LSP: manual

That forces the useful question: what did the plugin actually do?

If it is instructions, make a skill.
If it is automation, carry the scripts.
If it is policy, test the hooks.

Permissions are the dangerous part

Do not loosen permissions silently.

Map intent, not syntax:

Claude deny rules     -> Codex filesystem/rules/protected paths
Claude allow prefixes -> Codex allow rules
Claude ask rules      -> prompt rules or on-request approval
acceptEdits           -> workspace-write + approval policy
bypassPermissions     -> do not map without explicit approval

My conservative default:

approval_policy = "on-request"
sandbox_mode = "workspace-write"

Then add known-safe rules.

Not the other way around.

Zero prompts is not the goal. Correct blast radius is.

Sessions need handoff

Do not raw-copy session files.

Use handoff:

npx continues
continues list --source claude --json
continues inspect <session-id> --preset full --write-md handoff.md
continues resume <session-id> --in codex --preset standard
continues resume <session-id> --in codex --debug-prompt

bunx continues is fine if that is your runner. Verify it in your env.

A good handoff says: objective, current repo state, files changed, commands run, decisions made, failures, pending tasks, next action.

Long agent sessions rot. Handoff files keep the work inspectable.

Use the skill

I packaged the whole checklist as a skill so you run it instead of remembering it.

npx skills add https://github.com/nodeops-app/skills --skill claude-code-to-codex -y

It does the inventory, builds the conversion tables, flags partial and manual migrations, and ends with a written report you can diff and roll back.

The takeaway

Migrating agent CLIs is not moving a prompt.

It is moving the operating system around the prompt:

instructions
hooks
permissions
MCP servers
plugins
commands
subagents
sessions
team policy

Inventory first.
Convert behavior, not files.
Keep rollback.
Verify with real tool calls.

Boring on purpose.

Run NextDNS and Tailscale together without breaking MagicDNS

pratikbin — Wed, 06 May 2026 09:56:09 +0000

If you ssh into a tailnet host by name and your terminal answers Unknown host, NextDNS is probably eating your queries before Tailscale sees them.

Here's why, and a one-command fix.

The conflict

NextDNS ships an Apple Configuration Profile (apple.nextdns.io) that installs a system-wide DNS-over-HTTPS handler. The profile is tidy, signed, and ten-second setup — but it captures every query, including ones meant for your tailnet.

Tailscale's MagicDNS lives at 100.100.100.100. Names like mybox.tailnet-name.ts.net only resolve there. NextDNS doesn't know your tailnet, so it asks the public DNS root, gets NXDOMAIN, and your shell gives up.

The usual macOS escape hatch — dropping a file in /etc/resolver/ — looks like it works:

echo "nameserver 100.100.100.100" | sudo tee /etc/resolver/your-tailnet.ts.net

scutil --dns even shows the new resolver. But Configuration Profiles outrank /etc/resolver, so the file is ignored. Silent failure.

The fix

Replace the profile with NextDNS's CLI client. The CLI supports per-domain forwarders. Same profile, same blocklists, same dashboard — plus split DNS.

1. Remove the Apple profile

System Settings → General → Device Management → NextDNS → Remove

2. Install the CLI

brew install nextdns/tap/nextdns

3. Install with a forwarder for your tailnet

sudo nextdns install \
  -profile YOUR_PROFILE_ID \
  -forwarder 'your-tailnet.ts.net=100.100.100.100' \
  -forwarder 'ts.net=100.100.100.100' \
  -report-client-info

The format is DOMAIN=SERVER, not the dnsmasq-style /DOMAIN/SERVER — the CLI rejects slashes with a confusing "not a valid IP address" error.

The second forwarder catches MagicDNS short names and any other *.ts.net host. Drop it if you only want one tailnet.

4. Verify

dig mybox.your-tailnet.ts.net    # → 100.x.y.z (Tailscale)
dig example.com                   # → public IP via NextDNS
ping mybox.your-tailnet.ts.net    # works in your shell, not just dig

Last check matters. dig ignores the resolver chain; ping, curl, and your apps don't. If both pass, you're done.

Why this works

The NextDNS daemon listens on 127.0.0.1:53 and becomes the system resolver. For each query it checks forwarders first, then falls back to the NextDNS DoH endpoint. Tailscale's 100.100.100.100 only needs to be reachable, which it is whenever tailscaled is running.

NextDNS keeps filtering, analytics, and parental controls. Tailscale keeps MagicDNS. No fights.

The takeaway

System-wide DNS-over-HTTPS profiles are convenient until you need split DNS. If you run a VPN with its own resolver — Tailscale, ZeroTier, WireGuard with dns= — switch to a daemon that supports forwarders. Five minutes, no recurring pain.

Your product has an API. It still doesn't speak agent

pratikbin — Tue, 21 Apr 2026 11:30:00 +0000

Most developer tools are built for two audiences: humans and software. Humans get docs and dashboards. Software gets APIs, SDKs, and CLIs. Agents sit awkwardly in the middle. They can read, they can call APIs, they can follow instructions, but they still can't answer a basic question when they land on your domain: what does this product actually know how to help me do? Ready to copy prompt at the end

I think that's a real gap. We just closed it on CreateOS, and more companies should do the same.

Agents still need to be told everything

If Claude Code or OpenCode needs to deploy an app, it doesn't automatically know your platform exists. You can put instructions in CLAUDE.md; we do that too. But that only helps inside a specific repo. An MCP server helps, but only after someone installs it. llms.txt is useful, but it's descriptive. It tells an agent what your product is, not how to use it.

So today, most agents are still operating on tribal knowledge and prompt glue. That's not a great long-term interface.

The Agent Skills Discovery RFC is a clean fix for this. Think robots.txt, but for product capabilities.

What we shipped

Starting today, https://createos.nodeops.network/.well-known/agent-skills/index.json returns a machine-readable index of every skill our platform offers:

{
  "$schema": "<https://schemas.agentskills.io/discovery/0.2.0/schema.json>",
  "skills": [
    {
      "name": "createos",
      "type": "archive",
      "description": "Deploy ANYTHING to production on CreateOS cloud platform...",
      "url": "/.well-known/agent-skills/createos.tar.gz",
      "digest": "sha256:740b1bf8c464aa6148a3eb9d666f2542..."
    },
    {
      "name": "nodeops-auth",
      "type": "skill-md",
      "description": "Add NodeOps PKCE OAuth authentication to a Next.js app...",
      "url": "/.well-known/agent-skills/nodeops-auth/SKILL.md",
      "digest": "sha256:ca197a7ca9fbc5561a6b71c02ea3652..."
    }
  ]
}

An agent fetches one URL and gets the shortlist. Roughly 100 tokens per skill. No repo cloning, no scraping docs, no custom integration work just to understand what the platform can do.

When the task matches something specific like "deploy my app" or "add NodeOps auth," the agent pulls only that skill, verifies the sha256 digest, and moves on.

Progressive disclosure is the part they got right

The smartest part of the RFC is that it doesn't force you to dump everything into context up front.

Level	What	When	Cost
1	`name` + `description` from `index.json`	Startup / probing	~100 tokens/skill
2	Full `SKILL.md` body	When skill is activated	<5k tokens
3	Scripts, references, configs	On demand	As needed

Our createos skill bundles deployment scripts, API references, and config templates at about 28 KB. An agent working on a simple auth task never has to touch any of that. It grabs nodeops-auth/SKILL.md at about 6 KB and moves on. That's the whole point. Context is expensive. Don't waste it.

How we built it

We kept the implementation boring on purpose: one build-time script and static files.

No route handlers. No cold starts. No database read just to answer a well-known URL.

Build pipeline (scripts/build-agent-skills.mts, about 200 lines):

git clone --depth 1 the skills repo into .agent-skills-src/
For each skill directory, read SKILL.md, parse frontmatter (name, description)
If the skill is just a SKILL.md → copy it. If it has supporting files → tar.gz it
Compute sha256 digests, write index.json

It runs as a prebuild hook before every next build and build:cloudflare, so the published skills stay in sync with upstream.

Serving is just static files in public/.well-known/agent-skills/ behind Cloudflare Pages. Three header rules in next.config.ts handle the RFC requirements:

index.json → application/json; charset=utf-8 + CORS + max-age=60, s-maxage=300
.md → text/markdown; charset=utf-8 + CORS + max-age=300, s-maxage=3600
.tar.gz → application/gzip + CORS + max-age=300, s-maxage=3600

We keep the index TTL short so new skills show up quickly. Artifacts get a longer TTL because they're digest-pinned. If the sha256 matches, the file is still the file.

Serving cost is basically zero because it's static content on a CDN.

Why more companies should publish this

If you run a platform, an API, or a developer tool, you should seriously consider publishing /.well-known/agent-skills/.

First, it's cheap. A SKILL.md, some frontmatter, a small build step, and a few headers. We did the first version in an afternoon.

Second, it's standard. You're not inventing another one-off agent integration. You're publishing something any RFC-compliant client can understand.

Third, it composes well. Our createos skill teaches an agent how to deploy. Our nodeops-auth skill teaches it how to wire auth. Another vendor can publish observability or billing or incident-response skills the same way. The agent can mix and match without every platform building a separate integration for every other platform.

And finally, it's probably where this goes anyway. Claude Code doesn't auto-probe /.well-known/agent-skills/ yet. OpenCode doesn't either. Fine. robots.txt was useful before every crawler agreed on it too. Standards usually show up before the tooling catches up.

The trade-off

This is still pull, not push. Agents need to know your domain exists before they can ask for /.well-known/agent-skills/index.json.

So no, this doesn't solve discovery all by itself. There's no master directory yet. In practice, the bridge is simple: mention the URL in your docs, your AGENTS.md, your MCP server docs, or your llms.txt. That's enough to make it usable today while clients catch up.

How to add it to your product

Write your skills. Create a directory per skill with a SKILL.md. The frontmatter needs name and description. The body is instructions for the agent. That's the whole format.
Package and serve. Solo skills ship as plain SKILL.md. Skills with supporting files (scripts, references) ship as .tar.gz. Compute sha256 digests. Write index.json.
Add headers. Content-Type, Access-Control-Allow-Origin: *, and tiered Cache-Control. Three URL patterns.
Publish. Put it behind your domain. https://your-domain.com/.well-known/agent-skills/index.json. Done.

The RFC is here. The schema is here.

How to use it

# this will fail
npx skills add https://createos.nodeops.network -y

there is active issue on skills to add support for RFC v0.2.0

Or just hand this to your coding agent

Don't feel like reading a build script? Fill in the four inputs at the top, paste this into Claude Code, Cursor, or Copilot inside the target repo, and let it wire everything up.

Add [Agent Skills Discovery RFC v0.2.0](https://github.com/cloudflare/agent-skills-discovery-rfc) to this site so agents auto-discover our skills at `/.well-known/agent-skills/index.json`.

**Inputs** (fill in before pasting):
- **Skills source**: `<git URL of repo holding skills>` on branch `<ref>`, skills live under path `<skills/skills>` (each subdir = one skill with a `SKILL.md` containing `name` + `description` frontmatter).
- **Host framework**: `<Next.js app router | Astro | SvelteKit | plain static | Express | other>`.
- **Public dir**: `<public | static | dist | ...>`.
- **Production origin**: `<https://example.com>`.

**Deliverables** (do all):

1. **Build script** (zero new deps; Node built-ins + system `git` + system `tar`):
   - Clone the skills repo shallow (`git clone --depth 1 --branch <ref>`) into a gitignored `.agent-skills-src/`. Re-runs do `git fetch && git reset --hard FETCH_HEAD`.
   - Iterate each skill dir under the configured path. Read `SKILL.md`, parse YAML frontmatter, require `name` + `description`. Validate `name` against `^[a-z0-9]+(-[a-z0-9]+)*$`, length 1–64. Fail build on violation.
   - If dir contains only `SKILL.md` → copy to `<public>/.well-known/agent-skills/<name>/SKILL.md`, `type: "skill-md"`.
   - Else → `tar -czf <public>/.well-known/agent-skills/<name>.tar.gz --sort=name --mtime=@<upstream-commit-ts> -C <skill-dir> .`, `type: "archive"`. Fall back without repro flags on BSD tar.
   - Compute `sha256:<hex>` over the served artifact bytes.
   - Write `<public>/.well-known/agent-skills/index.json`:
     ```

json
     { "$schema": "https://schemas.agentskills.io/discovery/0.2.0/schema.json",
       "skills": [{ "name": "...", "type": "skill-md|archive", "description": "...", "url": "/.well-known/agent-skills/...", "digest": "sha256:..." }] }


     ```
   - Sort skills alphabetically by `name` (stable digest of `index.json`).

2. **Wire as prebuild hook** for the host framework (e.g. npm `prebuild` script, Astro integration, CI step) so output regenerates on every deploy.

3. **HTTP response headers** (per framework's config — `next.config.ts` `headers()`, `astro.config` headers adapter, `_headers` for Pages/Netlify, nginx/express middleware, etc.):
   - `/.well-known/agent-skills/index.json` → `Content-Type: application/json; charset=utf-8`, `Access-Control-Allow-Origin: *`, `Cache-Control: public, max-age=60, s-maxage=300`.
   - `/.well-known/agent-skills/*.md` → `Content-Type: text/markdown; charset=utf-8`, `Access-Control-Allow-Origin: *`, `Cache-Control: public, max-age=300, s-maxage=3600`.
   - `/.well-known/agent-skills/*.tar.gz` → `Content-Type: application/gzip`, `Access-Control-Allow-Origin: *`, `Cache-Control: public, max-age=300, s-maxage=3600`.

4. **Gitignore**: `.agent-skills-src/` and `<public>/.well-known/agent-skills/`.

5. **Verify locally**:
   - `curl -sS http://localhost:<port>/.well-known/agent-skills/index.json | jq .` returns valid JSON with `$schema` + `skills[]`.
   - Re-hash every served artifact and confirm match against `digest` in the index.
   - Response headers match section 3.

6. **Post-deploy check**: same three `curl` calls against the production origin.

**Constraints**:
- Do not add runtime dependencies. Hand-parse the two frontmatter fields (`name:` and `description:`) with a regex — full YAML is overkill here.
- Static output only. No server route handler — agents must be able to fetch from edge/CDN with no origin cold start.
- Never commit generated artifacts or the cloned skills repo.
- Skip skills with missing/invalid frontmatter with a warning. Hard-fail on bad `name` format.

**Reference implementation** (adapt, don't copy-paste): https://github.com/cloudflare/agent-skills-discovery-rfc/tree/main/examples

Return a summary of: files created, files modified, verification commands run, and production URL of the new `index.json`.
```

`

Four inputs in, working `/.well-known/agent-skills/` out. No runtime deps, static output, deploy hook wired.

Agents are already reading `llms.txt`. They're already talking to MCP servers. This fills a different gap: a standard way to publish what your product can help an agent do.

If you build developer tools, I think you should publish one.

Agents are already reading `llms.txt`. They're already talking to MCP servers. This fills a different gap: a standard way to publish what your product can help an agent do.

If you build developer tools, I think you should publish one.