DEV Community: Javier Leandro Arancibia

I let an AI agent write the tests for my access-control layer

Javier Leandro Arancibia — Thu, 25 Jun 2026 21:29:28 +0000

Fourth in the series. I've been giving mago — an autonomous agent team that runs over a GitHub repo on your own LLM key — real tasks on my own repos: a feature in Go, a crash fix in Zig, and a refactor in Python it wasn't allowed to cheat on. This time, the one that actually made me nervous: let it test my access-control layer.

The gap

superbackend is my Node.js backend toolkit. Its RBAC middleware — src/middleware/rbac.js: requireRight, requireModuleAccess, and a basic-auth super-admin bypass — had zero tests. Untested authorization is the scariest kind of untested code: a silent regression there is a security hole, not a bug report.

So I filed an issue: add a real unit-test suite for it. Mock the service so it needs no DB; cover the actual decisions; don't change the behavior.

What it shipped

src/middleware/rbac.test.js — 25 tests, +326 lines, covering each export against fake req/res/next with the rbac service mocked out:

requireRight — calls next() when the right is granted, returns 403 when denied, the super-admin basic-auth path bypasses the check, orgId resolves from params/query/body (and a custom resolver), and a service error is handled rather than crashing the request.
requireModuleAccess — allow/deny for both read and write actions.
isBasicAuthSuperAdmin — the true/false cases.

No change to rbac.js itself — tests only, exactly as asked.

Verified

$ npx jest src/middleware/rbac.test.js
Tests: 25 passed, 25 total

And the full suite stayed green — the PR is purely additive (+326/-0, one new file), introducing zero new failures. Merged. My access-control layer now has a safety net it didn't have this morning.

Four repos, four stacks, four kinds of work

Stack	Task
Go	a feature
Zig	a crash fix
Python	a refactor (without gutting the tests)
Node	test coverage for security-critical RBAC

Same loop every time: file an issue, the agent implements it on your own key, runs the repo's own tests, and opens a PR you review — verified, not blindly merged. Not pinned to a language, a framework, or a kind of task.

The honest framing hasn't changed: it's a reliable autonomous dev shop for well-scoped, verifiable work — features, fixes, refactors, and the tests you keep meaning to write. Not "build me a startup."

Try it

CLI-only, BYOK (your Claude Code or tau key — it never resells completions), €20/mo. First 10 founding operators free during the beta, with a direct line to me:

👉 https://mago.intrane.fr

curl -fsSL https://mago.intrane.fr/install.sh | sh
mago register

An AI agent fixed my red Python test suite — without gutting the tests

Javier Leandro Arancibia — Thu, 25 Jun 2026 21:16:17 +0000

This is the third in a short series. I've been pointing mago — an autonomous agent team that runs over a GitHub repo on your own LLM key — at my own repos and giving it real tasks. So far: a feature in Go and a crash fix in Zig. This time it's Python, and the question every skeptic asks: won't it just hack the tests to make them pass?

The setup

boilerplate-cli-ui-python is my agent-first Python CLI boilerplate. Its test suite was red — 4 failing tests:

FAILED tests/test_config.py::test_config_defaults - assert False == True
FAILED tests/test_output.py::test_output_formatter_json_mode - SystemExit: 0
FAILED tests/test_output.py::test_output_formatter_human_mode - SystemExit: 0
FAILED tests/test_output.py::test_output_error_json - SystemExit: 85

I filed an issue with one explicit instruction: fix the root cause, don't gut the tests.

What it actually did

It diagnosed two real bugs:

A config default mismatch (no_color defaulted to the wrong value).
The interesting one: OutputFormatter.output() and output_error() called sys.exit() inside the formatter, right after writing. That makes them impossible to unit-test — the test can't capture-then-assert because the process just exits — and it's a design smell: a formatter shouldn't own process control.

The fix wasn't "delete the assertions." It was a separation of concerns:

# before — formatter exits, untestable
def output_error(self, error, code):
    ...print json...
    sys.exit(code)

# after — the formatter just formats; the CALL SITE owns the exit
# src/output.py: no sys.exit at all
# src/main.py / src/cli.py, at every error path:
formatter.output_error(error.to_dict(), error.code)
sys.exit(error.code)

It pulled sys.exit() out of the formatter and added it at every error call-site in main.py/cli.py — so production exit codes are preserved while the formatter becomes testable. Then it rewrote the tests to assert the error payload instead of requiring the process to die.

I verified it — and checked specifically for cheating

$ python -m pytest
20 passed

Green is cheap if you gut tests, so I checked the diff:

grep sys.exit src/output.py → 0 (removed, as intended)
sys.exit(error.code) added at every error path in main.py/cli.py
the CLI still exits non-zero on errors

It fixed the design, kept the behavior, and made the tests meaningful. Merged.

Three repos, three stacks, one loop

Go — a feature (a command suggester).
Zig — a crash fix (red → green).
Python — a real refactor that made a broken suite green without weakening it.

Same loop every time: file an issue, the agent implements it on your own key, runs the repo's own tests, and opens a PR you review — verified, not blindly merged. It isn't pinned to a language or a framework, and — the part I actually cared about — when told not to cheat, it didn't.

Try it

CLI-only, BYOK (your Claude Code or tau key — it never resells completions), €20/mo. First 10 founding operators free during the beta, with a direct line to me:

👉 https://mago.intrane.fr

curl -fsSL https://mago.intrane.fr/install.sh | sh
mago register

If you point it at one of your own repos — any stack — I'd love to hear how it goes.

Same agent, different language: mago fixed a crashing test in my Zig CLI

Javier Leandro Arancibia — Thu, 25 Jun 2026 20:43:30 +0000

In a previous post I let mago — an autonomous agent team that runs over a GitHub repo — ship a feature to my Go deploy CLI. The obvious next question: does it generalize past Go, and past "add a feature"? So I pointed it at a Zig project and a crashing test.

The bug

tau is my agent-first AI CLI, reimplemented in Zig. Its test suite had one test that didn't just fail — it crashed:

error: 'agents_md.test.scanAgentsMd returns empty for bad path' terminated with signal ABRT
Build Summary: 1/3 steps succeeded (1 failed); 132/133 tests passed (1 crashed)

132 green, 1 hard abort — the kind of thing that sits red in CI because "it's just a test."

What mago did

It read the test and found the root cause, not the symptom. The test was passing undefined as the io: std.Io argument:

// before
const result = scanAgentsMd(undefined, std.testing.allocator, std.testing.allocator, "/nonexistent") catch ...

scanAgentsMd uses io, so in a debug build Zig traps the undefined access and aborts — before the test ever reaches the bad-path logic it meant to exercise. The fix is one line: pass a real io.

// after
const result = scanAgentsMd(std.testing.io, std.testing.allocator, std.testing.allocator, "/nonexistent") catch return;

It opened a PR with exactly that.

Verified

$ zig build test
Build Summary: 3/3 steps succeeded; 133/133 tests passed

Red → green. Merged.

Two repos, two stacks, one loop

Last time: a feature in Go (a "did you mean" command suggester for a 32-command CLI).
This time: a bug fix in Zig (a crashing test, root-caused and fixed).

Same loop both times: file a GitHub issue, the agent implements it on your own LLM key, runs the repo's own build/tests, and opens a PR — verified, not blindly auto-merged. It isn't tied to a language or a framework; it works against whatever your repo's build/test command says is green.

It's good at well-scoped, verifiable work — features, fixes, tests, refactors, DX. Not "build me a startup." Honest framing: a reliable autonomous dev shop you steer with a roadmap.

Try it

CLI-only, BYOK (your Claude Code or tau key — it never resells completions), €20/mo. First 10 founding operators free during the beta, with a direct line to me:

👉 https://mago.intrane.fr

curl -fsSL https://mago.intrane.fr/install.sh | sh
mago register

If you point it at one of your own repos — any stack — I'd love to hear how it goes.

I let an autonomous agent fix my own deploy CLI — here is the PR it shipped

Javier Leandro Arancibia — Thu, 25 Jun 2026 20:26:05 +0000

While building mago — a CLI that runs a small autonomous "company" of AI agents over a GitHub repo — I did the obvious test: point it at one of my own tools and give it a real task. No demo repo, no cherry-picked example. Here's exactly what happened, PR included.

The tool and the task

hotify-cli is my Go CLI for deploying web apps behind Traefik + Cloudflare (it's literally what deploys mago's own backend). It has 32 subcommands, and like a lot of CLIs, a mistyped command just dumped the entire ~130-line usage block and exited. Annoying for humans, noisy for agents.

So I filed a normal GitHub issue:

feat: suggest the nearest command on unknown input ("did you mean"), not a full-usage dump

…labeled it mago, and let the agent take it.

What the agent did, unattended

mago routed the issue to its implementer, which:

Cloned the repo and read the 32-command dispatch in main.go.
Wrote suggest.go — a rune-aware Levenshtein distance + a prefix-match shortcut + a canonical command list, with a length-aware threshold so unrelated typos don't surface a misleading suggestion.
Wrote suggest_test.go — 11 cases (typos, prefix matches, unrelated input).
Wired it into the default case of the dispatch.
Ran gofmt / go vet / go build / go test, then opened a pull request.

About 3 minutes, on Claude Sonnet, with no human in the per-step loop. Crucially it opened a PR — it didn't push to main. The merge stayed my call.

I verified it (don't trust the summary)

I checked out the branch myself:

gofmt: clean   go vet: clean   go build: OK   go test: ok

And the actual behavior:

$ hotify-cli statuss
hotify-cli: unknown command "statuss"
Did you mean "status"?
Run 'hotify-cli help' for usage.

$ hotify-cli deploi
Did you mean "deploy"?

Correct, scoped, tested, dependency-free. Merged.

Why it works — and where it doesn't

The thing that makes it trustworthy isn't the model, it's the loop:

Verified autonomy — a PR only auto-merges if review passes and the repo's own build/tests are green; failing or unverifiable work waits for you. (I kept this one review-only.)
GitHub-native — issues = tasks, labels = status, PRs = deliverables. The worker dials out, so there's no inbound webhook to expose.
BYOK — it drives your own Claude Code or tau key. It never resells completions.

It's genuinely good at well-scoped, verifiable work: tests, refactors, small features, CLI/UX fixes. It is not "build me a startup." Honest framing — it's a reliable autonomous dev shop you steer with a roadmap, not magic.

Try it

mago is CLI-only, BYOK, €20/mo. I'm onboarding the first 10 founding operators free during the beta, with a direct line to me:

👉 https://mago.intrane.fr

curl -fsSL https://mago.intrane.fr/install.sh | sh
mago register

If you try it on one of your own repos, I'd love to hear where it helped — and where it annoyed you.

Append-CLIP-Interrogation: A SuperCLI Plugin Guide

Javier Leandro Arancibia — Wed, 24 Jun 2026 03:44:35 +0000

Append-CLIP-Interrogation

Simple python script to automatically append caption generated by CLIP interrogator to an image file

Source

https://github.com/Append-CLIP-Interrogation/Append-CLIP-Interrogation

Installation

Check GitHub repository for installation instructions
Verify: Append-CLIP-Interrogation --version
supercli plugins install ./plugins/Append-CLIP-Interrogation --on-conflict replace --json

About SuperCLI

This plugin is part of the SuperCLI ecosystem - a unified CLI plugin manager for AI agents and developers.

Discover, install, and orchestrate thousands of command-line tools with SuperCLI.

App-Store-Connect-CLI: A SuperCLI Plugin Guide

Javier Leandro Arancibia — Wed, 24 Jun 2026 02:44:35 +0000

App-Store-Connect-CLI

Fast, scriptable CLI for the App Store Connect API. Automate TestFlight, builds, submissions, signing, analytics, screenshots, subscriptions, and more. JSON-first, no interactive prompts

Source

https://github.com/App-Store-Connect-CLI/App-Store-Connect-CLI

Installation

pip install app-store-connect-cli
Verify: app-store-connect-cli --version
supercli plugins install ./plugins/App-Store-Connect-CLI --on-conflict replace --json

About SuperCLI

This plugin is part of the SuperCLI ecosystem - a unified CLI plugin manager for AI agents and developers.

Discover, install, and orchestrate thousands of command-line tools with SuperCLI.

AiToEarn: A SuperCLI Plugin Guide

Javier Leandro Arancibia — Wed, 24 Jun 2026 01:44:35 +0000

AiToEarn

AI-powered earning platform CLI for monetizing AI capabilities and services

Source

https://github.com/AiToEarn/AiToEarn

Installation

pip install aitoearn
Verify: aitoearn --version
supercli plugins install ./plugins/AiToEarn --on-conflict replace --json

About SuperCLI

This plugin is part of the SuperCLI ecosystem - a unified CLI plugin manager for AI agents and developers.

Discover, install, and orchestrate thousands of command-line tools with SuperCLI.

Accurate-Cyber-Defense-Network-Penetration-Testing-Tool-CLI: A SuperCLI Plugin Guide

Javier Leandro Arancibia — Wed, 24 Jun 2026 00:44:36 +0000

Accurate-Cyber-Defense-Network-Penetration-Testing-Tool-CLI

Accurate-Cyber-Defense is a command-line penetration testing toolkit that performs automated network discovery, vulnerability scanning, exploit verification, and concise reporting to help security teams find and fix risks quickly.

Source

https://github.com/Accurate-Cyber-Defense-Network-Penetration-Testing-Tool-CLI/Accurate-Cyber-Defense-Network-Penetration-Testing-Tool-CLI

Installation

Check GitHub repository for installation instructions
Verify: Accurate-Cyber-Defense-Network-Penetration-Testing-Tool-CLI --version
supercli plugins install ./plugins/Accurate-Cyber-Defense-Network-Penetration-Testing-Tool-CLI --on-conflict replace --json

About SuperCLI

This plugin is part of the SuperCLI ecosystem - a unified CLI plugin manager for AI agents and developers.

Discover, install, and orchestrate thousands of command-line tools with SuperCLI.

7zr: A SuperCLI Plugin Guide

Javier Leandro Arancibia — Tue, 23 Jun 2026 23:44:34 +0000

7zr

CLI tool: 7zr

Source

https://manpages.debian.org/7zr

Installation

apt-get install 7zr 2>/dev/null || which 7zr
Verify: 7zr --version
supercli plugins install ./plugins/7zr --on-conflict replace --json

About SuperCLI

This plugin is part of the SuperCLI ecosystem - a unified CLI plugin manager for AI agents and developers.

Discover, install, and orchestrate thousands of command-line tools with SuperCLI.

7zip: A SuperCLI Plugin Guide

Javier Leandro Arancibia — Tue, 23 Jun 2026 22:44:34 +0000

7zip

7zip — high-ratio file archiver supporting 7z, ZIP, and RAR formats

Installation

brew install p7zip

About SuperCLI

This plugin is part of the SuperCLI ecosystem - a unified CLI plugin manager for AI agents and developers.

Discover, install, and orchestrate thousands of command-line tools with SuperCLI.

7za: A SuperCLI Plugin Guide

Javier Leandro Arancibia — Tue, 23 Jun 2026 21:44:35 +0000

7za

CLI tool: 7za

Source

https://manpages.debian.org/7za

Installation

apt-get install 7za 2>/dev/null || which 7za
Verify: 7za --version
supercli plugins install ./plugins/7za --on-conflict replace --json

About SuperCLI

This plugin is part of the SuperCLI ecosystem - a unified CLI plugin manager for AI agents and developers.

Discover, install, and orchestrate thousands of command-line tools with SuperCLI.

7z: A SuperCLI Plugin Guide

Javier Leandro Arancibia — Tue, 23 Jun 2026 20:44:35 +0000

7z

CLI tool: 7z

Source

https://manpages.debian.org/7z

Installation

apt-get install 7z 2>/dev/null || which 7z
Verify: 7z --version
supercli plugins install ./plugins/7z --on-conflict replace --json

About SuperCLI

This plugin is part of the SuperCLI ecosystem - a unified CLI plugin manager for AI agents and developers.

Discover, install, and orchestrate thousands of command-line tools with SuperCLI.

DEV Community: Javier Leandro Arancibia

I let an AI agent write the tests for my access-control layer

The gap

What it shipped

Verified

Four repos, four stacks, four kinds of work

Try it

An AI agent fixed my red Python test suite — without gutting the tests

The setup

What it actually did

I verified it — and checked specifically for cheating

Three repos, three stacks, one loop

Try it

Same agent, different language: mago fixed a crashing test in my Zig CLI

The bug

What mago did

Verified

Two repos, two stacks, one loop

Try it

I let an autonomous agent fix my own deploy CLI — here is the PR it shipped

The tool and the task

What the agent did, unattended

I verified it (don't trust the summary)

Why it works — and where it doesn't

Try it

Append-CLIP-Interrogation: A SuperCLI Plugin Guide

Append-CLIP-Interrogation

Source

Tags

Installation

About SuperCLI

App-Store-Connect-CLI: A SuperCLI Plugin Guide

App-Store-Connect-CLI

Source

Tags

Installation

About SuperCLI

AiToEarn: A SuperCLI Plugin Guide

AiToEarn

Source

Tags

Installation

About SuperCLI

Accurate-Cyber-Defense-Network-Penetration-Testing-Tool-CLI: A SuperCLI Plugin Guide

Accurate-Cyber-Defense-Network-Penetration-Testing-Tool-CLI

Source

Tags

Installation

About SuperCLI

7zr: A SuperCLI Plugin Guide

7zr

Source

Tags

Installation

About SuperCLI

7zip: A SuperCLI Plugin Guide

7zip

Tags

Installation

About SuperCLI

7za: A SuperCLI Plugin Guide

7za

Source

Tags

Installation

About SuperCLI

7z: A SuperCLI Plugin Guide

7z

Source

Tags

Installation

About SuperCLI