DEV Community

Selçuk GÜRAL
Selçuk GÜRAL

Posted on AI-assisted

Kule needs testers: run AI agents on a copy of your folder, then tell me what broke

#security #ai #opensource #showdev

Last week I wrote about Kule, the open-source tool I'm building. Your comments turned into real changes, so this time I'd like to ask for something more specific: try Kule on your own computer, and tell me what happened.

Kule in one paragraph

Kule is a control room for AI agents that work on your folders. You point an agent (Claude Code, Codex, Gemini, or any agent that speaks ACP) at a folder. Kule gives it a copy-on-write copy of that folder inside an OS sandbox, looks at what every command really changed, and lets nothing reach the real folder until a policy, or you, says so. Every commit can be undone. It has a CLI and a desktop app, it's written in Rust, and it's Apache-2.0.

What changed since the first post

That post covered 0.2.0-preview.1. The latest is 0.2.0-preview.14. Some of the bigger changes:

  • The default policy is an allow-list now. Files of a kind known not to run by themselves, or of a kind your project already tracks, pass on their own. Everything else waits for you, scripts and hidden files included.
  • Nothing an agent writes reaches your home. Agents keep their settings, history and transcripts in a home of their own inside Kule.
  • A task's commands can't read your other projects, or where apps keep their settings, sign-ins and data, beyond what toolchains and shells need.
  • Your terminals run in the task's sandbox too, not just the agent.
  • Kule tells you what it cut, ended or refused, where you're working: in the command's output, the terminal, and the app's Live view.
  • Releases are built on pinned images and can't be changed once they're published. Each one comes with checksums, a build provenance attestation and an SBOM.
  • 15 security advisories are published so far. Each one went out after its fix had shipped.

Some of these came straight from comments on the first post:

  • An approval card now says whether Kule looks inside the connection you're approving (#27).
  • I measured which tools send Encrypted Client Hello before deciding how Kule treats it (#34).
  • Holding a git push apart from fetches is next (#50).

Thank you to everyone who asked those questions.

What I'm asking for

I wrote a short testing guide with six scenarios. Each takes 5 to 20 minutes, and you can pick any of them:

  1. Install and check. Does the install guide hold up on your machine?
  2. The loop, without an agent. Make a task, change the copy, review, commit, undo. No network or agent needed.
  3. The app, with an agent. Your first task in the desktop app, from prompt to commit.
  4. Your own work. Ask an agent for something you'd ask it anyway, and see whether Kule gets in the way.
  5. Your own setup. This is where I need help most (more on that below).
  6. The boundaries. Try to get past them, and report what you find privately.

At the end of each one, there's only one question: what happened that you didn't expect? A confusing step, a policy that asks too often, a tool that breaks inside the sandbox, a slow copy: all of it is useful. "Everything worked" is useful too.

Setups I can't test myself

Kule is tested on a handful of machines. If yours is one of these, a short report is worth a lot:

  • a Linux distribution other than Ubuntu, or Ubuntu 24.04 with its user namespace restriction;
  • a file system other than APFS, Btrfs or XFS;
  • a large repository or a monorepo;
  • an Intel Mac;
  • an ACP agent other than Claude Code, Codex and Gemini.

Before you start

  • macOS 13+ and Linux x86_64 only. Windows support is in progress, but it's not ready for testing yet.
  • The macOS app isn't notarized. It's signed ad hoc, so macOS will refuse to open it the first time. The install guide shows how to check the download and open the app.
  • Start with a test folder, or with a project whose work is all committed and pushed. Kule doesn't touch your folder until you commit, but it's still a preview.
  • Agents use your own accounts, so a run spends your own quota or API credit, the same as it would without Kule.

How to report

Open an issue at github.com/kulehq/kule/issues/new/choose and pick one of the forms: Bug report, Test report or Idea or change. Questions are welcome in Discussions, or right here in the comments.

Issues are public, so please leave out API keys, tokens, private paths and anything else you wouldn't publish. The forms remind you.

If you find a way past one of Kule's boundaries, please don't post it publicly. Report it privately, as SECURITY.md explains. The fix is made in private and released first, and the advisory credits you unless you'd rather it didn't.

What happens next

I read every report. A fix ships in the next preview, and its issue says which one. The CHANGELOG lists what each preview changed.

Thanks for reading, and thanks in advance to anyone who gives it 15 minutes. 🙏

Top comments (0)