We've been building developer tools at Mininglamp for a while now, and one thing kept bugging us. The gap between writing a prompt and actually shipping something is still massive. You can get an LLM to spit out code all day long. Getting that code tested, deployed, and running without a human babysitting every step? Different story entirely.
So we built Mano-AFK, an autonomous execution framework that takes a natural language description and tries to go all the way to a working, deployed application. PRD generation, code, deployment, multi-level testing, automatic bug fixing. The full loop. We open-sourced it under Apache 2.0, and it lives under the Mano-P organization on GitHub.
This post is about what actually happened when we let it run on real tasks.
What the Pipeline Actually Does
Mano-AFK takes your prompt and runs through a chain of stages. Natural language in, and it generates a product requirements doc first. Then code. Then it deploys locally. Then it tests in three layers: lint checks, API tests, and full end-to-end browser-based testing. If something breaks, it loops back, reads the error, fixes the code, and tries again. There's also an adversarial review step at the end where a separate agent tries to poke holes in what was built.
You install it with brew:
brew install Mininglamp-AI/tap/mano-afk
The E2E testing part is where it gets interesting. Mano-AFK can use either our local Mano-P model or Claude CUA in the cloud for the browser testing. The local model is a 4B parameter VLA model that runs on Apple Silicon. We've benchmarked it across 100 test cases on 5 different web applications.
With W8A16 quantization: 58.0% accuracy on the CUA benchmark.
With W8A8 via our Cider SDK: 54.0% accuracy, but faster prefill at roughly 1,453 tokens per second.
That 4 percentage point drop for W8A8 might matter or might not, depends on whether speed or accuracy is more critical for your use case.
The Part That Worked Surprisingly Well
PRD generation was legitimately good. We gave it something like "build a project time tracker with team dashboards" and the PRD it produced was... actually usable? It broke things into features, defined API endpoints, thought about data models. Not perfect, but a solid starting point that would have taken a junior dev a couple hours to write.
The lint and API test stages ran smooth. When the generated code had syntax issues or basic logic bugs, the fix loop caught maybe 70% of them without any human input. Simple stuff like missing imports, wrong variable names, incorrect route definitions. It would read the traceback, understand what went wrong, and patch it.
The adversarial review step also caught things we didn't expect. On one run it flagged that the generated app had no input validation on a form field that accepted numbers. Tiny thing, but it ships to production all the time.
Where It Fell Apart
E2E testing with the local 4B model was hit or miss. 58% accuracy means it fails on nearly half the test cases. When it works, it genuinely navigates the browser, clicks buttons, fills forms, validates output. When it fails, it fails in weird ways. Clicking slightly off target. Getting confused by dropdown menus. Losing track of multi-step flows.
We noticed it struggled most with dynamic UIs. Anything with animations, loading spinners, or elements that shift position after render. The model is vision-based so it literally looks at the screen. If the screen changes between the screenshot and the action, things go sideways.
Complex multi-page flows also tripped it up. A simple "create account, log in, see dashboard" flow worked fine. But "create account, log in, create a project, add three tasks, assign one to a team member, export the report" was too many sequential steps. It would get 80% of the way there and then do something nonsensical on step 7.
One thing worth noting. The parent project Mano-P has a 72B model that scored 58.2% on OSWorld, ranking first among specialized models. But that 72B model is not open source. What you actually get to run locally is the 4B model. Make sure you internalize that distinction before setting expectations.
The Fix Loop: Clever but Not Magic
The automatic bug fixing deserves its own section because it's where most of the "autonomy" lives. When a test fails, AFK captures the error output and feeds it back in with context about what the code was supposed to do. It then generates a fix and reruns.
For straightforward bugs this is great. Missing dependency? It adds it. Wrong port configuration? Fixed. 404 on an API route because of a typo? Caught and patched.
But for architectural problems? Not so much. If the original code design was flawed, like choosing the wrong data structure or building a race condition into the async flow, the fix loop would just keep patching symptoms. We watched one run attempt 6 fix iterations on what was fundamentally a design issue before we killed it.
This maps to something obvious in retrospect. The fix loop is good at local fixes. Bad at structural rethinking.
The Adversarial Review
After all tests pass, a separate agent reviews the application from scratch. It gets the original prompt and the built application and tries to find problems.
In our testing it caught legitimate issues maybe 40% of the time that the test suite missed. Things like edge cases in form validation, missing error states, accessibility problems. The other 60% it either found nothing or raised false positives.
Still, having any automated adversarial step is more than most frameworks do. Most stop at "tests pass, ship it."
Cross-Project Learning
Mano-AFK maintains rules.md and preferences.md files that persist across projects. So if it learns that your team always uses Tailwind, or always structures API responses a certain way, it carries that forward. We found this actually made a noticeable difference after about 5 projects. The generated code started matching our conventions more closely without us explicitly specifying them each time.
Where the Boundary Actually Is
After running 30+ projects through this pipeline, simple CRUD apps and landing pages and internal tools worked most of the time without us touching anything. The sweet spot is stuff where the happy path is most of what matters.
Once you add complex state management, real-time features, or third-party auth flows, the success rate drops and you end up spending real time on fixes. Production systems with adversarial input or distributed architectures are a no-go for now.
Basically: describe what you want, walk away for 10 minutes, come back to something that works 60% of the time. For the other 40% you have a solid starting point with tests already written.
Trying It
Everything is Apache 2.0 on GitHub under the Mano-P org. Runs on Apple M4 with 32GB RAM.
If you're building anything in the autonomous coding space and want to compare notes, the repo issues are open.
Top comments (0)