It's been over three years since ChatGPT launched. Models have gotten dramatically better, dozens of Agent frameworks have shipped, and yet the number of AI applications that actually run complete business workflows in production without a human in the loop remains surprisingly small. GPT-4-class models can write code, analyze documents, and extract information at a level that would have been hard to imagine three years ago. Vector databases, tool calling protocols, multimodal reasoning — the infrastructure pieces are mostly there. So what's actually missing?
Filling out a web form is the kind of task most Agents can handle now — identify the fields, click the inputs, type the values. But what happens after you hit submit? Did the form go through? Did it throw an error? If it errored, was it a validation issue or a timeout? Do you need to go back and fix a field? Most Agents execute the action and stop. They wait for the next instruction. That "act, observe the result, decide if you're done, adjust if not" loop is the thing that separates a demo from something that actually does work. It sounds trivial when you write it out. It isn't.
When we started building Mano-AFK, our autonomous application builder, the initial assumption was that wrapping a while loop around a single-step Agent would be enough. It wasn't. Executing individual actions is not the bottleneck. The bottleneck is sustaining a decision loop across dozens or hundreds of steps — constantly evaluating how far you are from the goal, whether the last action moved you closer or further away, whether you need to backtrack. Short loops of 3 to 5 steps work fine. Getting a loop to run 50 or 100 steps without drifting off course is a different problem entirely.
The hardest part of that loop is verification. You clicked a button and the page navigated somewhere — was it the right page or the wrong one? Tests ran and failed — is there a bug in the code or is the test itself wrong? A build failed — missing dependency or bad config? There's no generic rule for these judgments. The model has to actually understand what the task is about. It can't just pattern-match.
The most common failure mode we saw in early Mano-AFK testing wasn't the Agent being unable to do something — it was the Agent doing something wrong and not realizing it, then compounding the error. A misconfigured field would cause a build to fail, the Agent would interpret it as a dependency issue and start reinstalling packages, and ten steps later it would be hopelessly far from the right path. We eventually added an adversarial reviewer — a separate Agent instance that independently evaluates whether the main Agent's decisions are aligned with the goal at each step, and forces a retry when things go off track. Stability improved dramatically after that. The mechanism isn't complicated. The difference it makes is bigger than swapping to a newer model.
Here's a finding that surprised us. Mano-CUA-4B running alone on 100 macOS GUI tasks hit 56% success rate. Add bash tool access and that jumped to 90%. The reason isn't that bash helped it execute more operations. Bash gave the Agent an external memory — it could write intermediate state, constraints, and completed checks to files and read them back when needed. Attention decay in long conversations is real. It's not that the context window is too small; models simply lose focus on early constraints 20-30 steps in. After seeing that result, Mano-AFK was redesigned to persist all intermediate state explicitly to the filesystem rather than relying on the model to remember things.
What happens if the process crashes at step 50? Single-step Agents don't care — you just retry. A loop Agent needs to serialize state at every step so it can resume from any point. It sounds like a boring engineering detail. It's also the kind of thing that determines whether something can run in production or not.
Benchmark leaderboards keep getting updated with higher scores, but most of them measure single-step action accuracy. That's a fundamentally different capability from completing an entire task end-to-end without human intervention. In Mano-AFK's CUA Benchmark tests, the W8A16 local 4B model achieved 58% end-to-end autonomous completion rate across 5 web applications and 100 test cases. With Cider W8A8 quantization that drops slightly to 54%, but prefill speed hits 1453 tokens/s. 58% doesn't look impressive on a leaderboard. But that's 58% of tasks completed fully autonomously from PRD through code generation, deployment, testing, and bug fixing — all running locally with zero human intervention. Digging into the failures, most weren't cases where the model couldn't execute a step; they were cases where a judgment call went wrong mid-loop and wasn't caught.
Cost is another dimension that doesn't show up in single-step thinking. A single API call costs a few seconds and a few cents. A loop that runs 100 steps multiplies that cost and latency by 100. Running locally on an M5 Pro, Mano-P decodes at roughly 80 tokens/s with zero API costs. For enterprise batch deployments that cost difference is decisive. Cider's INT8 quantization improves prefill by about 1.8x over W8A16 on M5 Pro — you barely notice that in a single call, but across a 100-step loop where prefill happens every turn, the compound effect is significant.
Waiting for the next model to solve all of this is a natural instinct. From what we've seen building Mano-AFK, the bottleneck isn't just raw model capability. It's how you structure the observe-plan-act-verify loop, where you place independent review, how you persist long-horizon state, how you handle recovery from failures, and how you catch errors before they cascade. Those are engineering problems, and they don't automatically resolve when parameter counts go up.
Mano-AFK and Cider SDK are open source under Apache 2.0 at https://github.com/Mininglamp-AI/Mano-P.
Top comments (0)