OpenHands started life as OpenDevin, the community answer to the closed Devin demo that made the rounds in 2024. Two years and a rename later, it is the most-watched open-source autonomous coding agent on GitHub, maintained by All Hands AI under an MIT license. We spent a week running it against real repositories — a Python API service, a small Astro site, and a deliberately broken test suite — to see what the agent actually does when you stop watching it.
This is not a demo recap. It is what happens when you hand the agent a Docker daemon, an API key, and a messy bug.
What OpenHands actually is
Strip away the branding and OpenHands is a loop: the agent reads your task, decides on an action, runs it inside a sandboxed runtime, reads the result, and repeats until it thinks it is done. The actions are the same ones you use — run a shell command, edit a file, execute a Python snippet, browse a web page. Nothing about the model is special-cased; the intelligence is whichever LLM you point it at.
That last part is the design decision that matters most. OpenHands is model-agnostic through LiteLLM, so you bring your own key. We ran it against Claude and GPT-class models and the agent code did not change — only the cost and the success rate did. If a better model ships next quarter, you swap one config line and keep the runtime, the GitHub integration, and the muscle memory. Closed agents do not give you that exit.
The runtime is the second thing to understand. Every action executes inside an isolated Docker container, not on your host. The agent can rm -rf its way through a problem and the blast radius stops at the sandbox. You mount your project in, the agent works on a copy, and you review a diff at the end. For anyone who has watched a free-running agent in a raw terminal, this isolation is the difference between trying it and refusing to.
The sandbox protects your filesystem, not your wallet or your secrets. An agent that loops on a hard problem will keep calling the model — we watched a single moderately complex bug burn through more tokens than a careful human session would have. Set spend limits at the provider, and never mount a directory containing production credentials or a real
.envinto the runtime.
Running it: three ways in, three different experiences
There are three front doors, and they are not interchangeable.
The CLI / headless mode is the one developers stick with. You give it a task string, it works in the sandbox, and it streams its reasoning and actions to your terminal. This is where OpenHands feels least like a toy — you can pipe it a GitHub issue and walk away. It is also the mode that exposes how often the agent narrates a plan, executes it, and then has to backtrack when a test fails. Watching that backtrack loop is the most honest benchmark you will get.
The web GUI runs locally via Docker and gives you a chat panel beside a live view of the agent's terminal and editor. It is the right place to start because you can interrupt. When the agent heads down a wrong path — and it will — you stop it, correct course, and let it continue. Treating the agent as a pair rather than a vending machine roughly doubled our completion rate on non-trivial tasks.
The managed cloud removes the Docker setup entirely and wires directly into GitHub: tag the agent on an issue or PR and it opens a branch. Convenient, and the fastest path to a first result, but you are now sending your code to someone else's runtime. For a public repo, fine. For your employer's monorepo, read the data policy first.
The project publishes results on SWE-bench Verified, the standard benchmark of resolving real GitHub issues, and it sits among the stronger open agents there. Treat that as a signal of direction, not a promise about your codebase — benchmark issues are curated and self-contained in a way your actual backlog is not.
Where it earns its keep, and where it doesn't
OpenHands is strong on tasks that are tedious but well-specified: add a field through a stack, write tests for an existing function, port a script, chase a failing test to its cause. Give it a clear acceptance check — a test that must pass — and it grinds toward it with a persistence that is genuinely useful. The sandbox means you let that grind run unattended.
It is weak exactly where every coding agent is weak. Ambiguous requirements produce confident wrong turns. Large refactors that touch architectural assumptions wander. And the agent will sometimes declare victory on a task that does not actually pass review, because its internal definition of "done" was looser than yours. The fix is the same as it is for a junior engineer: write the success criterion down as a test before you start, and the agent has something real to loop against.
The honest summary: OpenHands is a capable autonomous worker for scoped, verifiable tasks and an unreliable one for open-ended design. That is not a knock — it is the current ceiling for the whole category, and OpenHands hits it without charging you a per-seat subscription.
The highest-leverage habit we found: never give OpenHands a task without a way to check itself. "Fix the login bug" wanders. "Make
test_login_rejects_expired_tokenpass" converges. The agent is only as good as the goal you hand it, so spend your effort writing the goal, not babysitting the work.
How it compares to the IDE agents
OpenHands and an editor-native assistant solve different halves of the job. OpenHands is built to run unattended on a whole task; an in-editor agent is built to keep you in the loop on every line. If your workflow is "I am writing code and want fast, contextual help," a tight editor integration is the better fit, and the two coexist happily — many developers prototype interactively in their IDE and hand the repetitive, well-specified follow-up work to OpenHands in headless mode.
Neither replaces the other. The skill in 2026 is knowing which task belongs in which lane — and writing the test that tells the autonomous agent when it is actually finished.
Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.
Top comments (0)