Andrew

Posted on Aug 12 • Originally published at andrew.ooo

OpenHands Review: The 70K-Star Open AI Software Engineer

#aiagents #opensource #developertools #selfhosted

Originally published on andrew.ooo — visit the original for any updates, code snippets that aged out, or follow-up posts.

Every proprietary coding agent — Devin, Cursor's background agents, GitHub's Copilot Workspace — asks you to trust a black box with your source code, your API keys, and your bill. OpenHands is the bet that you shouldn't have to. It's MIT-licensed, self-hostable, and with 70,000+ GitHub stars it's the most popular open-source autonomous software engineer on the planet.

Formerly known as OpenDevin, OpenHands started in 2024 as the community's answer to Cognition's Devin demo. Two years later it isn't a demo clone — it posts SWE-bench Verified numbers within a few points of the best proprietary scaffolds, has been rebuilt around a composable Agent SDK, and ships a browser-based Agent Canvas that lets you drive OpenHands, Claude Code, Codex, or any ACP-compatible agent from one self-hosted control center.

TL;DR for Developers

Product:     OpenHands (All-Hands-AI/OpenHands)
Was called:  OpenDevin
What it is:  Autonomous AI software-engineering agent + control center
License:     MIT (fully open source)
SWE-bench:   ~68% Verified (open scaffold + Claude Opus 4.6)
Surface:     Agent Canvas (browser UI + built-in VSCode) — replaced legacy CLI/GUI
Sandbox:     Per-session Docker container (agent never touches your host)
Models:      Any LLM — Claude, GPT, Gemini, DeepSeek, local via Ollama/vLLM
Best for:    Issue resolution, greenfield builds, refactors you review before merge
Watch out:   Token burn on hard tasks, Docker overhead, occasional planning loops

If you want a capable "junior engineer" that runs on your own infrastructure with your own model keys, OpenHands is the strongest open option today. If you want zero-setup and someone else to own the infra, a managed product may still fit better. The rest of this review is the honest version of that trade-off.

What OpenHands Actually Is

OpenHands is an agent that reads a task in plain English, then acts on a real dev environment: it writes and edits files, runs shell commands, executes code, browses the web, and iterates on the results. The core research contribution is CodeAct — instead of choosing from a rigid menu of tools, the agent expresses its actions as executable Python/bash code. That single unified action space turns out to be more expressive than JSON tool-calling for real engineering work, which is a big reason the SWE-bench numbers hold up.

Two pieces matter in the 2026 architecture:

The OpenHands agent — the open-source CodeAct agent that does the actual coding.
Agent Canvas — a self-hosted browser UI and backend server that replaced the legacy CLI and Local GUI as the default surface. It's your control center: start conversations, watch the agent work in a built-in VSCode + terminal, track per-project token cost, and wire up automations (e.g. "when a GitHub issue is filed, decompose it into tasks and open a PR").

Crucially, Agent Canvas is agent-agnostic. It runs the OpenHands agent out of the box, but via the Agent-Client Protocol (ACP) it can also drive Claude Code, Codex, or Gemini. So even if you don't use the OpenHands agent itself, Canvas can be the single pane of glass for whatever agent you do use — across local, Docker, VM, or cloud backends.

Setup: From Zero to Working Agent

The self-hosted path needs Docker and a model API key. The pattern hasn't changed much — pull an image, mount your Docker socket so the agent can spin sandboxes, and open the UI:

# Pull the runtime sandbox image
docker pull docker.all-hands.dev/all-hands-ai/runtime:latest

# Run the app (Agent Canvas UI on http://localhost:3000)
docker run -it --rm --pull=always \
  -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:latest \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v ~/.openhands:/.openhands \
  -p 3000:3000 \
  --add-host host.docker.internal:host-gateway \
  --name openhands-app \
  docker.all-hands.dev/all-hands-ai/openhands:latest

Then point it at a model. Bring-your-own-key is a first-class feature — you set an LLM profile in Settings:

Provider:  Anthropic
Model:     claude-opus-4-6  (or claude-sonnet-4-5 for cheaper runs)
API Key:   sk-ant-...

Prefer local and private? Point the same profile at an OpenAI-compatible endpoint served by Ollama or vLLM:

Base URL:  http://host.docker.internal:11434/v1
Model:     qwen3-coder:30b
API Key:   ollama   (placeholder — local endpoints ignore it)

That's the whole promise: your code stays on your machine, the agent runs in a throwaway Docker sandbox, and the only thing leaving your network is the model call — which you can also keep local if you run open weights.

Real Usage: What a Task Looks Like

You don't script OpenHands; you brief it. A typical issue-resolution prompt:

Task: The /api/export endpoint times out on datasets over 10k rows.
Reproduce it with the seed data in tests/fixtures/large.json,
find the bottleneck, fix it, and add a regression test.
Run the test suite before you finish.

From there the agent works autonomously in its sandbox: it reads the repo, runs the failing case, forms a hypothesis (it'll often discover an N+1 query or an in-memory sort), edits the code, re-runs the tests, and iterates until green — narrating each step so you can interrupt. Because actions are code, you literally watch it type pytest tests/test_export.py -k large and read the traceback, rather than watching an opaque "tool call."

For automations, Canvas leans on the SDK. A minimal programmatic run looks like this:

from openhands.sdk import Conversation, LLM
from openhands.tools import BashTool, FileEditorTool

llm = LLM(model="anthropic/claude-opus-4-6")

convo = Conversation(
    llm=llm,
    tools=[BashTool(), FileEditorTool()],
    workspace="./repo",
)

convo.send_message(
    "Add type hints to utils/parsing.py and run mypy until it passes."
)
convo.run()

The November 2026 Software Agent SDK paper is the reason this feels production-shaped rather than demo-shaped: it bakes in native sandboxed execution, lifecycle control (pause/resume/step), model-agnostic multi-LLM routing, and built-in security analysis — the plumbing you'd otherwise reinvent to run an agent unattended. Compared with the SDKs shipped by OpenAI, Anthropic, and Google, that combination is what lets you run OpenHands as a durable background worker rather than a one-shot chat: you can pause a long refactor, swap the underlying model mid-run to save cost, and resume without losing the conversation state.

The automations layer is where this pays off in practice. In Agent Canvas you can register a workflow that fires on a schedule or a webhook — for example, decompose every new GitHub issue labeled agent-ready into subtasks, attempt a fix, and open a draft PR for a human to review. Because the same automation can post its results to Slack, Linear, or Notion, teams end up using OpenHands less like a chatbot and more like an always-on junior teammate that handles the boring 60% before anyone opens their editor.

Benchmarks: How Good Is It, Really?

SWE-bench Verified is the standard test — 500 real GitHub issues with hidden test suites. The honest, sourced numbers:

~68% SWE-bench Verified with the open CodeAct v3 scaffold + Claude Opus 4.6, per independent testing (SSOJet, June 2026) — within striking distance of proprietary scaffolds.
72% SWE-bench Verified reported with Claude Sonnet 4.5 + extended thinking in All Hands' own configuration.
53%+ on more conservative pairings — a reminder that your model choice moves the number more than anything else.

All Hands also publishes the OpenHands Index, a public benchmark launched in early 2026 that scores model+SDK combos across five real categories: Issue Resolution (aggregating SWE-bench and SWE-bench-multimodal), Greenfield, Frontend, Testing, and Information Gathering. It's unusually honest for a vendor benchmark because it grades their own scaffold against many models — including ones that beat the default.

The takeaway: OpenHands is not the bottleneck; the frontier model you plug in is. Pair it with a top model and it competes with anything. Pair it with a weak local model and it'll struggle on the hard 20%.

Community Reactions

The open-source crowd is genuinely enthusiastic, with the usual caveats:

"The most popular self-hosted autonomous agent, full stop." With 70k+ stars and 490+ contributors it's the reference implementation others benchmark against.
The Docker sandbox is both the headline feature and the headline cost. Testers love that the agent can't touch the host, but note the per-session container adds startup latency and needs Docker running with a few GB of headroom.
Token burn is the recurring complaint. From r/ChatGPTCoding: on ambiguous tasks it can "get caught in loops because of bad assumptions or bad planning, and burn through your tokens." The team ships loop-detection (stuck.py) to mitigate it, but reviewers still call it a hack more than a cure.
Cost transparency is a favorite. Per-project cumulative cost in Canvas is repeatedly praised by teams paying per token — you can see exactly which project is expensive.
"A capable junior engineer, not a senior." The consensus framing: great for well-scoped issues you review before merge; risky for unsupervised production changes on ambiguous requirements.

Honest Limitations

No hype here — these are the things that will actually bite you:

It burns tokens on hard problems. A single tricky bug can chew through real money if the agent loops. Scope tasks tightly, and prefer cheaper models (Sonnet-class) for exploration, saving the flagship for the final push.
Docker is a hard dependency. On locked-down corporate laptops with restricted Docker, setup can turn into a troubleshooting session. You need Docker Desktop (or a daemon), ~4GB+ RAM headroom, and comfort in a terminal.
Ambiguity produces confident wrong answers. Vague requirements yield partial or incorrect solutions delivered with full confidence. Treat every diff as junior-engineer output: review before merge, always.
Not for unsupervised production. It's explicitly not the right fit to run headless against prod without strong code review gates.
Managed alternatives are simpler. If you don't want to manage model spend or Docker infra, Devin (~$20/mo) or Copilot (~$10/mo) hand you a managed path — you trade openness and cost control for convenience.
Local-model quality gap. BYO local weights keep everything private, but a 30B coder won't hit the 68% number. Privacy and top-tier resolution rates are still somewhat in tension.

FAQ

Is OpenHands the same as OpenDevin?
Yes. OpenDevin was renamed OpenHands. Same project, same MIT license, same GitHub repo under All-Hands-AI — just a more grown-up name and a much larger scope than the original Devin-clone framing.

Is it actually free?
The software is free and MIT-licensed — you can self-host the entire stack at no cost. You still pay for model tokens (unless you run local open weights), and All Hands offers a paid OpenHands Cloud/Enterprise tier if you'd rather not host it yourself.

How does it compare to Cursor or Claude Code?
Different shape. Cursor is an IDE-first assistant; Claude Code is a terminal agent. OpenHands is an autonomous agent plus a self-hosted control center (Agent Canvas) that can also drive Claude Code or Codex via ACP. If you want one self-hosted pane of glass across multiple agents and backends, that's OpenHands' unique angle.

Can I run it fully offline with a local model?
Yes — point an LLM profile at an Ollama or vLLM endpoint and everything, including the sandbox, runs locally. Expect lower SWE-bench resolution than with a frontier API model; the trade is privacy and zero per-token cost.

What models get the best results?
Frontier models dominate: Claude Opus 4.6 and Sonnet 4.5 (with extended thinking) post the strongest numbers, with GPT and Gemini flagships competitive. Your model choice affects the outcome more than any OpenHands setting.

Is it safe to point at my real repo?
Reasonably — the agent executes inside a per-session Docker container, so it doesn't touch your host filesystem directly, and the SDK adds built-in security analysis. Still, review every diff and never auto-merge; give it a branch, not main.

The Verdict

OpenHands earned its 70,000 stars by being the rare open-source project that competes on capability, not just principle. Pair it with a frontier model and it resolves roughly two-thirds of real SWE-bench issues — good enough to genuinely offload well-scoped work. The Agent Canvas control center, the composable SDK, and the ACP support turn it from "Devin clone" into the self-hosted hub for any coding agent you run.

The catches are real and predictable: it costs tokens, it needs Docker, and it's a junior engineer whose work you must review. But for a team that wants autonomous coding on its own infrastructure — with its own model keys and full cost visibility — there is currently nothing else in the open ecosystem that does the job this well. Give it a branch, a tightly scoped task, and a strong model, and it earns its keep.

DEV Community