DEV Community

Cover image for The Part of Shipping a PR Nobody Automated Until We Had To
Muhammad Awais
Muhammad Awais

Posted on

The Part of Shipping a PR Nobody Automated Until We Had To

The Part of Shipping a PR Nobody Automated Until We Had To

You know this loop. You open a PR. CodeRabbit picks it up. You wait. It comes back with three comments one real, two nitpicks. You fix them, push, and wait again. Now it's rate limited because you pushed too soon, and the wait window is longer than it was five minutes ago. Twenty minutes later it finally reviews, approves, CI goes green, you click merge and by then you've context-switched away twice and lost the thread on what you were actually building.

None of that is CodeRabbit's fault. For what it's worth it's a genuinely good reviewer. The problem is the waiting. Every step in that loop is a human being asked to babysit a queue: is CI done yet, did the bot reply yet, is the rate limit over yet, did the reviewer request changes or just leave a nit. That's not engineering. That's polling with extra steps. We hit this exact loop enough times on our own Rails app that we stopped doing it by hand. Here's what replaced it.

The one command that ends the babysitting

claudetm merge-pr 172
Enter fullscreen mode Exit fullscreen mode

That's it. Point it at a PR number and walk away. It polls CI, and when a check fails, it doesn't just tell you. It reads the failure, fixes it, pushes, and keeps polling. When CodeRabbit leaves review comments, it reads them, decides which ones are real, fixes those, and responds. When the base branch moves underneath the PR and creates a conflict, it resolves the conflict instead of stopping to ask you. It loops through all of that up to 30 iterations until the PR is actually, genuinely mergeable, and then it merges it.

We've watched it do the exact rate-limit dance we used to do by hand: CodeRabbit comes back "Review rate limited," and instead of panic-pushing another commit (which, if you've been burned by this, you know makes the wait window longer, not shorter), it recognizes that specific response and waits it out rather than treating it as a real failure. That's not a generic "retry on error". That's the tool having actually been built by people who got burned by CodeRabbit's rate limiter and fixed it at the source.

What "give it a goal" actually looks like end to end

merge-pr is the finishing move, but the tool runs the whole thing from scratch too:

cd your-project
claudetm start "Add rate limiting to the public API"
Enter fullscreen mode Exit fullscreen mode

Output example:

▸ planning...
  → 3 tasks: add limiter middleware, wire config, add tests

▸ task 1/3: add limiter middleware
  → editing app/middleware/rate_limiter.py
  → running tests... 2 failed
  → fixing... re-running tests... passed
  → committed, pushed, PR #142 opened

▸ waiting for CI...
  → CI failed: lint error (unused import)
  → fixed, pushed, CI green

▸ waiting for review...
  → CodeRabbit: "Review rate limited. Next review available in: 12 minutes"
  → waiting (not nudging — pushing now only makes the window longer)
  → review landed: 1 real comment ("use a sliding window, not fixed bucket")
  → addressed, re-requested review
  → CodeRabbit: approved

▸ merged. verifying deploy...
  → health check passed. done.
Enter fullscreen mode Exit fullscreen mode

Read that log again and count how many times a human would normally have had to context-switch back in to check on something. In the loop above: zero.

The full lifecycle, as a diagram

PLANNING
│
├─ Read codebase
├─ Create task list
└─ Define success criteria
│
↓
WORKING (per task)
│
├─ Make changes
├─ Run tests
├─ Commit
├─ Push
└─ Create PR
│
↓
PR LIFECYCLE
│
├─ Wait for CI
├─ Fix failures
├─ Address reviews
├─ Resolve conflicts
└─ Merge
│
↓
VERIFICATION
│
├─ Run tests
├─ Check lint
├─ Verify criteria
└─ Done
Enter fullscreen mode Exit fullscreen mode

Most agent frameworks stop after the second box. You get code, you review it, you merge it yourself. The two boxes underneath that are where the actual tedium lives, and they're the ones we cared about closing.

A PR that goes stale doesn't rot

If you've run a busy repo you know this failure mode: your PR sits waiting on CI, someone else merges first, and now your branch is behind. Normally that's a "come back to this later" moment. Here, a PR that merely trails the base gets re-synced and re-tested before it's allowed to merge (sync-before-merge), and a PR that's outright conflicting gets handed to an agent session to resolve rather than blocking the run. Either way, the run doesn't stop. It notices for you.

Parallel work inside a single task, without git worktrees

When a task turns out to have independent, non-overlapping pieces, the session running it becomes a lead and hands pieces to hive-worker subagents. All sharing the same checkout, no worktrees, no clones. Only the lead touches git. Workers read, edit, and run narrow checks against a file set nobody else owns, and if a worker needs a file it doesn't own, it stops and reports instead of reaching across. The lead decides how many workers a task actually needs. Including zero, which is the right answer most of the time, since spawning four agents for four one-line edits costs more in cold-start overhead than it saves.

claudetm start "Port 20 view models to the new API" # parallel by default
claudetm start "Tweak the retry backoff" no-parallel # strictly one agent
Enter fullscreen mode Exit fullscreen mode

Running more than one subscription without them colliding

Profiles isolate credentials, so you can run multiple Claude subscriptions or point at a custom Anthropic-compatible endpoint entirely in parallel without one clobbering the other's session:

claudetm profile add work
claudetm profile login work

CLAUDETM_PROFILE=accountA claudetm start "..."   # in repo-a/
CLAUDETM_PROFILE=accountB claudetm start "..."   # in repo-b/
Enter fullscreen mode Exit fullscreen mode

Each profile gets its own credentials directory under ~/.claudetm/profiles/, so two subscriptions running at once don't invalidate each other's OAuth session. (Two copies of the same subscription running in parallel still can. That's an account-level limit, not something claudetm can route around. So give each concurrent run a distinct profile.)

The escape hatches say exactly what they bypass

admin merges past branch-protection rules that would otherwise block a finished, green PR. The situation you hit on a solo-authored PR against a main that requires an approving review nobody's around to give. It's a real override, not a workaround wearing a disguise. It does not skip CI, does not ignore a failing check, and does not merge over a reviewer who explicitly requested changes. If a human left CHANGES_REQUESTED, the run stops there even with admin on. That's a person pushing back, and the tool treats that differently from an unclicked "approve" button. If your branch protection exists because the review itself is the point, you leave admin off and merge by hand.

claudetm start "Your goal here" admin
claudetm merge-pr 42 admin # same override, single PR
Enter fullscreen mode Exit fullscreen mode

Under the hood

Python 3.10+, built on the Claude Agent SDK. The same functionality is exposed three ways behind one password: a REST API (FastAPI), an MCP server (FastMCP) for native IDE integration, and webhooks with HMAC signature verification for event notifications. Ships as a PyPI package or a multi-arch Docker image on GHCR. MIT licensed, so the whole thing including the CodeRabbit rate-limit handling described above is just Python you can go read.

What's not done yet

This is a young, fast-moving project (first commit January 2026), and being straight about the rough edges matters more to us than sounding finished:

  • No web UI. Everything is CLI, REST API, or MCP. A dashboard is something you'd build against the API yourself right now.
  • Costs real money by default. The smart tier defaults to Opus, the priciest model. budget caps spend per session, but it's opt-in, not a default guardrail. Set it if you're running this unattended.
  • You supply the trust boundary. admin and auto-merge exist because unattended operation is the point, but that means deciding which repos are safe to hand that much autonomy to is on you, not the tool.
  • It trusts "resolved," and that trust has one known blind spot. CodeRabbit auto-marks a review thread resolved once you push a new commit. Even if that commit didn't actually address the comment. We've seen a high-severity finding get auto-resolved that way once. It's worth a human glance at security/correctness comments specifically before a merge-pr run finishes, not blind trust of the "resolved" flag alone.

Try it

pip install claude-task-master
claudetm doctor # verifies Claude CLI creds, GitHub CLI auth, git config

cd your-project
claudetm start "Your goal here"
# ...or, on a PR that's already open and stuck in review purgatory:
claudetm merge-pr <PR#>
Enter fullscreen mode Exit fullscreen mode

Repo: https://github.com/developerz-ai/claude-task-master. MIT, issues and PRs genuinely welcome. If the CodeRabbit wait-loop at the top of this post is a loop you run by hand every week, I'd like to know if this actually closes it for you or just moves the tedium somewhere else.

Top comments (0)