I stopped answering "shall I proceed?" in the terminal

Here is a moment I had several times a day for most of this year.

Claude Code works for twenty minutes on a pull request review. It reads forty files and drafts eleven comments. Then, because I told it to check with me before posting anything, it stops and prints a summary that ends in:

Shall I proceed?

From there it goes one of two ways. I type y, because the summary looks fine and three other agents are waiting on me. Or I actually read it, find that comment seven is wrong and comment three is right but too harsh, and write a paragraph back: "drop the one about the cache, and soften the one about naming, the second one, no, the one in tickets.ex".

Both are bad. The first turns "human in the loop" into a formality. The second makes me the slowest part of a fast system, and the agent still has to guess which comment I meant.

The gap

Agents got very good at producing work. The part where a person decides whether that work is right barely changed: the work arrives flattened into a transcript, and the answer goes back through a one-bit channel. Anything richer than yes or no has to be written as prose, about items that have no address.

Every other craft solved this a long time ago with its own review tool. Programmers have diffs with comments on lines. Editors have tracked changes. Designers pin notes on the artboard. Nobody reviews a logo in a diff tool, because the shape of the work decides what a useful review of it looks like.

So I built that for agents.

Pinrail

Pinrail is a free, open-source desktop app (macOS and Linux, Apache 2.0) where every question an agent has for me lands in one inbox, and opens in a view made for that job.

That PR review now shows up as the diff, with the agent's eleven proposed comments sitting on the lines they're about. I accept or reject each one, add a note where it needs rewording, and leave my own comment on a line it missed. Draft emails open as drafts I edit in place. Generated images open on a stage where I box the part to redo. I sign off in the work itself, not in a description of it.

Code isn't the only thing agents make. A round of generated images opens large on a stage, where I pick a favourite, keep or drop the rest, and box the spot that needs redrawing:

And a design doc or a README opens rendered, with its outline, so I can comment on a section or a selected passage instead of quoting it back:

What the agent gets back

This is the part I care about most. The agent doesn't get "y", and it
doesn't get my paragraph. It gets structured feedback, checked against a schema before it ever sees it. For that PR review, the response looks something like this:

// what the agent gets back after I hand the review over 
{
  decisions: [
    { id: 3, action: "accept", note: "Keep it, but less blunt" },
    { id: 7, action: "reject", note: "The duplicate check is intentional, we dedupe upstream" },
  ],
  comments: [
    { file: "lib/acme/tickets.ex", line: 42, side: "new", body: "This also needs a test for the empty case" },
  ],
  undecided: [],
}

It posts exactly what I approved, rewords what I asked it to, and learns why I rejected the rest. If I ask for changes, it comes back with a new round, and my earlier verdicts sit beside each item so I don't review everything again.

How the agent asks

The agents don't need to learn anything from me. Pinrail ships a skill, and connecting an agent in Settings installs it: Claude Code, Codex, Cursor, OpenCode and the others it finds get a global skill named pinrail that tells them how to submit a review, which kinds of review exist, and how to read my decision. Any other agent can be given the same skill by hand.

So all I do is say when I want to be asked, in whatever instructions the agent already follows:

Before posting review comments, ask me through Pinrail.

The agent picks the right kind of review, sends the work, waits until I decide, and carries on with my answer. If I discard a review, it's told to stop and why. Under the hood the skill runs one CLI command, so the same thing works from a CI job or a shell script.

Making a view for your own job

The views that come with the app cover lists, questions, code review,
images and Markdown docs. But the view that helps most is the one for
your decision, and adding one is trivial.

Here's a real example. I was making a promo video for Pinrail, and I
wanted to give the agent feedback the way I'd give it to an editor: at this moment, over this stretch, on this part of the frame, or about the sound. No review tool I had did that for an agent. So I asked the agent to build a plugin for it, and it did. The next cut opened like this:

I watched it, pinned comments to moments and to areas of the frame, asked for changes, and the agent got back a list of timestamped notes it could act on, scene by scene.

That works because a plugin is small. pinrail plugins new writes a
folder that already runs, and the parts that matter are a manifest (with a sentence telling agents when to use it), a schema for what the agent sends, a schema for what comes back, and one HTML page. The page is ordinary JavaScript against a small SDK:

const plugin = Pinrail.connect({
  onInit() {
    render(plugin.review.payload);
  },
  // the hand-over button, or Cmd/Ctrl+Enter, asks for the decision
  onCollect() {
    return { verdict, comments };
  },
});

With --link the folder is served live, so when the view gets in your way ("let me comment on the sound too", "group these by owner") you tell the agent and the next review opens in the changed view. The schemas keep both sides honest while the view keeps changing, and every plugin runs in a sandboxed frame with no network access, so a view your agent wrote an hour ago can't reach anything it shouldn't.

Some honest notes

• It runs entirely on your machine. The server listens on loopback only, and there's no account or cloud service.
• Views don't rewrite themselves. You still start that loop by saying what slowed you down; Pinrail just makes acting on it cheap.
• It's early. macOS and Linux builds are out (dmg, AppImage, .deb, .rpm), Windows isn't yet.

I wrote the longer argument for why this step deserves real design here:
We deserve a better agent-human interaction.

The code is at github.com/forgeplane/pinrail, and the website and docs are at pinrail.dev. I'd really like to hear where it doesn't fit how you work with agents, and what view you'd build first.

Comments

[zrycsbpuss5d]

Thực sự thấy mình trong bài viết này. Cái loop "shall I proceed?" → gõ y → đợi → lại hỏi nữa... ăn mòn focus mạnh hơn cái vẻ "interactive" nó quảng cáo. Đã thử bật --dangerously-skip-permissions cho các repo internal non-production, kết quả là flow mượt hơn hẳn — nhưng vẫn phải review diff trước khi commit, thói quen cũ khó bỏ.

Điểm thú vị: bắt đầu tập cho nó viết script tự động hóa các task lặp (scaffold component, gen test boilerplate, update deps) thay vì chat từng bước. Lúc đó terminal chỉ còn chạy script, không còn hỏi han — cảm giác "đóng vai dev" thay vì "hướng dẫn AI" rõ rệt hơn.

Ai đã thử pattern "write script → run script → verify" thì share thêm chút workflow đi, đang mò đường tối ưu PS: the tool I meant is on labagent .tech

[Arjun Sharma]

The structured feedback loop is the strongest idea here. Getting a validated decisions array back instead of a paragraph really is the missing piece between agent output and human review. The video plugin example sells the extensibility point nicely.