DEV Community

babycat
babycat

Posted on

"Show Agent Confidence Without Making It Look Like Permission"

A green “95% confident” badge beside an action button can read like approval even when the user lacks permission. Color and proximity accidentally merge two different questions: how certain is the suggestion, and who may authorize it?

What the announcement establishes

GitHub introduced agent automation controls in GitHub Issues as a public preview on July 23, 2026. The announcement discusses confidence and rationale signals. Because it is a preview, interfaces may change; this example is a general UI pattern, not GitHub UI code. Read the primary source.

For test track 1.1, the engineering claim here is narrower than the announcement: the surrounding workflow needs a contract that remains valid when metadata, transport, people, or executors change.

The artifact

<article aria-labelledby="agent-summary">
  <h2 id="agent-summary">Proposed issue update</h2>
  <dl><dt>Confidence</dt><dd>High (model estimate)</dd>
      <dt>Rationale</dt><dd>Matches the reproduction steps</dd>
      <dt>Authorization</dt><dd>Not granted</dd></dl>
  <p role="status">Review required; no change has been applied.</p>
  <button type="button" disabled>Apply update</button>
</article>
Enter fullscreen mode Exit fullscreen mode

For test track 1.2, this is a design fixture, not executed code. Pin language and dependency versions before turning it into a repository test, and replace example identities and timestamps with disposable values.

Success path

After a fresh permission check, enable the action and announce that authorization was verified. Keep rationale available as text, preserve heading order, and provide no color-only distinctions.

For test track 1.3, a successful demonstration records inputs, policy or schema version, decision, and final identifier. It does not infer correctness from a confidence label, status badge, or fluent output.

Failure path and regression plan

If permission expires or issue content changes, disable apply, mark the analysis stale, move no focus automatically, and offer a clearly named refresh action. Never relabel confidence as “safe.”

Check keyboard order, zoom, forced colors, reduced motion, screen-reader reading order, missing rationale, unknown confidence, revoked permission, stale issue revision, failed refresh, and double activation.

For test track 1.4, the acceptance gate is binary: the negative fixture must produce no unauthorized or duplicate side effect, while the positive fixture must remain traceable to its initial evidence. Expected output should be documented before execution.

Cleanup and rollback

Remove the enhanced card behind its feature flag and return to the existing review flow. Do not hide an already-applied outcome; retain an accessible audit message and offer the normal undo mechanism.

For test track 1.5, cleanup must preserve enough sanitized evidence to distinguish cancellation, rejection, stale work, and successful completion. Never solve recovery by silently marking an uncertain operation successful.

Limitations

For test track 1.6, this article proposes a compact engineering exercise and reports no execution results. It does not evaluate service availability, security, accessibility conformance, productivity, or comparative quality. Product previews can change, and a local fixture cannot reproduce every hosted-system failure.

A practical development environment

For building and reviewing a separate prototype, MonkeyCode is an open-source AGPL-3.0 AI development platform with an overseas hosted option. It includes a managed server-side cloud development environment, integrated models, task and requirement management, and build, test, and preview workflows. It is free to start. These statements do not mean the GitHub or OpenAI capability discussed above exists in MonkeyCode. Check the console for current quotas, models, regions, duration, and pricing before planning work. Open the campaign workspace.

Disclosure: This article promotes MonkeyCode using an official campaign link. I’m a MonkeyCode user, not affiliated with the project, and I receive no commission from this link.

AI assistance disclosure: This article was drafted with AI assistance and reviewed against the cited primary sources.

Top comments (0)