"Make Agent Confidence Metadata, Not Authorization"

A worker receives an issue with high confidence, then writes to a protected path. If the authorization layer treats confidence as permission, explanatory metadata has silently become authority. The invariant is stricter: confidence may affect review priority, never the actions an actor can perform.

What the announcement establishes

GitHub announced agent automation controls in GitHub Issues as a public preview on July 23, 2026. The page describes confidence and rationale information plus repository controls; preview status matters because behavior may change. Read the primary source.

For test track 4.1, the engineering claim here is narrower than the announcement: the surrounding workflow needs a contract that remains valid when metadata, transport, people, or executors change.

The artifact

# risk-policy.yaml (illustrative, unexecuted)
paths:
  "docs/**": {risk: low, required_role: contributor}
  "infra/**": {risk: critical, required_role: maintainer}
agent_metadata:
  confidence: display_only
  rationale: audit_only
default: {risk: high, required_role: maintainer}

For test track 4.2, this is a design fixture, not executed code. Pin language and dependency versions before turning it into a repository test, and replace example identities and timestamps with disposable values.

Success path

A contributor requests a documentation edit. The policy matches docs/**, independently verifies the contributor role, records the confidence field for review, and permits only that path.

For test track 4.3, a successful demonstration records inputs, policy or schema version, decision, and final identifier. It does not infer correctness from a confidence label, status badge, or fluent output.

Failure path and regression plan

The same identity requests infra/prod.yaml. Even confidence 1.0 cannot satisfy required_role: maintainer; return a denial containing rule ID and actor, but no sensitive policy internals.

Use paired fixtures: low confidence plus valid authority must pass, while high confidence plus missing authority must fail. Add malformed confidence, absent rationale, path traversal, stale membership, and default-rule cases.

For test track 4.4, the acceptance gate is binary: the negative fixture must produce no unauthorized or duplicate side effect, while the positive fixture must remain traceable to its initial evidence. Expected output should be documented before execution.

Cleanup and rollback

Deploy in report-only mode, compare proposed decisions with current authorization, then enforce low-risk paths. Roll back by disabling the new evaluator, not by broadening roles; retain denial records and revoke temporary test credentials.

For test track 4.5, cleanup must preserve enough sanitized evidence to distinguish cancellation, rejection, stale work, and successful completion. Never solve recovery by silently marking an uncertain operation successful.

Limitations

For test track 4.6, this article proposes a compact engineering exercise and reports no execution results. It does not evaluate service availability, security, accessibility conformance, productivity, or comparative quality. Product previews can change, and a local fixture cannot reproduce every hosted-system failure.

A practical development environment

For a separate, controlled place to exercise repository policies, MonkeyCode is an open-source AGPL-3.0 AI development platform with an overseas hosted option. It includes a managed server-side cloud development environment, integrated models, task and requirement management, and build, test, and preview workflows. It is free to start. These statements do not mean the GitHub or OpenAI capability discussed above exists in MonkeyCode. Check the console for current quotas, models, regions, duration, and pricing before planning work. Open the campaign workspace.

Disclosure: This article promotes MonkeyCode using an official campaign link. I’m a MonkeyCode user, not affiliated with the project, and I receive no commission from this link.

AI assistance disclosure: This article was drafted with AI assistance and reviewed against the cited primary sources.