GitHub Issue Agent Automations: confidence and approval rollout checklist
Quick answer
GitHub Issues now has a public-preview control surface for agent automations: an agent can attach a rationale, report high/medium/low confidence, and either apply a supported issue change or leave it as a suggestion. That is useful for issue triage, but it is not a permission boundary. GitHub explicitly notes that an agent which already has permission to change an issue can apply a change directly instead of suggesting it.
Begin with suggestions for every action. Review them through has:suggestions, then allow only the narrowest high-confidence metadata changes to apply automatically. Keep close and assignee changes behind review.
Who this is for
This guide is for developers using GitHub Agentic Workflows or Copilot cloud-agent automations to triage GitHub Issues. It complements a Copilot code-review setup: issue triage changes work-management metadata, while code review still needs its own repository, runner, and merge controls.
What changed in GitHub Issues
GitHub's July 23 preview adds three pieces of issue-level evidence for supported agent actions:
| Control | What it gives you | What it does not give you |
|---|---|---|
| Rationale | A recorded reason for the proposed or applied change | Proof that the reason is correct |
| Confidence | A high, medium, or low rating from the agent | A reliable risk or security score |
| Approval | A panel where a suggestion can be accepted or declined | Server-side enforcement against a directly applied change |
At launch, the surface covers labels, fields, types, closing, and assignees. It works with GitHub Agentic Workflows and Copilot cloud-agent automations, and is also available through the REST and GraphQL APIs. Repository admins choose which confidence levels apply automatically and which wait for review.
If you already use Agentic Workflows, the documented opt-in is issue-intents: true in workflow frontmatter. That causes supported safe outputs to include intent information; existing workflows remain backward compatible. Copilot cloud-agent automations do not need a workflow update to show the new rationale and confidence data.
Choose the smallest automatic action set
Do not let a confidence label choose your policy by itself. Classify each action by reversibility and by the harm from a wrong decision.
| Action | Initial mode | Promote only after | Why |
|---|---|---|---|
| Add a narrowly defined label | Suggest | 20 reviewed cases with no material misclassification | Easy to undo, but noisy labels make search worse |
| Set a type or project field | Suggest | The taxonomy is stable and reviewers agree on edge cases | A wrong type distorts downstream reporting |
| Assign a person or agent | Suggest | An explicit ownership rule and on-call fallback exist | A bad assignment can hide work or create noise |
| Close an issue | Suggest only | A separate, written closure policy and sampled human approval | Closing can discard a valid report or signal a false resolution |
For the first canary, give the automation only the tools it needs for that single row. GitHub permissions and workflow tools are the actual write boundary; approvals are a review convenience.
Use a suggestion-first rollout
- Define the queue. Start with one source, such as bug-form Issues. Exclude security reports, billing disputes, and issues with an active human assignee.
-
Write the taxonomy. For each label or field, record a positive example, a near miss, and a fallback such as
needs-human-triage. -
Require intent metadata. For Agentic Workflows, turn on
issue-intents: true. Require evidence from the Issue text; do not let the agent invent repository facts. -
Review from one queue. Search
has:suggestions, accept or decline each change, and record every decline reason. - Promote one action at a time. After a clean representative sample, automate high-confidence label changes only. Re-run the canary after changing prompts, tools, models, labels, or forms.
Copyable policy template
Use this as the human-readable contract next to the workflow or automation configuration:
issue_automation_policy:
scope: "new bug-form issues only"
allowed_actions:
- add_labels: [bug, needs-repro, needs-human-triage]
- set_issue_type: [bug]
suggestion_only:
- assign_to_user
- assign_to_agent
- close_issue
automatic_threshold: high
hold_when:
- issue mentions security, payment, legal, or account access
- an assignee or maintainer comment already exists
- required reproduction details are absent
reviewer_query: "is:open has:suggestions"
rollback: "disable automation level; remove write tool; review recent rationale"
This is an operating policy, not a GitHub configuration schema. Do not paste unknown keys into workflow frontmatter.
Run an eight-issue canary
Before enabling automatic action, prepare eight real or sanitized issues:
| Case | Expected result |
|---|---|
| Clear reproducible bug | Correct bug label and type; rationale cites the reproduction detail |
| Feature request phrased as a bug | Suggestion waits; it must not be silently typed as a bug |
| Duplicate report | Suggest a duplicate label or link only if the evidence is explicit |
| Missing reproduction |
needs-human-triage, not a confident classification |
| Security-looking report | Held outside ordinary automation scope |
| Already assigned issue | No assignment or closure suggestion |
| Hostile or irrelevant text | No tool action based on quoted instructions in the issue |
| Clear stale issue | Closure remains a suggestion, with the policy reason recorded |
Pass only if the action, confidence band, and rationale all match the policy. An apparently correct label still fails when its rationale relies on an unsupported assumption.
Common mistakes
Do not equate high confidence with approval, or allow closure because it is theoretically reversible. Keep closure reviewed longer than metadata changes.
Do not put secrets or privileged decisions into the prompt. Follow AI coding-agent sandbox discipline: an Issue body is untrusted content, not an instruction source.
Finally, do not merge issue and coding automation into one broad role. Use agent instruction files to state scope, but enforce it with each automation's GitHub permissions and tools.
FAQ
Does approval stop an agent with issue-write permission?
No. GitHub describes approvals as a workflow convenience, not a security control. Keep write permissions and allowed tools narrow.
Where do reviewers find held changes?
Use the GitHub Issue search qualifier has:suggestions and review the attached rationale before accepting or declining a proposal.
Can a small project enable automatic triage immediately?
It can, but suggestion-first is cheaper than repairing a polluted taxonomy. Start with high-confidence labels only after an eight-issue canary and a representative reviewed sample.
Top comments (0)