DEV Community: Marcel.Lupo

Can Copilot Fix Its Own Security Findings? Testing GitHub Agentic Autofix

Marcel.Lupo — Mon, 17 Aug 2026 14:22:50 +0000

Can Copilot Fix Its Own Security Findings? Testing GitHub Agentic Autofix

GitHub can now assign a code scanning alert directly to Copilot cloud agent. The agent explores the repository, changes code, attempts validation, and opens a draft pull request. That is more ambitious than generating a patch beside one vulnerable line. It is also not the same as proving that a vulnerability is gone.

The practical answer to the title is therefore: Copilot can attempt end-to-end remediation, but the strength of the evidence depends on the scanner and your CI controls. A human still owns the security decision.

This deep dive goes beyond my earlier introduction to Copilot coding agent. Rather than surveying general coding tasks, it examines one narrow DevSecOps question: how should a team test, review, and govern agentic autofix without mistaking automation for assurance?

Evidence boundary: Agentic autofix is a paid public preview. I have not invented a successful run, timing result, credit charge, or vulnerability closure. The experiment below is a reproducible evaluation design for an eligible sandbox. Product behaviour is grounded in GitHub's 10 July 2026 announcement, its 16 July clarification, and linked GitHub documentation.

What GitHub Announced, and What It Clarified

On 10 July 2026, GitHub announced agentic autofix for code scanning alerts in public preview. The original workflow is straightforward:

Assign one or more code scanning alerts to Copilot.
Copilot cloud agent inspects relevant files across the codebase.
It proposes a fix and attempts to validate it.
It iterates when necessary.
It opens a draft pull request containing the changes, an explanation, and validation details.

GitHub says generation typically takes two to four minutes, but that is product guidance, not a result measured in this article.

The editor's note added on 16 July 2026 matters. It clarified that assignment works for all first-party and third-party code scanning alerts. In other words, eligibility is not limited to CodeQL findings. Alerts uploaded by integrated tools through SARIF can also be assigned.

That does not make all validation equal. GitHub's detailed autofix documentation says agentic autofix re-runs CodeQL using the code-scanning query suite on a best-effort basis. It cannot confirm alerts from custom CodeQL queries or the security-extended suite through that validation path, and fix quality for third-party alerts is not guaranteed.

This distinction is the centre of a sound evaluation:

Question	Answer in the preview
Can the alert be assigned?	Yes, for first-party and third-party code scanning alerts.
Can Copilot edit multiple files?	Yes. The cloud agent explores the wider codebase.
Does GitHub open a pull request?	Yes, normally a draft pull request with a summary and validation notes.
Is the originating finding always re-proven as closed?	No. Built-in validation has scanner and query-suite limits.
Is the pull request ready to merge without review?	No. Validation, behaviour, dependencies, and risk still require human review.

Do not extend the claim beyond code scanning. Dependabot and secret scanning are separate alert families with their own remediation workflows.

The Agent Loop Is the Real Change

Classic Copilot Autofix translates alert context into a single suggested change. Agentic autofix delegates a task to Copilot cloud agent, which can use repository context and tools over several iterations.

The loop is roughly:

alert context
    -> inspect related code and repository instructions
    -> form a remediation plan
    -> edit one or more files
    -> run available analysis and tests
    -> inspect failures or remaining findings
    -> revise the change
    -> open a draft pull request for review

Repository and organisation custom instructions apply while the agent works. That makes existing engineering guidance part of the control plane. A repository that tells agents to run focused tests, avoid new dependencies, and preserve public APIs gives the remediation loop better constraints than a repository with no executable acceptance criteria.

Validation must still be read scanner by scanner:

CodeQL code-scanning suite: Copilot can re-run CodeQL and use the result to iterate.
Custom or security-extended CodeQL queries: the documented built-in re-run does not prove those alerts are resolved. Re-run the repository's actual advanced setup in PR CI.
Third-party SARIF alerts: assignment is supported, but the scanner's own PR workflow remains the authoritative check. A clean CodeQL result does not close a Semgrep, Snyk, Checkmarx, or other third-party finding.
Functional behaviour: no static analyser proves that the application still meets its requirements. Unit, integration, contract, and negative security tests remain essential.

The agent may also report that it could not validate a fix or that an alert appears to be a false positive. That is useful evidence, not an invitation to auto-dismiss the finding.

Four Ways to Assign Work

GitHub exposes the same remediation idea through four entry points:

Individual alert UI: open a code scanning alert and choose Assign to Copilot.
Alert backlog batch: select between 1 and 25 alerts and assign them together. Copilot works on the selected set in one pull request.
Security campaign: select between 1 and 25 campaign alerts for one pull request. This fits an organised remediation deadline, but it increases review and rollback coupling.
REST API: update one code scanning alert and set assignees to ['copilot-swe-agent[bot]'].

The repository includes a safe PowerShell REST helper. Preview its target without making a request:

./code/assign-alert-to-copilot.ps1 `
  -Owner 'acme-security' `
  -Repository 'agentic-autofix-lab' `
  -AlertNumber 42 `
  -WhatIf

For an authorised run, set GITHUB_TOKEN in the process environment and omit -WhatIf. The helper sends:

PATCH /repos/acme-security/agentic-autofix-lab/code-scanning/alerts/42
Accept: application/vnd.github+json
X-GitHub-Api-Version: 2026-03-10

{"assignees":["copilot-swe-agent[bot]"]}

The Update a code scanning alert endpoint returns 200 OK on success. Classic personal access tokens need security_events for private or public repositories, or public_repo for public repositories only. Use the least-privileged supported token for your automation and never place it in the script, command history, or repository.

Start with one alert. A batch saves orchestration time, but it makes attribution harder: which edit fixed which alert, which check failed, and which change should be reverted? Batch only alerts that share a component, scanner, owner, and rollback unit.

Licence, Policy, and Cost Boundaries

The preview requires both sides of the product boundary:

An active GitHub Code Security or GitHub Advanced Security licence.
A GitHub Copilot licence with Copilot cloud agent enabled.
Copilot Autofix available in the repository.

Administrators can disable Copilot Autofix at enterprise, organisation, or repository level. Because agentic autofix relies on that setting, disabling classic Autofix also blocks the agentic experience. Administrators can separately opt repositories out of Copilot cloud agent. When the cloud agent is unavailable, assigning an alert falls back to classic Copilot Autofix instead of starting an agentic session. An enterprise policy set to not allowed cannot be overridden lower in the hierarchy, allowing it merely lets organisation and repository administrators choose.

The billing model is materially different from classic Autofix:

Route	Copilot licence	AI Credits	Actions minutes	Output
Classic Copilot Autofix	Not required	No	Not for generation	One suggested fix to review and apply
Agentic autofix	Required	Yes	Yes	Iterative agent session and draft PR
Manual remediation	No	No	Normal CI usage	Engineer-authored change and PR

During public preview, AI Credits are consumed only when a fix runs on alerts assigned to Copilot. The announcement says this usage is not itemised separately from other Copilot activity during the preview. Agent activity also consumes GitHub Actions minutes. Do not promise a fixed cost per alert: session depth, batch size, tests, runner type, and retries can change consumption.

Set a pilot budget before enabling broad campaigns. Track assigned alerts, agent sessions, Actions duration, PR outcome, and human review time together. A cheap generated patch that creates an expensive review queue is not a successful control.

A Reproducible Sandbox Evaluation

Use an organisation-owned sandbox that has the required licences and policies. Do not test with production secrets, customer data, or a live deployment path.

1. Prepare representative findings

Create three small, intentionally vulnerable examples with deterministic tests:

A CodeQL alert from the documented code-scanning suite.
A CodeQL alert produced only by your custom or security-extended configuration.
A third-party alert uploaded as SARIF by the scanner you actually use.

Keep each case on the default branch, record the alert number and tool, and ensure the scanner also runs on pull requests. Add functional tests that fail if a simplistic security patch changes intended behaviour.

2. Establish the baseline

Before assignment, record:

Commit SHA, scanner version, query suite, rule ID, severity, and alert URL.
Passing and failing test results.
Existing alert count for every enabled scanner.
Repository instructions and agent setup workflow revision.
The expected secure behaviour, including negative test cases.

Without that baseline, an alert disappearing could mean a real fix, a changed path, a scanner configuration change, or stale analysis.

3. Run isolated assignments

Assign one alert at a time through the UI. Reserve the API helper for a second pass after the manual flow is understood. Capture the agent session log, draft PR, changed files, explanation, validation commands, Actions runs, elapsed time, and AI Credit reporting available to your administrators.

Do not steer the first run unless it is blocked. You want to observe the default behaviour. In a later run, add precise repository instructions and compare whether the agent chooses better tests and smaller changes. Because model output is non-deterministic, repeat representative cases rather than treating one success as a rate.

4. Apply a merge gate

A candidate passes only when all applicable checks are true:

Gate	Reviewer question
Scanner closure	Did the original scanner, rule, configuration, and branch report the finding as fixed?
Regression scan	Did the change introduce new first-party or third-party alerts?
Functional safety	Do unit, integration, contract, and negative security tests pass?
Patch quality	Is the root cause fixed without broad rewrites or suppression?
Dependency integrity	Are new names real, supported, pinned appropriately, and licence-approved?
Scope	Can every changed file be traced to the assigned alert?
Reviewability	Is the PR small enough for a security-aware engineer to understand?
Operations	Are configuration, telemetry, runbooks, and rollback impacts addressed?

Report outcomes as validated, partially validated, not validated, or incorrect. “PR opened” and “agent says fixed” are activity measures, not security outcomes.

Review Risks and Rollback

GitHub's responsible-use guidance is direct: generated fixes can contain syntax errors, change semantics, address only part of a vulnerability, introduce another vulnerability, or suggest an unsupported or even fabricated dependency. Large files and repositories can also cause relevant context to be truncated. The same alert may produce different changes on repeated runs.

Review the data flow, trust boundaries, error paths, and tests, not just the highlighted line. Watch for common weak fixes such as swallowing an exception, removing functionality, adding a broad allow-list, weakening input validation, or suppressing the rule. Verify every dependency independently.

Before merge, rollback is simple: stop steering the session, close the draft PR, and retain the session log for evaluation. After merge, revert the remediation PR rather than hand-editing around it, then rerun the original scanner and functional tests. Never dismiss an alert merely to align the dashboard with the agent's claim.

One-alert PRs make this process much cleaner. For batch remediation, require a commit or clearly separable diff per alert and confirm the entire batch shares one rollback decision.

GitHub states that data handled by Copilot Autofix is not used for LLM training, but your normal source-code access, data classification, and third-party scanner policies still apply.

Troubleshooting the Preview

Assignment does not start an agentic session: check the Code Security or Advanced Security licence, Copilot licence, cloud agent policy, Copilot Autofix setting, repository opt-out, Actions availability, and your access to the alert. When cloud agent is unavailable, assignment can fall back to classic Copilot Autofix. Remember that upstream enterprise policy wins.

The REST helper returns an error: 400 usually points to the request, 403 can indicate an archived repository or missing Code Security entitlement, 404 can indicate an inaccessible repository or alert, and 503 is a service availability response. Confirm API version 2026-03-10, token scopes, alert number, and repository before retrying.

A PR opens without convincing validation: inspect the agent session log. Determine whether the alert came from a custom query, the security-extended suite, or a third-party scanner. Ensure the exact scanner runs on the PR and that its credentials and dependencies are available without weakening the agent environment.

The alert stays open after a plausible fix: compare branch, commit SHA, analysis category, and configuration. The same alert can be produced by multiple code scanning configurations, and stale configurations can disagree. Re-run the authoritative workflow rather than relying on the PR description.

The patch is too broad: ask Copilot in a PR comment to reduce scope, preserve the public API, and add a regression test. If the design remains difficult to verify, close the PR and remediate manually. Delegation is optional; accountability is not.

A Sensible Adoption Path

Start in observe mode with low-complexity, well-tested alerts in a non-production repository. Use individual assignments and require a security-aware reviewer. Compare agentic autofix with classic suggestions and manual fixes on patch size, scanner closure, regressions, review time, elapsed time, and cost.

Move to broader repositories only after the same scanner-specific gates are automated in branch protection. Introduce small homogeneous batches next. Security campaigns should come last, after ownership, budgets, concurrency, rollback, and audit evidence are established.

The feature is promising because it can do the tedious middle of remediation: investigate context, draft a multi-file change, run tools, and package evidence in a pull request. Its value is not that Copilot can mark its own work correct. Its value is that it can produce a testable candidate while deterministic scanners, repository tests, policy, and human review remain independent.

That is the right answer to “Can Copilot fix its own security findings?” Let it try. Make your delivery system prove the result.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 17-08-2026

Agentic DevOps Needs Observability: Trace GitHub Copilot with OpenTelemetry

Marcel.Lupo — Wed, 05 Aug 2026 07:16:58 +0000

Agentic DevOps Needs Observability: Trace GitHub Copilot with OpenTelemetry

An agent can read files, call a model, run tools, delegate to a subagent and edit code in one request. When that request feels slow or fails, a final chat message is not enough evidence to explain what happened.

On 8 July 2026, GitHub announced enterprise-managed OpenTelemetry export for VS Code and Copilot CLI. Administrators can now direct Copilot telemetry to an approved collector and govern the endpoint, transport, resource attributes, headers and content capture.

That enterprise control builds on the OpenTelemetry support already documented in VS Code. Copilot Chat can emit traces, metrics and events for agent orchestration, model calls and tool execution. The result is operational evidence for questions such as:

Which part of an agent request consumed the time?
Which model or tool is producing errors?
How many model turns does a task require?
Are subagents linked to the parent trace?
Is telemetry reaching the collector without capturing prompts or source code?

It is important to set the boundary early. Telemetry can show latency, errors, token use and observed user actions. It does not prove that generated code is correct, that an engineer was productive, or that an agent made a good decision. Tests, reviews and delivery outcomes remain separate evidence.

Current status: Product behaviour and managed-setting values in this tutorial reflect the official documentation available on 5 August 2026. Pilot managed settings against your supported client versions before broad deployment.

What You Will Build

You will run Jaeger locally in Docker, configure VS Code to export Copilot agent telemetry over OTLP/HTTP, produce an agent trace and inspect its hierarchy. Content capture remains disabled throughout the local walkthrough.

Copilot Chat in VS Code
        |
        | OTLP/HTTP on localhost:4318
        v
Jaeger all-in-one container
        |
        v
Trace UI on localhost:16686

The sample also includes an enterprise managed-settings.json shape for a later rollout. Local Jaeger uses transient in-memory storage, so this is a development exercise rather than a production observability platform.

Prerequisites

You need:

a current VS Code build with GitHub Copilot Chat
Docker Desktop or Docker Engine with Docker Compose
permission to run a local container and bind ports 16686, 4317 and 4318
a workspace where you can send a harmless agent request

No GitHub enterprise administration is required for the local exercise. Enterprise-managed delivery requires the appropriate GitHub plan and administrative access.

Step 1: Understand the Trace You Are Looking For

VS Code follows the OpenTelemetry Generative AI semantic conventions where a standard attribute exists. A typical request creates this span tree:

invoke_agent copilot
  |-- chat <model>              model requests one or more tools
  |-- execute_tool <tool-name>  VS Code runs the tool
  |-- chat <model>              model produces the next action or response
  `-- execute_hook <hook-name>  when a configured hook runs

invoke_agent wraps the full orchestration. Each chat span represents one model API call, while each execute_tool span records an individual tool invocation. A subagent receives the trace context, so its own invoke_agent span appears below the parent tool span when the runtime can propagate that context.

The attributes use three namespaces:

Namespace	Purpose
`gen_ai.*`	OpenTelemetry GenAI semantic conventions, including operation, provider, model and token attributes
`github.copilot.*`	Canonical Copilot-specific attributes for new queries and dashboards
`copilot_chat.*`	Legacy VS Code extension attributes retained for compatibility

Prefer gen_ai.* and github.copilot.* for new dashboards. The legacy namespace is still useful when an existing query depends on it.

Even with content capture off, metadata can include the repository remote, branch, commit SHA, GitHub organisation, model, token counts, tool names and errors. Treat that metadata according to your organisation's data classification rather than assuming that "no prompts" means "no sensitive context".

Step 2: Start Jaeger Locally

The tutorial pins the current Jaeger 2.20 all-in-one image and binds every port to the loopback interface:

services:
  jaeger:
    image: cr.jaegertracing.io/jaegertracing/jaeger:2.20.0
    ports:
      - '127.0.0.1:16686:16686'
      - '127.0.0.1:4317:4317'
      - '127.0.0.1:4318:4318'

From the post directory, start it:

docker compose -f code/compose.yaml up -d
docker compose -f code/compose.yaml ps

Open http://localhost:16686. The service list can be empty because no trace has arrived yet. Port 4318 receives OTLP/HTTP, 4317 receives OTLP/gRPC, and 16686 serves the Jaeger UI.

Step 3: Enable Copilot OpenTelemetry Without Content

Open Preferences: Open User Settings (JSON) in VS Code and merge these keys into your settings:

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.exporterType": "otlp-http",
  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4318",
  "github.copilot.chat.otel.captureContent": false
}

Reload VS Code after changing the configuration. Keeping captureContent false excludes full prompts, responses, system prompts, tool schemas, tool arguments and tool results. Metadata such as timing, model names and token counts is still emitted.

You can configure the same route with environment variables such as OTEL_EXPORTER_OTLP_ENDPOINT, but environment variables override user settings. A managed policy overrides both. The effective value order is managed policy, environment variable, user setting, then default.

Do not append /v1/traces to this base endpoint. An OTLP/HTTP exporter constructs signal-specific paths from http://localhost:4318.

Step 4: Generate and Inspect a Trace

Send a small agent request that causes at least one read-only tool call:

Read package.json, explain what each script validates, and do not edit any files.

Wait for the response, then return to Jaeger:

Select copilot-chat in Service.
Select Find Traces.
Open the newest invoke_agent trace.
Expand the chat and execute_tool children.

Inspect the root duration first. A long root with short model spans can indicate tool or orchestration time. A long chat span points towards model or network latency. Repeated chat and execute_tool pairs show how many reasoning turns the task needed. An error.type attribute marks a failed operation.

Useful attributes include:

gen_ai.request.model and gen_ai.response.model
gen_ai.usage.input_tokens and gen_ai.usage.output_tokens
gen_ai.tool.name and gen_ai.tool.type
github.copilot.agent.type
github.copilot.git.repository, branch and commit attributes
copilot_chat.time_to_first_token

You should not see gen_ai.input.messages, gen_ai.output.messages, tool arguments or tool results because content capture is disabled.

Jaeger is ideal for this trace walkthrough, but it is not the complete destination for every emitted signal. Use an OpenTelemetry Collector with a metrics and logs backend when you want durable metrics, event analysis, retention controls and alerting.

Step 5: Turn Signals Into Useful SLIs

Start with service level indicators that describe the system, not the developer. Set targets only after measuring a representative baseline.

Question	Signal	Example SLI
Are agent requests completing promptly?	`copilot_chat.agent.invocation.duration`	95th percentile end-to-end duration below a team-defined threshold
Are model calls responsive?	`gen_ai.client.operation.duration` and `copilot_chat.time_to_first_token`	95th percentile model duration and time to first token
Are tools reliable?	`copilot_chat.tool.call.count`, duration and `error.type`	Successful tool calls divided by all tool calls, grouped by tool
Are tasks looping?	`copilot_chat.agent.turn.count`	Percentage of invocations above a reviewed turn-count threshold
Is token demand changing?	`gen_ai.client.token.usage`	Input and output token distribution by resolved model
Are edits retained?	edit acceptance and survival metrics	Trend for investigation, never a correctness score

Events add detail around individual moments. Examples include copilot_chat.session.start, copilot_chat.tool.call, copilot_chat.agent.turn, edit feedback and cloud-session invocation. Use them to explain a metric change, such as which tool began failing after an extension update.

Avoid leaderboards based on token counts, accepted lines or session volume. More activity can mean harder work, unnecessary loops or a misconfigured agent. Edit acceptance and survival are behavioural observations, not proof of quality or productivity.

Step 6: Apply Privacy Before Scale

OpenTelemetry is off by default, and content capture is separately opt-in. Keep that separation in production.

A practical privacy review should cover:

data categories in attributes, events and optional content
approved collector endpoints and TLS validation
access control for traces, metrics and logs
retention, deletion and regional storage requirements
sampling and attribute limits
repository URLs, branch names and commit identifiers
whether prompts, file contents or tool results are ever justified

If content is required for a short investigation, use a restricted pilot, a defined expiry and a collector with suitable access controls. Return to metadata-only capture afterwards. Observability data should not become an ungoverned copy of source code and conversations.

Step 7: Manage Configuration Centrally

The 8 July announcement lets administrators deliver a telemetry block through native MDM, server-managed settings or a protected file. This example keeps content disabled and locked:

{
  "telemetry": {
    "enabled": true,
    "endpoint": "https://otel-gateway.corp.example",
    "protocol": "http/protobuf",
    "captureContent": false,
    "lockCaptureContent": true,
    "serviceName": "github-copilot",
    "resourceAttributes": {
      "deployment.environment": "production",
      "team.name": "developer-platform"
    },
    "headers": {
      "X-Scope-OrgID": "developer-platform"
    }
  }
}

The header is deliberately non-secret. Supply real collector credentials through your managed delivery system, never in a repository. Managed exporter headers are applied only to the Copilot Chat extension exporter and are not copied into environment variables, which prevents them leaking into tool subprocesses. In the current release, those managed headers are also not delivered to the agent host process, so test authentication for every client path.

Two precedence rules matter:

For managed channels, native MDM wins over server-managed settings, which wins over file-based settings. From VS Code 1.128, the highest channel that supplies any managed settings wins outright rather than merging with lower channels.
For each telemetry value, policy wins over environment variables, user settings and defaults.

On Windows, native settings use HKEY_LOCAL_MACHINE\SOFTWARE\Policies\GitHubCopilot. File-based settings use %ProgramFiles%\GitHubCopilot\managed-settings.json. Server-managed settings come from copilot/managed-settings.json in the selected .github-private governance repository.

File-based settings should be administrator-owned, not world-writable and not symlinked. Use Developer: Policy Diagnostics to confirm the active channel. Reload VS Code when telemetry policy changes because the agent host resolves its configuration at startup.

Enterprise team specialisation became available on 3 August 2026, but it does not make every managed key team-overridable. Teams can specialise the model and bypass-permission keys that an enterprise marks as overridable, and can add plugins and marketplaces. Telemetry remains an enterprise-level decision in the current reference.

There is one vocabulary caveat. The VS Code user setting and enterprise guide use exporter values such as otlp-http and otlp-grpc, while the current GitHub managed-settings reference lists http/json and http/protobuf as the accepted values for telemetry.protocol. Treat the live managed-settings reference and your deployed client build as the contract during a pilot rather than copying a user-setting value blindly.

Copilot CLI Transport Caveats

Not every CLI trace has the same parentage.

A background Copilot CLI agent hosted by VS Code produces a copilot-chat wrapper span with native github-copilot SDK spans beneath it in the same trace.
A terminal CLI session runs in a separate process. It emits independent root traces under service github-copilot, so it is not linked to the extension trace.
The terminal CLI runtime supports OTLP/HTTP only. If gRPC is configured, it still uses HTTP. Point it at an HTTP-capable port such as 4318 unless the backend deliberately serves both protocols on one port.

Search both copilot-chat and github-copilot before concluding that telemetry is missing. Also verify that collector authentication works for the agent host and terminal process, especially when the extension receives a managed header that those paths do not.

OpenTelemetry Is Not Session Audit Streaming

GitHub's Copilot agent session streaming is a separate enterprise capability in public preview for Enterprise Managed Users, including GitHub Enterprise Cloud with data residency. It sends Copilot usage records across supported clients to an audit stream or exposes the latest 48 hours through an enterprise REST API. It is designed for central auditability and can include session prompts, responses and tool calls when enabled.

OpenTelemetry is client-side operational instrumentation sent to an OTLP collector selected by the user or administrator. It gives detailed timing, hierarchy, errors, model metadata and tool telemetry.

Use OTel for reliability engineering and trace diagnosis. Use session audit streaming for enterprise governance, compliance and cross-client activity records. One does not replace the other, and their access, retention and privacy decisions should be reviewed independently.

Troubleshooting

Jaeger has no `copilot-chat` service

Confirm the container is running, the endpoint is http://localhost:4318, OTel is enabled and VS Code was reloaded. Send a new request after enabling telemetry because earlier sessions are not backfilled.

The collector returns `404` or connection errors

Use the base OTLP/HTTP endpoint without /v1/traces. Check that port 4318 is published and that a proxy or endpoint security rule is not intercepting loopback traffic.

CLI traces appear disconnected

That is expected for terminal CLI sessions. Search the github-copilot service. Only background CLI sessions hosted in the VS Code process share the extension trace.

Prompt or tool content is absent

That is the expected result with captureContent: false. Do not enable content merely to make a trace look richer. Start with metadata and use VS Code's Agent Debug Log for local diagnosis where appropriate.

A user setting has no effect

Check environment variables and Developer: Policy Diagnostics. Managed values override environment variables and user settings, while environment variables override user settings.

Metrics do not appear in Jaeger

Jaeger is the trace viewer in this tutorial. Route the emitted metrics and events through an OpenTelemetry Collector to a backend that supports those signal types.

Cleanup and Rollout

Stop and remove the local container:

docker compose -f code/compose.yaml down --remove-orphans

For enterprise rollout, begin with a small device group and metadata-only capture. Confirm endpoint authentication, both service names, CLI transport behaviour and policy precedence. Then define a short list of reliability SLIs, retention controls and alert owners. Review false alarms before expanding coverage.

Correlate telemetry with tests, review findings, incidents and deployment outcomes when evaluating an agentic workflow. Keep those evidence sources distinct. A fast, low-token trace can still produce an incorrect change, while a slow trace may reflect a genuinely difficult task.

Conclusion

Agentic DevOps needs the same operational discipline as any other distributed system. A connected invoke_agent trace makes model calls, tools, hooks and subagents inspectable without capturing conversation content by default. Start locally, establish reliable signals, govern the export path and keep correctness claims tied to engineering evidence rather than telemetry volume.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 05-08-2026

Human-in-the-Loop Agentic DevOps: Govern AI Automation in GitHub Issues

Marcel.Lupo — Mon, 27 Jul 2026 10:37:38 +0000

Human-in-the-Loop Agentic DevOps: Govern AI Automation in GitHub Issues

The biggest question in agentic DevOps is no longer, "Can an AI agent automate this?"

It is, "How much authority should we give it, and what happens when it is unsure?"

That distinction matters. An agent that suggests a label for an issue is useful. An agent that confidently closes the wrong production incident is a new operational problem.

On 23 July 2026, GitHub announced agent automation controls in GitHub Issues in public preview. These controls add confidence levels, rationales and human approval to agent-driven issue updates.

For DevOps teams, this is more important than another clever prompt. It gives us a practical way to introduce automation gradually, inspect why an agent wants to act, and keep people involved where judgement still matters.

In this post, we will explore how the controls work, where they fit into agentic DevOps, and how to build an adoption model that earns trust rather than assumes it.

Current status: Agent automation controls in GitHub Issues are in public preview and subject to change at the time of this writing - 27 July 2026.

The Automation Gap

Traditional DevOps automation is usually deterministic:

IF pull request has label "deploy-production"
AND required checks pass
THEN start the production deployment

The rule may be complicated, but the outcome is predictable. We can test the conditions and explain exactly why the automation ran.

Agentic automation is different:

Read this issue.
Understand what the author needs.
Choose the right service, severity and owner.
Decide whether it is a duplicate.
Take the appropriate action.

That is powerful because the agent can reason over messy, incomplete language. It is also probabilistic. Two similar issues may produce different recommendations, and a confident answer can still be wrong.

This creates an awkward gap:

Full manual triage is safe but slow and repetitive.
Full autonomous triage is fast but can make silent, high-impact mistakes.
Human-in-the-loop triage lets the agent do the reading and reasoning while a person retains control over uncertain actions.

The goal is not to insert an approval into every automated step. That would simply turn engineers into an expensive confirmation button. The goal is to automate according to confidence, impact and reversibility.

What GitHub Added to Issues

The public preview introduces three related controls for updates made by GitHub Agentic Workflows or Copilot cloud agent automations.

1. Confidence

The agent can associate a high, medium or low confidence level with a proposed issue action.

Repository administrators choose the minimum confidence required for changes to apply automatically. Actions below that threshold remain suggestions for a human to review.

This creates a useful middle ground:

Agent confidence	Example policy	Result
High	Auto-apply low-risk metadata	Change is applied
Medium	Require a maintainer decision	Suggestion waits for review
Low	Always require review	Suggestion waits for review

Confidence is not correctness. It is a routing signal. A high-confidence mistake is still a mistake, so the consequence of the action must influence your policy too.

2. Rationale

Every supported agent action can include a short explanation of why it was proposed or applied.

Instead of seeing only:

Added label: incident

you can see the reasoning:

Suggested incident because the issue reports an active production outage and includes failed health-check results.

That explanation helps reviewers make a faster decision and leaves a more useful trail in the issue timeline. It also gives teams evidence for improving prompts and automation rules when the same misunderstanding appears repeatedly.

3. Suggestions and approvals

Changes that do not meet the configured automation threshold appear as suggestions on the issue. Reviewers can accept or decline them individually or process multiple suggestions together.

At launch, the controls support agent actions such as:

changing labels
setting issue fields
changing the issue type
assigning or unassigning users and agents
closing issues

The has:suggestions issue search qualifier helps maintainers find issues waiting for human review.

GitHub also exposes these capabilities through its REST and GraphQL APIs, which makes it possible to build approval queues and reporting around the same model.

The Human-in-the-Loop Flow

A well-designed issue automation should separate observation, recommendation and authority:

New or updated issue
        |
        v
Agent reads repository context
        |
        v
Agent proposes action + confidence + rationale
        |
        +---------------------------+
        |                           |
        v                           v
Meets automation threshold     Below threshold
        |                           |
        v                           v
Low-risk action applied        Suggestion queued
        |                           |
        v                           v
Timeline records why           Human accepts or declines
        |                           |
        +-------------+-------------+
                      |
                      v
             Outcome is auditable

There are two important ideas in this flow.

First, the agent does not merely output an answer. It produces an action proposal with enough context to review.

Second, the threshold is not universal. A platform team may auto-apply a documentation label at high confidence but always require approval before closing an issue or assigning an on-call engineer.

That is policy based on risk, not enthusiasm for AI.

A Practical DevOps Scenario

Imagine an internal platform repository receiving these issues every day:

a deployment workflow timed out
a developer cannot authenticate to a test cluster
Terraform documentation is out of date
a production service is returning errors
a request duplicates an existing backlog item

An agent could inspect each issue, repository metadata and previous examples, then propose:

Issue signal	Proposed action	Sensible control
Mentions a Markdown typo	Add `documentation`	Auto-apply at high confidence
Includes failed workflow URL	Add `ci-cd` and assign platform team	Review at medium or low confidence
Reports active customer impact	Set type to incident and add `severity-1`	Always review
Matches an existing issue	Close as duplicate with reference	Always review
Requests a known self-service task	Assign a specialised agent	Review until the workflow is proven

Notice that the risky actions are not necessarily the most technically complicated. Closing an issue is easy to execute but can hide real work. Assigning a person is easy too, but repeatedly assigning the wrong on-call engineer creates noise and erodes trust.

Human-in-the-loop design is about consequence, not implementation difficulty.

Build a Trust Ladder, Not an Autonomy Switch

Teams often discuss AI automation as if it has two settings: on or off. A safer model is a trust ladder.

Level 1: Observe

The agent analyses issues and produces a report, but changes nothing.

Use this stage to compare recommendations with decisions your maintainers actually make. Record false positives, ambiguous categories and missing repository context.

Level 2: Suggest

The agent proposes labels, fields, assignments or closures. Humans review every action.

This is where rationales become valuable. Reviewers should be able to answer:

Did the agent identify the right evidence?
Is the confidence believable?
Is the proposed action reversible?
Would we make the same decision?

Level 3: Auto-apply low-risk actions

Allow high-confidence, reversible metadata changes such as adding a broad area label.

Keep monitoring declined or corrected actions. If maintainers routinely undo an automated label, the system has not earned that level of authority.

Level 4: Expand by evidence

Gradually add actions only after measuring their performance and operational impact.

Do not promote every action together. Your automation may be excellent at identifying documentation issues and poor at recognising duplicates. Those capabilities deserve different policies.

Level 5: Retain hard approval boundaries

Some decisions should remain human-owned even if the agent performs well:

closing security or incident reports
changing severity for active production events
assigning privileged remediation work
triggering deployments or infrastructure changes
making decisions with compliance or customer impact

Maturity is not maximum autonomy. Maturity is knowing which authority should never be delegated silently.

Confidence Is a Signal, Not a Control

This deserves special attention.

GitHub describes the approval experience as a workflow convenience, not a security boundary. If an automation has permission to update an issue directly, a suggestion panel does not remove that underlying authority.

That means confidence thresholds and approvals must sit inside a broader security model:

grant the smallest practical repository permissions
keep untrusted issue content away from secrets
constrain the tools available to the agent
record actions and rationales
use branch protection and environment protection for code and deployments
treat prompts as guidance, not access control
retain independent review for consequential changes

An issue comment can contain prompt injection just as a source file can. A malicious or accidental instruction may attempt to redirect the agent, expose available context or convince it to take an unrelated action.

Human review reduces risk, but permission design limits the possible damage.

Agent Automation Controls vs Agentic Workflows

These names are easy to mix up, so here is the distinction:

Capability	Purpose
Agent automation controls in Issues	Add confidence, rationale and review to supported issue actions
GitHub Agentic Workflows	Define reasoning-based repository automation in Markdown and run it through GitHub Actions
Copilot cloud agent automations	Run scheduled or event-driven background tasks with Copilot cloud agent
Copilot coding agent	Complete development tasks and open pull requests for review

The controls are the governance layer for agent-driven issue changes. Agentic Workflows and cloud agent automations are ways to produce those changes.

This is similar to familiar DevOps patterns. A deployment workflow performs work, while an environment protection rule controls whether that work may proceed. The implementation and the decision boundary are related, but they are not the same thing.

Where Human Review Can Fail

Adding a person does not automatically make a system safe. Poorly designed review processes create their own failure modes.

Approval fatigue

If every obvious label waits for approval, maintainers will accept suggestions without reading them.

Response: Auto-apply proven, low-impact actions and reserve attention for ambiguity or consequence.

Automation bias

Reviewers may trust a confident recommendation because it came from an agent and includes a polished rationale.

Response: Ask reviewers to verify the cited evidence, not the quality of the prose.

Stale suggestions

An issue can change while a recommendation waits in the queue.

Response: Re-evaluate suggestions after meaningful issue updates and avoid applying decisions based on outdated context.

Invisible corrections

Teams may fix wrong labels manually without recording that the agent was wrong.

Response: Track declined, reverted and corrected actions so automation quality can be measured.

Authority creep

An automation starts with triage, then gradually gains broader write permissions because they are convenient.

Response: Review permissions and action scope as part of the same change-control process used for production workflows.

The Bigger DevOps Lesson

Agentic DevOps changes the shape of automation.

We are moving from workflows that execute explicit instructions to systems that interpret intent and choose actions. That makes automation more capable, but it also means uncertainty must become visible.

Confidence, rationale and approval are useful because they turn hidden model judgement into an inspectable workflow:

confidence helps route the decision
rationale helps a reviewer understand it
approval keeps authority with the right person
audit history helps the team improve the system

None of these replaces permissions, sandboxing, protected environments or independent review. They complement those controls.

The best agentic DevOps system is not the one that removes humans from every loop. It is the one that removes people from repetitive work while bringing them back at exactly the points where context, accountability and judgement matter.

Start with suggestions. Measure the corrections. Expand only when the automation earns your trust.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 27-07-2026

Run GitHub Copilot CLI in GitHub Actions Without PATs or Runaway AI Costs

Marcel.Lupo — Mon, 20 Jul 2026 16:02:20 +0000

Run GitHub Copilot CLI in GitHub Actions Without PATs or Runaway AI Costs

AI automation in CI has always had two awkward questions: which long-lived token do you give the agent, and how do you stop an unattended run from spending more than expected?

GitHub has now addressed both. Since 2 July 2026, GitHub Copilot CLI can authenticate inside GitHub Actions with the workflow's built-in GITHUB_TOKEN. No personal access token (PAT) is required. Copilot CLI also supports per-session AI credit limits, so a non-interactive run can stop when it reaches the amount you set.

In this tutorial, we will build a manually triggered repository-review workflow that:

authenticates without a stored PAT
uses read-only repository permissions
caps each Copilot session with --max-ai-credits
writes the report to the GitHub Actions job summary
checks that the agent did not modify the checkout
limits duplicate and long-running jobs

The result is a useful starting point for repository health checks, CI failure analysis, release preparation, and scheduled engineering reports.

Current status: AI credit session limits are in public preview. The syntax and billing behaviour described here were verified against GitHub's documentation on 20 July 2026.

What You Will Build

The finished workflow has a deliberately small trust boundary:

Developer selects Run workflow
             |
             v
GitHub Actions creates a short-lived GITHUB_TOKEN
             |
             | contents: read
             | copilot-requests: write
             v
Copilot CLI reviews the ephemeral checkout
             |
             | --max-ai-credits 31, 50, or 100
             v
Markdown report is written to the job summary
             |
             v
Workflow verifies that the checkout is unchanged

There is no PAT in repository secrets. The workflow token exists only for the job, and its permissions are declared explicitly. AI credits used in an organisation-owned repository are billed to the organisation.

This tutorial invokes Copilot CLI directly because that makes the authentication, permissions, and cost boundary easy to inspect. GitHub recommends GitHub Agentic Workflows for most production automation because it adds safeguards designed for unattended agents. We will compare the two approaches later.

Prerequisites

You will need:

an organisation-owned GitHub repository with GitHub Actions enabled
Copilot CLI enabled for the organisation
permission to update the organisation's Copilot policies
the Allow use of Copilot CLI billed to the organization policy enabled
enough organisation AI credit budget for the test run

The session-limit feature requires Copilot CLI 1.0.66 or later. The workflow installs the latest version so that both GITHUB_TOKEN authentication and --max-ai-credits are available.

No PAT, API key, or custom Actions secret is required.

How AI credits translate to cost

GitHub defines one AI credit as $0.01 USD. A limit of 50 AI credits therefore represents a nominal session boundary of $0.50 USD.

Session limits are soft caps. Copilot only knows the cost of a response after that response completes, so an in-flight response can make the final usage slightly higher than the configured limit. Session limits complement organisation budgets and cost centres; they do not replace them.

GitHub also recommends setting a limit above 30 credits because many model calls cost more than 20 credits. That is why this tutorial offers 31 as the lowest choice rather than using a tiny value that may prevent useful work.

Step 1: Enable Organisation Billing for the Workflow

An organisation owner must allow Actions workflows to bill Copilot CLI usage directly to the organisation.

Open the organisation settings on GitHub.com.
Navigate to the organisation's GitHub Copilot policy settings.
Find the Copilot CLI section.
Confirm that Allow use of Copilot CLI billed to the organization is selected.

GitHub enables this setting by default when the existing Copilot CLI policy is enabled, but it is worth checking before debugging a failed workflow.

When this policy is active, a workflow can authenticate with GITHUB_TOKEN by declaring one additional permission:

permissions:
  contents: read
  copilot-requests: write

contents: read lets the checkout action read the repository. copilot-requests: write allows the token to make Copilot requests billed to the organisation. It does not grant write access to repository contents.

Step 2: Add the Bounded Repository Review Workflow

Create .github/workflows/copilot-ci-review.yml in the repository you want Copilot to inspect.

name: Copilot CLI repository review

on:
  workflow_dispatch:
    inputs:
      max_ai_credits:
        description: Maximum AI credits for this Copilot session
        required: true
        default: '50'
        type: choice
        options:
          - '31'
          - '50'
          - '100'

permissions:
  contents: read
  copilot-requests: write

concurrency:
  group: copilot-ci-review-${{ github.ref }}
  cancel-in-progress: true

jobs:
  review:
    name: Run bounded Copilot review
    runs-on: ubuntu-latest
    timeout-minutes: 10

    steps:
      - name: Check out repository
        uses: actions/checkout@v6

      - name: Install latest Copilot CLI
        run: |
          npm install --global @github/copilot@latest
          copilot --version

      - name: Review repository
        id: copilot
        shell: bash
        env:
          GITHUB_TOKEN: ${{ github.token }}
          MAX_AI_CREDITS: ${{ inputs.max_ai_credits }}
          REPORT_PATH: ${{ runner.temp }}/copilot-review.md
          PROMPT: >-
            Perform a read-only review of this repository at the current commit. Do not modify files. Produce a concise Markdown report containing: likely build and test commands based on repository evidence; the top three reliability or security risks with repository-relative file references; missing or weak automated checks; and three prioritised actions. Do not print secrets, tokens, or environment-variable values.


        run: |
          set -o pipefail
          copilot --yolo -p "$PROMPT" \
            --max-ai-credits "$MAX_AI_CREDITS" \
            | tee "$REPORT_PATH"

      - name: Publish Copilot report
        if: ${{ always() }}
        shell: bash
        env:
          REPORT_PATH: ${{ runner.temp }}/copilot-review.md
        run: |
          if [[ -s "$REPORT_PATH" ]]; then
            cat "$REPORT_PATH" >> "$GITHUB_STEP_SUMMARY"
          else
            echo "## Copilot CLI review" >> "$GITHUB_STEP_SUMMARY"
            echo "No report was produced. Check the workflow log." >> "$GITHUB_STEP_SUMMARY"
          fi

      - name: Verify Copilot made no file changes
        if: ${{ always() }}
        shell: bash
        run: |
          if [[ -n "$(git status --porcelain)" ]]; then
            echo "Copilot changed the checkout unexpectedly:"
            git status --short
            exit 1
          fi

You can also find the complete sample in code/copilot-ci-review.yml.

Let us unpack the controls that matter.

Manual trigger and bounded choices

The workflow only runs through workflow_dispatch. That avoids automatically feeding code from untrusted pull requests into an agent with broad local tool access.

The credit limit is a choice rather than free text. Users can select 31, 50, or 100 credits, but cannot inject arbitrary shell content through the workflow input.

Short-lived authentication

This line exposes the job's automatically generated token to Copilot CLI:

env:
  GITHUB_TOKEN: ${{ github.token }}

The token is created for the workflow job, constrained by the permissions block, and expires when the job ends. There is no secret to rotate and no individual developer identity tied to the automation.

Non-interactive execution

GitHub's direct-invocation example uses --yolo so Copilot does not pause for interactive tool approvals in Actions:

copilot --yolo -p "$PROMPT" --max-ai-credits "$MAX_AI_CREDITS"

The name is memorable because the risk is real. --yolo gives Copilot broad access to the workflow environment. A prompt saying "do not modify files" is guidance, not a security boundary.

The actual boundaries are the manual trigger, read-only repository permission, ephemeral runner, timeout, credit cap, and final clean-worktree check. Do not add deployment credentials or production secrets to this job.

Report outside the checkout

The report is written to ${{ runner.temp }} instead of the repository workspace. This lets the final step use git status --porcelain to detect any tracked or untracked file created by the agent.

The report step uses if: ${{ always() }}. If Copilot reaches the credit limit or exits with an error after producing partial output, the workflow still attempts to publish that output to the job summary.

Two independent limits

The workflow combines:

--max-ai-credits, which bounds model usage for the Copilot session
timeout-minutes: 10, which bounds total job runtime

The concurrency group also cancels an older run on the same branch when a replacement starts. This reduces accidental duplicate spend.

Step 3: Run the Workflow

Commit the workflow to the repository's default branch, then:

Open the repository on GitHub.com.
Select Actions.
Select Copilot CLI repository review.
Select Run workflow.
Keep the first run at 50 AI credits.
Select Run workflow again to confirm.

The job should:

check out the repository
install the latest Copilot CLI package
print the installed version
authenticate with GITHUB_TOKEN
inspect the repository within the selected credit limit
publish a Markdown report
verify that the checkout remains unchanged

Open the completed run and select Summary. A successful result will resemble:

## Repository review

### Likely build and test commands

- `npm ci`
- `npm test`
- `npm run lint`

### Reliability or security risks

1. The deployment workflow uses a floating third-party action version...
2. Integration tests do not cover the authentication failure path...
3. Dependency updates are not automated...

### Prioritised actions

1. Pin external actions to reviewed commit SHAs.
2. Add an authentication failure integration test.
3. Enable grouped dependency update pull requests.

The exact findings will depend on the repository and model. Treat the report as triage input, not as proof that the repository is secure or production-ready.

Step 4: Prove the Cost Boundary

Run the workflow again and select 31 AI credits. A sufficiently complex repository review may reach the limit before completing.

When a non-interactive session reaches its limit, Copilot stops cleanly and the command ends. Because the cap is soft, a response already in progress completes first and may take the final total slightly above 31 credits.

Do not rely on forcing the limit as a deterministic test. Model selection, token usage, and repository complexity all affect consumption. The repeatable checks are:

confirm the log shows --max-ai-credits receiving the selected value
confirm an over-limit run stops rather than waiting for human input
review organisation usage in GitHub's billing and usage dashboards
apply organisation budgets or cost-centre budgets as the outer spending boundary

User-level budgets are not considered when Actions usage is billed directly to the organisation because the activity is not attributed to an individual user.

Security Hardening Before Reuse

Direct Copilot CLI execution is powerful, but it needs the same threat modelling as any other privileged CI automation.

Risk	Control in this tutorial	Production recommendation
Long-lived credential theft	Uses job-scoped `GITHUB_TOKEN`	Do not replace it with a PAT unless there is a documented requirement
Repository modification	`contents: read` and clean-worktree check	Keep write operations in a separate reviewed job
Untrusted pull request content	Manual trigger only	Never run this pattern on forked PRs with sensitive credentials
Unbounded model usage	Selectable `--max-ai-credits` value	Add organisation budgets, alerts, and cost-centre attribution
Runaway execution	Ten-minute timeout and concurrency cancellation	Tune the timeout to the smallest useful value
Secret disclosure	No extra secrets and prompt forbids printing environment values	Keep deployment secrets out of the job entirely
Unsafe agent output	Human reads the job summary	Never execute commands copied from the report automatically
Dependency drift	CLI version is printed in the log	Pin a tested CLI version after validating new releases

For high-assurance environments, pin actions/checkout to a reviewed full commit SHA rather than a moving major-version tag. You can also pin @github/copilot after testing a specific release that supports both token authentication and session limits.

Remember that repository content can contain prompt-injection instructions. Read-only GitHub permissions stop a compromised prompt from pushing code, but the agent can still read files available in the checkout and interact with its local runner environment. Keep the environment intentionally sparse.

Direct Copilot CLI or GitHub Agentic Workflows?

GitHub's documentation recommends Agentic Workflows for most automation scenarios. Direct CLI invocation still has a useful place when you need to add one bounded reasoning step to an existing workflow.

Consideration	Direct Copilot CLI step	GitHub Agentic Workflows
Definition	Standard Actions YAML	Natural-language Markdown compiled to Actions YAML
Existing workflow integration	Straightforward	Better suited to agent-first workflows
Authentication	`GITHUB_TOKEN` with `copilot-requests: write`	`GITHUB_TOKEN` by default
Guardrails	You design them	Includes agent-focused integrity, firewall, safe-output, and threat-detection controls
Best fit	A bounded task inside an established pipeline	Issue triage, reporting, compliance, and change-producing autonomous workflows

Use the direct pattern when you understand and control the prompt, trigger, tools, and environment. Start with Agentic Workflows when the agent will process untrusted content, propose repository changes, or operate across a broader part of the software delivery lifecycle.

Practical Variations

Once the basic workflow is working, the same pattern can support several read-only DevOps tasks:

CI failure analysis

Download test results or build logs as artifacts, then ask Copilot to identify the likely failure chain and recommend the next diagnostic action. Avoid passing raw secrets or environment dumps into the prompt.

Release-readiness report

Ask Copilot to inspect changelogs, dependency updates, migrations, tests, and deployment definitions, then produce a checklist for a human release owner.

Infrastructure review

Ask Copilot to review Terraform, Bicep, Kubernetes, or workflow files for risky defaults, missing validation, and likely operational gaps. Keep cloud credentials out of the job and do not let the report apply changes automatically.

Scheduled repository health summary

Replace workflow_dispatch with a trusted schedule trigger after the prompt and cost profile are stable. Keep concurrency, timeout, permissions, and AI credit limits in place.

Validate the Result

Use this checklist before adopting the workflow more broadly:

[ ] The organisation policy allows Copilot CLI usage billed to the organisation.
[ ] The workflow contains no PAT or custom authentication secret.
[ ] Permissions are limited to contents: read and copilot-requests: write.
[ ] The installed Copilot CLI version is 1.0.66 or later.
[ ] The selected credit value appears in the Copilot command.
[ ] The report appears in the Actions job summary.
[ ] The final clean-worktree check passes.
[ ] The run appears in organisation billing and usage data.
[ ] A human reviews the output before taking action.

For a stronger validation, temporarily add a harmless tracked test file to the prompt's requested output. The clean-worktree step should detect the change and fail. Remove that test immediately afterwards; the production prompt should remain read-only.

Troubleshooting

Copilot returns an authentication or permission error

Check all three layers:

The organisation policy Allow use of Copilot CLI billed to the organization is enabled.
The workflow declares copilot-requests: write.
GITHUB_TOKEN: ${{ github.token }} is present on the Copilot step.

Also confirm that the repository belongs to the organisation whose policy and billing you configured.

`--max-ai-credits` is unknown

The installed CLI is too old. Session limits require Copilot CLI 1.0.66 or later. Check the version printed by the installation step and reinstall the latest package:

npm install --global @github/copilot@latest
copilot --version

The run exceeds the selected credit value

This can happen because the session limit is a soft cap. A model response already in progress is allowed to finish. Use the session limit together with organisation budgets and spending limits.

The report step runs after Copilot fails

That is intentional. if: ${{ always() }} preserves partial output and writes a diagnostic message when no report exists. The overall job still retains the Copilot step's failure state.

The clean-worktree check fails

Inspect the git status --short output. The prompt may have caused Copilot to create or edit a file despite the read-only instruction. Tighten the task, remove unnecessary tools or credentials, and consider moving the use case to Agentic Workflows before enabling it again.

Conclusion

Running an AI agent in CI no longer needs to mean storing a developer's PAT or accepting an open-ended model bill. GitHub Actions can issue Copilot CLI a short-lived, narrowly scoped GITHUB_TOKEN, while --max-ai-credits gives every run an explicit session boundary.

The important lesson is that authentication and cost controls are only part of the design. The trigger, repository permissions, runner contents, timeout, prompt, output handling, and human approval path all determine whether the automation is trustworthy.

Start with a manually triggered, read-only report like this one. Measure its output and cost, keep the environment free of sensitive credentials, and only expand its authority when the workflow has earned it.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 20-07-2026

Microsoft Build 2026: Top announcements from a DevOps lens

Marcel.Lupo — Wed, 03 Jun 2026 16:44:20 +0000

Microsoft Build 2026: Top announcements from a DevOps lens

Microsoft Build 2026 was not just another Copilot-heavy keynote. It felt like Microsoft drawing a new operating model for software delivery, where AI agents do real work across code, infrastructure, data, security, and operations.

For DevOps teams, the interesting question is not "what was announced?". It is "what changes how we build, ship, govern, and secure software?". I have ranked the ten announcements below through that lens, based on developer impact, platform maturity, and how often the same themes appeared across Microsoft, GitHub, Azure, and community recap coverage.

1. GitHub Copilot app becomes the control centre for agentic development

The biggest DevOps signal from Build was the new GitHub Copilot app. GitHub describes it as an agent-native desktop experience where developers can track sessions, issues, pull requests, background automation, and active work from one place.

This matters because agentic development can create a new kind of operational sprawl. One agent investigates a bug, another works through a backlog item, and another updates a pull request after review. Without a control plane, that quickly becomes hard to audit.

From a DevOps angle, the useful pieces are:

isolated git worktrees for parallel agent sessions
Agent Merge to monitor CI, checks, reviewers, and merge readiness
canvases that make plans, pull requests, terminal output, deployment state, and workflow progress inspectable
local and cloud sandboxes for bounded execution
expanded Copilot code review with custom skills, MCP servers, and workflow actions

This is GitHub moving from "AI writes code" to "AI participates in the delivery workflow". That is a much bigger shift.

2. Microsoft Foundry matures into a production agent platform

Microsoft Foundry had one of the most important Build updates for teams that want to move agents out of demos and into production. The Microsoft Foundry Build 2026 recap highlights hosted agents, Toolboxes, Memory, Foundry IQ, tracing, evaluations, Agent Optimizer, and governance improvements.

The headline for DevOps is that agents now have more of the platform services we expect from production workloads:

managed runtime and sandboxed sessions
filesystem access and durable state
scheduled routines for recurring automation
a single governed endpoint for tools through Toolboxes
memory for procedural, user, and session context
tracing, evaluations, guardrails, and optimisation loops
publishing into Teams and Microsoft 365 Copilot

If you are building platform engineering workflows, incident helpers, release assistants, or environment automation agents, this is the announcement to watch. Foundry is becoming the runtime layer where those agents can be deployed, observed, and governed.

3. Microsoft Execution Containers bring containment to local agents

Agents that can read files, call tools, browse, and run code are useful. They are also risky. The Windows developer announcements introduced Microsoft Execution Containers, or MXC, as a policy-driven execution layer for agents.

MXC lets developers declare what an agent can access, such as files or network resources, with containment boundaries enforced at runtime. Microsoft also announced Agent 365 integration with MXC, bringing Defender, Entra, Intune, and Purview protections to local agents.

For DevOps and security teams, this is the kind of boring infrastructure that makes agentic workflows viable:

explicit policy boundaries
OS-enforced containment
enterprise identity and manageability
safer local automation
a path for security teams to approve agent usage without blocking developers

The future of AI-assisted DevOps cannot be "let the model loose on my laptop". It needs identity, containment, permissions, and auditability. MXC is one of the most important steps in that direction.

4. Windows 365 for Agents gives agents managed Cloud PCs

Microsoft also announced Windows 365 for Agents, designed to give agents secure, managed Cloud PCs for enterprise workflows.

This is more interesting than it sounds. A lot of enterprise automation still depends on applications, browsers, internal portals, legacy workflows, and UI paths that do not have clean APIs. Giving agents a managed Cloud PC means they can interact with those systems inside a controlled environment rather than running ad hoc on a developer workstation.

For DevOps teams, likely use cases include:

release checklist automation across legacy systems
environment validation through browser-based admin portals
infrastructure support tasks where APIs are incomplete
secure no-code and pro-code agent execution
temporary automation environments that can be governed centrally

This is also a reminder that agentic automation will not replace every old system overnight. It may first wrap and operate them more safely.

5. Work IQ APIs make Microsoft 365 context available to agents

Microsoft announced that Work IQ APIs will be generally available on 16 June 2026. Work IQ gives agents access to business context from email, calendar, meetings, chats, files, people, collaboration patterns, and line-of-business systems.

For DevOps teams, this matters because software delivery is not only code. It is also conversations, approvals, calendars, incident notes, architecture decisions, change windows, ownership, and organisational context.

The Work IQ API domains are especially relevant:

Chat, for programmatic access to Microsoft 365 Copilot responses and agents
Context, for agent-ready context and source data
Tools, for actions like sending emails, scheduling meetings, and uploading documents
Workspaces, for storing intermediate state during long-running agent work

Imagine an incident agent that can read the meeting notes, identify the service owner, summarise the active change, draft a post-incident review, and schedule the follow-up. That only works if the agent has governed access to workplace context.

6. Rayfin targets prompt-to-production enterprise app backends

On the Azure side, one of the more practical announcements was Rayfin, an open-source SDK and CLI for building enterprise-grade application backends on Microsoft Fabric.

The pitch is simple. Coding agents can create applications quickly, but production applications still need databases, authentication, data models, access policies, operational state, and governance. Rayfin lets developers and agents define those backend pieces in code, then deploy them into Fabric.

That is valuable for DevOps because it gives teams a more structured path from prototype to production:

data models live in code
access policies become programmable
app data lands in OneLake
GitHub-based workflows can drive backend changes
Fabric provides enterprise security and scale

This could become an important bridge between AI-generated application code and the platform controls enterprises need before anything goes live.

7. Azure HorizonDB brings PostgreSQL into the AI application era

Databases were another major theme. Microsoft introduced Azure HorizonDB, a fully managed PostgreSQL-compatible database for AI-powered applications, now in public preview.

The DevOps relevance is architecture simplification. Agentic applications often need transactional data, vector search, semantic search, AI model access, and integration with analytics and orchestration systems. Teams frequently stitch those pieces together themselves, which increases operational complexity.

HorizonDB aims to reduce that stitching with:

PostgreSQL compatibility
zone resilience by default
elastic storage up to 128 TB
scale-out compute up to 3,072 vCores
vector search and integrated AI model management
connectivity to Microsoft Foundry and Fabric

For platform teams, this could reduce the number of bespoke data services needed for AI app patterns, especially when PostgreSQL is already the standard operational database.

8. Cosmos DB improves local development and agent memory

The same Azure data announcement also included useful updates for Azure Cosmos DB. The Linux Emulator is now generally available, which means developers can build and test Cosmos DB applications locally across Linux, macOS, and Windows without a cloud dependency.

That is excellent news for DevOps pipelines because local and CI-friendly emulators help teams:

shorten feedback loops
reduce cloud dependency during tests
improve repeatability across developer machines and build agents
catch data access issues earlier

Microsoft also previewed semantic reranking and an agent memory toolkit using Cosmos DB, Azure Durable Functions, and Foundry models. Persistent agent memory will become a common requirement for real operational agents, especially for support, incident, and customer-facing workflows.

9. Windows developer experience gets serious quality-of-life upgrades

Build also brought several developer experience updates for Windows that matter to DevOps practitioners:

Coreutils for Windows is generally available
WSL containers are coming to public preview
Windows Developer Configurations are generally available
Intelligent Terminal is in experimental preview
Windows 365 with Developer configuration is in public preview

The biggest practical win is consistency. DevOps teams often move between Windows, Linux, WSL, containers, CI runners, and cloud shells. Coreutils for Windows and WSL containers reduce the friction between those environments.

Windows Developer Configurations, powered by WinGet, also matters for platform engineering. A reproducible developer workstation is part of the software supply chain. If teams can bootstrap VS Code, GitHub Copilot, WSL, PowerShell 7, and developer-optimised settings with one configuration, onboarding and environment drift both improve.

10. MDASH and agentic security raise the bar for AI-era AppSec

Security also had a clear agentic theme. Microsoft highlighted MDASH, a multi-model agentic security system that uses teams of agents to find exploitable bugs.

For DevSecOps teams, this is less about replacing scanners and more about changing the shape of security automation. Traditional tools are good at known patterns. Agentic security systems can reason across code paths, dependencies, runtime behaviour, and exploitability hypotheses.

The lesson for engineering teams is clear:

expect more AI-generated code
expect more AI-reviewed code
expect attackers to use AI too
invest in automated review, policy, sandboxing, and exploitability analysis
keep humans in the loop for risk decisions, not repetitive triage

The DevOps teams that win here will combine AI speed with strong policy, testing, review, and traceability.

The pattern: Microsoft is building the agent operating model

Taken together, the announcements point to one big pattern. Microsoft is not only adding AI features to existing products. It is building an operating model for agents:

GitHub is the workflow and code control plane
Foundry is the production agent runtime
Windows and Windows 365 provide local and cloud execution environments
Work IQ, Foundry IQ, Fabric IQ, and Web IQ provide context
Fabric, HorizonDB, and Cosmos DB provide data foundations
MXC, Agent 365, Defender, Entra, Intune, and Purview provide governance

That is why Build 2026 is important for DevOps. The conversation has moved from "can AI write code?" to "can AI safely participate in the delivery system?".

What DevOps teams should do next

You do not need to adopt everything at once. I would start with four practical actions:

Map where agents already touch your delivery workflow, including IDEs, pull requests, tests, incident response, and documentation.
Define guardrails for agent access to repositories, environments, secrets, networks, and production data.
Experiment with agent workflows in GitHub and Foundry, but require traces, evaluations, and human approval for high-risk actions.
Review your platform architecture for AI workloads, especially databases, memory, retrieval, observability, and sandboxed execution.

The most exciting part of Build 2026 is not a single feature. It is that the pieces are starting to connect. For DevOps teams, that creates an opportunity to design AI into the software delivery lifecycle deliberately, rather than letting it arrive as another unmanaged tool.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 03-06-2026

GitHub Copilot CLI Plugins and Marketplaces: Extend Your Terminal Agent

Marcel.Lupo — Wed, 27 May 2026 14:02:50 +0000

GitHub Copilot CLI Plugins and Marketplaces: Extend Your Terminal Agent

GitHub Copilot CLI already gives you an AI assistant in the terminal. Plugins make it more powerful by letting you install reusable agents, skills, hooks, and tool integrations as packages.

Instead of manually copying prompt files, wiring MCP servers, or sharing setup notes in a wiki, you can package those capabilities once and install them from a marketplace. In this guide, we will look at how Copilot CLI plugins work, how to find and install them, how marketplaces are structured, and how to build a small plugin of your own.

Quick tip: if you ever forget a flag, run copilot plugin --help or copilot plugin <subcommand> --help. The CLI ships its own up to date reference.

How a plugin reaches your machine

Before diving in, it helps to see the bigger picture. Plugins are not installed at the repository level. They are installed per developer, into the local Copilot CLI folder on each machine. That means every developer chooses which plugins they want, and the Copilot CLI on their machine becomes aware of those agents, skills, hooks, and MCP servers.

 +-----------------------------+        +--------------------------------+
 |     Plugin Marketplace      |        |        Plugin Package          |
 |    (e.g. awesome-copilot)   | -----> |  agents | skills | hooks       |
 |                             |        |  MCP servers | LSP servers     |
 +-----------------------------+        +----------------+---------------+
                                                         |
                                       copilot plugin install <plugin>
                                       (run on each developer's machine)
                                                         |
                                                         v
 +-------------------------------------------------------------------+
 |                      Developer's local machine                    |
 |                                                                   |
 |   ~/.copilot/installed-plugins/<marketplace>/<plugin-name>        |
 |   ~/.copilot/installed-plugins/_direct/<source-id>                |
 |                              |                                    |
 |                              v                                    |
 |              Copilot CLI loads agents, skills, MCP                |
 +-------------------------------------------------------------------+

   Note: each developer installs independently. Nothing is stored
   at the repository level, so two engineers on the same repo can
   have completely different plugin sets active.

Key things to notice:

The marketplace hosts the plugin definitions, but nothing is active until a developer installs it.
copilot plugin install runs locally on a developer's machine, so each engineer ends up with their own set of installed plugins.
Installed contents live under the user's home directory at ~/.copilot/installed-plugins/, which is why two developers on the same repo can have completely different plugin sets.
The Copilot CLI on that machine then picks up those installed agents, skills, and MCP servers in future sessions.

If your team wants everyone to use the same set, you do not check the plugin into the repo. Instead, you publish a curated marketplace and ask developers to install from it, or you use enterprise plugin standards (covered later) to enable plugins centrally.

Want central, reproducible management? If you would rather pin and roll out a consistent set of agents across a team or organisation, instead of relying on each developer to run copilot plugin install, take a look at my previous post on the Agent Package Manager (APM): Agent Package Manager (APM): A DevOps Guide to Reproducible AI Agents. It complements the per-developer plugin model shown above by adding versioning and reproducibility on top.

Are these the same as GitHub Copilot Extensions?

This is a common point of confusion, so let us clear it up first. GitHub Copilot Extensions and Copilot CLI plugins are different systems.

	GitHub Copilot Extensions	Copilot CLI plugins
Where they run	VS Code, Visual Studio, JetBrains, github.com	Copilot CLI in your terminal
Distribution	GitHub Marketplace (GitHub Apps)	Git-repository based plugin marketplaces
Install command	Install on GitHub.com, enable in the IDE	`copilot plugin install`
What they extend	The Copilot chat experience in IDEs	The Copilot CLI agent, its tools, agents, skills, and MCP servers

Both extend Copilot, but they are not interchangeable. This post is about the CLI flavour.

What Is a Copilot CLI Plugin?

A Copilot CLI plugin is an installable package that extends GitHub Copilot CLI with reusable customisations. According to the official about CLI plugins page, plugins can bundle:

Component	Typical location	What it adds
Custom agents	`agents/*.agent.md`	Specialist agent modes for focused work
Skills	`skills/<name>/SKILL.md`	Reusable task instructions and workflows
Hooks	`hooks.json`	Commands that run at lifecycle events
MCP servers	`.mcp.json` or `.github/mcp.json`	External tools and data sources
LSP servers	`lsp.json` or `.github/lsp.json`	Language server integrations

That means a plugin is not just an MCP server. It is a distribution format. A plugin can contain only a custom agent, only a set of skills, a bundle of MCP servers, or a combination of all of these.

After installation, Copilot CLI stores plugin contents under the user's Copilot directory and loads them into future sessions. Marketplace plugins are installed under:

~/.copilot/installed-plugins/<marketplace>/<plugin-name>

Direct installs are stored under:

~/.copilot/installed-plugins/_direct/<source-id>

The plugin manifest itself (plugin.json) can live in any of these locations inside the plugin repo, checked in this order: .plugin/plugin.json, plugin.json, .github/plugin/plugin.json, or .claude-plugin/plugin.json. The last one exists for compatibility with Claude Code plugin layouts.

Plugins versus manual configuration

You can add custom agents, MCP servers, and skills manually without using a plugin at all. Plugins are simply a better way to distribute those things. The official docs summarise it like this:

Feature	Manual configuration in a repository	Plugin
Scope	Single repository	Any project
Sharing	Manual copy and paste	`copilot plugin install`
Versioning	Git history	Marketplace versions
Discovery	Searching repositories	Marketplace browsing

If you find yourself copying the same agents/ folder into multiple repositories, that is a strong signal it should be a plugin.

How Plugins Relate to MCP

MCP, or Model Context Protocol, is the open standard used by AI tools to connect models to external tools and data sources. GitHub documents MCP support across Copilot surfaces in About Model Context Protocol.

Copilot CLI can use MCP servers directly, without plugins. For example, you can add an MCP server interactively:

/mcp add

Or you can edit the global MCP configuration file:

~/.copilot/mcp-config.json

A plugin can wrap that configuration and make it reusable. Instead of telling every developer to paste the same JSON into their config file, you can ship the MCP server configuration inside a plugin and let them install it with one command.

For example, a plugin might include this .mcp.json file:

{
  "mcpServers": {
    "playwright": {
      "type": "local",
      "command": "npx",
      "args": ["@playwright/mcp@latest"],
      "env": {},
      "tools": ["*"]
    }
  }
}

That plugin would give Copilot CLI browser automation tools through the Playwright MCP server whenever the plugin is loaded.

Finding Plugins

Copilot CLI ships with two registered marketplaces by default:

Marketplace	Repository	Purpose
`copilot-plugins`	`github/copilot-plugins`	Official GitHub Copilot plugins
`awesome-copilot`	`github/awesome-copilot`	Community collection of plugins, agents, skills, prompts, and MCP integrations

You can list the marketplaces registered with your CLI:

copilot plugin marketplace list

Inside an interactive Copilot CLI session, use the slash command:

/plugin marketplace list

To browse plugins from a marketplace:

copilot plugin marketplace browse awesome-copilot

Or from inside the session:

/plugin marketplace browse awesome-copilot

The awesome-copilot project also has a web UI at awesome-copilot.github.com, which is useful when you want to search and filter through community entries before installing anything locally.

There is also a separate GitHub MCP Registry for discovering MCP servers. That registry is useful when you want raw MCP servers, but it is not the same thing as a Copilot CLI plugin marketplace. Plugins can bundle MCP servers, but marketplaces distribute complete Copilot CLI plugin packages.

Installing and Managing Plugins

The main install command is:

copilot plugin install <plugin-spec>

The plugin specification can point to a marketplace entry, a GitHub repository, a subdirectory, another Git URL, or a local folder.

Install source	Example
Marketplace entry	`copilot plugin install database-data-management@awesome-copilot`
GitHub repository root	`copilot plugin install johnpapa/ai-ready`
GitHub repository subdirectory	`copilot plugin install dotnet/skills:plugins/dotnet`
Git URL	`copilot plugin install https://github.com/owner/repo.git`
Local path	`copilot plugin install ./my-plugin`

You can also install from inside an interactive session:

/plugin install database-data-management@awesome-copilot

Useful management commands include:

copilot plugin list
copilot plugin update <plugin-name>
copilot plugin update --all
copilot plugin disable <plugin-name>
copilot plugin enable <plugin-name>
copilot plugin uninstall <plugin-name>

Marketplace management uses a nested command group:

copilot plugin marketplace add owner/repo
copilot plugin marketplace list
copilot plugin marketplace browse <marketplace-name>
copilot plugin marketplace remove <marketplace-name>
copilot plugin marketplace remove --force <marketplace-name>

The --force option matters. If a marketplace has installed plugins, Copilot CLI will not remove it unless you explicitly force removal, which also removes the plugins installed from that marketplace.

Good Plugins to Try

Here are a few useful examples from the official and community marketplaces. Always inspect a plugin before installing it, especially if it defines MCP servers or hooks.

Plugin	Marketplace	What it is useful for	Install command
`advanced-security`	`copilot-plugins`	GitHub Advanced Security workflows such as secret scanning and dependency scanning	`copilot plugin install advanced-security@copilot-plugins`
`spark`	`copilot-plugins`	GitHub Spark integration	`copilot plugin install spark@copilot-plugins`
`azure`	`awesome-copilot`	Azure skills and Azure MCP server integration	`copilot plugin install azure@awesome-copilot`
`microsoft-docs`	`awesome-copilot`	Microsoft Learn documentation through MCP	`copilot plugin install microsoft-docs@awesome-copilot`
`devops-oncall`	`awesome-copilot`	Incident triage chat mode, prompts, and instructions for DevOps on-call work	`copilot plugin install devops-oncall@awesome-copilot`
`dotnet`	`awesome-copilot`	Everyday .NET and C# development skills	`copilot plugin install dotnet@awesome-copilot`
`dotnet-test`	`awesome-copilot`	.NET testing, coverage, and framework-specific test guidance	`copilot plugin install dotnet-test@awesome-copilot`
`chrome-devtools-plugin`	`awesome-copilot`	Chrome DevTools and browser debugging workflows	`copilot plugin install chrome-devtools-plugin@awesome-copilot`
`ai-ready`	`awesome-copilot`	Repository analysis and AI-readiness configuration	`copilot plugin install ai-ready@awesome-copilot`

For DevOps engineers, the Azure, database, documentation, browser debugging, and security plugins are the most immediately practical. They extend the CLI from a general assistant into a specialist operator for the stack you actually use.

Building Your Own Plugin

Let us build a small plugin that gives Copilot CLI a specialised deployment helper agent and a deployment checklist skill.

Create this folder structure:

my-devops-plugin/
├── plugin.json
├── agents/
│   └── release-engineer.agent.md
└── skills/
    └── deployment-checklist/
        └── SKILL.md

The only required field in plugin.json is name, but real plugins should include a description, version, author, licence, keywords, and explicit component paths.

{
  "name": "devops-release-helper",
  "description": "Release engineering helpers for CI/CD, deployment checks, and incident-safe rollouts.",
  "version": "1.0.0",
  "author": {
    "name": "Example Platform Team",
    "email": "platform@example.com"
  },
  "license": "MIT",
  "keywords": ["devops", "deployment", "cicd", "release"],
  "agents": "agents/",
  "skills": "skills/"
}

Now create the agent file:

---
name: release-engineer
description: Helps plan, validate, and troubleshoot safe production releases.
tools: ['bash', 'view', 'rg', 'glob']
---

You are a release engineering assistant. Focus on safe deployments, rollback-readiness, observability, and clear operator hand-offs.

Before recommending a deployment, check for:

- CI status and failing tests
- Database migrations and rollback impact
- Feature flags and progressive rollout options
- Monitoring dashboards, alerts, and log queries
- Rollback commands and owner contact points

Then create the skill:

---
name: deployment-checklist
description: Create a deployment readiness checklist for the current repository.
---

Review the current repository and produce a deployment readiness checklist.

Include:

1. Build and test validation
2. Infrastructure or configuration changes
3. Database migration risk
4. Secrets and environment variables
5. Monitoring and alerting checks
6. Rollback plan
7. Post-deployment verification

Flag anything that looks risky or missing.

Install it locally:

copilot plugin install ./my-devops-plugin

Check that Copilot CLI can see it:

copilot plugin list

Then start an interactive session and inspect loaded agents and skills:

/agent
/skills list

If you change files inside a local plugin, reinstall it:

copilot plugin install ./my-devops-plugin

Plugin contents are cached at install time, so editing the source folder does not automatically update the installed copy.

Creating a Plugin Marketplace

A Copilot CLI marketplace is not the same as the traditional GitHub Marketplace. It is a Git repository based registry. Any repository can become a marketplace by adding a marketplace.json file at:

.github/plugin/marketplace.json

A simple marketplace might look like this:

{
  "name": "platform-team-plugins",
  "owner": {
    "name": "Example Platform Team",
    "email": "platform@example.com"
  },
  "metadata": {
    "description": "Curated Copilot CLI plugins for our engineering organisation.",
    "version": "1.0.0"
  },
  "plugins": [
    {
      "name": "devops-release-helper",
      "description": "Release engineering helpers for CI/CD and safe deployments.",
      "version": "1.0.0",
      "source": "./plugins/devops-release-helper"
    }
  ]
}

If that file lives in octo-org/platform-copilot-plugins, users can register the marketplace with:

copilot plugin marketplace add octo-org/platform-copilot-plugins

Then they can browse it:

copilot plugin marketplace browse platform-team-plugins

And install from it:

copilot plugin install devops-release-helper@platform-team-plugins

This is where marketplaces become powerful for teams. A platform team can curate approved plugins for cloud operations, security reviews, incident response, documentation, and release engineering. Developers get a simple install command, while the organisation keeps the source in Git where it can be reviewed, versioned, and audited.

Enterprise Plugin Standards

For larger organisations, GitHub documents enterprise plugin standards as a public preview feature. Enterprise administrators can publish standard marketplaces and enabled plugins through .github/copilot/settings.json in the enterprise .github-private repository.

An example configuration looks like this:

{
  "extraKnownMarketplaces": {
    "our-internal-marketplace": {
      "source": {
        "source": "github",
        "repo": "acme-corp/copilot-plugins"
      }
    }
  },
  "enabledPlugins": {
    "security-scanner@our-internal-marketplace": true,
    "code-standards@our-internal-marketplace": true
  }
}

This gives enterprises a central way to make known marketplaces available and enable approved plugins for users.

Security Considerations

Plugins can change what Copilot CLI can do, so treat them like developer tooling that runs with your local permissions.

Inspect before installing

Before installing a plugin, inspect:

plugin.json, to understand what components are included
.mcp.json, to see which external tools or endpoints are configured
hooks.json, to see commands that may run automatically
Agent and skill instructions, to understand behavioural changes

The github/awesome-copilot repository also warns that community customisations come from third-party developers and should be inspected before use.

Understand MCP policy

For Copilot Business and Enterprise, GitHub documents that MCP server use is controlled by organisation or enterprise policy and is disabled by default. If a plugin relies on MCP servers, users may need admin policy changes before it can work.

Be careful with tool approvals

Copilot CLI has a tool approval model for actions such as shell commands and file changes. Approving a broad command for the rest of a session can be convenient, but it also increases risk. Keep approval narrow when you are testing a new plugin, and avoid running the CLI with --allow-all-tools while you are evaluating an untrusted plugin. Use --allow-tool for the specific tools you actually need.

Prefer reviewed marketplaces

For teams, the best pattern is to create an internal marketplace. Put plugin changes through pull request review, pin versions where possible, and document what each plugin is allowed to do.

Practical DevOps Use Cases

Here are a few ways I would use Copilot CLI plugins in a DevOps workflow.

Use case	Plugin pattern
Cloud operations	Bundle Azure or AWS MCP servers with cloud runbook skills
Incident response	Ship an incident commander agent with log query and triage skills
Release engineering	Package deployment checklists, rollback prompts, and CI/CD helpers
Security review	Combine GitHub Advanced Security skills with secure coding agents
Documentation	Install documentation MCP plugins so Copilot can ground answers in official docs
Platform enablement	Publish internal standards as agents and skills through a team marketplace

The key point is repeatability. A good plugin turns tribal knowledge into a versioned package that every engineer can install and use in the same way.

Limitations and Current State

There are a few important details to keep in mind.

The docs do not show a copilot plugin search command. Discovery is through marketplace browsing and web UIs such as awesome-copilot.github.com.
Plugin files are cached at install time. Reinstall local plugins after editing them.
Plugin agents and skills do not override project-level or personal customisations. Project context still wins.
MCP servers from plugins may be affected by organisation or enterprise policy.
Enterprise plugin standards and the GitHub MCP Registry are documented as public preview features, so behaviour may change.
There is no separate plugin pricing model documented in the sources I reviewed. Plugins appear to be part of the Copilot CLI experience, but marketplace entries may depend on external services with their own costs.

Final Thoughts

Copilot CLI plugins are a practical way to make the terminal agent fit your real workflow. They turn agents, skills, hooks, MCP servers, and language integrations into installable packages. Marketplaces then give teams and communities a way to share those packages without copy-paste setup instructions.

If you are just getting started, browse awesome-copilot, install one or two plugins that match your stack, and inspect how they are structured. Then build a small internal plugin for a workflow your team repeats every week. Deployment checks, incident triage, and security review are all great first candidates.

The best plugins will not replace engineering judgement. They will capture your team's judgement and make it easier to apply consistently from the terminal.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 27-05-2026

Agent Package Manager (APM): A DevOps Guide to Reproducible AI Agents

Marcel.Lupo — Tue, 21 Apr 2026 14:21:12 +0000

Agent Package Manager (APM): A DevOps Guide to Reproducible AI Agents

If you have been customising GitHub Copilot, Claude Code, or Cursor for your team, you have probably hit the same wall I did. You spend a weekend crafting the perfect set of instructions, prompts, skills, and chat modes, you commit them to .github/ and .claude/, and then a teammate joins the project and has a totally different agent experience because their config drifted weeks ago.

There has been no package.json for AI agent configuration. Until now.

APM (Agent Package Manager) is an open-source project from Microsoft that treats agent configuration the same way npm, pip, or NuGet treat code dependencies. You declare what your project needs in an apm.yml file, commit a lockfile, and every developer or CI runner gets the exact same agent setup in seconds.

In this post we will look at APM from a DevOps angle, walk through a first install, and then wire it up to the awesome-copilot marketplace so you can pull in battle-tested plugins, skills, and agents without writing any of your own.

Why APM Matters for DevOps

The official tagline on the microsoft/apm README says it plainly:

APM – Agent Package Manager. An open-source, community-driven dependency manager for AI agents. Think package.json, requirements.txt, or Cargo.toml, but for AI agent configuration.

From a platform engineering perspective, that unlocks a few things we normally take for granted on the code side:

Reproducibility. An apm.lock.yaml pins every dependency to a full 40-character commit SHA, so a clone today and a clone in six months produce the same agent behaviour.
Portability. The same manifest works across GitHub Copilot, Claude Code, Cursor, OpenCode, and Codex. Write once, run in every editor on the team.
Governance. No central registry means there is no single point of compromise. Everything resolves from git, over SSH or HTTPS, and a built-in apm audit scans for hidden Unicode and prompt-injection payloads at install time.
CI friendly. There is an official microsoft/apm-action, a SARIF output for Code Scanning, and a bundle-and-ship flow (apm pack / apm unpack) for matrix and air-gapped builds.
No runtime footprint. Per the security docs, APM has no telemetry, no callbacks, and no arbitrary code execution. It is literally git clone plus cp plus a manifest.

APM is still a working draft (manifest schema 0.1, CLI in the 0.x range), so pin versions and expect the odd rough edge, but the shape of the tool is already very useful.

Core Concepts in Plain English

APM introduces a handful of concepts. Here is what they actually mean.

Concept	What it is	Where it lives
`apm.yml`	Manifest that declares your project's agent dependencies.	Repo root, committed.
`apm.lock.yaml`	Lockfile pinning every dep to an exact commit SHA.	Repo root, committed.
`apm_modules/`	Where downloaded packages are cached. Think `node_modules`.	Repo root, gitignored.
Deployed files	Primitives copied into `.github/`, `.claude/`, `.cursor/`, etc. after install.	Committed, so Copilot on github.com also sees them.
Primitives	The building blocks: instructions, prompts, agents, skills, chatmodes, hooks, plugins, MCP servers.	Inside packages.
Marketplace	A git-hosted index (e.g. awesome-copilot) you can search and install from.	Remote.

The important DevOps takeaway is that apm.yml, apm.lock.yaml, and the deployed files under .github/, .claude/, and .cursor/ all get committed. apm_modules/ does not. This is exactly the opposite of npm, and it is deliberate: it means Copilot on github.com and any teammate who has not run apm install yet still get the correct context.

Installing the APM CLI

APM ships native binaries for macOS, Linux, and Windows x86_64. Pick whichever installer suits your platform.

Linux and macOS:

curl -sSL https://aka.ms/apm-unix | sh

Windows (PowerShell):

irm https://aka.ms/apm-windows | iex

Homebrew, Scoop, or pip (alternative paths):

# Homebrew
brew install microsoft/apm/apm

# Scoop (Windows)
scoop bucket add apm https://github.com/microsoft/scoop-apm
scoop install apm

# pip (works but the native installer is the recommended path)
pip install apm-cli

Verify the install and check for updates:

apm --version
apm update --check

apm update will self-upgrade using the same native installer that put it on your machine.

Your First `apm.yml`

Scaffold a manifest in an existing repo:

cd my-devops-project
apm init -y

That creates an apm.yml that looks roughly like this. Here is the richer version straight out of the dependencies guide, annotated so you can see what each line buys you:

name: your-project
version: 1.0.0
dependencies:
  apm:
    # A whole Anthropic skill (folder with SKILL.md + assets)
    - anthropics/skills/skills/frontend-design

    # A plugin from the awesome-copilot marketplace
    - github/awesome-copilot/plugins/context-engineering

    # A single agent primitive file from any repo
    - github/awesome-copilot/agents/api-architect.agent.md

    # A full APM package pinned to a tag
    - microsoft/apm-sample-package#v1.0.0

    # Any git host (GitLab, Bitbucket, Azure DevOps, self-hosted)
    - https://gitlab.com/acme/coding-standards.git

    # Local path, useful for monorepos
    - ./packages/my-shared-skills

    # Object form when you need a sub-path and a ref
    - git: https://gitlab.com/acme/coding-standards.git
      path: instructions/security
      ref: v2.0

  mcp:
    # MCP server reference from the GitHub MCP Registry
    - io.github.github/github-mcp-server

The dependency string follows a consistent pattern: <host>/<owner>/<repo>[/<sub-path>][#<ref>]. If you omit the host, github.com is assumed. Ref pinning is standard git: tags (#v1.0.0), branches (#main), or raw commits (#a1b2c3…).

How to Use APM: A 5 Minute Walkthrough

Let us take the exact example the APM maintainer shared and run it end to end.

1. Install a single plugin on the fly

apm install github/awesome-copilot/plugins/context-engineering

Three things happen:

APM clones the github/awesome-copilot repo (shallow, depth 1) into apm_modules/.
It detects that plugins/context-engineering is a plugin folder (has a plugin.json) and copies its primitives into .github/, .claude/, .cursor/, or wherever your target agent expects them.
It adds an entry to apm.yml under dependencies.apm: and records the resolved commit SHA in apm.lock.yaml.

2. Or commit a full manifest and install everything

Hand-edit apm.yml with the dependencies you want, then run:

apm install

This resolves every dep in the manifest (including transitive ones), pins each to a commit SHA, and deploys the primitives into your project.

3. Inspect what you pulled in

apm deps list          # flat table, with primitive counts
apm deps tree          # hierarchical view of transitive deps
apm view github/awesome-copilot/plugins/context-engineering versions   # list tags/branches

4. Keep things fresh

apm outdated           # diff lockfile vs remote refs
apm deps update        # re-resolve and bump the lockfile
apm uninstall github/awesome-copilot/plugins/context-engineering
apm prune              # remove anything no longer referenced

5. Commit the right files

# .gitignore
apm_modules/

Commit apm.yml, apm.lock.yaml, and the deployed .github/, .claude/, .cursor/ directories. Your teammates now get the identical agent setup after git pull && apm install.

Wiring Up the awesome-copilot Marketplace

awesome-copilot is GitHub's community-curated library of Copilot agents, instructions, skills, prompts, chat modes, hooks, and plugins. It is pre-registered as the default marketplace in Copilot CLI and VS Code, and APM knows about it too.

Register and search

apm marketplace add github/awesome-copilot
apm marketplace list
apm search "terraform@awesome-copilot"
apm marketplace browse awesome-copilot

Install with the short `@awesome-copilot` suffix

apm install azure-cloud-development@awesome-copilot
apm install devops-oncall@awesome-copilot

This is equivalent to the long form apm install github/awesome-copilot/plugins/azure-cloud-development, but nicer to type and easier to share in docs.

Crucially, and this is the point the APM maintainer was making, awesome-copilot entries do not need to ship their own apm.yml. APM detects the shape of each folder (plugin, skill, hook package, single primitive file) and handles it correctly. You get to consume the full community catalogue without the authors having to adopt APM first.

DevOps picks worth installing today

Here are a handful of entries I reach for on every project. Verify names against the live marketplace before pinning, since the catalogue moves fast.

Entry	APM reference	Why DevOps engineers care
Azure cloud development plugin	`github/awesome-copilot/plugins/azure-cloud-development`	Bicep, Terraform, serverless, cost optimisation.
DevOps on-call plugin	`github/awesome-copilot/plugins/devops-oncall`	Prompts and a chat mode for incident triage on Azure.
Azure IaC generator agent	`github/awesome-copilot/agents/azure-iac-generator.agent.md`	Single-file agent, drops in as `.agent.md`.
Azure Policy analyser agent	`github/awesome-copilot/agents/azure-policy-analyzer.agent.md`	Reviews policy assignments and compliance.
Agent governance skill	`github/awesome-copilot/skills/agent-governance`	Guardrails and review checklists for agent rollouts.
Agent supply chain skill	`github/awesome-copilot/skills/agent-supply-chain`	Threat modelling for agent dependencies.
App Insights instrumentation skill	`github/awesome-copilot/skills/appinsights-instrumentation`	Adds telemetry calls to your code.

A realistic apm.yml for a DevOps repo might look like this:

name: platform-engineering-toolkit
version: 1.0.0
dependencies:
  apm:
    - github/awesome-copilot/plugins/azure-cloud-development#main
    - github/awesome-copilot/plugins/devops-oncall
    - github/awesome-copilot/skills/agent-governance
    - github/awesome-copilot/skills/appinsights-instrumentation
    - github/awesome-copilot/agents/azure-iac-generator.agent.md
  mcp:
    - io.github.github/github-mcp-server

Run apm install, commit the result, and every engineer on the team picks up the same Azure, DevOps, and governance tooling the next time they pull.

Running APM in CI

For pipeline reproducibility, use the official microsoft/apm-action (GitHub Actions) or the native installer in any other CI system.

A minimal GitHub Actions example that enforces lockfile integrity and runs the security audit:

name: Agent Config CI
on: [push, pull_request]

jobs:
  apm:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install APM
        run: curl -sSL https://aka.ms/apm-unix | sh

      - name: Resolve dependencies
        run: apm install --dry-run

      - name: Security audit
        run: apm audit --ci -f sarif -o apm-audit.sarif

      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: apm-audit.sarif

A few things to call out:

apm install --dry-run fails the job if the lockfile and manifest disagree, catching accidental drift in PRs.
apm audit scans downloaded primitives for hidden Unicode and prompt-injection characters (the "Glassworm" class of attacks) and can emit SARIF for GitHub Code Scanning, JSON for custom pipelines, or Markdown for step summaries.
For matrix or air-gapped builds, use apm pack --archive in one job to produce a tarball and apm unpack in downstream jobs to avoid re-cloning every dep.

Publishing Your Own APM Package

There is no registry and no apm publish. Publishing is literally "push to a git repo". The layout the first-package tutorial recommends is:

my-team-standards/
├── apm.yml
└── .apm/
    ├── instructions/
    ├── prompts/
    ├── skills/
    ├── agents/
    └── hooks/

Workflow:

apm init my-team-standards
# drop primitives under .apm/...
git init && git add . && git commit -m "Initial package"
git tag v1.0.0
git remote add origin https://github.com/my-org/my-team-standards.git
git push -u origin main --tags

Teammates or other repos consume it with:

apm install my-org/my-team-standards#v1.0.0

Tag releases with semver, keep a CHANGELOG, and treat breaking changes the way you would for any library.

DevOps Best Practices and Pitfalls

A short list of things I wish I had known before I started using APM in anger.

Pin with tags, not main. It is tempting to track a branch, but the whole point of the lockfile is that upgrades are deliberate. Use #v1.0.0 style refs and bump with apm deps update.
Commit the deployed files. If you do not commit .github/, .claude/, or .cursor/, Copilot on github.com (and first-time clones before apm install) will miss context.
Use devDependencies for authoring helpers. When you are building a plugin with apm init --plugin, anything you apm install --dev is excluded from the shipped bundle.
Keep MCP trust explicit. Transitive MCP servers require --trust-transitive-mcp to auto-register. That is a feature, not an annoyance: an .agent.md you installed should not be allowed to silently bring a new MCP server into your editor.
Treat APM deps like any other supply chain. Review the upstream repo, pin to a commit SHA for critical deps, and run apm audit in CI. The agent-supply-chain skill in awesome-copilot is a decent starting checklist.
Global installs have their place. apm install -g <pkg> deploys to ~/.copilot/, ~/.claude/, and friends, which is handy for personal utilities you want everywhere, without touching per-project manifests.

Conclusion

APM finally gives AI agent configuration the same treatment we have long given our code: a manifest, a lockfile, transitive resolution, a marketplace, audit tooling, and CI integration. For DevOps and platform engineering teams, that is the difference between "my Copilot is faster than yours" and "our whole team benefits from the same curated agent stack, reproducibly, every time".

Combine it with awesome-copilot and you get an instant on-ramp: hundreds of community-maintained plugins, skills, and agents you can pull into any project with one line of YAML.

My suggested next steps:

Install the APM CLI and run apm init in one of your repos.
Add one plugin from awesome-copilot, for example apm install devops-oncall@awesome-copilot.
Commit apm.yml, apm.lock.yaml, and the deployed files, then wire up microsoft/apm-action in CI.
Once you are comfortable, package your own team standards under .apm/ and share them across repos.

If you want to dig deeper, the APM docs, the CLI reference, and the enterprise security guide are the three pages worth bookmarking.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 21-04-2026

Steer GitHub Copilot CLI Sessions Remotely from Any Device

Marcel.Lupo — Wed, 15 Apr 2026 14:27:00 +0000

Steer GitHub Copilot CLI Sessions Remotely from Any Device

You kick off a complex refactoring task in Copilot CLI, then realise you need to leave your desk. Maybe it is time for lunch, a meeting just started, or you simply want to keep an eye on progress from your phone while you are on the couch. Until now, walking away from your terminal meant walking away from your session.

That changed on 13 April 2026 when GitHub shipped remote access for Copilot CLI sessions in public preview. You can now start a session on your workstation and pick it up, monitor it, and steer it from GitHub.com or GitHub Mobile, all in real time.

If you have been following the GitHub Copilot series, you will know we have covered the CLI in depth in the Copilot CLI Practical DevOps Guide. This post focuses entirely on the new remote access capability, how to enable it, what you can do with it, and how to make it work smoothly for everyday developer workflows.

What Is Copilot CLI Remote Access?

In a nutshell, remote access lets you stream a running Copilot CLI session to GitHub so you can interact with it from any browser or from the GitHub Mobile app.

Aspect	Detail
Status	Public preview (April 2026)
Available on	GitHub.com, GitHub Mobile (iOS beta via TestFlight, Android beta via Google Play)
Who can use it	Copilot Individual, Business, and Enterprise users (policy must be enabled for org/enterprise seats)
Session privacy	Only the account that started the session can view or interact remotely
Output limit	60 MB of session output streamed to the remote interface

The local CLI session continues to run on your machine. Remote access simply gives you a window into that session from another device. All shell commands, file operations, and tool executions remain local.

Prerequisites

Before you can use remote access, make sure you have the following in place.

1. Latest Copilot CLI version

Make sure you are running the latest version of Copilot CLI. Inside an interactive session, use the /update slash command:

/update

2. A GitHub repository

Your working directory must contain a Git repository hosted on GitHub.com. If you try to enable remote access outside a GitHub repository, the CLI will display:

Remote session disabled: not in a GitHub repository

3. Policy enabled (organisation and enterprise users)

For users with a Copilot seat from an organisation, the Remote Control policy must be enabled by an enterprise or organisation owner. This policy is off by default.

Note: Individual Copilot subscribers (Pro, Pro+) can use remote access without any additional policy configuration.

Organisation owners can enable the policy under:

Organisation Settings > Copilot > Policies > Remote Control

For enterprise-level policy management, see the GitHub docs on administering Copilot CLI.

4. An interactive session

Remote access is only available for interactive sessions. It is not available when you use the CLI programmatically with the --prompt command-line option, for example when using the CLI in a script.

5. Machine must stay online

The CLI session must be actively running on a machine with an internet connection. If the machine goes to sleep or loses connectivity, remote access is unavailable until it comes back online. We will cover how to handle this with the /keep-alive command later in this post.

How to Enable Remote Access

There are three ways to enable remote access, depending on your workflow preference.

Option A: The /remote slash command (mid-session)

Already in an interactive session? Simply type:

/remote

The CLI connects to GitHub.com and displays:

A URL in the format https://github.com/OWNER/REPO/tasks/TASK_ID that you can open in any browser.
A QR code you can scan with your phone to jump straight into GitHub Mobile.

Option B: The --remote flag (at startup)

If you know ahead of time that you want remote access, start your session with the flag:

copilot --remote

Remote access details are displayed as soon as the interactive session starts. You can redisplay them at any time by typing /remote.

Option C: Always-on configuration

If you always want remote access enabled, add this to your Copilot configuration file (typically ~/.copilot/config.json):

{
  "remoteSessions": true
}

Every interactive session will now start with remote access enabled automatically. To override for a specific session, use:

copilot --no-remote

Tip: The command-line options --remote and --no-remote always take precedence over the remoteSessions setting in the configuration file.

Accessing Your Session from GitHub.com

Once remote access is enabled, you have two ways to reach the session on the web.

Use the direct link

The CLI displays a URL when remote access is enabled. Open it in any browser and sign in with the same GitHub account that started the session.

Use the Copilot sidebar

Click the hamburger menu (top-left corner) on GitHub.com.
Click Copilot.
Under Recent agent sessions, find and click your CLI session.

The web interface shows the full session history in real time. You can scroll through output, read Copilot's responses, and interact exactly as you would in the terminal.

Accessing Your Session from GitHub Mobile

Remote sessions also appear in the GitHub Mobile app. This is particularly useful for quick check-ins and approving permission requests while you are away from your desk.

Finding your session

Open GitHub Mobile.
Tap the Copilot button in the bottom-right corner.
Your session appears under Agent sessions. Tap it to open.

Using a QR code for quick access

If you are at your terminal and want to quickly open the session on your phone:

Type /remote in the CLI to redisplay the session details.
Press Ctrl+E to toggle the QR code display.
Scan the QR code with your phone's camera.

Note: Mobile access is currently only available in the latest beta release of GitHub Mobile. Join via Apple TestFlight for iOS or Google Play for Android.

What You Can Do Remotely

The remote interface is not a read-only dashboard. It is a full steering wheel. Here is what you can do from GitHub.com or GitHub Mobile:

Action	Description
Respond to permission requests	Approve or deny tool, file path, and URL permission requests
Answer questions	Respond when Copilot asks for more information or a decision
Approve or reject plans	Respond to plan approval prompts in plan mode
Submit new prompts	Type new questions or instructions, just like in the terminal
Switch modes	Change between interactive, plan, and autopilot mode
Cancel work	End the agent's current operation

Both the local terminal and the remote interface are active simultaneously. You can type in either, and Copilot uses whichever response arrives first.

Note: Slash commands such as /allow-all are not currently available from the remote interface.

Keeping Your Machine Awake with /keep-alive

One of the most practical additions alongside remote access is the /keep-alive slash command. It prevents your machine from going to sleep while you are steering the session remotely.

Usage options

Command	Behaviour
`/keep-alive on`	Prevents sleep while the CLI session is active
`/keep-alive off`	Allows normal sleep behaviour
`/keep-alive busy`	Prevents sleep only while Copilot is actively working. Once the agent completes a task, the machine can sleep normally
`/keep-alive 30m`	Prevents sleep for 30 minutes
`/keep-alive 8h`	Prevents sleep for 8 hours
`/keep-alive 1d`	Prevents sleep for 1 day
`/keep-alive`	Displays the current keep-alive status

The busy option is particularly useful. It keeps the machine awake while Copilot is processing but lets it sleep once Copilot finishes and is waiting for input.

Reconnection and Resuming Sessions

Temporary disconnections

If the network drops temporarily, the remote interface reconnects automatically once connectivity is restored. The local session is completely unaffected by network interruptions.

Resuming a closed session

If you close a CLI session that had remote access enabled, the CLI displays a resume command:

copilot --resume=SESSION_ID --remote

You can also resume the most recent session with:

copilot --continue --remote

If you have "remoteSessions": true in your configuration file, you can omit the --remote flag when resuming:

copilot --continue

Important: When you resume a session, you must re-enable remote access (either via the --remote flag or the configuration file setting). Remote access does not persist automatically across session restarts.

A Practical Walkthrough

Here is a real-world scenario that demonstrates how remote access fits into a developer workflow.

Step 1: Start a session with remote access

You are at your workstation and want to refactor a Terraform module. Start the session with remote access:

cd ~/repos/my-infra
copilot --remote

The CLI displays the session URL and QR code.

Step 2: Give Copilot the task

Refactor the networking module to use for_each instead of count for subnet
creation. Update all references in the root module and ensure the plan shows
no destructive changes.

Copilot starts analysing the codebase and building a plan.

Step 3: Leave your desk

You have a meeting in five minutes. Before leaving, type:

/keep-alive busy

This ensures the machine stays awake while Copilot is working.

Step 4: Monitor from your phone

During the meeting, open GitHub Mobile and tap into your session. You can see Copilot's progress in real time. When it asks for plan approval, tap Approve right from your phone.

Step 5: Steer from the browser

Back at your desk (or from any computer), open the session link in a browser. Copilot has finished the refactoring and is asking whether to run terraform plan. Type your approval in the web interface.

Step 6: Review the results

Check the output in the web interface or switch back to your terminal. Both show the same session state.

Security and Privacy

Remote access has been designed with security in mind. Here are the key points:

Session-specific URLs. Each session gets a unique URL that only works for the authenticated user who started it.
No direct machine access. Remote access does not grant any access to your machine beyond what the CLI agent can do within the session. All operations run through the Copilot CLI's existing permission model.
Policy controls. Organisation and enterprise administrators can disable remote access entirely through the Remote Control policy.
Events streamed, not stored. Session events (messages, tool executions, permission requests) are streamed from your machine to GitHub for the remote interface. The CLI continues to run locally.

Tips and Best Practices

Here are some practical tips for getting the most out of remote access.

Use /keep-alive busy as your default. It prevents your machine from sleeping while Copilot works, but lets it rest once the agent is idle. This is the best balance between availability and power consumption.
Start with copilot --remote rather than adding it mid-session. This ensures the session is remotely accessible from the very first prompt, so you never miss a permission request.
Set remoteSessions: true in your config if you use this regularly. It removes the need to remember the flag, and you can always override with --no-remote for local-only sessions.
Bookmark the Copilot sidebar on GitHub.com. Navigate to the Copilot page and bookmark it. This gives you one-click access to all your recent sessions without needing to remember URLs.
Use the QR code for mobile. It is faster than searching for the session in the app, especially when you are in a hurry.
Be mindful of the 60 MB output limit. Very long-running sessions that generate large amounts of output may see reduced performance in the remote interface. The local terminal remains unaffected.

Conclusion

Remote access for Copilot CLI sessions solves a genuine pain point. Long-running tasks no longer tie you to your terminal. You can start a complex refactoring, infrastructure deployment, or code review task, walk away, and continue steering it from your browser or phone.

The feature is in public preview right now. Give it a try with copilot --remote and see how it fits into your workflow. If you are an organisation or enterprise user, ask your admin to enable the Remote Control policy so your team can start using it.

For more details, check the official documentation:

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 15-04-2026

Mastering Code Reviews with GitHub Copilot: The Definitive Guide

Marcel.Lupo — Fri, 13 Mar 2026 09:46:56 +0000

Mastering Code Reviews with GitHub Copilot: The Definitive Guide

Pull requests keep piling up. Your team reviews dozens a week, and each one needs careful attention to security, performance, style, and correctness. Human reviewers catch domain-specific issues that machines miss, but they also miss the mechanical things that machines are brilliant at, such as spotting unhandled edge cases, flagging deprecated API calls, or enforcing naming conventions across hundreds of files.

What if you could get an AI-powered first pass on every pull request before a human even looks at it? Better still, what if that AI reviewer lived in your editor, your terminal, your GitHub.com workflow, and your custom automation, all at once?

GitHub Copilot offers not one but eight distinct surfaces for AI-assisted code review. This guide maps every one of them, shows you how to configure each for maximum value, and walks through a real end-to-end review workflow that combines several surfaces together.

If you have been following the GitHub Copilot series, you have already seen individual pieces of this puzzle across previous posts on custom instructions, MCP, the coding agent, and the customisation guide. This post brings everything together through the lens of code review.

The 8 Surfaces of AI-Assisted Code Review

Before we dive into each method, here is the landscape at a glance. Every surface serves a different context in your workflow.

#	Method	Where It Runs	Best For	Setup Required	Automation Level
1	GitHub.com Native Review	Browser (github.com)	PR reviews at scale	Org/repo settings	High
2	VS Code Review Selection	Editor (VS Code)	In-flight code review	Settings toggle	Low
3	Custom Instructions	VS Code + GitHub.com	Team review standards	`.github/` config files	Medium
4	Prompt Files	VS Code	Repeatable review workflows	`.prompt.md` files	Medium
5	Custom Agents	VS Code	Specialised review roles	`.agent.md` files	Medium
6	MCP-Powered PR Review	VS Code	Deep PR analysis with live context	MCP server config	Medium
7	Coding Agent	GitHub Actions	Fully automated review tasks	Coding agent enabled	High
8	Copilot CLI	Terminal	Pre-commit local checks	Copilot CLI installed	Low

The rest of this guide walks through each surface in detail, then ties them together in a worked example.

1. GitHub.com Native Copilot Code Review

This is the highest-leverage surface for teams. Copilot acts as an automated reviewer directly on your pull requests in the browser, no editor or CLI needed.

How It Works

Navigate to any pull request on github.com.
Click Reviewers in the sidebar and select Copilot from the reviewer list.
Copilot analyses the diff and posts inline review comments, just like a human reviewer would.

You can also configure Copilot to automatically review every PR when it is opened or updated, so you never have to remember to request it.

What Feedback Looks Like

Copilot's review comments appear inline on the PR diff. Each comment includes:

A description of the issue found.
A suggested fix (where applicable) that you can accept with one click.
Severity context so you can prioritise critical issues over style nits.

Comments cover categories such as:

Bugs and logic errors (off-by-one, null dereferences, race conditions).
Security concerns (injection risks, hardcoded secrets, missing input validation).
Performance issues (unnecessary allocations, N+1 queries, blocking calls in async code).
Best practice violations (deprecated APIs, missing error handling, inconsistent patterns).

Custom Coding Guidelines

By default, Copilot reviews against general best practices. To tailor feedback to your team's standards, create a coding guidelines file:

.github/copilot-code-review-instructions.md

This file contains natural language descriptions of your team's review standards. For example:

## Security

- All database queries must use parameterised queries, never string concatenation.
- Secrets must be loaded from environment variables or a vault, never hardcoded.
- All public endpoints must validate and sanitise input.

## Naming

- Use PascalCase for public methods, camelCase for private methods.
- Infrastructure resources must follow the pattern: {env}-{region}-{service}-{resource}.

## Testing

- Every public method must have at least one unit test.
- Integration tests must clean up after themselves.

## Error Handling

- Never swallow exceptions silently. Log at minimum.
- Use typed errors rather than generic Error objects.

When Copilot reviews a PR in your repository, it reads these guidelines and incorporates them into its analysis. This is how you move from generic AI feedback to feedback that matches your team's actual standards.

Enabling at Organisation and Repository Level

Repository administrators can configure Copilot code review under Settings > Copilot > Code review:

Enable/disable Copilot as an available reviewer.
Auto-review: Automatically request Copilot review on every new or updated PR.
Custom coding guidelines: Point to your .github/copilot-code-review-instructions.md file.

At the organisation level, administrators can enable or enforce code review settings across all repositories, ensuring consistent review coverage.

Limitations

Language support: Copilot code review works best with widely-used languages (JavaScript, TypeScript, Python, Go, Java, C#, Ruby, etc.). Less common languages may receive less detailed feedback.
Context window: Very large PRs may exceed the context window. Copilot reviews what it can and notes when it could not review all files.
Not a replacement for human review: Copilot catches mechanical issues brilliantly but does not understand your business domain. It is a first pass, not the final word.

Tip: You can work around the language support limitation by pairing Copilot with an advanced reasoning model (such as Claude Opus or Gemini 3 Pro in extended thinking mode) or a custom agent that has access to online sources via MCP web tooling or Playwright. For example, create a .agent.md reviewer that uses a web search tool to look up the latest best practices, idioms, or security advisories for a less common language. The agent can research real time strategies, verify its findings against up to date documentation, and cite the references it used so you can check them yourself. This is especially useful for newer frameworks or niche languages where the model's training data may be limited. See Section 5: Custom Agents and Section 6: MCP-Powered PR Review later in this guide for setup details.

Official docs: GitHub Copilot code review

2. VS Code Review Selection

If GitHub.com native review is the "PR-level" surface, VS Code review selection is the "code-level" surface. It lets you review any block of code in the editor, at any time, not just when a PR is open.

How to Trigger It

Select code in the editor (a function, a class, a block, whatever you want reviewed).
Right-click and choose Copilot > Review and Comment.
Alternatively, open the Command Palette (Ctrl+Shift+P) and run GitHub Copilot: Review and Comment.

Copilot analyses the selected code and posts its findings directly as editor comments, inline in your code.

You can also trigger a review of all uncommitted changes directly from the Source Control panel.

Copilot posts its findings as inline comments with suggested changes that you can apply or discard with a single click.

Configuring Review Instructions

You can customise what Copilot looks for during VS Code reviews using two settings. These live in your VS Code settings.json file. To open it:

Press Ctrl+Shift+P (or Cmd+Shift+P on macOS) and run Preferences: Open Settings (JSON).
For workspace-level settings shared with your team, create .vscode/settings.json in your repository root.

Add or update the following:

{
  // Enable the review selection feature
  "github.copilot.chat.reviewSelection.enabled": true,

  // Define what Copilot should focus on during reviews
  "github.copilot.chat.reviewSelection.instructions": [
    {
      "text": "Check for security vulnerabilities including injection attacks, hardcoded secrets, and missing input validation.",
    },
    {
      "text": "Flag any functions longer than 50 lines and suggest decomposition.",
    },
    {
      "text": "Verify error handling covers all failure paths.",
    },
  ],
}

You can also point to an external file for review instructions:

{
  "github.copilot.chat.reviewSelection.instructions": [
    {
      "file": ".github/review-instructions.md",
    },
  ],
}

This keeps your review criteria version-controlled and consistent across the team.

When to Use This Surface

During development: Review your own code before you even commit.
Pair programming: Get an instant second opinion on a tricky function.
Learning: Understand unfamiliar code by asking Copilot to review and explain it.
Pre-PR polish: Catch issues before they reach the formal review stage.

For a deeper dive into VS Code Copilot settings, see my earlier post: Tune GitHub Copilot Settings in VS Code

3. Custom Instructions for Code Reviews

Custom instructions shape how Copilot behaves across your workspace. For code review, they let you embed your team's review standards so that every interaction, whether in chat, inline completion, or review, reflects your agreed-upon practices.

Where to Define Review Instructions

You have three options, each with a different scope:

File	Scope	Use When
`.github/copilot-instructions.md`	Every Copilot interaction	You want review standards applied everywhere
`.github/instructions/code-review.instructions.md`	Pattern-matched files	You want review rules for specific file types
`AGENTS.md`	Multi-agent workflows	You use multiple AI tools, not just Copilot

Example: A Code Review Instructions File

Create .github/instructions/code-review.instructions.md:

---
applyTo: '**/*.{ts,js,py,go,cs}'
---

# Code Review Standards

When reviewing code, always check:

## Security

- No hardcoded secrets, tokens, or connection strings.
- All user input is validated and sanitised before use.
- SQL queries use parameterised statements.
- No sensitive data logged at INFO level or above.

## Reliability

- All async operations have proper error handling.
- Network calls include timeouts and retry logic.
- Resource cleanup is handled in finally blocks or using statements.

## Maintainability

- Functions do one thing and are under 40 lines where practical.
- Variable names are descriptive and follow project conventions.
- No commented-out code left in place. Use version control instead.

## Performance

- No unnecessary allocations in hot paths.
- Database queries are indexed and avoid N+1 patterns.
- Large collections use streaming or pagination rather than loading everything into memory.

The applyTo glob ensures these instructions activate whenever Copilot works with source code files, without cluttering interactions with documentation or configuration files.

How Instructions Shape Review Output

When you use Copilot to review code, either via VS Code review selection, chat, or GitHub.com, it reads these instructions and adjusts its analysis accordingly. The instructions act as a persistent checklist that Copilot applies without you having to repeat it every time.

This is the foundation layer. The other surfaces (prompt files, agents, MCP) build on top of it.

For a full walkthrough, see: Instructions and Prompt Files to supercharge VS Code with GitHub Copilot

4. Prompt Files for Structured Reviews

While instructions define standards, prompt files create repeatable review workflows that you invoke on demand. Think of them as named review commands.

Creating a Review Prompt File

Create .github/prompts/security-review.prompt.md:

---
description: 'Run a security-focused code review on the selected code'
mode: 'ask'
tools: ['githubRepo', 'codebase']
---

Perform a thorough security review of the current code. For each issue found, provide:

1. **Severity**: Critical / High / Medium / Low
2. **Location**: File and line reference
3. **Issue**: What the problem is
4. **Risk**: What could go wrong if left unfixed
5. **Fix**: Specific code change to resolve it

Check for:

- Injection vulnerabilities (SQL, XSS, command injection)
- Authentication and authorisation gaps
- Hardcoded secrets or credentials
- Missing input validation
- Insecure cryptographic usage
- Sensitive data exposure in logs or error messages
- Missing rate limiting on public endpoints

Format output as a table sorted by severity.

Using It

In VS Code chat, type /security-review and Copilot runs the review following your exact template. The output is structured, consistent, and actionable every time.

More Review Prompt Ideas

Here are additional prompt files you might create:

Prompt File	Purpose
`performance-review.prompt.md`	Analyse code for performance bottlenecks, memory leaks, and scaling concerns
`test-coverage-review.prompt.md`	Identify untested code paths and suggest test cases
`accessibility-review.prompt.md`	Check UI code for accessibility compliance (WCAG)
`iac-review.prompt.md`	Review Terraform/Bicep for security, cost, and best practices
`api-review.prompt.md`	Review API endpoints for consistency, versioning, and documentation

Each prompt file becomes a slash command in VS Code chat. Your team builds a library of review plays that anyone can run.

For more on prompt files, see: GitHub Copilot Instructions vs Prompts vs Custom Agents vs Skills vs X vs WHY?

5. Custom Agents for Code Reviews

Custom agents take things further by creating a dedicated reviewer persona with specific tool access and boundaries. Instead of a general-purpose assistant that also reviews code, you build a specialist.

Why Use a Custom Agent for Reviews?

Role clarity: The agent only reviews. It does not generate new code, refactor, or make changes.
Tool restrictions: You can limit the agent to read-only tools so it cannot accidentally modify files during review.
Consistent voice: The agent's system prompt defines its review personality, checklist, and output format.
Team alignment: Everyone uses the same reviewer agent, so feedback is consistent regardless of who runs it.

Example: Security Review Agent

Create .github/agents/security-reviewer.agent.md:

---
description: 'A security-focused code reviewer that analyses code for vulnerabilities and compliance issues'
tools: ['codebase', 'fetch', 'githubRepo']
---

You are a senior security engineer performing code review. Your role is strictly read-only. You analyse code but never modify it.

## Review Process

1. Examine the code provided or referenced in the conversation.
2. Check against the OWASP Top 10, CWE Top 25, and project-specific security policies.
3. For each finding, provide severity, location, description, risk, and remediation guidance.

## Output Format

Present findings in a structured report:

### Summary

- Total issues found: X
- Critical: X | High: X | Medium: X | Low: X

### Findings

For each finding:

- **ID**: SEC-001
- **Severity**: Critical/High/Medium/Low
- **Category**: (e.g., Injection, Auth, Crypto)
- **Location**: File and line
- **Description**: What the issue is
- **Risk**: Impact if exploited
- **Remediation**: How to fix it

## Rules

- Never suggest "it looks fine" without evidence of thorough checking.
- Always check for hardcoded secrets, even in test files.
- Flag any use of deprecated cryptographic algorithms.
- If you cannot determine security posture from the available context, say so explicitly.

Using It

Switch to the security-reviewer agent in VS Code chat. Everything you ask is filtered through the security reviewer's lens. Ask it to review a file, a diff, or a PR, and you get structured security feedback every time.

You can create different review agents for different concerns: architecture-reviewer.agent.md, performance-reviewer.agent.md, accessibility-reviewer.agent.md, each with its own checklist and tool set.

For a deeper look at custom agents: GitHub Copilot Instructions vs Prompts vs Custom Agents vs Skills vs X vs WHY?

6. MCP-Powered PR Reviews

The Model Context Protocol (MCP) lets Copilot connect to external tools and data sources. For code review, the GitHub MCP server is the key enabler. It gives Copilot live access to pull request data: diffs, comments, linked issues, CI status, and more.

Setting Up the GitHub MCP Server

If you followed my earlier post on MCP setup, you already have this configured. The GitHub MCP server provides tools such as:

get_pull_request to fetch PR metadata and description.
get_pull_request_diff to retrieve the actual code changes.
list_pull_request_comments to see existing review feedback.
get_issue to pull in linked issue context.
list_pull_request_files to see which files changed.
create_pull_request_review_comment to post review comments back.

Review Workflow with MCP

Here is how you can run a PR review entirely from VS Code chat using MCP:

Step 1: Fetch the PR context

Look at PR #42 in my repository. Fetch the diff, the PR description, and any linked issues.

Copilot uses the MCP tools to pull all the context it needs.

Step 2: Analyse the changes

Review the changes in PR #42 for security issues, bug risks, and adherence to our coding standards.
Cross-reference with the linked issue to ensure all requirements are addressed.

Step 3: Post feedback

Post your review findings as inline comments on PR #42, grouped by severity.

Copilot can post review comments directly on the PR via MCP, so the feedback appears on github.com where the rest of the team can see it.

Why MCP Changes the Game

Without MCP, Copilot in VS Code only sees the files open in your editor. With MCP, it can:

Pull in the full PR diff, even files you have not opened.
Read existing review comments so it does not duplicate feedback.
Check CI status to correlate test failures with code changes.
Follow linked issues to understand the intent behind the changes.
Post comments back to the PR so everything stays in one place.

This turns VS Code chat into a full PR review workstation.

For MCP setup details, see: Supercharge VSCode GitHub Copilot using MCP

7. Coding Agent for Automated Reviews

The GitHub Copilot coding agent is an autonomous AI that runs in a GitHub Actions environment. While it is primarily designed for coding tasks, you can use it for review-adjacent automation.

How It Applies to Code Review

The coding agent works through GitHub Issues. You describe a task, assign it to Copilot, and it opens a PR with the result. For review workflows, consider these patterns:

Pattern 1: Fix issues found in review

Title: Address review feedback on PR #42

Review the comments on PR #42 and implement the requested changes:
- Fix the SQL injection vulnerability in UserService.cs
- Add input validation to the CreateUser endpoint
- Add unit tests for the new validation logic

Assign this issue to Copilot, and it will create a new PR addressing the review feedback.

Pattern 2: Review-driven refactoring

Title: Refactor authentication module per review guidelines

The authentication module in src/auth/ has accumulated technical debt.
Apply our coding standards from .github/copilot-code-review-instructions.md:
- Extract duplicated validation logic into shared utilities
- Add proper error handling to all OAuth flows
- Ensure all endpoints validate JWT tokens consistently

Pattern 3: Pre-review preparation

Title: Add missing tests before review

Before PR #42 is reviewed, ensure:
- All new public methods have unit tests
- Integration tests cover the new API endpoints
- Test coverage for the changed files is above 80%

The coding agent handles the mechanical work so that human reviewers can focus on design decisions and domain logic.

Key Considerations

The coding agent cannot approve or merge PRs. It always produces a PR for human review.
Every PR it creates is scanned by CodeQL and secret scanning.
It runs in an isolated container, so it cannot affect your production environment.
One task, one PR. Complex reviews may need to be broken into separate issues.

For full setup and use cases, see: Using GitHub Copilot Coding Agent for DevOps Automation

8. Copilot CLI for Local Reviews

GitHub Copilot CLI is a standalone terminal agent that replaces the retired gh copilot extension. You invoke it with the copilot command. It can read files, run shell commands, and interact with GitHub, making it ideal for reviewing changes before you commit or push.

Quick Local Review

Start an interactive session and ask Copilot to review your staged changes:

copilot

Then type your review prompt:

Review my staged changes for bugs, security issues, and style problems

Or use the programmatic interface for a one-shot review:

copilot -p "Review my staged changes for bugs, security issues, and style problems" --allow-tool='shell(git)'

Targeted Reviews

Review a specific file:

copilot -p "Review src/auth/login.ts for security vulnerabilities" --allow-tool='shell(cat)'

Review changes between branches:

copilot -p "Review the changes between main and feature/auth-refactor, focusing on authentication security" --allow-tool='shell(git)'

Check the changes in a pull request directly from the terminal:

copilot -p "Check the changes made in PR #42. Report any serious errors you find" --allow-tool='shell(gh)'

When to Use CLI Reviews

Pre-commit check: Catch issues before they enter version control.
Quick sanity check: Get a fast opinion on a small change without opening an editor.
CI integration: Add a review step to your pipeline that flags issues early.
Remote work: Review code on a server where you only have terminal access.

The CLI is a full agentic experience. It can read your codebase, run shell commands, and even create pull requests. For review purposes, use --allow-tool='shell(git)' to give it read access to your Git history without granting broader permissions.

For CLI setup and more examples, see: GitHub Copilot CLI: A DevOps Engineer's Practical Guide

The Comparison Cheat Sheet

Use this table to pick the right review surface for your situation:

Method	Where	Best For	Setup Effort	Automation	Team Visibility
GitHub.com Native	Browser	PR reviews at scale	Low (org setting)	High (auto-review)	High (comments on PR)
VS Code Selection	Editor	In-flight code	Minimal (built-in)	Manual trigger	Low (local only)
Custom Instructions	Both	Team standards	Low (one file)	Passive (always-on)	Medium (shared via repo)
Prompt Files	VS Code	Repeatable plays	Low (per prompt)	On-demand	Medium (shared via repo)
Custom Agents	VS Code	Specialised roles	Medium (agent config)	On-demand	Medium (shared via repo)
MCP PR Review	VS Code	Deep PR analysis	Medium (server setup)	On-demand	High (can post to PR)
Coding Agent	GitHub	Automated fixes	Low (already enabled)	High (issue-driven)	High (creates PRs)
Copilot CLI	Terminal	Pre-commit checks	Low (install `copilot`)	Manual/scriptable	Low (local only)

Choosing Your Starting Point

Team just getting started? Enable GitHub.com native review. It requires no editor changes and covers every PR automatically.
Individual developer wanting better habits? Start with VS Code review selection. It is built in and instant.
Team with established coding standards? Add custom instructions. Your standards become Copilot's standards.
Mature team wanting structured workflows? Build prompt files and agents. Create a library of review plays.
Dealing with complex PRs and cross-references? Set up MCP. Live context makes reviews dramatically better.

Worked Example: End-to-End PR Review

Let us walk through a realistic review workflow that combines multiple surfaces. Imagine you have a teammate's PR that adds a new user registration endpoint with database access and email notifications.

Step 1: Pre-Push Local Check (CLI)

Before the PR even exists, the developer reviews their own changes locally:

copilot -p "Review my staged changes for security issues, particularly around user input handling and database access" --allow-tool='shell(git)'

Copilot flags that the email field is not validated and the SQL query uses string interpolation. The developer fixes both before pushing.

Step 2: PR Created, Copilot Auto-Reviews (GitHub.com)

The developer pushes and creates a PR. Copilot automatically reviews it (auto-review is enabled) and posts inline comments:

High: Missing rate limiting on the registration endpoint.
Medium: The error response leaks internal database column names.
Low: Variable usr should be user per naming conventions.

The team's .github/copilot-code-review-instructions.md includes rate limiting and error handling standards, so Copilot catches these against the team's own rules.

Step 3: Deep Dive in VS Code (Review Selection + MCP)

You open the PR branch in VS Code. You select the registration handler function and use Copilot > Review and Comment for a focused analysis.

Then you use MCP to pull in broader context:

Fetch PR #87, including the diff, linked issue #156, and any existing review comments.
Review the entire PR for security compliance. Consider the requirements in issue #156
and check that all acceptance criteria are met.

Copilot correlates the PR changes with the issue requirements and identifies that one acceptance criterion (email verification flow) is missing from the implementation.

Step 4: Structured Security Review (Prompt File)

You run /security-review in VS Code chat. The prompt file produces a structured table:

Severity	Location	Issue	Fix
Critical	`register.ts:45`	No rate limiting	Add express-rate-limit middleware
High	`register.ts:62`	SQL string interpolation	Use parameterised query
Medium	`register.ts:78`	Error leaks DB schema	Return generic error message
Low	`email.ts:12`	SMTP credentials in env check	Add startup validation

Step 5: Automated Fix (Coding Agent)

For the mechanical fixes, you create a GitHub Issue:

Title: Address security review findings on PR #87

Fix the following issues found during code review:
1. Add rate limiting middleware to POST /api/register (max 5 requests per minute per IP)
2. Convert SQL query on line 62 to use parameterised statements
3. Replace detailed error responses with generic messages
4. Add SMTP credential validation at startup

The coding agent picks up the issue, implements the fixes, and opens a follow-up PR.

Step 6: Human Review

With the mechanical issues resolved, the human reviewer focuses on what matters most:

Does the registration flow make business sense?
Is the email notification content appropriate?
Does the database schema support future requirements?
Are the architectural decisions sound?

AI handled the mechanical first pass. The human focuses on design, domain, and judgement.

Tips and Best Practices

Layer your review surfaces. No single surface catches everything. Use GitHub.com auto-review as your always-on baseline, prompt files for structured deep dives, and MCP for complex PRs.
Write clear review instructions. Whether in .github/copilot-code-review-instructions.md, VS Code settings, or agent definitions, specific instructions produce specific feedback. "Check for security issues" is vague. "Ensure all user input is sanitised, all queries are parameterised, and no secrets are hardcoded" is actionable.
Treat AI review as a first pass, not the final word. Copilot is exceptional at catching mechanical issues (bugs, security patterns, style violations) but does not understand your business domain. Human reviewers should always have the last say on design, architecture, and domain logic.
Keep review instructions version-controlled. Store them in .github/ so they evolve with your codebase and are reviewed like any other code change.
Start small, expand gradually. Enable GitHub.com auto-review today. Add custom instructions next week. Build your first prompt file next month. You do not need all eight surfaces on day one.
Use different agents for different review concerns. A security review agent should not worry about variable naming. A style review agent should not flag architectural choices. Specialised agents give focused, high-quality feedback.
Combine MCP with prompt files for maximum context. A prompt file defines the review structure. MCP provides the live data (PR diff, comments, issues). Together, they produce the most thorough reviews.
Review the reviewer. Periodically check Copilot's review output against your team's actual standards. If it is flagging too many false positives or missing real issues, update your instructions.

Conclusion

Code review is one of the highest-value activities in software engineering, and also one of the most time-consuming. GitHub Copilot provides eight distinct surfaces for AI-assisted review, each suited to a different moment in your workflow:

GitHub.com catches issues at the PR level, automatically.
VS Code catches issues at the code level, on demand.
Custom instructions make your team's standards automatic.
Prompt files turn review workflows into repeatable commands.
Custom agents create dedicated reviewer personas.
MCP connects reviews to live PR data and project context.
The coding agent automates fixes for review findings.
The CLI enables fast local checks before code leaves your machine.

The real power comes from combining them. Start with GitHub.com auto-review as your baseline, layer on custom instructions for your team's standards, and add prompt files and agents as your review workflows mature.

Your reviewers will thank you. Your codebase will thank you. And the bugs that never make it to production? They will never know what hit them.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 13-03-2026

Evaluating LLM Models in GitHub Copilot. A Practical Scoring and Assessment Guide

Marcel.Lupo — Fri, 06 Mar 2026 18:12:20 +0000

Evaluating LLM Models in GitHub Copilot. A Practical Scoring and Assessment Guide

GitHub Copilot gives us access to a fast-moving set of LLMs from multiple providers. That is great for innovation, but it also creates a practical problem for teams. Which model should you use for a specific task, and how do you justify that decision with evidence instead of gut feel?

This guide is a practical framework you can use with your own network and team. We will cover how model evaluation works, how to build your own scoring approach, and how to run repeatable comparisons so you can choose models with confidence as new releases arrive.

Why Model Evaluation Matters

Choosing a model is no longer a one-time decision.

Models are specialised: some are better for fast lightweight tasks, others for deeper reasoning and debugging.
Cost matters: different models have different premium request multipliers in Copilot.
Model catalogues change frequently: new models are added, and older ones are retired.
Teams need consistency: shared evaluation criteria helps avoid random model switching and uneven output quality.

If you do not evaluate, you usually optimise for what feels fastest in the moment. That often creates hidden costs later in rework, review cycles, and reliability issues.

Models Available in GitHub Copilot Today

GitHub maintains a live reference of supported AI models. The most important point is this. Treat the docs as the source of truth because availability can vary by client, plan, and release cycle.

Supported models: https://docs.github.com/en/copilot/reference/ai-models/supported-models
Model comparison: https://docs.github.com/en/copilot/reference/ai-models/model-comparison

At the time of writing, Copilot includes models from:

OpenAI (for example GPT family and Codex variants)
Anthropic (Claude family)
Google (Gemini family)
xAI (for example Grok Code Fast 1)
Copilot-tuned options in preview

A practical way to think about model choice:

Fast/simple tasks: quick syntax help, small edits, repetitive transformations
General coding/writing: day-to-day coding, docs, refactoring support
Deep reasoning/debugging: multi-step investigations, architecture-level decisions, complex defect analysis
Agentic workflows: long-running coding tasks in chat/agent modes

Also note:

Auto model selection is available in supported IDE chat experiences, and can choose a model automatically.
You can still override manually when you have evidence from your own evaluation.

How LLMs Are Evaluated in Industry

Public benchmark scores are useful signals, but they are not the whole story.

Common Benchmark Families

Benchmark	What it tests	Typical use
MMLU	Broad knowledge and reasoning	General model capability checks
HumanEval / MBPP	Code generation correctness	Coding assistant comparisons
SWE-bench	Real software issue resolution	Engineering task realism
GSM8K / MATH	Mathematical reasoning	Structured reasoning quality
ARC / HellaSwag	Commonsense and reasoning	General reasoning robustness
TruthfulQA	Truthfulness and hallucination tendency	Reliability risk checks
MT-Bench / Arena-style pairwise	Conversational quality and preference	Human preference alignment

Useful references:

MMLU paper: https://arxiv.org/abs/2009.03300
HumanEval repo: https://github.com/openai/human-eval
SWE-bench: https://www.swebench.com/
Chatbot Arena: https://lmarena.ai/
Hugging Face leaderboard: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard

Why Benchmarks Alone Are Not Enough

Benchmarks can miss your local context:

your coding standards
your cloud/platform stack
your repo structure
your incident patterns
your definition of acceptable risk

A model can rank highly and still underperform on your real workloads. That is why self-assessment is essential.

A Practical Evaluation Framework You Can Run Yourself

Use a lightweight internal framework you can rerun monthly or quarterly.

Step 1: Define Evaluation Scenarios

Create 10 to 20 prompts based on real work. Keep them representative and repeatable.

Examples for DevOps/platform teams:

Fix a failing GitHub Actions workflow from logs
Refactor a Terraform module and preserve backwards compatibility
Generate tests for a PowerShell script with edge cases
Write an incident summary from deployment telemetry

Step 2: Define a Scoring Rubric

Score each response against the same dimensions.

Dimension	1 (Poor)	3 (Acceptable)	5 (Excellent)
Correctness	Wrong or unusable output	Works with fixes	Works first time, production-ready
Reasoning quality	No clear logic	Basic explanation	Clear reasoning with trade-offs
Instruction adherence	Misses key constraints	Follows most constraints	Follows all constraints precisely
Security and safety	Introduces risky patterns	Neutral/safe enough	Proactively applies safer patterns
Maintainability	Hard to read/operate	Adequate quality	Clear, testable, maintainable output
Latency usefulness	Too slow for value	Acceptable speed	Fast with strong quality

Step 3: Run Blind Comparisons

Do not show model names to reviewers during scoring.

Use the same prompt set for each model.
Randomise response order.
Have at least two reviewers score each output.
Average the scores to reduce individual bias.

Step 4: Add Cost-Performance Scoring

Quality alone is not enough to pick a model. A model that scores slightly higher but costs several times more per request may not be worth it for everyday tasks. Adding a cost dimension keeps your evaluation grounded.

Where to find cost data. GitHub publishes premium request multipliers for each model in the Copilot docs. These multipliers tell you how many premium requests a single interaction with that model consumes relative to the baseline. Use this as your cost proxy.

Premium request costs: https://docs.github.com/en/copilot/managing-copilot/monitoring-usage-and-entitlements/about-premium-requests

Calculate a value score. Divide your average quality score by the model's premium multiplier:

Value Score = Average Quality Score / Premium Multiplier

Worked example. Suppose three models score as follows after your blind evaluation:

Model	Avg quality (1-5)	Premium multiplier	Value score
Model A	4.4	1x	4.4 / 1 = 4.4
Model B	4.7	3x	4.7 / 3 = 1.57
Model C	4.1	0.33x	4.1 / 0.33 = 12.42

Model B scores highest on raw quality, but its value score is the lowest because each request costs three times the baseline. Model C delivers almost the same quality at a fraction of the cost, making it the best value pick for lightweight tasks.

When to ignore the value score. Cost efficiency matters most for high-volume, routine tasks. For critical debugging or architecture decisions where correctness has outsized impact, pick the highest quality model regardless of cost. Use the value score as a guide, not a rule.

Practical Ways to Run Your Own Evaluation

There are several free approaches you can use depending on your team size and automation needs. Here are two that work well with GitHub Copilot.

Option A: Manual Side-by-Side Comparison in Copilot Chat

The simplest method is to use GitHub Copilot Chat directly. No extra tooling required.

Open Copilot Chat in VS Code and select your first model.
Run a prompt from your evaluation set and capture the response.
Switch to the next model using the model picker and run the same prompt.
Score each response using your rubric and record the results.

A simple markdown table works well for tracking:

| Prompt | Model A | Model B | Model C | Notes |
| --- | --- | --- | --- | --- |
| Fix failing workflow | 4 | 5 | 3 | B caught root cause immediately |
| Refactor Terraform | 3 | 4 | 4 | A missed a variable dependency |
| Generate Pester tests | 5 | 4 | 3 | A covered more edge cases |

This approach works best for:

Individual contributors evaluating models for their own workflow.
Small teams running a quick monthly check.
Getting started before investing in automation.

Tips for a fair manual comparison:

Use the exact same prompt text for each model.
Do not reveal the model name to the scorer if more than one person is reviewing.
Record latency (how long you waited) alongside quality scores.

Option B: Automated Evaluation with DeepEval

If you want repeatable, scriptable evaluation that integrates with CI/CD, DeepEval is a practical open-source choice. It uses pytest-style test cases and supports custom scoring criteria via an LLM-as-a-judge approach.

Website: https://deepeval.com/
GitHub: https://github.com/confident-ai/deepeval

1. Install

pip install -U deepeval

2. Create a test file

Create test_copilot_eval.py with test cases that match your evaluation scenarios:

import pytest
from deepeval import assert_test
from deepeval.metrics import GEval
from deepeval.test_case import LLMTestCase, LLMTestCaseParams


correctness = GEval(
    name="Correctness",
    criteria="The response must correctly identify the root cause and provide a secure, working fix.",
    evaluation_params=[
        LLMTestCaseParams.ACTUAL_OUTPUT,
        LLMTestCaseParams.EXPECTED_OUTPUT,
    ],
    threshold=0.5,
)

maintainability = GEval(
    name="Maintainability",
    criteria="The response must produce clear, readable, and well-structured code.",
    evaluation_params=[LLMTestCaseParams.ACTUAL_OUTPUT],
    threshold=0.5,
)


def test_workflow_fix():
    test_case = LLMTestCase(
        input="Given this GitHub Actions error: 'OIDC token audience invalid', identify root cause and fix.",
        actual_output="<paste model response here>",
        expected_output="The OIDC token audience must match the value configured in your identity provider. Update the audience field in your federated identity settings.",
    )
    assert_test(test_case, [correctness])


def test_terraform_refactor():
    test_case = LLMTestCase(
        input="Refactor this Terraform snippet to reduce duplication while preserving behaviour.",
        actual_output="<paste model response here>",
    )
    assert_test(test_case, [maintainability])

3. Run the evaluation

deepeval test run test_copilot_eval.py

4. Interpret results

When you run the evaluation, DeepEval produces terminal output similar to the following:

Running 2 test(s)...

  test_workflow_fix
    ✅ Correctness (score: 0.82, threshold: 0.5, passed: True)

  test_terraform_refactor
    ✅ Maintainability (score: 0.71, threshold: 0.5, passed: True)

======================= Results =======================
Tests run: 2, Passed: 2, Failed: 0
Overall pass rate: 100.0%

Metric scores:
  Correctness       avg: 0.82   min: 0.82   max: 0.82
  Maintainability   avg: 0.71   min: 0.71   max: 0.71
=======================================================

Each test case is scored on a 0 to 1 scale against your criteria. A score above the threshold you set counts as a pass. If a test fails, the output shows the reason so you can see where the model response fell short:

  test_workflow_fix
    ❌ Correctness (score: 0.34, threshold: 0.5, passed: False)
       Reason: The response did not identify the OIDC audience
       mismatch as the root cause and suggested unrelated
       permission changes instead.

To compare models, run the same test file once per model (swapping in each model's responses for actual_output) and record the results side by side:

Test case	Metric	Model A	Model B	Model C
test_workflow_fix	Correctness	0.82	0.91	0.64
test_terraform_refactor	Maintainability	0.71	0.78	0.69
Overall pass rate		100%	100%	50%

This gives you a quantitative basis for model comparison that you can track over time. You can also:

Add multiple metrics per test (correctness, security, maintainability).
Integrate tests into your CI/CD pipeline for regression checks when new models are released.
Export results to JSON for further analysis or dashboard reporting.

Define custom GEval metrics for each dimension in your scoring rubric to get automated scoring that aligns with your team's specific criteria.

Other Free Tools Worth Knowing

A few other open-source options are available if you want to explore further:

OpenAI Evals (https://github.com/openai/evals): a framework and registry of benchmarks for evaluating LLMs and LLM systems. Good for standardised benchmark testing.
LM Evaluation Harness (https://github.com/EleutherAI/lm-evaluation-harness): the backend behind the Hugging Face Open LLM Leaderboard. Supports 60+ academic benchmarks with API model support. Best suited for formal benchmark comparisons.

Recommended Team Operating Model

Use this cadence so your model decisions stay current:

Run a small benchmark set every month.
Run a fuller evaluation set each quarter.
Re-run when GitHub adds or retires major models.
Publish a one-page internal scorecard for your team.

A simple scorecard format:

Model	Avg quality	Avg latency	Cost proxy	Value score	Best-fit tasks
Model A	4.4	1.2s	1x	4.4	General coding
Model B	4.7	2.1s	3x	1.57	Deep debugging
Model C	4.1	0.9s	0.33x	12.42	Fast lightweight edits

This makes model selection transparent and easier to defend in architecture and governance reviews.

Final Thoughts

The model landscape in GitHub Copilot will keep moving. That is a feature, not a problem, if your team has a repeatable evaluation method.

Start with a small prompt set, apply a consistent scoring rubric, and review quality, latency, and cost together. Once your network sees the method in action, model choice becomes a practical engineering decision instead of a subjective debate.

Additional Resources

Supported AI models in Copilot: https://docs.github.com/en/copilot/reference/ai-models/supported-models
AI model comparison in Copilot: https://docs.github.com/en/copilot/reference/ai-models/model-comparison
Changing model in Copilot Chat: https://docs.github.com/en/copilot/using-github-copilot/ai-models/changing-the-ai-model-for-copilot-chat
DeepEval docs: https://deepeval.com/docs/getting-started
DeepEval GitHub: https://github.com/confident-ai/deepeval
OpenAI Evals: https://github.com/openai/evals
LM Evaluation Harness: https://github.com/EleutherAI/lm-evaluation-harness

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 06-03-2026

GitHub Copilot Instructions vs Prompts vs Custom Agents vs Skills vs X vs WHY?

Marcel.Lupo — Thu, 26 Feb 2026 09:09:32 +0000

GitHub Copilot Instructions vs Prompts vs Custom Agents vs Skills vs X vs WHY?

If you have been following my recent GitHub Copilot posts, you might have noticed a pattern. We have covered instructions, prompt files, skills, MCP, coding agents, and more. Each feature is powerful on its own. The confusing part is deciding which one to use for a specific problem.

This post is your practical decision guide.

We will compare:

Custom Instructions
Prompt Files
Custom Agents
Skills
MCP Servers
Hooks

The 60-Second Cheat Sheet

Primitive	Typical location	Scope	Best for	When not to use
Custom Instructions	`.github/copilot-instructions.md`, `.github/instructions/*.instructions.md`, `AGENTS.md`	Always-on or pattern-based	Team standards and default behaviour	One-off tasks or named workflows
Prompt Files	`.github/prompts/*.prompt.md`	Manual, on-demand slash command	Repeatable one-shot tasks	Multi-step runbooks with assets
Custom Agents	`.github/agents/*.agent.md`	Selected agent mode/persona	Specialised role plus tool control	General coding help or tiny tweaks
Skills	`.github/skills/<name>/SKILL.md`	On-demand and auto-discovered	Reusable multi-step workflows with resources	Simple standards or hard policy enforcement
MCP Servers	`.vscode/mcp.json` or user `mcp.json`	Tool and data connectivity	Live access to external systems and APIs	Static guidance that does not need external context
Hooks	`.github/hooks/*.json`	Agent workflow lifecycle events	Hard policy gates and safety controls	Soft guidance or stylistic preferences

If you remember one thing, remember this:

Instructions for always-on guidance.
Prompts for named one-off tasks.
Skills for reusable workflows.
Custom Agents for role and tool boundaries.
MCP for external live context.
Hooks for hard stop enforcement.

1) Custom Instructions: "Always apply these rules"

Use custom instructions when you want Copilot to behave consistently without repeating yourself in every chat.

What they are

Markdown instructions that are automatically added to context.
Three flavours:
- Global: .github/copilot-instructions.md applies to every chat request in the workspace.
- File/task-targeted: *.instructions.md files with an applyTo glob pattern or task description match. Store them in .github/instructions/.
- Multi-agent compatible: AGENTS.md in the workspace root is recognised by multiple AI agents (not only Copilot). Supports subfolder-level scoping for monorepos (experimental).
Instructions can also be shared at GitHub organisation level, so every repository in the org inherits a common baseline. (Note: Organisation-level instructions require a Copilot Business or Enterprise plan.)
Priority order when conflicts occur: personal (user-level) > repository > organisation.

Suitale Usage Examples

You want all IaC suggestions to follow your team's naming and tagging conventions, e.g. every resource must include environment and cost-centre tags.
You want consistent code style across the repo, e.g. British English in comments, four-space indentation, or a preferred import order.
You want secure defaults baked into every suggestion, e.g. private endpoints by default, no public IPs without justification, secrets always pulled from a vault.
You work with multiple AI agents and want a single AGENTS.md recognised by all of them, so rules are defined once and shared.

Do not use when

You only need a task once.
You need a named slash command.
You need a full runbook that includes extra scripts and templates.

Quick guide: which instruction type?

Type	When to reach for it
`copilot-instructions.md`	Single project, Copilot-only
`.instructions.md` + `applyTo`	Different rules for different file types or frameworks
`AGENTS.md`	Multi-agent workflows, or subfolder-level monorepo rules
Organisation instructions	Shared baseline across all repos in a GitHub org

DevOps example

"For all cloud IaC, always use least privilege, avoid hardcoded secrets, and follow approved resource naming conventions."

For a deeper walkthrough, see my earlier post: Instructions and Prompt Files to supercharge VS Code with GitHub Copilot

Official docs: Use custom instructions in VS Code

2) Prompt Files: "Run this specific play"

Prompt files are reusable slash commands for recurring tasks.

What they are

Markdown prompt files, usually in .github/prompts/*.prompt.md.
Invoked with / in chat, just like slash commands.
YAML frontmatter can define the agent, tools, model, and description.
Support built-in variables such as ${selection}, ${file}, and ${input:variableName} for dynamic context.
Can reference a custom agent via the agent field, inheriting that agent's tool set.

Suitale Usage Examples

You want a quick /security-review command that scans the current file for common misconfigurations such as open ports or missing encryption.
You want /generate-module to scaffold a new IaC module with consistent structure, inputs, outputs, and documentation every time.
You want /release-notes that always follows a fixed changelog format, pulling context from recent commits.
You want to override the default agent for a specific task by setting agent: plan in the frontmatter, e.g. a /design-review prompt that runs inside a planning-only agent.

Do not use when

You need always-on behavioural standards.
You need automatic discovery of a rich workflow with supporting resources.
You need deterministic enforcement logic.

DevOps example

Create /pipeline-hardening that checks secrets handling, approval gates, artifact integrity, and rollback readiness. Use ${input:environment} to let the user specify which environment to review.

Official docs: Use prompt files in VS Code

3) Custom Agents: "Use this specialist persona and tool set"

Custom Agents were previously known as custom chat modes.

What they are

Agent definition files in .github/agents/*.agent.md.
Define a specialist persona, instructions, allowed tools, and optional handoffs.
Designed for role-specific flows such as planning, review, or implementation.
Handoffs let you chain agents into guided workflows. After one agent finishes, a handoff button appears to transition to the next agent with pre-filled context. For example: Plan > Implement > Review.
Can run as subagents and can also be reused in background agents and cloud agents. (Note: The coding agent requires a Copilot Pro+, Business, or Enterprise plan.)
Can be shared at GitHub organisation level, so every repo in the org gets the same agent definitions. (Note: Organisation-level agent sharing requires a Copilot Business or Enterprise plan.)

Suitale Usage Examples

You want a Security Reviewer agent that can only read code and run linters but never edit files, e.g. a pre-merge review step that flags risks without changing anything.
You want a Planner agent that outputs a structured implementation plan and then hands off to an Implementation agent, giving the team a review checkpoint before code is written.
You want strict tool boundaries, e.g. an Auditor agent with access to cloud cost APIs and read-only file tools, but no ability to execute commands or apply changes.
You need a guided multi-step workflow where each stage has a different persona and tool set, such as Design > Build > Test.

Do not use when

You just need a short reusable prompt.
You only need default team standards.
You are trying to solve external data access without MCP.

DevOps example

A cost-optimiser.agent.md that only has read-only cloud and repo tools, then hands off to a separate implementation agent for approved changes. The handoff button appears automatically after the cost analysis completes.

Official docs: Custom agents in VS Code

4) Skills: "Package a reusable runbook"

Skills are where things get interesting for SRE and DevOps teams.

What they are

Folder-based capability packages with a required SKILL.md.
Stored in .github/skills/<skill-name>/ (workspace) or ~/.copilot/skills/ (personal, available across all workspaces).
Can include scripts, templates, references, and examples alongside the instructions.
Loaded progressively: Copilot reads name and description first, loads full instructions only when relevant, and accesses bundled resources on demand.
Portable: Skills follow the open Agent Skills standard. The same skill works across VS Code, GitHub Copilot CLI, and the GitHub Copilot coding agent.

Suitale Usage Examples

You want a repeatable incident triage workflow that walks through impact assessment, timeline capture, and owner assignment using a bundled template.
You want a postmortem assistant that generates a structured report with root-cause analysis, action items, and SLA impact, pulling from scripts and examples in the skill folder.
You want a CI/CD troubleshooting playbook the whole team can invoke, e.g. a skill that runs diagnostic scripts against a failed pipeline and suggests fixes.
You want cross-tool portability so the same workflow works in VS Code, the CLI, and the coding agent without maintaining separate definitions.

Do not use when

You only need a global coding rule.
You only need a tiny slash command with no supporting resources.
You need hard policy blocking behaviour.

DevOps example

A incident-triage skill that collects service impact, timeline, likely root causes, next actions, and owner assignments using a consistent template. The same skill can be invoked from the CLI during on-call triage and from VS Code during post-incident review.

My practical Skills deep dive: GitHub Copilot Skills: Reusable AI Workflows for DevOps and SREs

Official docs: Use Agent Skills in VS Code

5) MCP Servers: "Connect Copilot to live systems"

MCP is often the missing piece when people expect Copilot to access live external context.

What it is

MCP (Model Context Protocol) is an open standard for connecting AI models to external tools and services.
Configured in .vscode/mcp.json (workspace) or user profile mcp.json. Can also be installed from the VS Code MCP server gallery.
MCP servers can provide four categories of capability:
- Tools: Actions the agent can invoke (e.g. create an issue, query a database).
- Resources: Data the agent can pull into context (e.g. files, API responses, database rows).
- Prompts: Pre-configured prompt templates contributed by the server.
- MCP Apps: Interactive UI components such as forms and visualisations rendered directly in chat.
Organisations can centrally manage MCP server access via GitHub policies. (Note: Organisation-level MCP policy management requires a Copilot Business or Enterprise plan.)

Suitale Usage Examples

You need live GitHub data inside chat, e.g. listing open issues, checking workflow run status, or creating a PR directly from a conversation.
You need live cloud metadata, e.g. pulling resource tags, SKU availability, or cost estimates from Azure, AWS, or GCP during an architecture review.
You need external system context such as querying a database, pulling monitoring dashboards, or fetching ticket details from a service management tool.

Do not use when

Static instructions are enough.
The task is purely local code guidance.
You have not reviewed trust and security implications.

DevOps example

Use GitHub MCP to create issues and inspect workflow runs, then use a cloud-compatible MCP server to pull environment metadata for triage context.

My earlier MCP setup guide: Supercharge VSCode GitHub Copilot using Model Context Protocol (MCP) - Easy Setup Guide

Official docs: Add and manage MCP servers in VS Code

6) Hooks: "Enforce policy at execution time"

If instructions are advice, hooks are enforcement.

What they are

Deterministic shell commands that run at key strategic points in an agent's workflow.
Configured as JSON files in .github/hooks/*.json in your repository.
Available hook types include: sessionStart, sessionEnd, userPromptSubmitted, preToolUse, postToolUse, agentStop, subagentStop, and errorOccurred.
The preToolUse hook is the most powerful because it can approve or deny tool executions before they happen.
Hooks receive detailed JSON input about the agent's actions, enabling context-aware automation.
Work with both the Copilot coding agent on GitHub and GitHub Copilot CLI. (Note: The coding agent and CLI require a Copilot Pro+, Business, or Enterprise plan.)

Suitale Usage Examples

You need a hard control, not a recommendation, e.g. blocking any file edit that adds a public IP to IaC without an approved exception.
You must block risky operations unless a condition passes, e.g. deny bash commands that target production databases or delete cloud resource groups.
You need consistent compliance checks before agent actions proceed, e.g. validating that every new service deployment includes a required cost tag.
You want immutable audit logging of every tool invocation for regulatory or internal compliance, e.g. recording who ran what command and when.

Do not use when

A style guide or prompt convention is enough.
You do not need strict allow or deny behaviour.
You only need static behavioural guidance (use instructions instead).

DevOps example

A preToolUse hook that runs a security check script before any bash or edit tool call, blocking risky infrastructure changes in production unless required checks pass.

Official docs: About hooks for Copilot coding agent

So, Why This Many Primitives?

Because they solve different control layers:

Behaviour layer: Instructions
Task invocation layer: Prompt files
Persona and tool boundary layer: Custom agents
Workflow packaging layer: Skills
External connectivity layer: MCP
Policy enforcement layer: Hooks

This is not feature overlap. This is a composable system. You can mix and match:

A custom agent can reference instructions files via Markdown links so you do not duplicate rules.
A prompt file can set agent: to run inside a specific custom agent, inheriting its tool set.
A skill can bundle scripts that a custom agent or prompt file invokes.
An MCP server provides tools that any agent, prompt, or skill can call.
Hooks sit underneath everything, enforcing policy regardless of which primitive triggered the action.

The key insight is that each primitive has a different lifetime and trigger. Instructions are always on. Prompts are invoked manually. Skills are discovered automatically or by slash command. Agents set a persistent persona for a session. MCP provides live external data. Hooks enforce hard gates at execution time.

A Practical "When to Use What" Decision Flow

Walk through these questions in order. Pick the first match:

Do you need a rule applied to every chat request without anyone remembering to activate it? Instructions.
Do you need a named, repeatable task that people invoke on demand? Prompt File.
Do you need a specialist persona with a limited set of tools, or a multi-step handoff workflow? Custom Agent.
Do you need a reusable multi-step runbook with scripts, templates, or reference files bundled together? Skill.
Does the task require live data from an external system (GitHub API, cloud metadata, databases)? Add MCP.
Must a policy be enforced deterministically with no chance the model ignores it? Add a Hook.

Notice that 1 to 4 are "pick one" decisions, while 5 and 6 are additive layers you stack on top.

What a Mature DevOps Setup Looks Like

A practical stack for a platform engineering team might be:

Instructions: Workspace-level copilot-instructions.md for coding and security defaults. Scoped *.instructions.md files for IaC, pipeline YAML, and language-specific conventions.
Prompt files: 5 to 10 prompt files for common operational tasks such as /security-review, /release-notes, /changelog, and /generate-module.
Custom agents: 2 to 3 agents (planning, implementation, security review) connected via handoffs so the team flows from plan to code to review in one conversation.
Skills: 3 to 6 skills for triage, postmortems, runbook generation, and IaC change risk analysis. These work the same in VS Code, the CLI, and the coding agent.
MCP servers: GitHub MCP for issue and PR management, plus cloud-context MCP servers for pulling live environment metadata during triage.
Hooks: preToolUse hooks to block dangerous commands and postToolUse hooks for audit logging in autonomous coding agent workflows.

This combination gives you speed, consistency, and safety without forcing everything into one mechanism.

Final Thoughts

If you have ever asked "Should I use instructions, prompts, agents, or skills?", the answer is usually: use the smallest primitive that solves the actual problem.

Start simple:

Add instructions first for always-on standards.
Add prompt files when you catch yourself typing the same prompt repeatedly.
Add skills when a workflow needs bundled scripts and templates.
Add custom agents when you need role boundaries and handoff workflows.
Add MCP when the task requires live external data.
Add hooks where policy must be enforced, not suggested.

That is the difference between experimenting with Copilot and operationalising Copilot. Each primitive has a clear purpose, and together they form a composable customisation system.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 26-02-2026

GitHub Copilot Skills: Reusable AI Workflows for DevOps and SREs

Marcel.Lupo — Tue, 24 Feb 2026 16:35:29 +0000

GitHub Copilot Skills: Reusable AI Workflows for DevOps and SREs

If you're a DevOps engineer or SRE, you probably have a handful of repeatable tasks that keep coming back: triaging failed pipelines, checking for risky Terraform changes, writing runbooks, and turning messy incident notes into something your team can actually use.

Until recently, you could get part of the way there with custom instructions and prompt files. They are both great, but they do not fully solve the same problem: packaging a repeatable, multi-step workflow with its own supporting assets.

This is where Agent Skills comes in. Agent Skills is an open standard (see agentskills.io) that works with GitHub Copilot in VS Code, Copilot CLI, and the Copilot coding agent.

In this post we will cover what Skills are, how to set them up in VS Code, and how to use them from beginner scenarios to more advanced DevOps and SRE use cases.

What Are GitHub Copilot Skills?

A Skill is an on-demand, reusable workflow for Copilot. A Skill lives in a folder, has a required SKILL.md, and can include supporting resources such as scripts, references, and templates.

At a high level, it is designed for:

Repeatable workflows you want to reuse across a team
Multi-step procedures that benefit from checklists and branching logic
Bundled assets such as scripts, templates, and short reference docs

The key idea is progressive loading:

Copilot first uses the Skill name and description for discovery.
If the request matches, it loads the Skill instructions.
It only loads extra resources when the Skill references them.

This makes Skills a good fit for DevOps because you can keep the default Copilot experience lean, then load a specialised runbook only when you need it.

Skills vs Prompts vs Instructions vs Agents vs Hooks

Skills sit in the same overall customisation system as instructions, prompt files, custom agents, and hooks. They are not a replacement. They are a different primitive.

Here is a practical DevOps-oriented way to think about them:

Primitive	Best for	DevOps example
Workspace instructions	Always-on standards	"Tag every Azure resource with `owner` and `env`"
File instructions	Standards for certain files	Helm chart defaults for `**/values.yaml`
Prompt files	One-shot tasks	"Summarise this sprint's deployment changelog"
Skills	Reusable workflows with assets	Kubernetes rollback playbook + kubectl scripts
Custom agents	Specialised role and tool limits	Cost-optimisation advisor (read-only)
Hooks	Deterministic enforcement	Reject Terraform plans that drop deletion protection

If you're new to Copilot customisation, my rule of thumb is:

Use instructions when you want Copilot to behave consistently all the time.
Use a prompt file when you want a reusable one-off command.
Use a Skill when you want a repeatable workflow that feels like a runbook.

Prerequisites

To use Skills effectively, you will want:

VS Code with GitHub Copilot enabled
A repo (or workspace) where you can commit the Skill folder so your team can share it

In VS Code chat, type /skills to open the Configure Skills menu and confirm VS Code can see your workspace Skills.

Skills can be stored in a few locations. For team usage, the simplest is the repository:

.github/skills/<skill-name>/SKILL.md

VS Code also recognises .claude/skills/ and .agents/skills/ as project skill directories. For personal skills that follow you across workspaces, use ~/.copilot/skills/.

If you want to share Skills across multiple repos, VS Code also supports additional search locations via the chat.agentSkillsLocations setting.

Your First Skill (Beginner Setup)

1) Create the folder

Create this structure:

.github/skills/incident-triage/
└── SKILL.md

2) Add SKILL.md

The front matter must match the folder name.

---
name: incident-triage
description: 'Triage production incidents and failed deployments. Use when: a pipeline fails, an alert fires, or you need an incident update for stakeholders.'
argument-hint: 'Optional: service name, environment, and alert link'
---

# Incident Triage

## When to Use

- Pipeline failed in `prod`
- Pager/alert fired and you need a first response
- You need a status update for stakeholders

## Procedure

1. Confirm the affected service, environment, and impact
2. Collect signal: alerts, recent deploys, logs, and error budgets
3. Identify likely root cause categories (deploy, dependency, infra, config)
4. Recommend next actions: rollback, mitigation, escalation
5. Write an incident update in a consistent format

## Output Format

Return:

- Impact summary
- Timeline
- Suspected causes
- Next actions
- Owner assignments

That is enough to get started. The Skill is now discoverable based on the description.

Using Skills in Chat

There are two ways Skills help you day-to-day:

Manual invocation: you can run the Skill as a slash command.
Automatic discovery: Copilot can decide to load the Skill when the request matches the description.

To invoke a Skill manually, type / in the chat input, pick the Skill, and then add any extra context. For example: /incident-triage payments-api prod https://alert-link.

You can control this with optional front matter:

user-invokable: false hides it from / commands
disable-model-invocation: true prevents automatic loading, but keeps slash use

This matters for teams. Some Skills are safe to auto-load. Others might be better as explicit, opt-in runbooks.

DevOps and SRE Use Cases (Practical)

Here are a few Skills that are genuinely useful in real operations.

Use Case 1: CI/CD Failure Triage

Goal: Standardise how you investigate failures and what you record.

A Skill can:

Ask for the workflow URL and failing job name
Require you to record whether it looks transient or deterministic
Produce a consistent issue template for follow-up

Use Case 2: Terraform Change Risk Review

Goal: Make IaC reviews consistent across a team.

A Skill can:

Require you to list affected environments
Check for risky patterns (public exposure, identity changes, state migration)
Output a simple go/no-go summary and required approvals

Use Case 3: Runbook Generator

Goal: Turn tribal knowledge into a maintained operational runbook.

A Skill can:

Enforce sections (Symptoms, Checks, Rollback, Escalation)
Add a "Safety" section (blast radius, data loss risk)
Produce a markdown runbook that matches your house style

Use Case 4: Postmortem Assistant

Goal: Reduce the time from incident to learning.

A Skill can:

Convert incident notes into a timeline
Extract contributing factors and remediation items
Enforce blameless language and clear action items

Advanced Patterns (Where Skills Get Interesting)

Once the basics work, Skills shine when you treat them like reusable operational modules.

1) Bundle references and templates

Keep SKILL.md short and link out to extra files in the same folder:

references/ for short docs you want to load on demand
assets/ for templates (issue templates, runbook templates)
scripts/ for small utilities

2) Keep descriptions keyword-rich

The description is the discovery surface. If you want the Skill to load for DevOps/SRE workflows, include words like:

incident, outage, on-call, pipeline, deployment, rollback
terraform, kubernetes, azure, aws, gcp
security review, compliance, audit

3) Decide which Skills auto-load

For teams, I recommend:

Auto-load for safe, low-risk guidance Skills (triage checklists)
Manual-only for high-impact Skills that might lead to edits or deployments

Real Examples You Can Explore

This repo already contains a working Skill you can inspect:

.github/skills/new-blog-post/SKILL.md in this repository

If you want ready-made public examples to learn from (without relying on GitHub code search), start here:

https://github.com/github/awesome-copilot (community collection)
https://github.com/anthropics/skills (reference skills)

Copy any skill folder into .github/skills/, then tweak the wording and guardrails until it matches how your team works.

If you want to find more public examples on GitHub, search for the folder pattern:

path:.github/skills SKILL.md

Note that GitHub code search may require you to sign in.

For the authoritative reference, see the VS Code documentation:

https://code.visualstudio.com/docs/copilot/customization/agent-skills

Tips for Adoption in DevOps Teams

Start with one runbook: pick a repetitive task (CI triage) and ship one Skill.
Treat Skills as code: version them, review changes, keep them short and intentional.
Measure outcomes: shorter time to triage, fewer repeated mistakes, better incident updates.
Keep it safe: do not turn every Skill into an autonomous actor. Default to guidance and checklists.

Conclusion

GitHub Copilot Skills are a practical way to turn the workflows in your head into something your whole team can reuse. For DevOps and SREs, that means more consistent triage, better runbooks, and faster, calmer incident response.

If you want, I can also turn the example incident-triage Skill into a full folder with templates and a lightweight script so you can drop it straight into .github/skills/.

Author

Marcel.LupoFollow

Microsoft MVP in DevTech - DevOps | DevOps Architect | Technical speaker focused on Microsoft technologies, Agentic AI, IaC & automation in Azure. Find me on GitHub: https://github.com/Pwd9000-ML

Like, share, follow me on: 🐙 GitHub | 🐧 X | 👾 LinkedIn

Date: 24-02-2026

DEV Community: Marcel.Lupo

Can Copilot Fix Its Own Security Findings? Testing GitHub Agentic Autofix

Can Copilot Fix Its Own Security Findings? Testing GitHub Agentic Autofix

What GitHub Announced, and What It Clarified

The Agent Loop Is the Real Change

Four Ways to Assign Work

Licence, Policy, and Cost Boundaries

A Reproducible Sandbox Evaluation

1. Prepare representative findings

2. Establish the baseline

3. Run isolated assignments

4. Apply a merge gate

Review Risks and Rollback

Troubleshooting the Preview

A Sensible Adoption Path

Author

Marcel.LupoFollow

Agentic DevOps Needs Observability: Trace GitHub Copilot with OpenTelemetry

Agentic DevOps Needs Observability: Trace GitHub Copilot with OpenTelemetry

What You Will Build

Prerequisites

Step 1: Understand the Trace You Are Looking For

Step 2: Start Jaeger Locally

Step 3: Enable Copilot OpenTelemetry Without Content

Step 4: Generate and Inspect a Trace

Step 5: Turn Signals Into Useful SLIs

Step 6: Apply Privacy Before Scale

Step 7: Manage Configuration Centrally

Copilot CLI Transport Caveats

OpenTelemetry Is Not Session Audit Streaming

Troubleshooting

Jaeger has no copilot-chat service

The collector returns 404 or connection errors

CLI traces appear disconnected

Prompt or tool content is absent

A user setting has no effect

Metrics do not appear in Jaeger

Cleanup and Rollout

Conclusion

Author

Marcel.LupoFollow

Human-in-the-Loop Agentic DevOps: Govern AI Automation in GitHub Issues

Human-in-the-Loop Agentic DevOps: Govern AI Automation in GitHub Issues

The Automation Gap

What GitHub Added to Issues

1. Confidence

2. Rationale

3. Suggestions and approvals

The Human-in-the-Loop Flow

A Practical DevOps Scenario

Build a Trust Ladder, Not an Autonomy Switch

Level 1: Observe

Level 2: Suggest

Level 3: Auto-apply low-risk actions

Level 4: Expand by evidence

Level 5: Retain hard approval boundaries

Confidence Is a Signal, Not a Control

Agent Automation Controls vs Agentic Workflows

Where Human Review Can Fail

Approval fatigue

Automation bias

Stale suggestions

Invisible corrections

Authority creep

The Bigger DevOps Lesson

Author

Marcel.LupoFollow

Run GitHub Copilot CLI in GitHub Actions Without PATs or Runaway AI Costs

Run GitHub Copilot CLI in GitHub Actions Without PATs or Runaway AI Costs

What You Will Build

Prerequisites

How AI credits translate to cost

Step 1: Enable Organisation Billing for the Workflow

Step 2: Add the Bounded Repository Review Workflow

Manual trigger and bounded choices

Short-lived authentication

Non-interactive execution

Report outside the checkout

Two independent limits

Step 3: Run the Workflow

Jaeger has no `copilot-chat` service

The collector returns `404` or connection errors

`--max-ai-credits` is unknown

Your First `apm.yml`