DEV Community: Ve Sharma

[Boost]

Ve Sharma — Mon, 06 Jul 2026 13:43:34 +0000

Ve Sharma

Jul 6

Make Every Token Count: Why Agent Quality Is the Better Way to Optimize Spend

#ai #github #githubcopilot #claude

20 min read

Make Every Token Count: Why Agent Quality Is the Better Way to Optimize Spend

Ve Sharma — Mon, 06 Jul 2026 13:43:19 +0000

GitHub Copilot switched from premium requests to usage-based billing. That single change set off a wave of questions from customers, and most of them sounded like this:

"How do we cut our token spend?"

It's the wrong question. Or at least, it's the small version of the right question.

The better question is: how do we make the most out of the tokens we spend? The goal isn't fewer tokens per request. The goal is fewer wasted tokens per successful outcome. Once you frame it that way, you stop chasing cost and start chasing quality, and the cost takes care of itself because most token waste comes from failed runs and stuffed contexts.

This post is the long-form version of a workshop I run on agent quality and token optimization. We'll cover the why, the foundations, and the practical controls you can apply today.

Slide images in this post are inpired the GitHub workshop deck of a similar name, used with credit to GitHub. The interpretation and prose are mine.

The workshop this post is based on.

TL;DR for the Busy Dev

If you only have two minutes:

Optimize for cost per successful outcome, not raw token count. Most token waste comes from failed runs and stuffed contexts.
Quality compounds. At 95% accuracy per step, a 50-step agent workflow lands at 8%. Every quality lever pays back hard.
Context is the main lever. Reasoning models for planning, smaller models for implementation. Never stuff "might-need" files. Use /clear often.
Use deterministic guardrails. Tests, linters, security scans. The Copilot CLI team ships 500 PRs a week and 53% of their codebase is tests. That's the move.
Treat agent configs like engineering. A small, human-written copilot-instructions.md beats any clever prompt.

The five concrete actions to take today live at the bottom.

The rocket problem

Stay with me on an analogy. It's the easiest way I've found to explain what has changed.

The Sputnik era

Call it the Sputnik era: early, experimental, cheap-feeling, launch-and-learn energy. Rockets are cheap, you build twenty in a weekend, you point them roughly in the right direction because you just need one to land.

That's exactly how a lot of us have been using agents. You give a little context. You write a quick prompt. You send the agent off. If it comes back with something good, you ship it. If not, you fire another one. Then another. The cost is invisible, so the gambling feels free.

How most teams still use agents: launch more, hope one hits.

The Falcon era

Now imagine launches cost what real launches cost. A SpaceX Falcon isn't cheap. You get one or two shots, not twenty. So the engineering job becomes: build a rocket that actually reaches the destination and lands back on the pad so you can reuse it.

That's where we are with agents now. Usage-based billing didn't make tokens expensive in absolute terms. It made the gambling expensive. The fix isn't cheaper fuel. The fix is fewer, better-engineered launches.

The expensive-rocket-that-explodes-on-the-pad era

There's a third failure mode worth naming, and it's the most painful one. You can absolutely build a very heavy, very expensive agent that burns a fortune in tokens and still explodes on the launch pad. Massive context window stuffed with every file that might be relevant. Every MCP turned on. Three reasoning models chained together. No tests in the loop. No stop signal. No definition of done.

The agent reasons in circles. It calls tools you didn't need. It "fixes" things that weren't broken. By turn 40 it's drifting from the original goal, hallucinating helpfully, and burning through your monthly budget. The output, when it finally arrives, is wrong.

That's the worst of the three. You paid for the moon mission and you blew up the pad.

I'll come back to this one throughout the post. Most of the controls that follow are about not building this rocket.

Why counting tokens is the wrong question

Quality drives ROI, not token frugality

Apply the standard ROI formula to agents:

Agent ROI = (Value of Agent Output − Token Cost) / Token Cost × 100%

You can't calculate this cleanly. Nobody can. How do you quantify the value of an agent that wrote a feature for you? A lot of companies are still figuring this out. But the formula doesn't have to be measurable to be useful as guidance.

The key insight: if the value is zero, the result is −100%. Optimizing the denominator from there is pointless. A free wrong answer is still a wrong answer. Even worse, optimizing for low cost often causes low value. Cheap models and skimpy context produce bad output that you then pay to debug. Now you've spent tokens on the bad run, tokens on the debug session, tokens on the fix, plus your own time.

The flip side: when you reduce waste at the source, both quality and cost improve together. Most of what we do wrong as developers is:

Stuffing irrelevant files into the context window
Letting conversations compound with useless turns we keep paying for in every following turn
Asking one model to do work that a smaller, cheaper one would handle better

Trim that, and the bill shrinks while the quality goes up.

To be clear: some of the controls in this post will increase the tokens of a single run. A reasoning model costs more than a small one. Tests in the loop add tool calls. Subagents spin up a second context. The trade is total cost across attempts, not raw tokens per request.

The standard ROI math, applied to agents: net value over cost, as a percentage.

The compounding error problem

There's one more reason quality matters more than people think. LLMs are non-deterministic. They have an error margin. They're never 100 percent accurate per step. In a multi-step agent workflow, that error compounds.

Here's the math:

Accuracy per step	After 10 steps	After 50 steps
99%	90%	61%
95%	60%	8%
90%	35%	0.5%

Read that table twice. At 95 percent per step, which feels great, you're at 8 percent after a long workflow. Most real agent sessions have many more than 50 internal steps.

This doesn't mean every agent will fail. It means every quality improvement you make doesn't give you a linear gain. It gives you a compounding one. And every miss costs you real money: the wasted tokens, the bug fix run, the review cycle, the human time, and sometimes the incident in production.

Compounding accuracy: 99% per step still only lands you at 61% over 50 steps.

In classical software we addressed this with the shift-left movement. Shift-left quality, testing, security. That movement matters even more in agentic systems. The cost of being right early is much smaller than the cost of being wrong late.

The summary in one line

Instead of counting tokens, make every token count.

Send fewer rockets with higher accuracy and the fuel cost shrinks on its own.

The whole pitch in one line.

Foundations: how agents actually work

You can't optimize what you don't understand. Here's the short version. Skip ahead if you live in this stuff already.

LLMs are token probability machines

An LLM is a text-in, text-out machine. Under the hood, it's a token probability machine. Given everything you fed it plus its training data, it predicts the most probable next token. Then the next. Then the next. "Next word" is good enough as a mental model.

Newer models add reasoning, tool use, larger context, and biases toward code. The underlying engine is the same.

The core principle: context balance

The single most important rule of working with LLMs:

Provide as little context as possible, but as much as required.

Miss it in either direction and you lose quality.

Too much context biases the model toward irrelevant information. The math doesn't distinguish "useful" from "noise". Stuff a giant codebase into context for a small change and the model is fighting hundreds of competing signals.

Too little context is just as bad. The model fills the gap with what it knows from training, which means it may hallucinate to compensate. There's no error message when this happens. The model doesn't say "I needed more information." It just makes something up. The math doesn't distinguish between hallucination and fact.

Context engineering is the fundamental skill. Everything that follows is a flavor of it.

Harness, LLM, and your config

When you "talk to Copilot" or any other coding agent, three things are in play:

You and your project: prompts, files, instructions, skills, MCPs.
The agent (the harness): VS Code chat, Copilot CLI, Copilot's cloud agent, Claude Code, OpenAI Codex. Just code. No magic.
The LLM: GPT, Claude, Gemini. A pure model API.

Two things matter about how the harness talks to the LLM:

It's just text. No magic protocol. The harness assembles plain text, sends it to the model, parses what comes back, and decides what to do next.
It's stateless. The LLM doesn't remember anything between calls. "Having a conversation" means re-sending the entire previous exchange every single turn. Tokens compound.

That second point is what makes context engineering matter so much. Every turn you take pays for every turn before it. Input tokens can be cached on repeated turns, but caching isn't guaranteed. Don't plan around it.

Harness, LLM, and your project: three independent layers.

Context rot: lost in the middle, recency bias

Two things degrade in long context windows. Both are real, both are documented, and both should change how you work.

Lost in the middle (below 50 percent context fill): models favor content at the beginning and end of the window over the middle. Usually fine, because your instructions sit at the start and your current work sits at the end.

It hits you when you switch tasks mid-session. You start with a bug fix, then halfway through you say "now let's implement that feature instead." As the window grows, the model can pull back toward the original bug fix because it weights the start of the conversation more than recent input. Fix: start a new session for each distinct task.

Recency bias (above 50 percent context fill): past that threshold, the model starts heavily favoring the end of the conversation. It can forget your system instructions, your custom instructions, your original prompt. The model drifts. You start seeing it do things you don't understand because it's operating mostly on recent context.

Fix: don't let the window grow past 60-70 percent unless you really need to. Divide and conquer from the start. Compact only when you must. Reach for /clear more often than you currently do.

None of this is absolute. "Bias" doesn't mean the model will forget the middle. There are real scenarios where you need to be at 70 or 80 percent. Just be aware of it as a knob you can turn.

Context rot: lost-in-the-middle bias below 50% fill, recency bias above it.

The controls you actually have

This is the practical core. Tips are ordered by impact. Earlier ones matter more for almost everyone. Later ones are power-user territory.

Where you sit on the maturity spectrum

How much should you care about optimization? It depends on where you sit:

AI-assisted engineer: one agent at a time, mostly synchronous. Maybe ten agent runs a day. If you spend 20 dollars a month on tokens, even cutting that in half saves you 10 dollars. Not worth massive effort.
AI engineer: orchestrator of multiple async agents, dozens to hundreds per day. Every percentage point of quality compounds. Every percentage point of token savings multiplies. Optimization is a serious investment.

Most people are moving from the first to the second whether they realize it or not. Read on with that in mind.

The maturity spectrum: from one synchronous agent at a time to orchestrating many at once.

Lever 1: model choice and auto mode

The biggest single lever. When tokens felt free, everyone defaulted to the largest available model for everything, including typo fixes. The waste was significant.

The cost gap between the smallest and largest options can be 20x or more. Bigger doesn't always mean better, either.

Reasoning models (the current top tier like Claude Opus and GPT-5.5): planning, architecture, debugging complex bugs, synchronous work where you're driving, anything that needs to think before it acts.
Mid-tier models (Sonnet, GPT-5.4): async implementation after a plan exists. Most day-to-day code work.
Smaller models (Haiku, GPT-mini): small refactorings, repetitive tasks, doc updates, anything where the answer is mostly clear.

Counter-intuitive note: using a reasoning model for implementation can actually hurt quality. If you have a tight spec, a reasoning model may second-guess it, re-open the plan, and go rogue. "Actually, this spec is wrong, I'll do it my way." That's great when you want it to challenge your thinking. It's terrible when you just want it to ship the plan you agreed on.

The lazy answer is Auto Mode. As of mid-2026, auto mode has task intent detection and picks the model for you. Stay lazy. Make deliberate model choices only when it matters.

Lever 2: provide only relevant context

The second biggest lever, and the one with the most room to improve.

Stop stuffing the prompt with "might-need" information. Stop attaching your entire project for a small change. Let the agent discover what it needs. It can find files on its own.

A few tactical rules:

Compacting helps token count but loses information. If the lost information was relevant, you've traded tokens for misses. Use it carefully.
Use /clear liberally. Start fresh for each distinct task. Don't try to steer a bloated 80-percent-full session with a new prompt. Recency bias will eat your guidance.
Context doesn't carry across sessions. Your bill still counts every token you send, but a fresh session stops the old context from being resent on every turn. Throw context away without hesitation when you switch tasks.

Model choice and relevant context are the two biggest levers you have.

Your prompt: the always-on context

Your prompt sits at the top of every turn. Combined with the system prompt and tool descriptions, it has outsized influence. That "lost in the middle" bias works in your favor here. The model weights your opening heavily.

Don't optimize prompts for fewer tokens. Optimize them to steer the agent correctly.

Be precise.

Vague:

Fix the bug.

Precise:

Issue #45 in this repo describes a bug where editing a user's email triggers a 500 from POST /users/:id.
Reproduce it locally with the test in tests/users/email-update.test.ts, then fix it in src/users/handlers.ts.

The second one is longer in tokens. It saves you many turns. Worth it.

Add stop signals.

Tell the agent when it's done. Otherwise it'll keep going.

Once the test in tests/users/email-update.test.ts passes and `npm run lint` is clean, stop.
Do not commit, push, or open a PR.

Without that stop signal, agents often keep working past the actual goal: running extra linters, committing, pushing, opening PRs, updating unrelated docs. Every one of those is a step toward the exploding rocket.

Add known context up front.

If you know where the relevant files are, name them. If you want the agent to read a docs page, hand it the URL. If a skill applies, invoke it. Anything you can supply up front saves discovery cycles.

Research, plan, implement

The single workflow change that has helped me most. Don't do everything in one session.

Phase 1: Research. Load the files, read the issue, understand the problem. A lot of files come into context here. Most of them won't matter for implementation. Use a model that handles big context well.
Phase 2: Plan. Take what you learned and produce a precise spec. A detailed to-do list of what needs to change, where, and why. Use a reasoning model. This is where the expensive models earn their cost.
Phase 3: Implement. With a clear spec, you can switch to a smaller model. You can even split the work and run multiple agents in parallel, one per layer (frontend, backend, database) or per slice, with explicit contracts between them. Each agent only carries the context that matters for its slice.

Yes, there's some duplication when you move between phases. You'll paste in the plan you already created. That tiny duplication is worth it. The alternative is one giant session carrying all of Phase 1's research through Phase 3's typing, hurting both quality and cost. That single bloated session is, again, the exploding rocket.

Research → Plan → Implement, with the right model for each phase.

Deterministic controls: tests, linters, security scans

Not strictly context engineering. It's the most powerful counter to non-determinism we have. Tests are the landing pad for the Falcon.

The Copilot CLI team ships 500 PRs a week. Their number-one engineering practice for working with agents is tests. 53 percent of their codebase is tests. Why?

Because a test is a deterministic control. It either passes or it fails. There's no probability distribution. A failing test pulls the agent back from drift and restarts the accuracy curve. If after ten steps the agent has drifted to 50 percent accuracy, the failing test forces a correction and the next step starts fresh at ~99 percent again.

Picture two timelines:

With tests in the loop. Buggy change. Failing test. Correction. Stable working base. Next change. Failing test. Correction. Stable working base. Done.

Without tests. Buggy change. Buggy change on top. Buggy change on top of that. "Done." Then an incident in production, plus a debugging session that fills the next context window, plus CI/CD minutes burned, plus review cycles, plus human hours, plus the accumulation of tokens to clean up the mess.

The "no tests, faster" path is a mirage. The total token and time cost is higher every time.

Tests aren't the only deterministic control. Linters, type checkers, security scanners, schema validators, contract tests, all of them work the same way. Anything you can give the agent that produces a binary pass/fail and that it can run on its own is a quality multiplier.

Tests bring the agent back on track. No tests means buggy-on-buggy until it explodes.

Agent configs: where context engineering lives

When you talk about context engineering in practice, you're mostly talking about agent configs. These are the markdown files and controls you set up once and that agents pick up automatically.

The full menu:

Config	What it is	When to reach for it	Token risk
Persistent instructions (`copilot-instructions.md`)	Always-on rules for every session	Project guard rails, agent-miss log, output trimming	Pays itself off if kept small
Custom agents (`.github/agents/*.agent.md`)	Manually invoked workflows with scoped tools	Reusable roles like a TDD-red runner or a research subagent	Low
Skills (`.github/skills/*/skill.md`)	Lazy-loaded capabilities the model can pull in	Domain knowledge or workflows the model wouldn't know	Low when unused, full size when invoked
MCP servers	External APIs exposed as tools	Capabilities you genuinely need on tap	High. Bloats tool descriptions, tempts the agent into off-task calls
Subagents	A second context window for a focused task	Research-heavy work that would pollute the main session	Trades main-session quality for total tokens
Scoped instructions (`.github/instructions/*.instructions.md`)	Instructions that fire by file path pattern	Monoliths with distinct sections	Low
Prompt files (`.github/prompts/*.prompt.md`)	Manually invoked reusable prompts	Common starting points. Not in Copilot CLI	Low
Copilot Memory	Auto-generated always-on instructions from your behavior	Set-and-forget. Check periodically	Low

The top three are where you should spend most of your time. Here's how to actually use them.

Persistent instructions (your `copilot-instructions.md`)

These sit in the context window for every agent session, every interaction. That makes them powerful and dangerous. Powerful because they steer every run. Dangerous because every token in them is a token you pay for in every session.

Rules I live by:

Keep them small. Treat each line as a decision. If it isn't going to fire on most runs, it doesn't belong here.
Don't put full docs in them. No human-readable guides. No design philosophy essays. Distill ruthlessly.
Don't generate them with AI. This is your one chance as a human to fill the gaps the model can't know on its own. AI-generated instructions tend to be wordy and imprecise. Write them yourself.
Use them as an agent-miss log. When you see the agent reach for the wrong testing framework or the wrong build command, write a one-liner that prevents it next time.
Trim outputs. "Be concise", "drop niceties", "only return code". Output tokens cost more than input tokens. This matters.
Recreate them often. The Copilot CLI team throws away their entire instructions file every three months and starts fresh. Models change. Projects change. Old instructions accumulate and stop being relevant. Treat it as a living, breathing document.

A starter copilot-instructions.md for a Node.js project might look like this:

# Project guard rails

- Use vitest for tests, not jest. Test files live next to source as *.test.ts.
- Build command is `pnpm build`. Not npm. Not yarn.
- All exported functions must have a JSDoc comment with @param and @returns.
- Do not commit, push, or open PRs. Stop after the change is made and tests pass.
- Be concise in your replies. Skip the recap of what you did unless asked.

That's roughly 80 tokens. It'll pay for itself many times over.

Persistent instructions sit in every session, every interaction.

Custom agents

A custom agent is a markdown file that defines a role and the tools the agent should use when invoked.

Think of them as manually invoked workflows. You write the prompt once and reuse it. You also get to scope which tools the agent has access to, which prevents it from going down paths you didn't want.

A simple TDD-style custom agent might look like this (treat the tool names as illustrative, check current Copilot custom-agent docs for the real ones):

---
name: tdd-red
description: Add a failing test for a feature, then stop.
tools: [edit, run_command]
---

You are a test-first developer. The user will describe a feature.

Your job:
1. Write one failing test that captures the expected behavior.
2. Run the test suite to confirm the new test fails.
3. Stop. Do not implement the feature itself.

You invoke it with something like /tdd-red add API endpoint for user search. The harness loads the agent file, narrows the available tool set, and runs the workflow.

The token savings from tool scoping are real but small. The big benefit is structural. You prevent the agent from doing things you don't want it to do.

Custom agents wrap a reusable role with a scoped tool set.

Skills

A skill is something you offer to the agent. The harness lists the skill's description in context, and the model decides when to pull the full skill in. That makes skills lazy-loaded context. They cost almost nothing when unused. They cost their full size when invoked.

A few rules:

Don't overdo it. You don't need fifty skills.
Watch out for redundant skills. Does the model really need a "React skill" when it already knows React? Probably not.
Reserve skills for capabilities the model doesn't have. Domain knowledge, your specific deployment process, your internal API conventions.
Maintain them. Models get better. A skill that was essential last year may be redundant this year. Prune.

MCPs and subagents (in brief)

MCPs expose external APIs as agent tools. Once active, the harness puts their tool descriptions in context, which is overhead even when you don't call the tools. They also tempt the agent into off-task calls. A read_webpage tool in scope means the agent might decide to read a webpage for a task where reading a webpage isn't useful. This is how you build the exploding rocket: every MCP turned on, every tool tempting an extra detour.

The pattern that works: keep MCPs deactivated by default. Activate them inside a custom agent that actually needs them. The Playwright MCP is a great example. Powerful for serious frontend work. Wasteful for everything else.

Subagents open a second context window for a focused task, usually research. The subagent reads documents, builds a summary, and hands the summary back to the main session. Your main session stays clean. You spend more total tokens to get there. Use them deliberately for research tasks where the alternative is polluting your main session with many irrelevant files.

Power user moves

Advanced territory. Worth it if you orchestrate many agents. Probably not worth it if you're running ten a day.

Think in code. When you need to filter or transform large outputs, prefer writing a small script over feeding raw data to the model. Example: filter a GitHub REST API response down to the three fields you actually need before showing it to the agent.
CLI versus MCP. A CLI tool the model already knows (gh, kubectl, aws) can sometimes be leaner than the equivalent MCP. The MCP adds a static tool description in context. The CLI is part of the model's training. Experiment.
Trim shell outputs. Long shell output is a context killer. Tools like rtk strip CLI output down to agent-relevant information only.

A couple of niche ones if you're really pushing: run /chronicle tip regularly inside Copilot CLI to surface prompt patterns you could improve, and look at plugins like copilot-codeact-plugin that collapse multiple tool calls into one round trip.

What to invest in for the long term

Three things matter most if you want to be ahead of where this is going.

Build your analytical skills. What set developers apart was never just writing code. It was analytical skills. Building domain knowledge fast. Understanding what customers actually need. Translating requirements into systems. That's the part agents can't do. They'll execute. They won't strategize. Lean into that work.

Apply good architecture. More important than ever. Good architecture reduces agent misses, keeps the agent in its lane, and keeps the codebase legible enough for a non-human collaborator to navigate. Domain-Driven Design, Hexagonal Architecture, CQRS, Event-Driven Design. What these share: they cleanly separate low-level technology from the differentiating domain core. Old debates about function length, comment style, or semicolons in JavaScript matter less than they used to. Architecture is what matters now.

Iterate on prompts and configs as engineering work. You're a context engineer now. Keep configs fresh. Treat agent misses like incidents. Set agents up for success the same way you'd set up a junior teammate. Clear scope, clear definition of done, clear guard rails, room to do good work.

5 things you can start doing today

If you do nothing else from this post, do these:

Choose the right model for the right task. Stop defaulting to the biggest one. Auto mode if you can't be bothered to pick.
Provide clear guidance in your prompts. Be precise. Add stop signals. Supply known context up front.
Research, plan, implement. Split phases into separate sessions. Reasoning model to plan, smaller one to implement.
Provide deterministic guardrails. Tests, linters, security scans. The agent will use them. They'll pull it back from drift.
Maintain a concise, human-written copilot-instructions.md. Use it as an agent-miss log. Use it to trim outputs. Recreate it every few months.

Five things you can start doing today.

If you take one thing out of this whole post, take this:

Provide as little context as possible, but as much as required.

That's the principle. Everything else is how to apply it.

Send fewer rockets. Make each one land.

This post is based on a workshop I run on agent quality and token optimization. Slide images are from the GitHub workshop deck of the same name, credit to GitHub. If you want to dig deeper or share notes from your own teams, find me on GitHub.

I’m Ve Sharma, a Solution Engineer at Microsoft focusing on Cloud & AI working on GitHub Copilot. I help developers become AI-native developers and optimize the SDLC for teams. I also make great memes. Find me on LinkedIn or GitHub.

Squad vs Fleet Mode in GitHub Copilot CLI: What's the Difference?

Ve Sharma — Sun, 07 Jun 2026 17:28:35 +0000

TL;DR For the Busy Dev

Fleet Mode (/fleet) is built into Copilot CLI. It auto-decomposes your task and runs stateless sub-agents in parallel. Zero setup. No memory between sessions.
Squad is an open-source framework by Brady Gaster (Principal PM Architect at Microsoft). It installs a persistent team of named AI specialists into your repo. They remember decisions, enforce review protocols, and learn your codebase over time.
They are different layers that solve different problems. Fleet is a dispatch primitive. Squad is a coordination framework built on top of the same sub-agent primitives.
Squad is not redundant because of Fleet. Brady explicitly evaluated using Fleet as Squad's core and decided against it.
They work better together. Squad v0.10.0 ships a hybrid dispatch mode that uses Fleet for read-heavy batch work (2.9x faster) and its own charter-aware dispatching for writes.
Pick Fleet for one-off parallel tasks. Pick Squad for projects where agents need to accumulate knowledge over days or weeks. Pick both if you want the speed of Fleet and the governance of Squad.

The confusion is real

I kept seeing /fleet and squad referenced in the same conversations. Both involve multiple agents. Both parallelize work. Both run inside the Copilot CLI. So I dug into the docs, the source code, and the GitHub issues to figure out what actually differs.

The short answer: they operate at different levels. Fleet gives you raw parallelism. Squad gives you a coordinated team. Understanding that distinction changes how you think about multi-agent workflows.

What is Fleet Mode?

Fleet is a built-in Copilot CLI slash command. Type /fleet followed by a prompt, and the orchestrator breaks your task into independent sub-tasks, then runs them in parallel.

/fleet refactor the auth module, add tests for each endpoint, and update the README

The orchestrator figures out what can run at the same time. It dispatches background sub-agents, waits for them to finish, then synthesizes the results.

You can also trigger it from plan mode. Write a plan with Shift+Tab, then choose "Autopilot + /fleet" to run everything hands-off.

Here's what you need to know about Fleet's sub-agents:

Each one gets its own isolated context window. They don't see what the other agents are doing.
They cannot talk to each other. All coordination flows through the parent orchestrator.
They share the filesystem but there's no file locking. If two agents write the same file, the last one wins. Silently.
They are stateless. When the session ends, everything they learned is gone.

Fleet is fast. It's free of setup. And it works well for tasks that split cleanly into independent pieces.

What is Squad?

Squad is an open-source framework by Brady Gaster (Principal PM Architect at Microsoft, working on CoreAI Apps & Agents). Install it globally, run squad init, and you get a persistent AI development team living inside your repo as plain Markdown files.

npm install -g @bradygaster/squad-cli
squad init
copilot --agent squad --yolo

That squad init command scaffolds a .squad/ directory. Inside it:

team.md lists who's on the team
routing.md defines who handles what kind of work
decisions.md records team decisions that every agent reads before starting work
agents/{name}/charter.md defines each agent's identity, expertise, and constraints
agents/{name}/history.md stores what each agent has learned about your specific project

Agents get persistent names drawn from fictional universes. Apollo 13 mission control callsigns. Characters from The Usual Suspects. The names stick across sessions and live in .squad/casting/registry.json.

A coordinator agent (.github/agents/squad.agent.md) acts as the router. It reads your request, checks the routing table, and spawns the right specialists. It never does domain work itself. It delegates.

A silent agent called Scribe runs in the background after every work batch. Scribe merges agent decisions, updates history files, and archives old entries. You never interact with Scribe directly.

Another agent called Ralph watches your GitHub issue board and can auto-triage and dispatch agents on a polling interval.

All of this state lives in git. Clone the repo and you get the team with all their accumulated knowledge.

Side-by-side comparison

	Fleet Mode	Squad
Setup	None. Built into Copilot CLI	`npm install` + `squad init`
Agent identity	Anonymous, generic	Named specialists with charters
Memory	None. Starts fresh every session	Three tiers: skills, decisions, personal history
Routing	Auto-decomposition from your prompt	Explicit routing rules in `routing.md`
Quality gates	None	Reviewer lockout, design reviews, retros
Custom agents	Ignores `.github/agents/` charters in practice	Charter-driven, enforced
Cross-client	CLI only	CLI + VS Code + GitHub.com
File safety	Shared filesystem, no locking	Write-authority model per agent
Governance	None	RAI agent, audit trails, write restrictions
Automation	Not designed for it	Ralph daemon for issue triage

The key difference

Fleet answers the question: "How do I run multiple agents in parallel?"

Squad answers a different question: "How do I run a persistent team of specialists that learn my project and enforce quality over time?"

Fleet is a dispatch primitive. Squad is a coordination framework. Squad happens to use the same underlying task tool that Fleet uses to spawn sub-agents. But it layers six things on top:

Persistent identity. Agents have names, charters, and histories that survive sessions.
Cross-session memory. Decisions and skills compound over time. Clone the repo and a new contributor gets the team's full knowledge.
Explicit routing. Domain-aware routing rules in a durable file. Named routing, domain matching, and skill-aware routing, in that priority order.
Quality gates. A reviewer lockout protocol prevents an agent from revising its own rejected work. A different agent must step in.
Ceremonies. Design reviews auto-trigger before multi-agent tasks touching shared systems. Retrospectives auto-trigger after failures.
Governance. A write-authority model controls which agent can write to which file. A dedicated RAI agent issues traffic-light verdicts.

Brady Gaster put it simply in a GitHub issue:

"Fleet mode is a Copilot CLI feature, whereas Squad is a Copilot Agent in the context of the CLI. We investigated using Fleet for our spawn at one point but didn't. Fleet is awesome sauce!"

They work better together

As of Squad v0.10.0, Fleet is integrated as an optional dispatch mode inside Squad's watch system.

squad watch --execute --dispatch-mode hybrid

In hybrid mode, Squad's Ralph daemon auto-classifies incoming GitHub issues by scanning the title:

Issue type	Example keywords	Dispatched via
Read-heavy	research, review, analyze, audit	`/fleet` (parallel)
Write-heavy	fix, implement, create, build	`task` tool (sequential, charter-aware)

The classification defaults to "write" when it's ambiguous. That's the safer path because write-heavy work needs charter enforcement.

Why not pure Fleet?

Testing revealed a critical limitation. Fleet ignores custom agent charters. When you reference @my-agent in a /fleet prompt, Copilot CLI spawns a generic explore agent instead. Your agent's identity, constraints, and expertise get skipped.

That makes Fleet great for read-only work like triage and audits. But for code changes that need governance, you want Squad's charter-aware dispatch.

The benchmarks

Squad's team benchmarked hybrid mode against sequential dispatch on 4 real issues:

Metric	Fleet (parallel)	Sequential	Improvement
Total time	116s	332s	2.9x faster
Premium requests	12	~16	~25% fewer
MCP startup overhead	1x (shared)	4x (per process)	4x less

Fleet correctly triaged all four issues with proper urgency classification and actionable next steps.

When to use which

Pick Fleet when:

You have a one-off task that splits into independent pieces. Multi-file refactor, test generation across modules, docs for several components.
You don't need agents to remember anything after the session.
You don't need review protocols or quality gates.
You want zero setup and maximum speed.

Pick Squad when:

You're working on a project over days or weeks and want agents to build up knowledge.
You need quality enforcement. Reviewer lockout, design reviews before complex changes, retrospectives after failures.
You want named specialists whose domain expertise persists across sessions.
You need it working in VS Code or GitHub.com too, beyond just the CLI.
You want automated issue triage running on a polling interval.
Your team has conventions that agents should follow consistently.

Pick both when:

You already run Squad and want to speed up batch operations. Issue triage, code audits, research.
You want Fleet's speed for reads and Squad's governance for writes.

The mental model

Fleet is like hiring freelancers for a day. They show up, you explain the task, they work fast in parallel, and they deliver. Tomorrow you'd have to explain everything again from scratch.

Squad is like having a permanent team. They know your codebase, your conventions, and your past decisions. They review each other's work. When someone new joins, they read the decision log and get up to speed. They run design reviews and retros. They get better over time.

Fleet gives you speed. Squad gives you continuity. The hybrid mode gives you both, applied where each one is strongest.

GitHub Copilot SDK vs Azure AI Foundry Agents: Which One Should Your Company Use?

Ve Sharma — Tue, 10 Feb 2026 04:45:04 +0000

TL;DR for the Busy Person

GitHub Copilot SDK gives you the same agentic core that powers Copilot CLI — context management, tool orchestration, MCP integration, model routing — so you can embed it in any app without building an agent platform from scratch. Best for: developer tools, Copilot Extensions, and any software where the user is a developer.
Azure AI Foundry Agents is a full platform for building, deploying, and governing general-purpose enterprise AI agents across any business domain. Best for: Product & end-user facing apps, customer support, document processing, ops automation, and multi-agent workflows outside of dev.
They're complementary. Most enterprises will use both — Copilot SDK for the developer layer, Foundry for the business layer.
Both are enterprise-ready. GitHub's platform carries SOC 2 Type II, ISO 27001, IP indemnification, and has feature set that covers the entire SDLC.

The Problem: Building Agentic Workflows From Scratch Is Hard

Building agentic workflows from scratch is hard.

You have to manage context across turns, orchestrate tools and commands, route between models, integrate MCP servers, and think through permissions, safety boundaries, and failure modes. Even before you reach your actual product logic, you've already built a small platform.

Most teams don't want to build a platform. They want to build a product.

This is the core problem both GitHub Copilot SDK and Azure AI Foundry Agents solve — but they solve it for different audiences, in different contexts, with different trade-offs.

The Confusion

After talking to customers and internal teams, one theme keeps coming up:

"We want to build AI agents. Do we use the GitHub Copilot SDK or Azure AI Foundry? What's the difference? Can we use both?"

The short answer: they solve different problems. But the overlap in marketing language ("agents," "AI," "SDK") makes it murky. Let's fix that.

What Is the GitHub Copilot SDK?

The GitHub Copilot SDK (now in technical preview) removes the burden of building your own agent infrastructure. It lets you take the same Copilot agentic core that powers GitHub Copilot CLI and embed it in any application.

What the SDK handles for you:

You used to build this yourself	Copilot SDK handles it
Context management across turns	✅ Maintained automatically
Tool/command orchestration	✅ Automated invocation and chaining
MCP server integration	✅ Built-in protocol support
Model routing (GPT-4.1, etc.)	✅ Dynamic, policy-based routing
Planning and execution loops	✅ Multi-step reasoning out of the box
Permissions and safety boundaries	✅ Enforced by the runtime
Streaming responses	✅ First-class support
Auth and session lifecycle	✅ Managed under the hood

What you focus on: Your domain logic. Your tools. Your product.

Available in: TypeScript/Node.js, Python, Go, .NET

Designed for: Developer-facing applications — Copilot Extensions, dev portals, CLI tools, code review bots, internal productivity tools.

Why This Matters

Without the SDK, you're stitching together an LLM API, a context window manager, a tool registry, an execution loop, error handling, and auth — before writing a single line of product code. The SDK collapses all of that into an import.

Your App
    │
    ├── Your product logic + custom tools
    ├── Copilot SDK (agentic core)
    │       │
    │       └──── handles context, orchestration,
    │             MCP, model routing, safety
    │       │
    │       └──── HTTPS ──▶ GitHub Cloud (inference)
    │
    └── Your UI / API / Extension

What Is Azure AI Foundry Agent Service?

Azure AI Foundry is a full platform for building, deploying, and governing enterprise AI agents for any business domain — not just developer workflows.

What Foundry gives you:

Multi-agent orchestration (multiple specialized agents coordinating on a workflow)
Deep data connectors (SharePoint, SQL, M365, external APIs, Logic Apps)
Bring your own model (Azure OpenAI, open-source, frontier, or custom)
Goal-driven autonomy with thread-based memory
Full governance stack (Entra ID, Purview, Defender)
One-click deploy to Teams, M365 Copilot, web apps

Designed for: Business-wide automation — customer support agents, document processing pipelines, HR/finance/IT workflows, knowledge management with RAG.

When to Use Which

Use the Copilot SDK when:

Your user is a developer. You're building tools that live in the developer workflow — IDE extensions, CLI tools, Copilot Chat extensions, internal dev portals.
You want to ship fast. The SDK gives you a production-tested agentic core on day one. No need to build context management, tool orchestration, or model routing.
You're extending GitHub Copilot. Building a @my-tool extension for Copilot Chat is a first-class use case.
You want to stay in the GitHub ecosystem. Your code, your PRs, your CI/CD, and now your AI agent — all on the same platform.
You need something lightweight. Import the SDK, register your tools, start a session. That's it.

Use Azure AI Foundry when:

Your user is not a developer. You're building for support teams, ops, finance, HR, or customers.
You need multi-agent orchestration. Multiple specialized agents collaborating on a complex workflow (e.g., triage → investigate → resolve → notify).
You need deep data integration. Connecting to SharePoint, SQL, CRM, ERP, or other enterprise data sources via built-in connectors.
You want to bring your own model. Fine-tuned models, open-source models, or models from different providers.
You need VNet isolation or customer-managed encryption. For workloads where network-level isolation or key management is a hard requirement.

Use both when:

You're an enterprise with developer teams AND business teams that both need AI agents.
Your dev team uses Copilot SDK for internal tooling, while your platform team uses Foundry for customer-facing agents.

┌──────────────────────────────────────────────┐
│             ENTERPRISE AI STACK              │
│                                              │
│  ┌───────────────────┐ ┌──────────────────┐  │
│  │  Developer Layer  │ │  Business Layer  │  │
│  │                   │ │                  │  │
│  │  Copilot SDK      │ │  Azure AI        │  │
│  │  • @extensions    │ │  Foundry Agents  │  │
│  │  • Dev portals    │ │  • Product apps  │  │
│  │  • CLI agents     │ │  • Biz pipelines │  │
│  │  • Ops workflows  │ │  • Ops workflows │  │
│  └───────────��───────┘ └──────────────────┘│
│                                              │
│  Shared: Microsoft identity, models, trust   │
└──────────────────────────────────────────────┘

Trade-Offs Worth Knowing

Dimension	Copilot SDK	Azure AI Foundry
Time to first agent	Fast — import SDK, register tools, go	Slower — more config, but more control
Agentic core included?	✅ Yes — same engine as Copilot CLI	You build or configure your own agent logic
Model choice	GitHub-hosted models (GPT-5.3, etc.) + BYO Model	BYO model, Azure OpenAI, open-source, frontier
MCP support	✅ Built-in	Supported via configuration
Multi-agent orchestration	Early / evolving	First-class, production-ready
Data connectors	You build your own (via custom tools)	Built-in (SharePoint, SQL, M365, Logic Apps)
Deployment surface	Your app, VS Code, GitHub.com	Teams, M365 Copilot, web apps, containers
Network isolation (VNet)	Not available	✅ Full VNet / private endpoint support
Customer-managed keys	Microsoft-managed	✅ Azure Key Vault
Infrastructure ownership	GitHub-managed (less to operate)	Your Azure subscription (more control, more ops)
Billing model	Per Copilot seat	Azure consumption-based
Best for	Developer tools and workflows	Business-wide automation at scale

A Note on Enterprise Readiness & Compliance

There's a misconception that GitHub is only "good enough" for compliance compared to Azure. That's not accurate.

GitHub is an enterprise-grade platform trusted by the world's largest companies and governments. The compliance posture is strong and getting stronger:

Certification / Control	GitHub (incl. Copilot)
SOC 2 Type II	✅ Available (Copilot Business & Enterprise in scope)
SOC 1 Type II	✅ Available
ISO/IEC 27001:2022	✅ Copilot in scope
FedRAMP Moderate	🔄 Actively pursuing
IP Indemnification	✅ Included for enterprise plans
No training on your code	✅ Copilot Business/Enterprise data is never used for model training
Duplicate detection filtering	✅ Available to reduce IP risk

GitHub's story is the end-to-end developer platform — code, security, CI/CD, and now AI — all enterprise-ready under one roof. The Copilot SDK extends that story into your own applications.

On data residency specifically: yes, Azure AI Foundry offers more granular region-bound controls (VNet isolation, customer-managed keys, explicit region pinning). This matters for certain regulated workloads. But for many enterprises — especially in the US and Canada — data-in-transit concerns with GitHub Copilot are well-addressed by existing encryption, privacy controls, and contractual terms. Data residency is worth understanding, but it shouldn't be the sole deciding factor. Evaluate it alongside your actual regulatory requirements, not as a blanket blocker.

Decision Framework

What are you building?
│
├── A developer tool, extension, or dev or ops workflow?
│   └── ✅ GitHub Copilot SDK
│       • You get a production-tested agentic core out of the box
│       • Ship fast, stay in the GitHub ecosystem
│       • Enterprise-ready compliance
│
├── A product, business/ops/customer-facing agent?
│   └── ✅ Azure AI Foundry
│       • Multi-agent orchestration, data connectors, BYO model
│       • Full Azure governance and network isolation
│
├── Both?
│   └── ✅ Use both — they're complementary
│
└── Not sure yet?
    └── Start with the user:
        • Developer or internal? → Copilot SDK
        • End-facing user or Non-developer? → Foundry

Key Takeaways

The Copilot SDK removes the hardest part of building agents. Context, orchestration, MCP, model routing, safety — it's all handled. You focus on your product.
Foundry is for the business. When your agents are for an user facing product or orchestrate across departments, or serve non-developer users, Foundry is the right tool.
GitHub is enterprise-ready. SOC 2, ISO 27001, IP indemnification, no training on your data.
They're complementary, not competing. Copilot SDK for the developer/ops layer. Foundry for the product/business layer. Same Microsoft ecosystem underneath.
Start with the user, not the technology. Who is the agent serving? That answer picks your tool.

Resources

Building Context-Aware CI with GitHub Copilot SDK and Microsoft WorkIQ

Ve Sharma — Mon, 26 Jan 2026 04:42:57 +0000

Overview

Building Context-Aware GitHub Actions with the Copilot SDK and Microsoft WorkIQ

TL;DR for the Busy Dev

Problem: Enterprise devs waste hours on rework because they miss decisions made in meetings they didn't attend
Solution: GitHub Copilot SDK + Microsoft WorkIQ = CI that queries your M365 meetings and fails PRs that violate team agreements
Stack: GitHub Actions + @github/copilot-sdk npm package + @microsoft/workiq via MCP
Outcome: Three-state CI check (PASS/WARN/FAIL) with PR comments linking violations to specific meeting decisions
When to use: Enterprise teams already on M365 who want to enforce institutional knowledge automatically
When to skip: Small teams, async-first orgs, or repos without feature-scoped meetings

Show me the code: Full implementation on GitHub

The $50,000 Problem Nobody Talks About

Picture this: Your team spends 2 hours in a planning meeting deciding to use PostgreSQL for the new user service. Three developers nod along. Two weeks later, someone opens a PR adding MongoDB.

The code review catches it. Eventually. After 40 hours of development.

This isn't a made-up scenario. According to a 2024 study by Stripe, developers spend 17.3 hours per week on maintenance tasks, with a significant portion attributed to "undoing work that shouldn't have been done in the first place."

The root cause? Context fragmentation.

Your decisions live in:

Meeting transcripts (Teams)
Email threads
Team messages
Loop pages & power point decks
Someone's head

Your CI/CD pipeline? It only knows about your code.

What if your CI could attend your meetings?

Enter the Dynamic Duo: Copilot SDK + WorkIQ

GitHub Copilot SDK

The Copilot SDK (@github/copilot-sdk) lets you build agentic workflows that leverage GitHub's AI infrastructure. Think of it as programmable Copilot—you define the prompt, the tools, and the outcome.

import { CopilotClient } from "@github/copilot-sdk";

// Create a session with MCP servers for tool access
const session = await CopilotClient.createSession({
  token: process.env.COPILOT_GITHUB_TOKEN,
  mcpServers: {
    workiq: {
      command: "npx",
      args: ["-y", "@microsoft/workiq", "mcp"],
    },
  },
});

// Send prompt and wait for response
const response = await session.sendAndWait([
  { role: "system", content: "You are a compliance analyst..." },
  { role: "user", content: "Analyze this PR for meeting compliance" },
]);

What it brings to the table:

Structured prompts with MCP tool access
File system and repository awareness
Configurable MCP servers for external integrations
Runs in CI environments with sendAndWait() for synchronous responses

Microsoft WorkIQ

WorkIQ is Microsoft's natural language interface to your M365 data. It's currently in public preview and supports:

Data Source	Example Query
Emails	"What did Sarah say about the deadline?"
Meetings	"Summarize decisions from yesterday's sprint planning"
Documents	"Find PowerPoints about Q4 roadmap"
Teams	"What blockers came up in #engineering today?"
People	"Who's working on Project Alpha?"

npm i -g @microsoft/workiq

# Query your M365 data
workiq ask -q "What architectural decisions were made this week?"

What it brings to the table:

Natural language queries to M365
Meeting transcripts and decisions
Cross-platform context (Teams, Outlook, SharePoint)
MCP server mode for tool integration

Why Together They're Batman and Robin

Separately, they're powerful. Together, they're transformative.

Copilot SDK Alone	WorkIQ Alone	Combined
Analyzes code	Queries meetings	Validates code against meeting decisions
Generates reports	Summarizes context	Links violations to specific discussions
Runs in CI	Requires human invocation	Fully automated compliance
No organizational context	No code awareness	Full-stack intelligence

The magic: WorkIQ provides the institutional memory. Copilot SDK provides the reasoning engine. Together, they close the loop between "what we decided" and "what we built."

Real Implementation: Decision Compliance CI Check

Here's what we built—a GitHub Action that fails PRs violating team agreements.

Architecture

┌──────────────────────────────────────────────────────────────┐
│                     Pull Request Opened                       │
├──────────────────────────────────────────────────────────────┤
│  1. Extract keywords from branch (feature/user-auth → auth)  │
│  2. CopilotClient.createSession() with WorkIQ MCP server     │
│  3. session.sendAndWait() → WorkIQ queries M365 meetings     │
│  4. Analyze PR diff against those decisions                  │
│  5. Generate structured report (JSON + Markdown)             │
│  6. Post PR comment with findings                            │
│  7. CI outcome: ✅ PASS / ⚠️ WARN / ❌ FAIL                   │
└──────────────────────────────────────────────────────────────┘

The Workflow (Simplified)

# .github/workflows/workiq-decision-compliance.yml
name: Meeting Decision Compliance

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  compliance-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 22

      - name: Install and build SDK script
        working-directory: .github/scripts/workiq-decision-compliance
        run: |
          npm ci
          npm run build

      - name: Run Decision Compliance Agent
        working-directory: .github/scripts/workiq-decision-compliance
        env:
          COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
          WORKIQ_TENANT_ID: ${{ secrets.WORKIQ_TENANT_ID }}
          PR_TITLE: ${{ github.event.pull_request.title }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
          CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
          LOOKBACK_DAYS: "7"
        run: node dist/index.js

The SDK Script (Key Sections)

// src/index.ts - Main entry point
import { CopilotClient } from "@github/copilot-sdk";
import { configSchema } from "./config";
import { buildUserPrompt, SYSTEM_PROMPT } from "./prompts";
import { parseComplianceResult } from "./types";

async function main() {
  const config = configSchema.parse(process.env);

  // Create session with WorkIQ as MCP server
  const session = await CopilotClient.createSession({
    token: config.COPILOT_GITHUB_TOKEN,
    mcpServers: {
      workiq: {
        command: "npx",
        args: ["-y", "@microsoft/workiq", "mcp", "-t", config.tenantId],
      },
    },
  });

  // Send prompt and wait for structured response
  const response = await session.sendAndWait([
    { role: "system", content: SYSTEM_PROMPT },
    { role: "user", content: buildUserPrompt(config) },
  ]);

  // Parse and act on results
  const result = parseComplianceResult(response.text);

  // Exit code based on status
  process.exit(result.status === "FAIL" ? 1 : 0);
}

What Developers See

When a PR is analyzed, they get a comment like:

## 📋 Meeting Decision Compliance Report

| Status | Decisions Checked | Violations | Warnings |
|--------|-------------------|------------|----------|
| ⚠️ WARN | 3 | 0 | 1 |

### Decisions Analyzed
1. **2026-01-20 Sprint Planning**: Use Zod for all API validation
2. **2026-01-18 Architecture Review**: REST endpoints use noun-based naming

### Warnings
- **Naming Convention** (2026-01-18): Endpoint `/api/getUsers` uses verb prefix
  - 💡 Recommendation: Rename to `/api/users` for REST consistency

### Compliance Evidence
- ✅ Found Zod schemas in src/validators/
- ✅ Database uses PostgreSQL via Drizzle ORM

The Enterprise Impact

For teams already invested in M365, this is a force multiplier:

Before

Developer → Writes code → PR review → "Wait, didn't we decide X?"
→ Rework → Re-review → Merge (3-5 days)

After

Developer → Writes code → CI checks decisions → Fix before review
→ PR review → Merge (1-2 days)

ROI Calculation (Conservative)

Metric	Before	After	Savings
Rework incidents/month	8	2	6 incidents
Hours per incident	4	0	24 hours/month
Developer cost/hour	$75	-	$1,800/month
Team of 10 devs	-	-	$18,000/month

And that's just direct rework. The indirect benefits—faster onboarding, preserved institutional knowledge, reduced meeting FOMO—compound over time.

More Use Cases (Beyond Compliance)

Once you have this pipeline, the possibilities expand:

Use Case	WorkIQ Query	Copilot SDK Action
Sprint Alignment	"What's committed for this sprint?"	Verify PR implements sprint items
Stakeholder Notification	"Who discussed this feature?"	Auto-tag relevant people on PR
ADR Validation	"What architecture decisions exist?"	Check code against ADRs
Onboarding Context	"What decisions led to this code?"	Generate context for new devs
Risk Assessment	"Were there concerns about this approach?"	Surface historical objections
Meeting Prep	"What PRs relate to tomorrow's review?"	Auto-generate meeting agendas

Honest Pros and Cons

✅ Pros

Benefit	Impact
Automated institutional memory	Decisions become enforceable, not just documented
Reduced meeting FOMO	Miss a meeting? CI has your back
Faster code reviews	Reviewers focus on logic, not policy
Self-documenting PRs	Context travels with the code
Graceful degradation	WARN state handles ambiguity without blocking

❌ Cons

Challenge	Mitigation
M365 admin consent required	One-time setup, but needs IT involvement
Meeting quality matters	Garbage in, garbage out—transcripts need clear decisions
False positives possible	WARN state prevents blocking on uncertain cases
Keyword matching is imperfect	Future: Use embeddings for semantic matching
Public preview (WorkIQ)	APIs may change; not for mission-critical yet
Cost	Copilot license + M365 license + compute time

When to Use This (And When Not To)

✅ Good Fit

Enterprise teams (50+ devs) on M365
Regulated industries needing audit trails
Teams with frequent architectural discussions
Repos with long-lived feature branches
Organizations fighting "tribal knowledge" loss

❌ Skip It If

Small teams (< 10 devs) with high context sharing
Async-first orgs (decisions in docs, not meetings)
Open source projects (no M365 integration)
Repos with no feature-scoped meetings
Teams that don't record/transcribe meetings

Getting Started

Prerequisites

GitHub Copilot license
Microsoft 365 with Teams/Outlook
WorkIQ admin consent (guide)
Node.js 22+

Quick Setup

# 1. Clone the reference implementation
git clone https://github.com/VeVarunSharma/contoso-vibe-engineering
cd contoso-vibe-engineering/.github/scripts/workiq-decision-compliance

# 2. Install dependencies
npm install

# 3. Accept WorkIQ EULA (first time only)
npx -y @microsoft/workiq accept-eula

# 4. Test the WorkIQ connection
npx -y @microsoft/workiq ask -q "What meetings do I have this week?"

# 5. Add secrets to your repo
gh secret set COPILOT_GITHUB_TOKEN
gh secret set WORKIQ_TENANT_ID

Full Implementation

The reference implementation includes:

# Key files:
# .github/workflows/workiq-decision-compliance.yml  - GitHub Actions workflow
# .github/scripts/workiq-decision-compliance/       - TypeScript SDK script
#   ├── src/index.ts    - Main entry point
#   ├── src/config.ts   - Zod config validation
#   ├── src/prompts.ts  - System and user prompts
#   └── src/types.ts    - TypeScript interfaces
# .github/skills/workiq-copilot/                    - Documentation

What's Next?

This is just the beginning. The Copilot SDK + MCP ecosystem is evolving fast. Expect more business layer and enterprise value context integrations in the pipeline.

Wrapping Up

The Copilot SDK gives you programmable AI reasoning. WorkIQ gives you access to your organization's collective memory. Together, they bridge the gap between what your team decided and what your code does.

For enterprise developers drowning in meetings, context switches, and "I didn't know we decided that" moments—this is the escape hatch.

The future of agentic devOps & CI isn't just testing code—it's validating intent.

GitHub + Azure DevOps: The Better Together Story (And Why GitHub Should Be Your Future

Ve Sharma — Mon, 19 Jan 2026 05:14:46 +0000

🧭 TL;DR for the Busy Person — Why GitHub Should Be Your Long‑Term SDLC Home (Even If You're Using Azure DevOps Today)

Azure DevOps isn’t going away — it remains excellent for Boards, Pipelines, Test Plans, enterprise workflow, and hybrid/legacy workloads.
GitHub is where Microsoft is investing for AI‑native software development — Copilot Workspace, repo‑wide reasoning, agents, automated fixes, and modern CI/CD workflows.
Copilot works great with Azure DevOps, but advanced AI features only unlock when repos live in GitHub.
The best path? Start Copilot inside Azure DevOps → Migrate repos gradually → Move new projects to GitHub by default.
This keeps risk low while letting teams benefit from GitHub’s AI automation, security tooling, and unified developer experience.

🚀 The Future of Software Development: GitHub as the AI-Native Platform

Over the past two years, GitHub has transformed from a developer tool into a full AI‑powered software development platform.

Microsoft & GitHub are pouring much more resources and getting some of the smartest minds at the company to improve the GitHub platform, including the Copilot.

GitHub now offers:

Agentic workflows (multi-step reasoning, automated refactors, multi-file edits, across the SDLC.)
Top Tier LLMs - There's large variety of top off the shelf models (Gemini, Anthropic, OpenAI, etc) for developers, and the ability to use your own models via Microsoft Foundry.
Copilot Workspace (task planning → implementation → PR → CI)
Copilot Autofix (security + code quality fixes generated automatically)
Graph-based repo understanding
GitHub Actions ecosystem (90K+ reusable workflows)
Deep GitHub Advanced Security integration

This is a non-exhaustive list as there would be too many features to list. See the GitHub Changelog for yourself here.

Azure DevOps is still strong, but built on a services model (Repos, Boards, Pipelines) rather than a unified AI runtime.

GitHub is now effectively the AI layer for Microsoft‑based development.

🔍 GitHub vs Azure DevOps — A Balanced, Honest Comparison

High-Level Summary

Capability	GitHub	Azure DevOps
AI & Copilot depth	Full repo reasoning, Workspace, Agents, PR intelligence, Autofix	IDE‑only Copilot, limited PR intelligence
Repo intelligence	Cloud‑based semantic graph, multi-file context	No server-side intelligence
CI/CD	GitHub Actions (huge ecosystem, cloud-native)	Azure Pipelines (mature, enterprise, hybrid)
Security	GitHub Advanced Security built-in; Autofix; dependency insights	Scanning available but no Autofix, fewer AI-powered fixes
Dev Experience	Unified repos → PR → CI → Security → AI	Split experience across services
Ecosystem	Largest open-source + enterprise dev community	Strong enterprise workflows, compliance, approvals
Planning	GitHub Projects (improving rapidly)	Azure Boards (richer today)
Test Management	Integrations / GitHub apps	Azure Test Plans best-in-class
Best Fit For	Modern, cloud-native, AI-assisted teams	Enterprise, hybrid, legacy, structured planning

Azure DevOps is still excellent.

GitHub is simply where the future direction of AI-driven DevOps is going.

🤖 GitHub Copilot: GitHub vs Azure DevOps (Important Differences)

Copilot works in both environments… but the experience is not equal.

💡 Copilot with Azure DevOps (what you get)

Code completions in VS Code / Visual Studio
Copilot Chat in IDE
Basic explanations + code edits
Some PR help (summaries)
No repo-level graph or reasoning
No agents
No Workspace
No AI-powered Autofix

🔮 Copilot with GitHub (what unlocks)

Repo-wide reasoning (Copilot Workspace)
Multi-step planning → coding → PR creation
AI-generated PR summaries + inline reviewing
Copilot Autofix (security, quality, dependency fixes)
Agentic workflows across repos + CI/CD
Better diffs, test generation, refactor support
Native integration with GitHub Actions

Bottom Line:

If your code lives in GitHub → Copilot becomes 10x more powerful.

Azure DevOps users still get value — great place to start — but the ceiling is lower.

⚖️ Pros & Cons — A Balanced View

The AI-native development platform inside Microsoft
Copilot Workspace + Agents + Autofix
Superior PR + review experience
Best-in-class security tooling
GitHub Actions ecosystem
Developers overwhelmingly prefer GitHub
Faster onboarding of new hires

GitHub — Cons

GitHub Projects still catching up to Azure Boards
Migration effort required for pipelines and YAML normalization
Some enterprise compliance workflows still maturing

Azure DevOps — Pros

Azure Boards: still best enterprise planning tool
Azure Pipelines: powerful, hybrid, legacy-ready
Test Plans: far ahead for structured QA
Enterprise approvals & audit trails robust
No need to migrate everything at once

Azure DevOps — Cons

AI capability capped at IDE level
No server-side repo reasoning
Multiple services lead to tool fragmentation
Long-term innovation emphasis has shifted toward GitHub

🔄 A Practical, Low-Risk Transition Strategy (Used by Real Customers)

Microsoft field guidance now follows this pattern:

1️⃣ Start with Copilot — inside Azure DevOps

This reduces friction and avoids platform conversations too early.

No migration
No workflow disruption
Fastest path to measurable productivity gains

Goal: show value → build internal pull.

**2️⃣ Move one high-value repo to GitHub**

Usually:

A microservice
A heavily changed repo
A team that is cloud-native
A repo with active CI/CD challenges

Reason: teams instantly feel improved PR + automation experience.

3️⃣ Expand GitHub footprint as value becomes undeniable

Developer-led, not top-down.

Most orgs follow:

Copilot adoption
Selective repo moves
Standardize new projects on GitHub
Integrate Azure Boards with GitHub
Shift CI/CD to GitHub Actions over time

This avoids big-bang migrations.

🏗️ Recommended Hybrid Model (Best of Both Worlds)

Many customers land here during transition:

Keep using Azure Boards (excellent planning tool)
Move source code to GitHub (AI-native)
Use GitHub Actions for modern workflows
Keep Azure Pipelines for complex/legacy workloads
Integrate Test Plans as needed

This lets teams modernize without breaking what works.

💼 Business & Technical Benefits of Moving to GitHub

Business Benefits

Faster delivery = reduced time-to-market
Better quality reduces production risk
Developers happier → talent retention improves
Lower tool fragmentation
AI-assisted automation reduces cost of delivery
Aligns with Microsoft’s investment strategy

Technical Benefits

More automation with Copilot Workspace & Agents
Richer PR reviews (summaries, test suggestions, change reasoning)
Repo-wide graph understanding improves refactors
GitHub Advanced Security detects + fixes issues automatically
GitHub Actions easier to maintain than Pipelines YAML
Huge marketplace ecosystem
Better alignment with open-source standards

💭Final Thoughts

You don’t need to choose GitHub or Azure DevOps.

Most organizations start hybrid and let developer experience drive the long-term destination.

The truth is simple:

Azure DevOps is excellent at planning + enterprise workflows.
GitHub is the long-term AI-native engineering platform.

Start with Copilot where you are.

Move code when it makes sense.

Let productivity metrics guide the rest.

What If Your CI Pipeline Could catch regulatory compliance violations of your code?

Ve Sharma — Tue, 13 Jan 2026 03:35:41 +0000

TL;DR for the Busy Dev:

Traditional CI checks (linting, tests, SAST) are deterministic—same input always produces the same output
AI-powered CI checks using GitHub Copilot CLI can catch nuanced compliance violations that rule-based scanners miss
We built a PIPA BC (Privacy) compliance gate that analyzes code for consent verification, data minimization, PHI logging, and more
Non-compliant code fails the pipeline with a detailed report of violations
Best used alongside (not replacing) deterministic checks for defense in depth
ROI: Catching a privacy violation in CI costs ~$100. Catching it post-breach? $4.45 million average (IBM 2023)
Example: Here are 2 examples in PR formats using the compliance workflow: Compliant version, and the Non-compliant version failing in the CI.

The Problem: Privacy Compliance is Hard to Automate

If you've worked with healthcare, finance, or government data, you know the drill. Regulations like PIPA BC, HIPAA, GDPR, and PIPEDA require specific handling of personal information:

✅ Verify consent before data access
✅ Minimize data exposure (return only what's needed)
✅ Never log sensitive values like SIN, PHN, or medical records
✅ Maintain audit trails
✅ Implement proper authentication and authorization

Traditional static analysis tools can catch some of these—hardcoded secrets, missing auth middleware, SQL injection. But they struggle with contextual violations:

"Is this console.log() statement printing a patient's Social Insurance Number, or just a debug message?"

That's where non-deterministic, AI-powered checks come in.

Deterministic vs. Non-Deterministic CI Checks

Aspect	Deterministic	Non-Deterministic (AI)
Output	Same input → Same result, always	May vary slightly between runs
Examples	ESLint, Jest, Trivy, CodeQL	GitHub Copilot CLI, LLM analysis
Strengths	Predictable, fast, cheap	Understands context, catches nuance
Weaknesses	Pattern-based, easy to bypass	Can hallucinate, slower, costs $
Best for	Syntax, known vulnerabilities	Compliance, code review, intent

The key insight: These aren't competing approaches—they're complementary layers of defense.

Our Use Case: PIPA BC Compliance for Healthcare APIs

British Columbia's Personal Information Protection Act (PIPA) governs how private-sector organizations handle personal information. For healthcare apps, this means:

PIPA Section	Requirement	What to Check
Section 6	Consent	Is consent verified before PHI access?
Section 4	Data Minimization	Are we returning only necessary fields?
Section 11	Purpose Limitation	Is a purpose required and validated?
Section 34	Security	Auth middleware? No PHI in logs?
Section 34	Audit Trail	Are accesses logged (without PHI values)?

Let's build a CI gate that checks all of this.

The GitHub Action: How It Works

Here's our workflow that runs on every push to main:

name: PIPA BC Compliance Check

on:
  pull_request:
    paths:
      - "services/medical-api/**"
    types:
      - opened
      - synchronize
      - reopened
  workflow_dispatch: # Allows manual trigger for testing

# Ensure only one compliance check runs at a time per PR
concurrency:
  group: pipa-compliance-${{ github.event.pull_request.number || github.run_id }}
  cancel-in-progress: true

permissions:
  contents: read
  pull-requests: write

env:
  COMPLIANCE_THRESHOLD: 80

jobs:
  pipa-compliance-check:
    name: PIPA BC Compliance Analysis
    runs-on: ubuntu-latest
    timeout-minutes: 15

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          fetch-depth: 0 # Full history for accurate diff

      - name: Get changed files
        id: changed-files
        uses: tj-actions/changed-files@v44
        with:
          files: |
            services/medical-api/**/*.ts
            services/medical-api/**/*.tsx
            services/medical-api/**/*.js
            services/medical-api/**/*.jsx

      - name: Check if relevant files changed
        id: check-files
        run: |
          if [ -z "${{ steps.changed-files.outputs.all_changed_files }}" ]; then
            echo "skip=true" >> $GITHUB_OUTPUT
            echo "No relevant TypeScript/JavaScript files changed in medical-api"
          else
            echo "skip=false" >> $GITHUB_OUTPUT
            echo "Files to analyze:"
            echo "${{ steps.changed-files.outputs.all_changed_files }}"
          fi

      - name: Setup Node.js
        if: steps.check-files.outputs.skip != 'true'
        uses: actions/setup-node@v4
        with:
          node-version: 22

      - name: Install GitHub Copilot CLI
        if: steps.check-files.outputs.skip != 'true'
        run: npm i -g @github/copilot

      - name: Run PIPA BC Compliance Agent
        if: steps.check-files.outputs.skip != 'true'
        env:
          COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
          GITHUB_REPOSITORY: ${{ github.repository }}
        run: |
          set -euo pipefail

          # Build the list of changed files
          CHANGED_FILES="${{ steps.changed-files.outputs.all_changed_files }}"

          # Read the agent prompt
          AGENT_PROMPT=$(cat .github/agents/pipa-bc-compliance.agent.md)

          # Build the full prompt
          PROMPT="$AGENT_PROMPT"
          PROMPT+=$'\n\n## Context\n'
          PROMPT+="- Repository: $GITHUB_REPOSITORY"
          PROMPT+=$'\n- Service: services/medical-api'
          PROMPT+=$'\n- Changed Files: '"$CHANGED_FILES"
          PROMPT+=$'\n\n## Task\n'
          PROMPT+='1. Read the PIPA BC compliance documentation at services/medical-api/PIPA_COMPLIANCE.md'
          PROMPT+=$'\n2. Analyze the changed files for PIPA BC compliance'
          PROMPT+=$'\n3. Check each file against all PIPA BC requirements (consent, data minimization, purpose limitation, security, audit logging, access control)'
          PROMPT+=$'\n4. Generate a compliance report at services/medical-api/pipa-compliance-report.md'
          PROMPT+=$'\n5. The report MUST include a JSON block with compliance_score and status fields'

          # Run Copilot CLI with the agent
          copilot --prompt "$PROMPT" --allow-all-tools --allow-all-paths < /dev/null

. . .

You can find the full file here: .github/workflows/pipa-bc-compliance.yml

What's Happening Here?

Checkout & Setup: Standard CI setup
Install Copilot CLI: The @github/copilot npm package provides terminal access to Copilot
Load the Agent Prompt: We read our compliance rules from a markdown file
Run Analysis: Copilot analyzes the entire codebase against our rules
Fail on Critical: If critical violations are found, the pipeline fails

The magic is in the agent prompt—a markdown file that tells Copilot exactly what to look for.

Example 1: Compliant Code ✅

Here's a PIPA-compliant patient endpoint:

// ✅ Auth middleware applied to all routes
app.use("/*", requireAuth);

app.get("/:id", zValidator("query", getPatientQuerySchema), async (c) => {
  const patientId = c.req.param("id");
  const { purpose } = c.req.valid("query"); // ✅ Purpose required
  const user = c.get("user");

  // ✅ Verify role has permission for this purpose
  if (!ROLE_PERMISSIONS[user.role].includes(purpose)) {
    await createAuditLog({ action: "ACCESS_DENIED", ... });
    return c.json({ error: "Access denied" }, 403);
  }

  // ✅ Verify consent before data access
  const consentResult = await verifyConsent(patientId, purpose, user.id);
  if (!consentResult.valid) {
    await createAuditLog({ action: "ACCESS_DENIED", ... });
    return c.json({ error: "Consent verification failed" }, 403);
  }

  const patient = await db.query.patients.findFirst({
    where: eq(patients.id, patientId),
  });

  // ✅ Apply data minimization based on purpose
  const filteredData = filterPHI(patient, purpose, user.role);

  // ✅ Audit log with field names only, no PHI values
  await createAuditLog({
    action: "PATIENT_ACCESS",
    fieldsAccessed: filteredData._accessedFields, // Names only!
  });

  return c.json({ data: filteredData });
});

CI Result: ✅ PASS

Here are some examples of the GitHub Compliance CI check in action taken from the example repo.

Example 2: Non-Compliant Code ❌

Here's intentionally bad code for demo purposes:

// ❌ VIOLATION 8: Hardcoded credentials
const API_SECRET_KEY = "sk-prod-12345-abcdef-secret-key";
const DATABASE_PASSWORD = "super_secret_password_123";

// ❌ VIOLATION 1: No authentication middleware
app.get("/:id", async (c) => {
  const patientId = c.req.param("id");

  // ❌ VIOLATION 7: No purpose validation
  const purpose = c.req.query("purpose"); // Optional and ignored!

  // ❌ VIOLATION 2: No consent verification
  const patient = await db.query.patients.findFirst({
    where: eq(patients.id, patientId),
  });

  // ❌ VIOLATION 4: Logging PHI values
  console.log(`[PATIENT] SIN: ${patient.socialInsuranceNumber}`);
  console.log(`[PATIENT] Health Card: ${patient.healthCardNumber}`);
  console.log(
    `[PATIENT] Medical History: ${JSON.stringify(patient.medicalHistory)}`
  );

  // ❌ VIOLATION 6: No audit logging

  // ❌ VIOLATION 3: Returns ALL fields (no data minimization)
  return c.json({
    data: patient,
    _internal: {
      apiKey: API_SECRET_KEY, // ❌ Exposing secrets!
    },
  });
});

// ❌ VIOLATION 5: Bulk export with no access controls
app.get("/export/all", async (c) => {
  const allPatients = await db.query.patients.findMany();
  return c.json({ patients: allPatients });
});

CI Result: ❌ FAIL

The report is uploaded as an artifact, and it's attached right in the Pull Request for other agents & humans to review, and to possibly to other agentic actions from this type of trigger.

The report identifies all 8 violations with specific line numbers and PIPA sections violated.

Step-by-Step Setup Guide

Prerequisites

GitHub repository with Actions enabled
GitHub Copilot subscription (Business or Enterprise)
Fine-grained Personal Access Token with Copilot permissions

Step 1: Create the Copilot Token

Go to GitHub Settings → Developer settings → Personal access tokens → Fine-grained tokens
Create a new token with:
- Repository access: Your target repo
- Permissions: Copilot → Read-only
Copy the token

Step 2: Add the Secret

Go to your repo's Settings → Secrets and variables → Actions
Create a new secret: COPILOT_GITHUB_TOKEN
Paste your token

Step 3: Create the Agent Prompt

Create .github/agents/pipa-bc-compliance.agent.md:

# PIPA BC Compliance Agent

You are a privacy compliance auditor for British Columbia's PIPA.

## Your Task

Analyze all TypeScript/JavaScript files for PIPA violations:

### Critical Violations (Immediate Fail)

- [ ] PHI values logged to console (SIN, PHN, medical records)
- [ ] No authentication on endpoints accessing PHI
- [ ] No consent verification before data access
- [ ] Hardcoded credentials or API keys
- [ ] Bulk data export without access controls

### Major Violations

- [ ] Missing purpose validation on data requests
- [ ] No data minimization (returning all fields)
- [ ] Missing audit logging

## Output Format

Generate a report with:

1. Overall compliance score (0-100)
2. List of violations with file, line, and PIPA section
3. Remediation guidance for each violation

If any CRITICAL violations exist, include this line:
"THIS ASSESSMENT CONTAINS A CRITICAL VULNERABILITY"

You can find the full file for this compliance agent here: .github/agents/pipa-bc-compliance.agent.md

Step 4: Create the Workflow

Create .github/workflows/pipa-compliance.yml with the workflow shown above.

Step 5: Test It

Push compliant code → Should pass ✅
Push non-compliant code → Should fail ❌

Pros and Cons

✅ Pros

Benefit	Impact
Catches contextual violations	Understands "is this log statement printing PHI?"
Natural language rules	No regex patterns or AST parsing required
Adapts to your codebase	Learns patterns from your actual code
Reduces manual review time	Flags issues before human reviewers see them
Documentation as code	Agent prompts are readable, version-controlled

❌ Cons

Drawback	Mitigation
Non-deterministic output	Run multiple times, require consensus
Can hallucinate	Human review of flagged issues
Slower than linting	Run on specific paths, use caching
Costs money	Only run on PRs to protected branches
Requires Copilot license	ROI justifies for compliance-heavy orgs

When to Use Each Type

Use Deterministic Checks For:

✅ Syntax errors and formatting (ESLint, Prettier)
✅ Type checking (TypeScript)
✅ Known vulnerability patterns (CodeQL, Trivy)
✅ Unit and integration tests (Jest, Playwright)
✅ Secret scanning (git-secrets, Gitleaks)
✅ Dependency vulnerabilities (Dependabot, Snyk)

Use Non-Deterministic (AI) Checks For:

✅ Privacy compliance (PIPA, HIPAA, GDPR intent)
✅ Business logic validation
✅ Code review augmentation
✅ Documentation completeness
✅ API contract compliance
✅ Security posture assessment

The Hybrid Approach (Recommended)

jobs:
  # Fast, deterministic checks first
  lint-and-test:
    runs-on: ubuntu-latest
    steps:
      - run: pnpm lint
      - run: pnpm test
      - run: pnpm typecheck

  # AI compliance check after deterministic passes
  compliance-check:
    needs: lint-and-test
    runs-on: ubuntu-latest
    steps:
      - run: copilot --prompt "..."

Only run the expensive AI check if the cheap checks pass first.

Why This Matters: The Business Case

The Cost of Getting It Wrong

Stage	Cost to Fix
Development (caught in CI)	~$100
QA/Staging	~$1,000
Production (pre-breach)	~$10,000
Post-breach (notification, legal, fines)	$4.45M average (IBM 2023)

For PIPA BC specifically:

Fines: Up to $100,000 per violation
Reputation: Healthcare data breaches make headlines
Lawsuits: Class actions for privacy violations

ROI Calculation

Annual Copilot Business license: ~$228/developer
PRs per developer per year: ~200
Cost per AI compliance check: ~$1.14

Violations caught per 100 PRs: ~5
Cost to fix in CI: $100 × 5 = $500
Cost to fix post-production: $10,000 × 5 = $50,000

Savings per 100 PRs: $49,500
ROI: 4,342%

Even if AI checks only catch one violation per quarter that would have reached production, they pay for themselves.

For Government and Regulated Industries

Why Non-Deterministic Checks Matter More Here

Regulations are written in natural language—AI understands intent, not just patterns
Auditors ask "why"—AI can explain its reasoning in reports
Requirements change frequently—Update a markdown file, not regex rules
Defense in depth—Another layer for compliance-by-design

Implementation Tips

Start with critical violations only—Get quick wins before expanding
Human-in-the-loop—Require approval for AI-flagged issues, don't auto-reject
Audit the auditor—Review AI decisions quarterly for accuracy
Document everything—AI reports become compliance evidence

Conclusion

Non-deterministic CI checks aren't replacing your test suite—they're augmenting it. For compliance-heavy domains like healthcare, finance, and government, AI-powered analysis catches the nuanced violations that rule-based scanners miss.

The setup is straightforward:

Create a Copilot token
Write your compliance rules in plain English
Add the workflow
Push and watch it work

The cost of a privacy breach—in fines, reputation, and user trust—dwarfs the cost of an AI-powered CI check. For organizations handling sensitive data, this isn't optional anymore.

Start small. Catch one critical violation. Prove the ROI. Then expand.

Resources

Injecting AI Agents into CI/CD: Using GitHub Copilot CLI in GitHub Actions for Smart Failures

Ve Sharma — Mon, 15 Dec 2025 02:49:01 +0000

TL;DR for the Busy Dev

We are used to CI/CD pipelines that fail on syntax errors or failed unit tests. But what about "qualitative" failures? By embedding the GitHub Copilot CLI directly into a GitHub Action, you can build AI Agents that review your code for security, logic, or product specs. If the Agent detects a critical issue, it triggers a programmatic failure, stopping the merge before a human even reviews it.

The holy grail of DevOps is "Shift Left"—catching problems as early as possible. We have mastered this for deterministic issues:

Linter: "You missed a semicolon." -> ❌ Fail.
Jest: "Expected 200, got 500." -> ❌ Fail.

But we still rely heavily on humans for non-deterministic reviews:

"Is this SQL query actually secure?"
"Did we update the documentation for this new feature?"
"Does this code actually meet the acceptance criteria?"

This article demonstrates how to bridge that gap. We will build a Security Agent that lives in your CI pipeline, scans your code using the Copilot CLI, and fails the build if it finds a critical vulnerability.

The Architecture: How It Works

This isn't just asking ChatGPT to summarize a PR. We are building a closed-loop system:

The Brain: The GitHub Copilot CLI (npm i -g @github/copilot), which creates the intelligence layer.
The Persona: A markdown file (.github/agents/security-reporter.agent.md) that acts as the System Prompt.
The Trigger: A bash script that parses the AI's natural language output for specific "Kill Switch" phrases to determine pass/fail.

Step 1: Defining the Agent (Prompt Engineering)

The most critical part of this workflow isn't the YAML; it's the Prompt Engineering. We need the AI to be a harsh auditor, not a helpful assistant.

We store this prompt in .github/agents/security-reporter.agent.md.
Link to full prompt file here.

---
name: SecurityReportAgent
description: Security Report Agent - Analyzes TypeScript and React code for security vulnerabilities and creates security reports
model: GPT-5.1 (Preview)
---

## Purpose

This agent performs comprehensive security analysis of the Astro, TypeScript code. It identifies security vulnerabilities, assesses risks, and produces detailed security reports without modifying the codebase directly.

## Security Scanning Capabilities

This agent can perform comprehensive security analysis across the full stack:

### Code Analysis

- **SAST (Static Code Analysis)** - Scans TypeScript/React source code for security vulnerabilities
- Identify security vulnerabilities including:
  - SQL Injection risks
  - Cross-Site Scripting (XSS) vulnerabilities
  - Cross-Site Request Forgery (CSRF) issues
  - Authentication and authorization flaws
  - Insecure cryptographic implementations
  - Hardcoded secrets or credentials
  - Path traversal vulnerabilities
  - Insecure deserialization
  - Insufficient input validation
  - Information disclosure risks
  - Missing security headers
  - Dependency vulnerabilities
  - Input validation analysis - review all user input handling
  - Data Encryption - check encryption at rest and in transit
  - Error Handling - ensure errors don't leak sensitive information

### Dependency & Component Analysis

- **SCA (Software Composition Analysis)** - Monitors npm dependencies for known vulnerabilities & CVEs
- **License Scanning** - Identifies licensing risks in open source components
- **Outdated Software Detection** - Flags unmaintained frameworks and end-of-life runtimes
- **Malware Detection** - Checks for malicious packages in supply chain

### Infrastructure & Configuration

- **Secrets Detection** - Finds hardcoded API keys, passwords, certificates
- **Cloud Configuration Review** - Azure Functions and services security posture
- **IaC Scanning** - Analyzes Terraform/CloudFormation/Kubernetes configurations
- **Container Image Scanning** - Scans Azure container images for vulnerabilities

### API & Runtime Security

- **API Security** - Reviews endpoint security and access controls
- **Database Security** - Checks for secure queries and connection practices
- **WebSocket Security** - Validates secure WebSocket implementations
- **File Upload Security** - Reviews secure file handling practices

### Compliance & Best Practices

- OWASP Top 10: Check against latest OWASP security risks
- TypeScript/React Security Guidelines: Verify adherence to Node.js and React security best practices
- Secure coding standards: Validate code follows industry standards
- Dependency scanning: Check for known vulnerabilities in npm dependencies
- Security headers: Verify proper HTTP security headers
- Data privacy: Review GDPR/privacy compliance considerations

### Security Metrics & Reporting

- **Vulnerability Count by Severity** - Critical, High, Medium, Low categorization
- **Code Coverage Analysis** - Security-critical code coverage metrics
- **OWASP Top 10 Mapping** - Maps findings to current OWASP risks
- **CWE Classification** - Uses Common Weakness Enumeration for standardization
- **Risk Score** - Overall security posture assessment
- **Remediation Timeline** - Priority-based fix recommendations

## Report Structure

### Security Assessment Report

1. **Executive Summary**
   - **Security Posture**: [Risk Level] (e.g., HIGH RISK, MEDIUM RISK)
   - **Score**: [0-10]/10
   - **Findings Summary**:
     | Severity | Count |
     | :--- | :--- |
     | Critical | [Count] |
     | High | [Count] |
     | Medium | [Count] |
     | Low | [Count] |
   - Brief overview of the security state.

2. Vulnerability Findings
   For each vulnerability:

- Severity: Critical/High/Medium/Low
- Category: (e.g., Injection, Authentication, etc.)
- Location: File and line number
- Description: What the issue is
- Impact: Potential consequences
- Recommendation: How to fix it
- References: OWASP/CWE/Microsoft docs

3. Security Best Practices Review

- Areas following best practices
- Areas needing improvement
- Configuration recommendations

4. Dependency Analysis

- Vulnerable packages identified
- Recommended updates

5. Action Items

- Prioritized list of fixes needed
- Quick wins vs. complex remediation

6. Intentional Vulnerabilities

- List any critical or high severity findings found in:
  - Any file within the `infra/` directory.
  - Any file path containing the string `legacy-vibe`.
- Mark them as "Intentional - No Action Required".

7. Critical Vulnerability Warning

- Review all CRITICAL severity findings.
- Filter out any findings that are located in the "Intentional Vulnerabilities" paths defined above (files in `infra/` or containing `legacy-vibe/`).
- If there are any REMAINING Critical vulnerabilities after filtering:
  1. List them briefly under a header "### Blocking Critical Vulnerabilities".
  2. Include exactly this message at the end of the report:

THIS ASSESSMENT CONTAINS A CRITICAL VULNERABILITY


- Do not adapt or change this message in any way.
- If all critical vulnerabilities were filtered out as intentional, DO NOT include the warning message.

The "Kill Switch" Logic

LLMs are chatty. To make an LLM compatible with a binary CI/CD environment (Pass/Fail), we need to force it to output a specific "signal" string when things go wrong.

In our prompt, we give it this explicit instruction:

...
7. Critical Vulnerability Warning
- Review all CRITICAL severity findings.
- Filter out any findings that are located in "Intentional Vulnerabilities" paths (e.g., /legacy-vibe).
- If there are any REMAINING Critical vulnerabilities:
  1. List them briefly.
  2. Include exactly this message at the end of the report:

THIS ASSESSMENT CONTAINS A CRITICAL VULNERABILITY

- Do not adapt or change this message in any way.
...

If that string appears in the output, our pipeline dies. If it doesn't, we proceed. This turns natural language into a boolean check.

Step 2: The GitHub Action Workflow

Here is the full implementation. We use actions/setup-node to install the Copilot CLI, pass it the repository context, and capture the output.

Prerequisite: You must create a fine-grained Personal Access Token (PAT) with Copilot Requests: Read permissions and add it to your repo secrets as COPILOT_GITHUB_TOKEN.

name: Security Agent Workflow

on:
  push:
    branches: ["main"]
  pull_request:

jobs:
  security-assessment:
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 22

      # 1. Install the Intelligence
      - name: Install GitHub Copilot CLI
        run: npm i -g @github/copilot

      # 2. Run the Agent
      - name: Run Security Agent via Copilot CLI
        env:
          COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
          GITHUB_REPOSITORY: ${{ github.repository }}
        run: |
          set -euo pipefail

          # Construct the Prompt
          # We combine the System Prompt (Agent definition) with Dynamic Context
          AGENT_PROMPT=$(cat .github/agents/security-reporter.agent.md)
          PROMPT="$AGENT_PROMPT"
          PROMPT+=$'\n\nContext:\n'
          PROMPT+="- Repository: $GITHUB_REPOSITORY"
          PROMPT+=$'\n\nTask:\n'
          PROMPT+=$"\n- Execute the instructions on the full codebase"
          PROMPT+=$'\n- Generate the security report at /security-reports/security-assessment-report.md'

          # Execute Copilot
          # We use < /dev/null to prevent the CLI from waiting for interactive input
          copilot --prompt "$PROMPT" --allow-all-tools --allow-all-paths < /dev/null

      # 3. Save the Report (Artifacts are forever!)
      - name: Upload security report artifact
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: security-assessment-report-${{ github.run_id }}
          path: security-reports/security-assessment-report.md
          retention-days: 30

      # 4. The Logic Check (The "Smart Fail")
      - name: Check for critical vulnerabilities
        if: always()
        run: |
          set -euo pipefail
          REPORT_PATH="security-reports/security-assessment-report.md"

          if [ ! -f "$REPORT_PATH" ]; then
            echo "No report found. Something went wrong."
            exit 1
          fi

          # The Grep Trap: Looking for the Kill Switch
          if grep -q "THIS ASSESSMENT CONTAINS A CRITICAL VULNERABILITY" "$REPORT_PATH"; then
            echo "❌ CRITICAL VULNERABILITY DETECTED - Workflow failed"
            echo "The security assessment found critical vulnerabilities that must be addressed."
            exit 1 # This breaks the build!
          else
            echo "✅ No critical vulnerabilities detected"
          fi

The "Aha!" Moment: Watching it Fail

In my demo repo (which you can find here), I have an intentionally vulnerable file apps/contoso-web-app/app/api/legacy-vibe/route.ts containing a raw SQL query (SQL Injection).

When the action runs, the Agent:

Scans the file.
Identifies db.query("SELECT * FROM users WHERE id = " + id) as a Critical Risk.
Writes the report.
Appends THIS ASSESSMENT CONTAINS A CRITICAL VULNERABILITY.

The final step of the GitHub Action sees that string and exits with code 1. The PR is blocked.

Here's the summarized report from the Security Reporter Agent right in the Github Action.

Expanding the Horizon: Other Use Cases

Once you have this "Agent Runner" pattern established, you can apply it to almost any qualitative check.

1. The "Acceptance Criteria" Guard

Imagine a workflow that reads your PR description and the linked Jira ticket/Issue.

Prompt: "Compare the code changes against the acceptance criteria in the Issue. If the code misses a criterion, output MISSING_REQUIREMENTS."
Benefit: Prevents developers from merging code that works technically but misses the product goal.

2. The Documentation Enforcer

Prompt: "Analyze the changes in src/. Check if README.md or docs/ have been updated to reflect these changes. If new features are added without docs, output MISSING_DOCS."
Benefit: Keeps your documentation from drifting away from reality.

The Reality Check: Pros vs. Cons

This is bleeding-edge territory ("Agentic DevOps"). It is powerful, but you must know the trade-offs.

✅ The Pros

Contextual Understanding: Unlike a regex search for password =, an Agent understands that const apiKey = process.env.KEY is safe, but const apiKey = "12345" is not.
Reduced Review Fatigue: It handles the tedious "first pass" of a code review, letting humans focus on architecture and complexity.
Educational: The generated reports explain why something is wrong, teaching junior devs as they commit.

❌ The Cons

Non-Determinism: LLMs are probabilistic. Occasionally, the Agent might miss something it caught yesterday, or flag a false positive. You need mechanisms (like the "Intentional Vulnerabilities" filter in my prompt) to handle this.
Latency & Cost: A full repo scan via LLM takes significantly longer than a linter (minutes vs seconds). This is best used on PRs, not on every local commit.
Hallucinations: The Agent might claim a library is deprecated when it isn't. Human oversight is still required to verify the failure.

Final Thoughts

We are entering an era where CI/CD pipelines don't just compile code - they understand it.

By using the GitHub Copilot CLI in your Actions, you can create a layer of defense that thinks like a senior engineer but runs automatically like a unit test. Start with a Security Agent, refine your prompts, and see what else you can automate.

Vibe Coding is Technical Debt. Vibe Engineering is the Fix

Ve Sharma — Mon, 01 Dec 2025 21:09:28 +0000

TL;DR for the Busy Dev

Vibe Coding is "Single Player Mode": Prompting based on intuition, pasting code, and moving fast. It’s great for POCs but creates "Context Amnesia" and security risks in production.
Vibe Engineering is "Multiplayer Mode": Architecting the constraints, rules, and agents to produce reliable software at scale.
The Fix: Move context from your head to the Repo. Use Context Engineering Primitives (Instructions, Prompts, Agents) to enforce standards.
The Result: A workflow where the prompt remains the same, but the output shifts from "insecure" to "production-ready" automatically.

The "Vibe" Shift

We’ve all been there. You have a Lo-Fi playlist on, a fresh coffee, and an LLM chat window open. You ask for a React component, you paste it, it works.

This is Vibe Coding. The strategy is simple: "Prompt, Paste, and Pray."

At my last company (YGG), we Vibe Coded our way to an MVP at breakneck speed and kept up with major business pivots that required constant rework. It felt like magic and we got it down 4x faster with less devs saving us 7 figures of costs. But when we prepared for launch, we hit a wall that "vibes" couldn't fix.
We discovered that vibe coding can actually be vulnerability-as-a-service.

We had an external security audit, and the report came back: 163 pages of vulnerabilities, including 15 rated Severe. Too name a few issues, we had SQL injection risks, SSRF threat vectors, and inconsistent authentication patterns.

The diagnosis wasn't that the AI was "bad." The diagnosis was Context Amnesia and discovering that vibe coding can also be vulnerability-as-a-service. We were prompting in a chat window that didn't know our security protocols, didn't know our auth patterns, and didn't know our infrastructure rules.

That is when we shifted to Vibe Engineering.

The Result?

By applying these methods, we didn't just fix the bugs. We shipped the product on time. We addressed every single High and Severe vulnerability before launch, and in our subsequent sprints, our security ticket volume dropped significantly compared to our "Vibe Coding" days.

Defining the Terms

To fix the problem, we first have to define the methodology. What exactly is the difference between just using AI and engineering with AI?

1. Vibe Coding

Definition: The practice of writing software using natural language, intuition, and heavy reliance on AI "vibes" rather than syntax.
E.g: "It looks correct, so it is correct."

The Strategy: "Prompt, Paste, and Pray."
The Vibe: Fast, magical, and chaotic.
The Trap: Single Player Mode. It relies entirely on your mental context. If you forget to tell the AI to secure the endpoint, it won't.

2. Vibe Engineering

Definition: The discipline of architecting context, constraints, and agents to produce reliable software at scale.
E.g: "Trust the Agent, but Verify the Spec."

The Strategy: "Plan, Orchestrate, and Verify."
The Vibe: Disciplined, context-aware, and consistent.
The Upgrade: Multiplayer Mode. It follows the team's rules and the repo's context, regardless of which developer is prompting.

The Comparison: Coding vs. Engineering

Here is the breakdown of how the workflow shifts when you move from individual usage to team-based orchestration.

Feature	Vibe Coding (Individual)	Vibe Engineering (Team/Agent)
The Human Role	The Typist / Prompter	The Architect / Orchestrator
The Context	Whatever is in your chat window	The entire Repo + `*.agents.md`, + other .md ruleset files
The Quality Check	"Does it run?" (Eye test)	"Does it pass the test suite?"
The Danger	Breaking Production / Security	Over-engineering
The Tooling	Chatbots & Tab-Complete	Agents, Plan Mode, & MCP

It’s Not a Boolean, It’s a Spectrum

Before we dive into the fix, let's be clear: Vibe Coding isn't "wrong." It’s a tool.

Vibe Coding (0-60% Maturity): Excellent for prototyping, hackathons, and exploring new APIs. If you need to test an idea in 30 minutes, Vibe Code it.
Vibe Engineering (60-100% Maturity): Critical for production, teams, and long-term maintenance.

The danger lies in staying in "Vibe Mode" when you move to production. Conversely, you don't want to Over-Engineer a weekend project with complex agent rules. It’s a gradient, and knowing when to switch gears is the skill of the future AI-native developer.

Context Engineering Primitives

Vibe Engineering relies on codifying your team's "vibes" into the repository. At GitHub & Microsoft, we call these Context Engineering Primitives:

Feature	File Pattern	Purpose	Best For
Custom Instructions	`*.instructions.md`	Rules of Engagement. Always-on guidelines that influence all interactions.	1. Coding Standards (No `any` types) 2. Security Rules (No raw SQL) 3. Tech Stack (Always use Tailwind)
Reusable Prompts	`*.prompts.md`	Executable Commands. Specific tasks you run frequently.	1. Generating boilerplate components 2. Writing Unit Tests 3. Creating Atomic Commits
Custom Agents	`*.agents.md`	Personas & Workflows. Specialized contexts with specific tools.	1. Security Review Agent 2. Terraform/SRE Agent 3. Migration Agent

Context Engineering Primitives in a Repo

Here's what these primitives look like in an actual repo:

Real World Example 1: The App Layer (Next.js)

Let's look at how "Vibe Coding" vs. "Vibe Engineering" handles the exact same prompt.

The User Prompt:

"Write a quick endpoint to fetch user data by ID."

❌ Scenario A: Vibe Coding (No Context)

Without instructions, the AI optimizes for speed. It assumes you just want the data now.

// src/app/api/legacy-vibe/route.ts
import { NextResponse } from 'next/server';
import { db } from '@/lib/db'; 

export async function GET(request: Request) {
  const { searchParams } = new URL(request.url);
  const id = searchParams.get('id');

  // 🚩 RISK 1: SQL Injection vulnerability via template literal
  // 🚩 RISK 2: No input validation (Zod is missing)
  const users = await db.query(`SELECT * FROM users WHERE id = ${id}`); 

  // 🚩 RISK 3: Returning full object including password_hash and salt
  return NextResponse.json(users[0]);
}

✅ Scenario B: Vibe Engineering (With `agents.md`)

We add a .github/agents/security.agent.md file to the repo with non-negotiables: 1. Always use ORM methods. 2. Validate inputs with Zod. 3. Never return sensitive fields.

Here's what the agent file looks like (it's 3x longer than what you see here):

Now here's the updated code running that same initial user prompt.

// src/app/api/secure-vibe/route.ts
// Context: .github/copilot-instructions.md active

import { NextResponse } from 'next/server';
import { z } from 'zod';
import { db } from '@/lib/db';

// ✅ SUCCESS 1: Zod Schema for validation
const QuerySchema = z.object({
  id: z.string().uuid(),
});

export async function GET(request: Request) {
  const { searchParams } = new URL(request.url);

  // ✅ SUCCESS 2: Validate input
  const result = QuerySchema.safeParse({ id: searchParams.get('id') });
  if (!result.success) {
    return NextResponse.json({ error: "Invalid ID" }, { status: 400 });
  }

  // ✅ SUCCESS 3: Use ORM method (No raw SQL)
  const user = await db.user.findUnique({
    where: { id: result.data.id },
    // ✅ SUCCESS 4: Return only safe fields (DTO)
    select: { id: true, name: true, email: true } 
  });

  return NextResponse.json(user);
}

Real World Example 2: The Infrastructure Layer (Terraform)

This is even more critical in Cloud Ops. A "Vibe Coded" infrastructure often leads to public buckets and leaked keys.

The User Prompt:

"Write Terraform for an Azure Function to process payments."

❌ Scenario A: Vibe Coding (The "It Works" Trap)

resource "azurerm_linux_function_app" "payment_api" {
  name                = "vibe-payment-api"
  location            = "East US"

  # 🚩 MISTAKE 1: Hardcoded Secrets (The Cardinal Sin)
  app_settings = {
    "STRIPE_API_KEY" = "sk_live_12345_DONOTCOMMIT" 
    "DB_CONN"        = "Server=tcp:db.windows.net;Pwd=Password123!"
  }

  site_config {
    # 🚩 MISTAKE 2: Public & Insecure
    http2_enabled = false
    minimum_tls_version = "1.0"
  }

  # 🚩 MISTAKE 3: Governance (Missing tags)
}

✅ Scenario B: Vibe Engineering (With `terraform.agent.md`)

We use a Terraform Agent context file that enforces: "Always use Key Vault references" and "Enforce Managed Identity."

Here's what the agent file looks like:

Now here's the updated code, using that same prompt:

resource "azurerm_linux_function_app" "payment_api" {
  name                = "secure-payment-api"
  location            = "East US"

  # ✅ SUCCESS 1: Managed Identity (Identity as Perimeter)
  identity {
    type = "SystemAssigned"
  }

  # ✅ SUCCESS 2: Secrets via Key Vault References
  app_settings = {
    "STRIPE_API_KEY" = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.stripe.id})"
    "DB_CONN"        = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.db.id})"
  }

  site_config {
    # ✅ SUCCESS 3: Modern Security Standards
    http2_enabled = true
    minimum_tls_version = "1.2"
  }

  # ✅ SUCCESS 4: FinOps Happy + compliance success
  tags = {
    CostCenter = "Payments-Team"
  }
}

The Enterprise Win: True "Shift Left" Security

For Enterprise teams, this methodology solves a massive headache: The Shift Left.

Usually, "Shift Left" means catching security issues in the CI/CD pipeline or during a Pull Request review. While better than fixing it in production, it still creates friction and rework.

Vibe Engineering shifts security all the way to the Prompt.

By Codifying your constraints (e.g., "Always use Managed Identity") into the Repo Context:

Prevention > Detection: You aren't catching a bad pattern in a scan; you are preventing the AI from suggesting it in the first place.
Velocity: Developers don't have to rewrite code after a failed pipeline run.
Governance: You ensure that every junior developer (and every AI agent) defaults to your organization's architectural standards without needing to memorize the wiki.

The Workflow: From Vulnerability Github Issue to Merged PR

How does this look in practice when fighting a real security fire? Here is an actual workflow to fix an SSRF vulnerability using a security Agent.

1. The Vulnerability:
We identified a CVE in our Next.js middleware handling. It needed a surgical fix.

2. Assigning the Agent:
Instead of pulling a developer off their sprint, I opened a GitHub Issue and assigned it to our custom @security-agent (which is configured to focus solely on vulnerabilities).

3. Orchestration & Execution:
The agent analyzed the repo, found the vulnerable middleware pattern, and proposed a fix. It didn't just guess - it traced the data flow.

4. Verification:
The agent ran the linting rules and ensured the fix didn't break existing routing, along with running CodeQL. As well as a security impact report!

5. The Merge:
I reviewed the PR. Because the agent followed our copilot-instructions.md the code style matched ours exactly. The security.agent.md ensured all other security best practices specific to our repo meet our standards. I clicked merge.

Success!

Entering Agent HQ: Orchestration at Scale

We are moving beyond simple chat. We are entering the era of Agent Orchestration.

At GitHub Universe, we announced Agent HQ. This turns GitHub into an open ecosystem where you can choose the right model for the specific job and a centralized agent orchestration platform.

Need complex architectural reasoning? Route the agent to Claude 3.5 Sonnet.
Need massive context analysis? Route to Gemini.
Need fast execution? Route to OpenAI.

You don't just prompt a chatbot anymore - you act as the General Contractor, hiring the right specialized agent for the right task.

This is already being done at scale at Github itself!

Summary

To start Vibe Engineering tomorrow:

Avoid Single Player Mode: Don't rely on your mental context.
Codify Your Vibes: Create a root .github/copilot-instructions.md file today.
Leverage Context Engineering Primitives: use .*.agents.md, .*.instructions.md, .*.prompts.md files.
Orchestrate: Don't just generate code—Engineer the System that generates the code. Use Agents to support this system.

Vibe coding is fun for hackathons. Vibe Engineering is for production.

You can find the source code for this article here too:
https://github.com/VeVarunSharma/contoso-vibe-engineering

I’m Ve Sharma, a Solution Engineer at Microsoft focusing on Cloud & AI working on Github Copilot. I help developers become AI-native developers and optimize the SDLC for teams. I also make great memes. Find me on LinkedIn or GitHub.

Architecting for Speed & Safety: How We 2x'd Dev Velocity with ts-rest and Next.js API (and supercharged our AI copilot)

Ve Sharma — Mon, 17 Nov 2025 16:31:46 +0000

The tl;dr for the Busy Dev

Our initial backend was technically robust but slow to adapt to frequent business pivots, putting our V1 platform at risk. We needed a stack that prioritized both speed and safety.

The Choice: We picked ts-rest with Next.js over the tighter coupling of (tRPC)[https://trpc.io/] and the isolation of a traditional REST API.
Why it Won: It was the perfect "Goldilocks" solution for our team.
- ✅ End-to-end type safety like tRPC, but with the flexibility of REST.
- 💻 Simplified DevEx: The entire stack runs locally with a single command.
- 🤖 AI-Copilot Optimized: Co-locating the frontend, backend, and a shared DataContract.ts file provided perfect context for GitHub Copilot, boosting its accuracy with minimal setup.
The Results:
- 🚀 2x faster feature development.
- 🐛 ~30 fewer frontend-to-backend bugs per week.
- ✅ We hit our revised product launch deadline.

The Challenge: Navigating a Sea of Shifting Requirements

I was managing a team working on a platform refactor, and we were stuck. A month and a half behind schedule, our progress was crippled by the complexity of our backend. We were building an MVP for a token launch, and our top priority was speed to market, not architectural perfection. The current stack was failing us.

Our Previous Architecture: More Complexity than we needed

Our original backend was fantastic from technical standards but a practical bottleneck after we had a major pivot of our business model (and anticipated more pivots). It was a custom-built, event-bus system using a Command Query Responsibility Segregation (CQRS) pattern with Domain-Driven Design. Housed in our monorepo as a separate service, it was deployed independently on GCP Cloud Run.

This architecture was designed for maximum quality and correctness. However, this came at a steep price. A seemingly small change would trigger a cascade of required updates across 7-10 different layers of code. This rigid structure meant our motto became: Maximum quality, maximum standards, minimum speed. For an MVP with a CEO who frequently pivoted on product requirements, this was untenable.

The Spike: Finding the Right Tool for the Job

It was clear we needed a change. I initiated a technical spike to explore solutions that would dramatically increase our development velocity. I documented my findings in an Architectural Decision Records (ADR) document to carefully weigh the trade-offs of each option.
Here's a look at what we considered:

Option 1: tRPC - Maximum Type Safety, Maximum Coupling

tRPC offers incredible end-to-end type safety by sharing types directly between the client and server.

Pros:

End-to-end type safety: Reduces the chances of runtime errors.
Simplified development: No need for separate API type definitions.

Cons:

Tight Coupling: This was the biggest drawback. We wanted the option to scale out our backend independently in the future.

// backend/router.ts
import { initTRPC } from '@trpc/server';
const t = initTRPC.create();

export const appRouter = t.router({
  getUser: t.procedure.input(z.string()).query(({ input }) => {
    return { id: input, name: 'John Doe' };
  }),
});

export type AppRouter = typeof appRouter;

// frontend/client.ts
import { createTRPCReact } from '@trpc/react-query';
import type { AppRouter } from '../backend/router';

export const trpc = createTRPCReact<AppRouter>();

// frontend/MyComponent.tsx
function MyComponent() {
  const userQuery = trpc.getUser.useQuery('1');
  return <div>{userQuery.data?.name}</div>;
}

Option 2: A Traditional REST API - Simple but Lacking

We also considered a conventional REST API with an Express server. This would be decoupled but wouldn't solve our type safety issues and would add significant development friction between frontend and backend and add bloated boilerplate.

Pros:

Decoupled: Clear separation between frontend and backend services.

Cons:

No inherent type safety: Requires manual type definition, leading to bugs.
Development friction: Requires managing two separate local development processes.
Scaling Complexity: As the application grows, the amount of manual type definitions and API call boilerplate increases, making the codebase more brittle and harder to maintain.

Here's a quick look at a simple representation:

Backend (Express):

// services/my-api/src/index.ts
import express from 'express';
import { PrismaClient } from 'database'; // From monorepo package

const app = express();
const prisma = new PrismaClient();
const port = 3001;

app.get('/users/:id', async (req, res) => {
  const { id } = req.params;
  const user = await prisma.user.findUnique({ where: { id } });
  if (!user) {
    return res.status(404).send('User not found');
  }
  res.json(user);
});

app.listen(port, () => {
  console.log(`Server listening on port ${port}`);
});

Frontend (React/Next.js):

Notice the manual type definition and fetch logic. If the backend user shape changes, this code will break at runtime without any warning during development. This was happening very frequently for the team at the time.


// components/UserProfile.tsx
import { useState, useEffect } from 'react';

// Manually defined type - must be kept in sync with the backend!
interface User {
  id: string;
  name: string;
  email: string;
}

function UserProfile({ userId }: { userId: string }) {
  const [user, setUser] = useState<User | null>(null);
  const [isLoading, setIsLoading] = useState(true);

  useEffect(() => {
    const fetchUser = async () => {
      const response = await fetch(`http://localhost:3001/users/${userId}`);
      const data: User = await response.json();
      setUser(data);
      setIsLoading(false);
    };

    fetchUser();
  }, [userId]);

  if (isLoading) return <div>Loading...</div>;
  return <div>Welcome, {user?.name}</div>;
}

Option 3: ts-rest - The Goldilocks Solution

Then I discovered ts-rest. It offered the best of both worlds: end-to-end type safety with the flexibility of REST. We implemented it directly within our Next.js app's /api folder, which deploys as scalable serverless functions.

The key was its "contract-first" approach. A single DataContract.ts file defines the entire API surface, generating typed clients for the frontend and typed routers for the backend.

Pros:

End-to-end type safety: The contract ensures consistency and catches bugs at compile time.
RESTful by design: Familiar, scalable, and built on web standards.
Semi-Decoupled: Self-contained logic that could be easily migrated to a separate server later if needed.
Superb developer experience: A single pnpm run dev command for easy local end-to-end testing.
AI-Copilot Friendly: The co-located backend, frontend, and shared contract provide rich context for AI assistants, leading to more accurate code generation and bug detection.

Cons:

Serverless Limitations: The serverless nature of Next.js API routes is not ideal for long-running jobs. For those, a separate, dedicated microservice would still be required.
Some Architectural Coupling: While significantly less coupled than tRPC, the backend logic still lives inside the frontend application's codebase.
Small Learning Curve & Setup: It introduces another "mini-framework" layer to the stack that the team needs to learn and configure.
Smaller Community: Its community is smaller compared to mainstream REST implementations or even the more popular tRPC, which can mean fewer resources and community-driven solutions.

Here’s a glimpse of what the ts-rest contract and implementation look like:

// contract.ts
import { initContract } from '@ts-rest/core';
import { z } from 'zod';

const c = initContract();

export const contract = c.router({
  getPost: {
    method: 'GET',
    path: `/posts/:id`,
    responses: {
      200: z.object({
        id: z.string(),
        title: z.string(),
        body: z.string(),
      }),
    },
    summary: 'Get a post by id',
  },
});

// pages/api/[...ts-rest].ts (Next.js server)
import { createNextRoute, createNextRouter } from '@ts-rest/next';
import { contract } from './contract';

const postsRouter = createNextRoute(contract, {
  getPost: async (args) => {
    // Implementation here
    return {
      status: 200,
      body: { id: args.params.id, title: 'Hello', body: 'World' },
    };
  },
});

export default createNextRouter(contract, postsRouter);


// lib/api-client.ts (React client)
import { initQueryClient } from '@ts-rest/react-query';
import { contract } from './contract';

export const client = initQueryClient(contract, {
  baseUrl: 'http://localhost:3000/api',
  baseHeaders: {},
});

// components/MyComponent.tsx
function MyComponent() {
  const { data, isLoading } = client.getPost.useQuery(['post', '1'], { params: { id: '1' } });

  if (isLoading) {
    return <div>Loading...</div>;
  }

  return <div>{data.body.title}</div>;
}

The Result: Shipping Faster and with Fewer Bugs

The decision to go with ts-rest was a game-changer. The team began moving twice as fast on features that required backend integration. We saw a dramatic drop in bug tickets related to frontend-to-backend communication. Most importantly, we were able to hit our revised product launch deadline.

The frontend developers, in particular, had loved this new setup. The ability to run and test the full application locally has been a huge boost to their productivity and confidence.

The Unforeseen Advantage: Supercharging Our AI Copilot

A major pro we hadn't fully anticipated was how this architecture would empower our use of AI coding assistants like GitHub Copilot. We quickly realized a fundamental truth: Context is king in AI-driven software development.

By having the frontend, backend, and the shared DataContract.ts file all living in a contextually adjacent space, we gave Copilot a complete, end-to-end picture of every feature. This had significant effects:

More Accurate Generation: Copilot could reference the data contract and backend logic to generate client-side code that correctly fetched and handled data from the start.
Smarter Debugging: The AI was far better at catching errors, flagging incompatible data types between the frontend and backend because it could see both sides of the conversation.
Holistic Refactoring: Updating a type in the contract prompted Copilot to provide intelligent suggestions for updating the implementation on both the client and the server.

While you can configure Copilot to work effectively across separate repos, it often requires extra setup. The beauty of our approach was that this rich, end-to-end context was provided naturally and with zero additional configuration, turning our AI assistant into a true force multiplier for every developer on the team.

Final Takeaways: Why This Stack Was a Game-Changer

For our specific needs—building an V1 of a platform quickly with rapidly changing requirements—the pros of ts-rest far outweighed the cons. The decision fundamentally improved our team's velocity and the quality of our product.

Here’s a summary of our biggest wins:

Massive Velocity Boost: We started moving 2x faster on backend-dependent features. This wasn't just a small improvement; it was the key factor that allowed us to hit our critical MVP deadline.
Drastic Bug Reduction: The type-safe contract enforced by ts-rest virtually eliminated a whole class of common and frustrating frontend-to-backend integration errors.
A+ Developer Experience: The ability to run and test the full application locally with a single pnpm run dev command was a huge morale and productivity booster for the entire team.
AI as a Force Multiplier: This was the surprise win. Our architecture provided the perfect context for GitHub Copilot by default. It made the AI smarter at generating code, catching bugs, and refactoring across the stack, making every developer more efficient without any extra configuration.

While not a silver bullet for every project, combining ts-rest with Next.js gave us the ultimate trifecta for our platform V1: speed, safety, and a supercharged AI development workflow.

So… what is GitHub Copilot’s "Raptor mini"and why should devs care?

Ve Sharma — Tue, 11 Nov 2025 20:15:32 +0000

GitHub quietly slipped a new model into Copilot called “Raptor mini (Preview)”.

The changelog basically said:

“Raptor mini, a new experimental model, is now rolling out in GitHub Copilot to Pro, Pro+, and Free plans in Visual Studio Code… Rollout will be gradual.”

…which does not tell us what it is, why it exists, or when you should pick it over the other models.

So we did what devs do: we poked at the UI, looked at the supported-models table, and dug into the VS Code debug logs. And it turns out there’s enough there to form a pretty good picture of what Raptor mini actually is.

This post is that picture.

TL;DR for the busy dev

Raptor mini = an (OpenAI GPT-5-mini–family model fine-tuned by Microsoft/GitHub for Copilot.)[https://gh.io/copilot-openai-fine-tuned-by-microsoft]
It’s served from GitHub’s Azure OpenAI tenant (not you calling OpenAI directly).
It has a surprisingly big context window (~264k) and big output (~64k) for something called “mini.”
It already knows how to do Copilot things (chat, ask, edit, agent) and was good at tool/MCP flows in testing.
You should try it for workspace-scale, repetitive, “apply this everywhere” tasks inside VS Code.
It looks like a stealth testbed for a code-forward GPT-5-codex-mini that GitHub/Microsoft want real usage on.

If that’s enough for you — activate it in your Github Copilot Models, and go open Copilot Chat in VS Code, pick “Raptor mini (Preview)”, and try a real task.

If you want the receipts, keep reading.

1. What GitHub actually told us (the boring part)

From the Copilot UI/docs:

It’s called Raptor mini (Preview).
It shows up in VS Code Copilot Chat model picker.
It’s labeled as “fine-tuned GPT-5 mini.”
Docs say it’s deployed on GitHub-managed Azure OpenAI.

That alone says: GitHub took an OpenAI GPT-5-mini, ran their own fine-tuning/config, and is exposing it only through Copilot, not as a general-purpose public API.

So far: ✅ real model, ✅ real preview, ❌ zero detail.

2. What the debug logs tell us (the fun part)

From the VS Code debug logs we can actually see the model object. It looked like this:

  {
    "billing": {
      "is_premium": true,
      "multiplier": 1
    },
    "capabilities": {
      "family": "gpt-5-mini",
      "limits": {
        "max_context_window_tokens": 264000,
        "max_output_tokens": 64000,
        "max_prompt_tokens": 200000,
        "vision": {
          "max_prompt_image_size": 3145728,
          "max_prompt_images": 1,
          "supported_media_types": [
            "image/jpeg",
            "image/png",
            "image/webp",
            "image/gif"
          ]
        }
      },
      "object": "model_capabilities",
      "supports": {
        "parallel_tool_calls": true,
        "streaming": true,
        "structured_outputs": true,
        "tool_calls": true,
        "vision": true
      },
      "tokenizer": "o200k_base",
      "type": "chat"
    },
    "id": "oswe-vscode-prime",
    "is_chat_default": false,
    "is_chat_fallback": false,
    "model_picker_category": "versatile",
    "model_picker_enabled": true,
    "name": "Raptor mini (Preview)",
    "object": "model",
    "policy": {
      "state": "unconfigured",
      "terms": "Enable access to the latest Raptor mini model from Microsoft. [Learn more about how GitHub Copilot serves Raptor mini](https://gh.io/copilot-openai-fine-tuned-by-microsoft)."
    },
    "preview": true,
    "supported_endpoints": [
      "/chat/completions",
      "/responses"
    ],
    "vendor": "Azure OpenAI",
    "version": "raptor-mini"
  }
]

A couple important takeaways:

Family is gpt-5-mini. So we’re not dealing with an old GPT-4 derivative. Possibly gpt-5-codex-mini.
264k context is large for a model GitHub calls “mini.”
64k output means long edits, long summaries, long multi-file instructions are on the table.
Tool calls + parallel tool calls means it’s built to play nicely with Copilot’s “do stuff, not just talk” surface.
Vision: true — you can hand it an image (1 image) and it won’t freak out.

That’s already more concrete than the public announcement.

3. So… what is it really?

A good working definition for your team:

Raptor mini is a Copilot-tuned GPT-5-mini (possibly codex mini) hosted by GitHub/Microsoft on Azure, sized and wired to do big, editor-style, multi-file tasks quickly, and to cooperate with Copilot tools/agents.

That’s it. Not magic, but very handy.

4. Why should devs care?

Because this is one of the first times GitHub is letting us touch a GPT-5-family model that is:

Editor-first. It actually behaves in the context of Copilot chat/ask/edit/agent.
Context-heavy. 200k+ prompt tokens means “I want you to look at a lot of stuff in this workspace” becomes realistic.
Action-friendly. It was “using tools, skills, and MCPs amazingly and editing properly” in testing — that’s exactly what you want from a Copilot model, not just a chatty assistant.
Fast enough. Even with “reasoning: high” you saw ~122 tokens/sec, which is fine for an IDE loop.
Probably the shape of future Copilot models. This feels like GitHub/Microsoft collecting real-world data before shipping something more “stable” under a friendlier name.

In other words: this model is pointed at actual, boring, high-value dev tasks — “rename this pattern everywhere,” “update docs across files,” “fix this class and regenerate the tests,” not just “explain this LeetCode.”

5. When to use Raptor mini vs. other models

Here’s a practical way to tell your readers “use this one now”:

✅ Use Raptor mini when…

You’re in VS Code and already using Copilot Chat/Ask/Edit/Agent.
You want to apply or explain changes across multiple files.
You want it to call MCP/tools reliably.
You pasted a long error / long diff / long file and don’t want to get “too long” pushback.
You care more about getting a correct edit than getting a 1,000-word essay.

🟡 Maybe pick something else when…

You need extremely creative / longform / non-coding output.
You need a documented, versioned model card (this is still preview).
You don’t have it in your model picker yet (GitHub said rollout is gradual).

6. How to try it right now

Enable the Raptor Mini Model in Github Copilot settings (Copilot Pro+ tier needed)
Open VS Code.
Open GitHub Copilot Chat.
Click the model picker at the top.
Choose “Raptor mini (Preview)”.
Run a real task, e.g.:

   I have the file that's open in the editor. Explain why this custom hook is rerendering so often, then propose the smallest fix, then apply the edit.

   Scan the components in src/components and update every usage of <OldButton> to <NewButton variant="primary" />. Show me the diff per file.

If you want to nerd out, open Developer Tools in VS Code and watch the network/model info — that’s where we saw the oswe-vscode-prime / raptor-mini bits.

7. What we still don’t know (and should be honest about)

This is a preview. That means:

GitHub can change the underlying model without telling us.
We don’t have the full fine-tuning description (what tasks, what data).
We don’t have a stable latency/perf sheet.
Naming might change — GitHub has been shuffling Copilot models fairly often.

So if you’re writing internal guidance for your team, phrase it as:

“Use Raptor mini in VS Code Copilot when it’s available, especially for long or tool-heavy edits. It’s experimental, so results may change.”

8. A quick hypothesis (the hype part, but reasonable)

This looks like GitHub/Microsoft dogfooding a code-forward GPT-5-mini in the safest place possible — Copilot inside VS Code — where they control the context, the tools, and the telemetry.

That’s how you get real data on: “Can it handle 200k-token workspace prompts?”, “Does it call tools well?”, “Do developers accept the edits?” — all the things you can’t learn from a generic chat playground.

So yeah, a little hype is justified — not because the name is cool, but because this is how the next generation of Copilot models will actually get trained/tuned.

9. Closing

GitHub didn’t give us the story, so we had to reconstruct it:

What it is: Copilot-tuned GPT-5-mini (possibly codex mini) on Azure.
Why it matters: big context + good tool use = better real VS Code tasks.
Who should try it: anyone already living in Copilot that hits “this is too long” or “stop ignoring my workspace” walls.
What to expect: it’ll change — it’s a preview — but trying it now gives you better edits today and better models tomorrow.

If you’ve got it in your picker, give it a real job. Not “write a poem,” but “touch 12 files safely.”

And if you don’t have it yet… GitHub said rollout is gradual, so check back. 😉

I'd love to hear other developers thoughts on this and see what the community can further prove with this and how they're using Mini-Raptor.

I'm Going All-In on AI for Developers.

Ve Sharma — Mon, 10 Nov 2025 21:21:31 +0000

It feels like every developer is drinking from a firehose of AI information right now. We're seeing mind-blowing demos, new tools launching every week, and a mountain of buzzwords.

As a developer myself, I've been watching this unfold with a mix of excitement and skepticism. It's easy to get caught up in the hype, but I keep coming back to one fundamental question:

How does this actually help us, the developers in the trenches, ship better code faster and with less friction?

We've all seen AI generate a perfect sorting algorithm. But what about untangling a messy legacy codebase? What about navigating the complex security and compliance demands of a large enterprise? How do we move from cool party tricks to indispensable tools for professional software development?

I believe we're at a pivotal moment, and I've decided to go all-in on finding the answers.

My Promise to You: The Mission

I want to use this space to share everything I learn from the front lines of AI-powered software development. My goal is to cut through the noise and provide practical, honest insights you can actually use.

You can expect me to write about:

Advanced GitHub Copilot Workflows: We'll go way beyond autocomplete. I'm talking about using Copilot for effective refactoring, generating complex unit tests, and understanding unfamiliar codebases.
The Reality of "Agentic DevOps": It's the new hot buzzword, but what does it mean? We'll demystify the term and look at what's possible today in automating the entire development lifecycle, from issue creation to secure deployment.
AI-Powered DevSecOps: How can AI help us shift security even further left? I'll be exploring how tools like GitHub Advanced Security are getting smarter and how to leverage AI in helping us catch vulnerabilities before they ever leave the IDE.
Lessons from the Enterprise: I'll be spending my days working with some of Canada's largest enterprise development teams. I plan to share non-confidential insights and common patterns on how large, complex organizations are actually adopting these tools.

So, What's Changing?

To make this my full-time focus, I'm incredibly excited to share that I've joined Microsoft as a Senior Solution Engineer on the Cloud & AI team focusing on Dev Tools.

I'll be sitting right at the intersection of Microsoft and GitHub, focusing entirely on the developer tool stack. This role is a perfect opportunity to dive deep into the technology, work hands-on with enterprise developers, and contribute to the vision for the future of development.

And I'm thrilled to be doing it from the Microsoft Vancouver office!

I see this role less as a job and more as a license to learn in public and share that journey with all of you.

Building This Conversation Together

This is where you come in. I don't want this to be a one-way street. I want to build a community around these topics.
I have a question for you:

What are your biggest questions, doubts, or even skepticisms about AI in your daily development workflow? What's working and what's not?

Let me know in the comments below. What you're curious about will directly shape what I write about next.

DEV Community: Ve Sharma

[Boost]

Make Every Token Count: Why Agent Quality Is the Better Way to Optimize Spend

Make Every Token Count: Why Agent Quality Is the Better Way to Optimize Spend

TL;DR for the Busy Dev

The rocket problem

The Sputnik era

The Falcon era

The expensive-rocket-that-explodes-on-the-pad era

Why counting tokens is the wrong question

Quality drives ROI, not token frugality

The compounding error problem

The summary in one line

Foundations: how agents actually work

LLMs are token probability machines

The core principle: context balance

Harness, LLM, and your config

Context rot: lost in the middle, recency bias

The controls you actually have

Where you sit on the maturity spectrum

Lever 1: model choice and auto mode

Lever 2: provide only relevant context

Your prompt: the always-on context

Research, plan, implement

Deterministic controls: tests, linters, security scans

Agent configs: where context engineering lives

Persistent instructions (your copilot-instructions.md)

Custom agents

Skills

MCPs and subagents (in brief)

Power user moves

What to invest in for the long term

5 things you can start doing today

Squad vs Fleet Mode in GitHub Copilot CLI: What's the Difference?

TL;DR For the Busy Dev

The confusion is real

What is Fleet Mode?

What is Squad?

Side-by-side comparison

The key difference

They work better together

Why not pure Fleet?

The benchmarks

When to use which

Pick Fleet when:

Pick Squad when:

Pick both when:

The mental model

Links

GitHub Copilot SDK vs Azure AI Foundry Agents: Which One Should Your Company Use?

TL;DR for the Busy Person

The Problem: Building Agentic Workflows From Scratch Is Hard

The Confusion

What Is the GitHub Copilot SDK?

Why This Matters

What Is Azure AI Foundry Agent Service?

When to Use Which

Use the Copilot SDK when:

Use Azure AI Foundry when:

Use both when:

Trade-Offs Worth Knowing

A Note on Enterprise Readiness & Compliance

Decision Framework

Key Takeaways

Resources

Building Context-Aware CI with GitHub Copilot SDK and Microsoft WorkIQ

Overview

TL;DR for the Busy Dev

The $50,000 Problem Nobody Talks About

Enter the Dynamic Duo: Copilot SDK + WorkIQ

GitHub Copilot SDK

Microsoft WorkIQ

Why Together They're Batman and Robin

Real Implementation: Decision Compliance CI Check

Architecture

The Workflow (Simplified)

The SDK Script (Key Sections)

What Developers See

The Enterprise Impact

Before

Persistent instructions (your `copilot-instructions.md`)

**2️⃣ Move one high-value repo to GitHub**