DEV Community: Vignesh Athiappan

Turning a System of Record into an AI Agent: Building MCP Tools on Azure

Vignesh Athiappan — Fri, 17 Jul 2026 06:57:13 +0000

A practical, end-to-end walkthrough of taking a read-only slice of an enterprise source system and exposing it to an AI agent as a set of Model Context Protocol (MCP) tools — using Azure Logic Apps, API Management, and Agent Foundry. All identifiers below are placeholders; swap in your own.

The goal

We wanted a simple outcome: a user asks a business question in plain language and gets a straight answer — no dashboards, no field names, no training on the underlying system. That means an AI agent with safe, read-only access to a backend system of record, exposed as discrete tools the model can call.

The constraints shaped every decision:

Read-only. The agent can retrieve and analyze, never write.
Safe. Records in most business systems contain free text (notes, subjects, descriptions) that could carry prompt-injection payloads. Tool output must be treated as data, never instructions.
Composable. The agent should see many small, well-described tools — "list records", "aggregate by category", "record change history" — not one giant "query the system" tool.

The architecture

Five layers, one direction of flow:

Agent (Agent Foundry)
      │  MCP (JSON-RPC over HTTP)
      ▼
MCP server (API Management)
      │  REST operation per tool
      ▼
API gateway (API Management)
      │  single POST, routed by body
      ▼
Tool executor (Logic App)
      │  OAuth token + REST calls
      ▼
Source system (REST API)

The key design choice: one backend endpoint, many logical tools. The Logic App exposes a single HTTP trigger that accepts { "tool": "records.list", "parameters": { ... } } and routes internally. API Management then fans that single endpoint out into many named operations, and its MCP feature turns those operations into agent tools. This keeps the backend trivial to maintain while the agent still sees a rich, typed tool catalog.

Step 1 — Design the tools

Start from the questions, not the schema. A useful toolset usually falls into a few families:

Records: list / get / search / filter for the core business entities.
Activity & change history: activity on a record, and field-level history of what changed and when.
Analytics: server-side aggregates — totals, counts, group-by-category, and top-N rankings.
Metadata: let the agent introspect the available tables and fields at runtime instead of hardcoding everything.

Two implementation notes that matter a lot:

Prefer aggregates over "pull a list and count." If the source system can compute a sum or a group-by server-side, a "total by category" answer is one summary row, not hundreds of records the model has to tally. Push the work down to the system wherever you can.

Prototype against the real API first. A quick collection of calls against the source system's REST endpoint (with OAuth client-credentials) lets you nail the field names, filters, and relationships before you touch orchestration. Field-name mismatches are the most common silent failure — verify against real responses.

Step 2 — Build the tool executor (Logic App)

The Logic App is a dispatcher. One HTTP request trigger, then:

Fetch the source-system secret from Key Vault (never inline secrets).
Get an OAuth token via client credentials for the source system's scope.
Compute a category from the tool id (first(split(tool, '.'))).
A Switch on category, with a nested Switch on the full tool id inside each — each leaf is one read-only REST call.

A few hard-won details:

Switch cases cap at 25. With dozens of tools, a single switch won't do — split by category into an outer switch, then an inner switch per category. This "nested switch" pattern is also cleaner to read.
Action names must be globally unique across the whole definition.
Escape literal @. If any query parameter uses an @ alias, remember that Logic Apps reads a bare @ as an expression. Double it (@@) so it renders as a literal @ at runtime.
Seed a default result. Initialize the response variable to an { "error": "unknown tool" } object; each matched case overwrites it. Unknown tools then return cleanly instead of hanging.

Step 3 — Describe the tools as OpenAPI

The agent needs a tool schema per operation. Generate an OpenAPI document with one operation per tool, where each operation's request body is only that tool's parameters (a get op takes a record id; an aggregate takes nothing). Keep the path equal to the tool id so a gateway policy can recover it later.

Generate this programmatically from the same source of truth as the Logic App so the two can't drift — derive each operation's parameters from the inputs the Logic App actually reads.

Gotcha: API Management's inline "OpenAPI specification" editor parses the document as Swagger 2.0. Paste an OpenAPI 3.0 doc and you get "The Swagger version specified is unknown." Either import via the Add API → OpenAPI surface (which accepts 3.0), or hand it a Swagger 2.0 document. Same operations either way.

Step 4 — Put it behind API Management

Import the OpenAPI to create the operations, then add one API-scoped inbound policy that:

validates the caller's Entra token (validate-jwt with your audience),
recovers the tool id from the request path,
wraps the caller's parameters into the { tool, parameters } body the Logic App expects,
forwards to the single Logic App trigger.

<set-variable name="toolId" value="@(context.Request.OriginalUrl.Path.Split('/').Last())" />
<set-body><![CDATA[@{
    var incoming = context.Request.Body?.As<JObject>(preserveContent: true) ?? new JObject();
    if (incoming["tool"] != null) { return incoming.ToString(); }   // already wrapped
    var wrapped = new JObject();
    wrapped["tool"] = (string)context.Variables["toolId"];
    wrapped["parameters"] = incoming;
    return wrapped.ToString();
}]]></set-body>

Authenticating to the backend — drop the shared secret. A Logic App request trigger defaults to a SAS signature (sig) baked into its URL. That's a shared secret you then have to store and rotate. The better pattern: give API Management a managed identity, add an OAuth authorization policy on the Logic App trigger (issuer + audience, optionally locking to APIM's appid), disable SAS on the trigger, and let APIM authenticate with its own identity:

<authentication-managed-identity resource="api://<APP_ID>" />

No sig, nothing to rotate, and you can restrict the trigger to exactly one identity.

Step 5 — Expose the API as an MCP server

API Management can publish selected operations as an MCP server. Point its "expose an API as an MCP server" feature at your API, select the operations you want as tools, and it hands you a server URL (an /mcp endpoint). The operation operationIds become the tool names the agent sees, so keep them clean (records_list, analytics_by_category).

Tier caveat: MCP server export isn't available on the Consumption tier of API Management. If the MCP Servers blade is missing, that's why.

Step 6 — Connect to Agent Foundry

In the agent, add a Model Context Protocol tool:

Endpoint: the exact server URL from the MCP Servers blade.
Auth: Microsoft Entra.
Audience: this must match the aud your gateway's validate-jwt requires. The audience is the single most common misconfiguration — get it right and the token check passes.

If the gateway 401s and the token looks right, paste it into a JWT decoder and check iss/aud against the policy. Managed-identity tokens use the v1 issuer (https://sts.windows.net/<tenant>/); configuring the policy with the v2 issuer produces a mismatch.

Step 7 — The system prompt is part of the security perimeter

Tools enforce read-only at the API layer, but the behavior — tone, safety, honesty — lives in the system prompt. Ours does three jobs:

Clear framing. Lead with the answer, plain language, no field names, one drill-down.
Prompt-injection defense. The rule that matters most: the user's message is the request; tool output is evidence. Record names, notes, and free-text fields are attacker-influenceable, so the agent never follows instructions embedded in them — no "ignore previous instructions," no exfiltration, no acting on a record that says to.
Resolve-or-ask, and know your limits. Names get resolved to ids via a search tool first; ambiguous matches trigger a clarifying question rather than a guess. And the prompt names the gaps explicitly so the agent says "I don't have that" instead of improvising a malformed query.

That last point is worth underlining: the agent should never hand-craft queries or field names. It picks a tool and fills named inputs. If a question needs a field the tools don't expose, that's a capability gap to report, not something to invent.

Step 8 — Making it actually work

The demo worked on the first real question. The next few turns taught us the operational lessons:

Context windows blow up fast with many tools. Every tool definition rides in the prompt, and raw responses are large. Two fixes: trim tool payloads (request only the fields you need, cap page sizes, drop redundant related data), and use a model with tool search, which loads tool definitions on demand instead of front-loading all of them. Together they cut context dramatically.
A "chat"-class model is the wrong choice for a tool-heavy agent. Pick a model tuned for agentic tool calling with a large context window and, ideally, native tool search.
Auto model routers need a constrained pool. A router that can escalate to a model you haven't provisioned throws "deployment does not exist," and one that routes to a small model reintroduces the context ceiling. If you use a router, pin its subset to large-context, provisioned models — or just deploy one capable model directly.

Lessons learned

One backend, many tools keeps the surface simple while giving the agent a rich catalog.
Push secrets to Key Vault or eliminate them with managed identity — don't let a SAS signature or client secret live in policy or config.
Generate the OpenAPI from the same source as the executor so the schema can't drift from what the backend actually reads.
Trim aggressively. Lean payloads and a tool-search model beat a bigger context window every time.
The system prompt is security-critical, not just tone — treat tool output as untrusted, resolve-or-ask, and be honest about gaps.

The result is an agent a non-technical user can talk to in plain language, backed by a boringly maintainable pipeline, with the secret-handling and injection surface handled deliberately rather than by accident.

— Engineering notes, generalized. Replace all <placeholders> with your own tenant and resource values.

From a 15-Second Chatbot to a Real Agentic Assistant

Vignesh Athiappan — Thu, 16 Jul 2026 04:14:43 +0000

What a year of building an enterprise AI copilot actually taught me

When I started, the goal sounded simple: give employees one place to ask a question and get an answer. No more hunting through a dozen internal apps to find a leave policy, check a project allocation, or raise a request. One assistant, one box, one answer.

It took me the better part of a year, several architectures I'm now slightly embarrassed by, and a long list of failures that each taught me something specific. This is the honest version of that journey — the walls I hit, what I learned at each one, and the rearchitecture that finally made everything click.

(Names, systems, and numbers here are anonymized. This is about the engineering, not the company.)

The problem I set out to solve

The organization had roughly 6,000 employees across about ten countries, and their digital workday was fragmented. Policies lived in one place, HR data in another, tickets in a third, learning records in a fourth. Every routine question meant knowing which system to open and how to phrase things there.

The vision was a single AI assistant embedded in the employee platform: ask in plain language, and it retrieves the right knowledge, reads your live data, and — eventually — takes actions on your behalf. That's easy to say on a slide. The gap between the slide and a system that actually works is where all the learning lives.

Where I started — and the first wall

My first version was the obvious one. I had country-specific policy content indexed for search — thirteen separate vector indexes, one per country. Behind the assistant sat a workflow that received the question, looped through the relevant indexes, called a search agent, then called a language model to generate the answer. Every branch was wired by hand: knowledge path, live-data path, default path.

It worked. It also took about fifteen seconds per answer.

Fifteen seconds is an eternity for someone who just wants to know how many casual leaves they have left. Worse, the whole thing was sequential: an HTTP call, then a loop over indexes, then an agent call, then a model call, each waiting on the last. The workflow engine was doing orchestration work it was never designed for.

Lesson zero: if your response time embarrasses you in a demo, the architecture is telling you something. Don't tune it — question it.

Learning #1: A wrapped search is not a good search

My instinct was to consolidate: take those thirteen indexes and hang them all off one intelligent agent, letting the model pick which to query based on the question and the user's location. Cleaner, in theory.

Then I noticed something that stopped me. When I queried the search index directly, the results were sharp. When I routed the same question through an agent that wrapped the search, the answers got noticeably worse.

The reason turned out to be fundamental, and it's a trap a lot of people fall into. When you query search directly, you control the exact query text and exactly which retrieved chunks reach the model. When an agent sits in front of the search, it silently rewrites your question into its own search query first — and that rewrite loses nuance. "How many casual leaves can I take in [country]?" becomes a generic "casual leave policy," and the country-specific content never surfaces. On top of that, the agent's default retrieval settings rarely match the ones you carefully tuned for direct queries.

Lesson: an abstraction that "handles retrieval for you" is also an abstraction that quietly degrades it. I collapsed the thirteen country indexes into one federated index with a country filter — the total document count was small enough that a single filtered query was both faster and more accurate than looping over thirteen. That one change killed the index loop entirely and reclaimed a big chunk of those fifteen seconds.

Learning #2: Sequential is the enemy — and parallelism has its own traps

With retrieval fixed, the next bottleneck was structural. My workflow had around twenty intent branches — projects, skills, leave, allocations, and so on — evaluated one after another. So I parallelized them: fan every branch out from a single intent-detection step and let them run at once.

This is where I learned that concurrency in a low-code workflow engine has sharp edges nobody warns you about:

Shared variables aren't concurrency-safe. My branches each appended their result to one shared string variable. Under parallel execution, writes stepped on each other and results came back garbled. The fix was to stop using a mutable shared variable inside parallel branches entirely and give each branch its own scoped output object, then aggregate once at the end with a single atomic write.
A variable cannot reference itself in its own assignment. I had a fallback that set a variable to a value derived from that same variable — and the platform rejected it outright with a "self-reference is not supported" error. The fix was to wrap the assignment in a condition: only overwrite the variable when the upstream step actually succeeded; otherwise leave it holding whatever it already had.
Every action name must be globally unique, even across nested scopes. Duplicate an action into a second path and forget to rename it, and the whole definition fails to save.
Branch containers have hard caps. The switch construct I was leaning on maxes out at twenty-five cases. The moment my tool set grew past that, I had to nest switches inside switches — which is exactly as fun to hand-maintain as it sounds. (More on how I stopped hand-maintaining it below.)

Lesson: parallelism buys you latency but charges you correctness. Every shortcut that was fine in a sequential flow — shared state, casual naming, one big switch — becomes a bug the instant things run at the same time.

Learning #3: Routing is a first-class problem, not an afterthought

Once the assistant could answer fast, it started answering wrong — not factually wrong, but routed to the wrong place. A question like "who is the admin for the [city] office?" got sent to a live-data lookup instead of the knowledge base. "How do I apply for leave?" got answered from policy documents instead of the how-to system manual, because both mention the word "leave."

I'd been routing on keywords. Keywords can't tell the difference between asking about a topic and asking how to do something in that topic.

Two changes fixed this:

First, intent-first routing. Before matching any topic, classify what the user actually wants — a how-to, a policy lookup, a data read, an action. Classify intent, then pick the source. Ambiguous words like "leave" stop being landmines once you've already decided the user wants instructions rather than policy.

Second, a hybrid answer pattern instead of a single winner-take-all route. The knowledge base runs as a default for every question. A specialist path runs only when the router judges it relevant. Then a final model scores both outputs and either picks the stronger one or merges them when both clear a relevance bar. I also added a dedicated query-rewriting step before every search so the query hitting the index was optimized rather than raw.

Lesson: don't make routing an all-or-nothing bet on one classifier. Let a cheap default always run, let specialists run when warranted, and let a synthesizer decide at the end. It's more forgiving of the router being occasionally wrong — which it will be.

Learning #4: Identity is the hard part nobody puts on the roadmap

This one cost me more time than any clever routing logic. The assistant doesn't just need to know what you asked — it needs to know who you are across every system, and every system identifies you differently.

The core HR system keyed people one way. The skills and project systems used a different internal integer ID. The engagement platform used yet another. Passing the wrong identifier to the wrong API didn't throw a clean error — it silently returned someone else's data, or nothing, or results from years ago.

I learned to treat identity as its own resolution step. Resolve the user once, up front, using a universal lookup key (email worked as the fallback that every system understood), map that to each system's internal ID, and then inject the correct ID into every downstream call. I also established one firm rule: there is exactly one canonical cross-system employee identifier, and local, system-specific IDs never leave their own system — no matter how tempting the parameter name makes it look.

Lesson: in any multi-system integration, identity mapping is not plumbing you can bolt on later. It's foundational. Get it wrong and every feature built on top inherits a silent data-correctness bug.

And a related one that kept biting me: field-name mismatches fail silently. An API expects emp_id, you send employee_id, and instead of an error you get an empty result you'll misread as "no data." I started defending against this with fallback chains over the likely field names — but the real fix was always to stop guessing and verify the actual field names from run history.

Learning #5: When the definition has 54 tools, stop editing it by hand

By this point the assistant had grown from a handful of branches into a genuine orchestrator exposing dozens of tools — read-only lookups and read-write actions merged into a single definition with more than fifty tools and nested switches several layers deep.

You cannot hand-edit a file like that reliably. One duplicated action name, one branch that points to a step that no longer exists, one switch that quietly grew past its case limit, and the whole thing refuses to deploy — usually with an error that points nowhere near the actual cause.

So I stopped writing the definition by hand and started generating it with a script that validated as it built. Every build ran the same checks before producing output: no duplicate action names anywhere in the tree, no branch referencing a non-existent predecessor, no switch over its case cap, no variable referencing itself. Structural errors got caught at build time, at my desk, instead of at deploy time in front of everyone.

Lesson: past a certain complexity, your configuration is code, and it deserves the same treatment — generated, validated, and tested before it ships. Catching a structural error in a build script is cheap. Catching it in production is not.

The rearchitecture that finally clicked

Every fix above made the system better, but they were all improvements to the same fundamental shape: a big hand-orchestrated workflow with an intermediate agent standing between the user and the tools. That intermediate layer was expensive — it consumed tokens on every single turn, it could only really handle one question at a time, and it leaned on data that was up to a day stale because it read from a batch-loaded store.

The unlock was adopting the Model Context Protocol (MCP) and letting the model talk to the tools directly through a standard interface, removing the intermediate orchestrator entirely. Three things fell out of that at once:

Multi-question support. A user could now ask "what's my leave policy and my current balance?" in one message and get both, because the model could invoke multiple tools in a single turn instead of being funneled through a single-intent pipeline.
Lower cost. Deleting the intermediate agent deleted its per-turn token overhead.
Real-time data. Direct tool calls hit live sources instead of a day-old snapshot.

That was the moment the thing stopped feeling like a chatbot bolted onto some search indexes and started feeling like an assistant that could actually reason about which tools to use and combine their results.

Where it landed

The assistant went from a fifteen-second single-answer bot to something most of the workforce uses, resolving a large share of routine queries on its own, in a fraction of the original response time, across every country the organization operates in. It reads live data, answers policy questions grounded in the right regional content, and is moving steadily toward safely taking actions on the user's behalf.

The number I'm proudest of isn't a latency figure. It's that the architecture is now boring in the right way — a clean tool interface, generated and validated configuration, identity resolved once and correctly, routing that fails gracefully. Boring, at this scale, is the achievement.

The lessons that generalize

If you're building something similar, here's what I wish I'd known at the start, in the order the project taught them to me:

A slow demo is an architecture problem, not a tuning problem. Question the shape before you optimize the parts.
Convenience abstractions over retrieval quietly degrade retrieval. Know exactly what query is hitting your index and what reaches your model.
Parallelism trades latency for correctness. Shared state, loose naming, and one giant branch are all fine until things run concurrently — then they're bugs.
Route on intent, not keywords, and don't bet everything on one classifier. A cheap always-on default plus a scored synthesizer is far more forgiving.
Identity mapping across systems is foundational, not plumbing. One canonical ID; local IDs never travel; resolve the user once, up front.
Silent failures are the expensive ones. Empty results from a field-name mismatch will mislead you longer than any exception.
Past a certain size, your config is code. Generate it, validate it, and catch structural errors before deploy.
The right abstraction removes a layer instead of adding one. The biggest win of the whole project came from deleting the orchestrator in the middle, not making it smarter.

The learning curve wasn't a smooth ramp. It was a series of walls, and each wall taught me exactly one thing I couldn't have learned any other way. That's the part worth writing down.

One pane over 258 resources — and every one of them talking back.

Vignesh Athiappan — Wed, 15 Jul 2026 07:00:54 +0000

Fifteen enterprise systems, fifty-plus integration workflows, a CI/CD estate, and an Azure bill that grows on its own. Here is what changed when all of it started reporting to a single operations layer — including the line most teams never watch: the invoice.

Vignesh Athiappan
Engineering & Enterprise Systems
~14 min read

A modern enterprise platform doesn't fail loudly. It fails quietly — a sync lane that stops moving records at 2 a.m., a pipeline that has been red for three days, a single AI resource quietly eating a quarter of the monthly bill. None of these page you. All of them cost you. The gap between "something is wrong" and "we know what's wrong" is where operational credibility lives or dies, and for most teams that gap is measured in days because the truth is scattered across a dozen portals.

I run an integration estate that connects fifteen-plus enterprise systems through more than fifty Azure Logic App workflows — an applicant tracking system, a CRM, a service desk, a document store, a financials platform, and the internal products stitched on top of them. On paper it's healthy. In practice, answering a simple question like "is everything moving right now?" used to mean opening the Azure portal, then Azure DevOps, then three run-history blades, then Cost Management, then a spreadsheet. By the time you'd assembled the picture, it was stale.

So I built a console that assembles it for you. This is a write-up of what that unified operations layer watches, why a single pane of glass is a genuine capability rather than a dashboard vanity project, and — the part teams consistently under-invest in — how putting the Azure bill on the same screen as the health signals turns cost from a monthly surprise into an operational instrument.

15+
enterprise systems under one integration fabric
50+
Logic App workflows moving records across lanes
258
Azure resources billed on a single subscription
69%
of spend concentrated in the top 10 resources
The problem

The estate outgrew any single portal
Scale doesn't announce itself as complexity. It announces itself as surface area. Every system you integrate adds a place where things can silently stall, a new failure mode, a new credential that quietly expires, and a new line on the invoice. Individually each is manageable. Collectively they exceed what any one native tool was built to show you, because the native tools are organized around resources — this Logic App, that pipeline, this SQL database — and incidents are organized around flows: a candidate that didn't reach the ATS, a project that didn't sync to financials, a report that came back empty.

The console reorganizes the estate the way incidents actually arrive. Instead of a resource inventory, it's a set of questions, each with its own view:

Integration · Jobs
Is everything moving, and did last night's batch finish?
Live run status across the workflow fabric, with the scheduled jobs that feed it.
Platform ↔ ERP
Are projects and financials in agreement?
The sync lane between the core platform and the financials system, record-for-record.
CRM ↔ ERP · CRM ↔ Platform
Is the commercial side reconciled everywhere it needs to be?
Two more lanes, watched with the same lens so a break in one doesn't hide behind another.
Credentials
What is about to expire and take a lane down with it?
The failure mode that never shows up in a health check until it's already an outage.
Autofill · Automation
Are the automations still doing their job, or silently no-op?
The quiet helpers that only get noticed when they stop.
Observability · Data Lake
How is each product actually performing under load?
Each product surface, the REST APIs, and the Bronze→Silver→Gold lake, each with its own performance view.
The point of the grid isn't that it's comprehensive. It's that each tile answers a question an engineer or an executive would otherwise have to reconstruct by hand. A view earns its place on the pane only if it collapses a five-portal investigation into a glance.

Native tools are organized around resources. Incidents are organized around flows. The distance between those two shapes is where time-to-detect goes to hide.
Delivery is a health signal too

Your pipelines are telling you something — read them
Runtime health is only half the story. The other half is whether the team can safely change the system, and that lives in the delivery pipeline. The console pulls a read-only view straight from the Azure DevOps REST API — a personal access token with read scopes only, nothing stored — and turns raw build history into an operational read on the delivery org itself.

Five numbers sit at the top: pipeline runs in the window, success rate, failed runs, open pull requests, and PRs completed. On their own those are trivia. What makes them operational is what sits underneath:

Failure reasons, not just failure counts. For every failed run, the view reads the build timeline, finds the task that actually broke, and surfaces the first line of its error message — so "17 failures" becomes "the same missing environment variable, seventeen times." That is the difference between an alert and a diagnosis.
A pipeline heartbeat. Each pipeline shows its last fifteen runs as a strip of green-and-red ticks. A wall of green is trust. An alternating stripe is a flaky pipeline you'd never catch from an aggregate pass rate. Your eye finds the pattern before any threshold would fire.
People and repositories. Who triggered runs, who opened, reviewed, and completed PRs, and where the review load actually falls — the org's real bus-factor, made visible instead of assumed.
Detection is the whole game. A failed deploy that a human notices on Thursday when a report looks wrong is an incident with a three-day fuse. The same failure, shown as a red tick beside its own error message the moment it happens, is a five-minute fix. The console doesn't make the pipelines more reliable — it makes their unreliability impossible to ignore, which is what actually shortens mean-time-to-resolve.

Follow the money

The one line nobody watches: the Azure bill
Here's the view that changes how leadership sees the whole thing. Cloud cost is usually treated as an accounting artifact — a monthly PDF someone forwards with a raised eyebrow. Put it on the operations pane, in the same visual language as everything else, and it becomes a signal you can act on in real time. For the first two weeks of the billing period, the picture looks like this:

A pie chart is not an insight. What the console does is read three signals out of that spend, and each one is a decision waiting to be made:

1 · Concentration — where a small change moves the whole bill
Ten resources out of 258 account for 69% of the spend, and a single resource group — the one hosting the AI/inference stack — is 27% on its own. That's not a problem; it's leverage. It means cost optimization is a targeted exercise, not a 258-item audit. You tune the top ten and you've touched two-thirds of the invoice. The concentration view exists to point the flashlight exactly there.

2 · Zero reserved capacity — money left on the table
Every dollar here is pay-as-you-go. Reserved and savings-plan commitments sit at 0%. For steady-state workloads — the SQL estate at a quarter of spend, the always-on compute — pay-as-you-go is the most expensive way to buy predictable capacity. Surfacing "0% pre-paid" next to "100% on-demand" reframes the bill as an open savings lever, typically a double-digit percentage on the reservable share, rather than a fixed cost of doing business.

3 · Footprint sprawl — 153 workflows on one line
Logic Apps are 19% of spend across 153 resources — by far the largest resource count on the estate. That's the integration fabric doing its job, but it's also the place where an orphaned trigger or a runaway polling interval hides in plain sight. Watching the count and the cost together is how you catch the workflow that's been retrying every thirty seconds since a deploy three weeks ago.

The AI and Search slice deserves its own note: at a combined 34% it's already the largest category, and it's the one most likely to scale non-linearly as agent and vector workloads grow. Watching it weekly rather than monthly is the difference between a managed ramp and a bill shock.

Cost isn't a report you receive. On the operations pane it's a gauge you read — and the three signals it shows are all decisions, not just numbers.
Why one pane changes the work

A broken sync, a red pipeline, and a cost spike are the same incident
The real payoff of a single pane isn't convenience. It's correlation. Consider a genuinely common Tuesday: a deploy goes out, a Logic App starts failing, its retries spike the run count, the run count spikes the Logic Apps cost line, and a downstream sync lane quietly stops reconciling records. In a fragmented world those are four tickets in four tools, investigated by three people, connected by nobody until Friday.

On one pane they're one story, visible in one scroll: the red tick on the pipeline heartbeat, the failure reason from the build log, the spend line bending upward, the lane going stale. You don't infer the connection — you see it. That is the entire value proposition, and it shows up in numbers that leadership cares about:

Time-to-detect collapses. Silent failures — the expiring credential, the flaky pipeline, the runaway workflow — surface at a glance instead of at the next complaint.
Time-to-resolve shrinks. Failure reasons arrive pre-diagnosed from the logs, so the first ten minutes of every incident aren't spent finding the tool that has the answer.
Bus-factor becomes visible. The people and review views turn "only one person understands this" from an assumption into a metric you can plan around.
Cost becomes a lever, not a surprise. Concentration, reservation gaps, and sprawl are shown as actions, and the monthly finance conversation starts from a shared, current picture instead of a stale PDF.

Field notes on platform & enterprise-systems engineering. · Figures are month-to-date on a single Azure subscription and reflect a 14-day window. · Delivery metrics via Azure DevOps REST 7.1, read-only.

From 15+ Manual Deployments to 2 YAML Templates: Scaling Azure DevOps Pipelines for IIS-Hosted Apps

Vignesh Athiappan — Tue, 14 Jul 2026 14:35:47 +0000

The Problem

I manage 8 web applications hosted on a single Windows VM running IIS. Each application has 2–3 deployable components:

An Angular frontend
A .NET backend API
Often a separate integration API

That's roughly 20 deployable units on one server. For years, every deployment was manual — build locally or on the server, copy files over, restart IIS, pray.

When I finally decided to automate with Azure DevOps pipelines, I hit the obvious trap immediately: the naive approach means one pipeline per component. That's 20 pipelines today — and every time a new application is built (which happens regularly), I'd be creating 2–3 more. Each pipeline would duplicate the same build/copy/restart logic. Fixing a deployment bug would mean fixing it 20 times.

On top of the pipeline-count problem, the manual process had three landmines baked into it:

iisreset kills everything. The old scripts restarted the entire IIS server to deploy one app — taking down all 8 applications for every single deployment.
Config file overwrites. Each app has server-side config (appsettings.json for .NET, assets/config.json for Angular) that must never be replaced by whatever is in source control. One careless copy and production points at the wrong database.
No consistency. Every app had slightly different variable names, folder conventions, and deploy steps. Nothing was reusable; everything was tribal knowledge.

So the real requirement wasn't "create pipelines." It was:

Build a deployment system where adding a new application requires zero new pipeline logic.

The Solution: Template-Based Pipeline Architecture

Azure DevOps supports YAML templates — reusable pipeline definitions that live in one repository and are referenced by other repositories. That's the entire foundation of this design.

The architecture

flowchart TD
    subgraph TPL["📦 pipeline-templates repo (logic lives here, ONCE)"]
        T1["build-deploy-dotnet.yml"]
        T2["build-deploy-angular.yml"]
    end

    subgraph APPS["Application repos (thin callers, ~20 lines each)"]
        A1["Portal<br/>azure-pipelines.yml"]
        A2["Orders<br/>azure-pipelines.yml"]
        A3["Ops<br/>azure-pipelines.yml"]
        A4["Billing / Reports / Notify / ...<br/>azure-pipelines.yml"]
    end

    subgraph LIB["🔑 Variable groups (one per app)"]
        V1["PORTAL-PROD"]
        V2["ORDERS-PROD"]
        V3["OPS-PROD"]
        V4["..."]
    end

    A1 -->|references| TPL
    A2 -->|references| TPL
    A3 -->|references| TPL
    A4 -->|references| TPL
    A1 -.->|reads paths from| V1
    A2 -.->|reads paths from| V2
    A3 -.->|reads paths from| V3
    A4 -.->|reads paths from| V4

    TPL ==>|deploys via self-hosted agent| VM["🖥️ Windows VM — IIS<br/>(all 8 apps)"]

Three building blocks:

1. One shared pipeline-templates repository. It contains exactly two templates:

Template	Handles
`build-deploy-dotnet.yml`	Restore → build → publish → protect configs → stop app pool → robocopy deploy → start app pool
`build-deploy-angular.yml`	Node setup → `npm ci` → `ng build` → protect configs → robocopy deploy → recycle app pool

2. A thin azure-pipelines.yml in each app repo. ~20 lines. It declares the templates repo as a resource, points at the app's variable group, and passes a handful of parameters (app pool name, artifact name). It contains zero deployment logic.

trigger: none   # manual, deliberate deployments only

resources:
  repositories:
    - repository: templates
      type: git
      name: pipeline-templates

parameters:
  - name: deployBackend
    type: boolean
    default: false
  - name: deployFrontend
    type: boolean
    default: false

variables:
  - group: PORTAL-PROD

stages:
  - ${{ if eq(parameters.deployBackend, true) }}:
    - template: templates/build-deploy-dotnet.yml@templates
      parameters:
        artifactName: Portal_API_Drop
        appPoolName: PortalAPIService
  - ${{ if eq(parameters.deployFrontend, true) }}:
    - template: templates/build-deploy-angular.yml@templates
      parameters:
        artifactName: Portal_Web_Drop
        appPoolName: PortalFrontEnd

The checkbox parameters (deployBackend, deployFrontend, deployIntegration) mean one pipeline per app covers all its components — the person running it just ticks what they want deployed.

3. Standardized variable groups. Every app gets its own variable group (PORTAL-PROD, ORDERS-PROD, …) — but the variable names inside are identical across all apps. Only the values differ:

BACKEND_SOURCE_PATH        (where the .sln lives in the repo)
BACKEND_DEPLOYMENT_PATH    (C:\inetpub\wwwroot\<App>\...)
FRONTEND_SOURCE_PATH
FRONTEND_DIST_PATH
FRONTEND_TARGET_PATH
NODE_VERSION
AGENT_POOL_NAME / AGENT_NAME
INTEGRATION_* (only if the app has one)

This is the trick that makes templates truly generic: the app's identity lives in the group name, never in the variable names. The template just reads $(BACKEND_DEPLOYMENT_PATH) and works for any app.

The Deployment Flow

Here's what one deployment actually does, end to end:

flowchart TD
    A["▶️ Run pipeline manually<br/>(tick Backend / Frontend / Integration)"] --> B["Checkout app repo +<br/>templates repo"]
    B --> C["Build<br/>(dotnet publish / ng build)"]
    C --> D["🛡️ Layer 1: delete protected configs<br/>from BUILD OUTPUT<br/>(appsettings.json, assets/config.json)"]
    D --> E["Publish pipeline artifact"]
    E --> F["⏸️ Stop ONLY this app's<br/>IIS app pool"]
    F --> G["🛡️ Layer 2: robocopy /XF<br/>excludes protected files<br/>during copy"]
    G --> H{"robocopy<br/>exit code < 8?"}
    H -->|yes| I["▶️ Start app pool"]
    H -->|no| J["❌ Fail the pipeline<br/>(pool restarted, investigate)"]
    I --> K["✅ App live —<br/>other 7 apps never blinked"]

Two design decisions here are worth calling out because they came directly from the failure modes of the manual era:

App-pool-level recycle, not `iisreset`

The template stops and starts only the app pool of the component being deployed. The other applications on the VM stay up throughout. Deploying Portal no longer causes an outage for Orders, Billing, Reports, and everyone else. This alone changed deployments from "schedule it after hours" to "just deploy it."

Config protection at two independent layers

Server-side config files are protected with defense in depth:

Layer 1 — build output cleanup: protected files are deleted from the build output before anything is copied. If it's not in the artifact, it can't overwrite anything.
Layer 2 — copy-time exclusion: the deploy uses robocopy with the /XF flag listing protected filenames, so even if layer 1 misses something, the copy itself refuses to touch appsettings.json or config.json at the destination.

Either layer failing alone still leaves production config intact. Both would have to fail simultaneously — and the post-deploy verification step is simply checking the config file's Date Modified is old.

Why robocopy over xcopy? Reliable retry behavior (/R:2 /W:5), sane exit codes you can gate on, and the /XF exclusion list lives visibly in the pipeline YAML instead of in a mystery exclude-file sitting somewhere on the server.

The Rollout: Pilot → Validate → Replicate

Rolling out an unproven template to 8 production apps simultaneously is how you take prod down 8 times in one afternoon. Instead:

flowchart LR
    P["🧪 Pilot: Portal<br/>(prove the templates)"] --> V["Validate<br/>configs untouched?<br/>only its pool recycled?<br/>app responds?"]
    V --> R["🏭 Assembly line:<br/>Ops → Reports →<br/>Notify → Orders → Billing →<br/>Gateway"]
    R --> S["All pipelines registered<br/>with SAVE only —<br/>team runs them deliberately"]

Portal was the pilot — including its separate integration-API repo (which gets its own thin caller file, same templates).
Any template bug found during the pilot gets fixed once, and every future app inherits the fix before it even onboards.
Every subsequent app was pure assembly line: clone the variable group, update values, drop in the thin YAML, register the pipeline with Save (not Run). Nothing touches production until someone deliberately runs it, one component at a time, simplest apps first.

Onboarding time per app after the pilot: ~10 minutes.

Real-World Gotchas (the stuff no tutorial mentions)

Angular 20's application builder changes the dist path. Modern Angular builds output to dist/<project>/browser — that extra /browser folder broke the copy path until FRONTEND_DIST_PATH accounted for it. Older Angular apps output straight to dist/<project> or plain dist. This is exactly why dist path is a per-app variable, not template logic.

Variable group names must match the YAML exactly. The #1 source of "variable group could not be found" errors was a mismatch between - group: PORTAL-PROD in YAML and the actual group name in the Library (including accidental suffixes). Character-for-character.

Variable group pipeline permissions. New pipelines fail with authorization errors until the group grants access. During rollout, setting the group's pipeline permissions to Open access (fine when the group holds no secrets — ours hold only paths) removes the friction; you can tighten to per-pipeline authorization later.

Nested and non-standard repo layouts. One app had its Angular project buried at Frontend/Client/<AppName>; another had a frontend-only repo; one had suspicious folder names on the server (...Working) left over from manual-deploy days. Templates survive all of this because paths are data, not logic — you verify what IIS actually points at, put the truth in the variable group, done.

robocopy /XF matches filenames anywhere in the tree. Excluding config.json protects assets/config.json but would also skip any other config.json in the build. For config-protection purposes that's exactly the behavior you want — just know it.

How Generic Is This, Really?

Very. Nothing in this design is specific to my apps. The abstraction boundary is clean:

Lives in templates (shared, fixed)	Lives in variable group / parameters (per app)
Build steps for a .NET API	Source path, .NET version
Build steps for an Angular app	Node version, dist path
Config protection (both layers)	Which files are protected (parameter)
App pool stop/copy/start sequence	App pool name
robocopy flags, error gating	Source and target paths

So this pattern transfers directly if you have:

Any number of apps on shared IIS servers — the pool-level recycle is what makes multi-tenant VMs safe to deploy to.
Different stacks — a build-deploy-node.yml or build-deploy-python.yml template slots in beside the existing two; app repos just reference a different template.
Multiple environments — the same templates work for staging/UAT: create PORTAL-UAT variable groups pointing at different paths/servers and pass a different agent. The template never changes.
Multiple servers — the agent name/pool is a variable too. New VM = new agent registration + new values in the groups.

The invariant that makes it all work: templates contain behavior, variable groups contain facts. As long as you never let an app-specific fact leak into a template, the system scales linearly with zero added complexity.

What's Next

The system is live but young. The roadmap:

Finish onboarding the last apps — a couple were deferred (unclear IIS pool mappings needed verification first). They're a 10-minute job each once verified.
Run-and-validate pass — pipelines were registered Save-only; each needs its first deliberate run with the post-deploy checklist (config timestamps unchanged, neighbors still up, endpoint responds).
CI triggers — everything is manual today (trigger: none) by design. Once trust is earned, branch-based triggers for lower environments are a one-line change in the thin YAML.
Approval gates — ADO Environments support manual approval checks before prod deploys, with no extra pipeline logic.
Smarter rollback — a backup/restore step existed in early versions and was removed for speed; a versioned-artifact rollback (redeploy the previous pipeline artifact) is the cleaner long-term answer.
Secrets to Key Vault — variable groups currently hold only paths; when secrets enter the picture, they'll link to Azure Key Vault rather than living in the Library.
Deployment visibility — an executive dashboard over the ADO REST APIs (build runs, PR activity, failure reasons) is already in progress, closing the loop from "we deploy reliably" to "leadership can see it."

Takeaways

Don't count pipelines — count places logic lives. 8 apps, 20 deployables, and the deployment logic exists in exactly 2 files.
Standardize variable names, vary the values. That's the entire trick behind generic templates.
Protect production config with two independent layers. One layer is a hope; two is a design.
Recycle app pools, never the server. Shared IIS VMs and iisreset don't mix.
Pilot on one app before replicating. Fix template bugs once, inherit fixes everywhere.
Register with Save, run deliberately. Automation should reduce risk, not add surprise.

Onboarding the next application my team builds will take 10 minutes and zero new pipeline logic. That was the whole point.

Built with Azure DevOps YAML templates, variable groups, a self-hosted agent, robocopy, and a healthy fear of iisreset.

DEV Community: Vignesh Athiappan

Turning a System of Record into an AI Agent: Building MCP Tools on Azure

The goal

The architecture

Step 1 — Design the tools

Step 2 — Build the tool executor (Logic App)

Step 3 — Describe the tools as OpenAPI

Step 4 — Put it behind API Management

Step 5 — Expose the API as an MCP server

Step 6 — Connect to Agent Foundry

Step 7 — The system prompt is part of the security perimeter

Step 8 — Making it actually work

Lessons learned

From a 15-Second Chatbot to a Real Agentic Assistant

What a year of building an enterprise AI copilot actually taught me

The problem I set out to solve

Where I started — and the first wall

Learning #1: A wrapped search is not a good search

Learning #2: Sequential is the enemy — and parallelism has its own traps

Learning #3: Routing is a first-class problem, not an afterthought

Learning #4: Identity is the hard part nobody puts on the roadmap

Learning #5: When the definition has 54 tools, stop editing it by hand

The rearchitecture that finally clicked

Where it landed

The lessons that generalize

One pane over 258 resources — and every one of them talking back.

From 15+ Manual Deployments to 2 YAML Templates: Scaling Azure DevOps Pipelines for IIS-Hosted Apps

The Problem

The Solution: Template-Based Pipeline Architecture

The architecture

The Deployment Flow

App-pool-level recycle, not iisreset

Config protection at two independent layers

The Rollout: Pilot → Validate → Replicate

Real-World Gotchas (the stuff no tutorial mentions)

How Generic Is This, Really?

What's Next

Takeaways

App-pool-level recycle, not `iisreset`