DEV Community: Vignesh Athiappan

PDF Rendering for HR at Scale: Why Your Simple Test Fails (and How to Fix It)

Vignesh Athiappan — Mon, 21 Sep 2026 03:42:35 +0000

PDF Rendering for HR at Scale: Why Your Simple Test Fails (and How to Fix It)

We spent two weeks evaluating PDF-generation vendors for our talent acquisition platform. Here's what we learned, why a naive test gives you garbage results, and the template anyone can use to pick the right engine.

The Problem

Our platform generates offers, appointment letters, and onboarding docs for candidates across India, Mexico, and the US. These PDFs carry salary and personal data. We needed:

Fidelity. Multi-page documents with headers, footers, page numbers, table headers that repeat across pages, and no orphaned headings.
Security. Data stays encrypted; no vendor logging of content.
Cost. We're bootstrapped; $10k/yr on a renderer we could build ourselves is hard to justify.

We looked at Gotenberg (self-hosted), PDFShift, and DocRaptor.

How We Broke Our Own Test

Day one, we grabbed a screenshot of our profile screen from our HR system (Angular + PrimeNG), pasted the raw HTML into DocRaptor's try-it-out page, and got back a PDF full of grey boxes where icons should be.

Conclusion: "PDF generation looks crap; all engines probably suck."

We were completely wrong. We'd tested nothing.

The screenshot HTML had:

Zero inline CSS (all styles lived in component stylesheets on the page)
Icon fonts and SVGs that never loaded (no external URLs allowed)
Angular metadata (_ngcontent-wij-c34) that meant nothing in a PDF
No @page rules, no print CSS, no structure

Rendering a screen UI as a PDF is not the same as rendering a print template. Every engine — Prince, Chromium, even LibreOffice — gives the same result from garbage input: garbage output.

The lesson: Your test isn't testing the engine. It's testing your understanding of what a PDF template is.

The Right Test

A real PDF is self-contained HTML with embedded CSS. No external stylesheets. No dynamic JavaScript. No assumptions about available fonts.

We built a single .html file that:

Contains everything. Inline <style> block with @page rules, print CSS, margins, headers/footers, page numbers.
Uses print-specific CSS. @page { size: A4; margin: 28mm 20mm; }, @top-center { content: element(pageHeader); }, counter(page), page-break-inside: avoid.
Tests the hard cases:
- Multi-page table with <thead> that repeats
- Rows that must never split across pages
- Running headers and footers
- Glyph rendering: ₹ (rupee), € (euro), ñ (Spanish n), and em-dashes
- Long numbered lists that span pages
- Signature blocks that shouldn't split
Uses dummy data, not real PII. Template goes into version control; you can iterate on it without legal questions.

Here's the structure:

<!DOCTYPE html>
<html>
<head>
  <style>
    @page {
      size: A4;
      margin: 28mm 20mm 24mm 20mm;
      @top-center { content: element(pageHeader); }
      @bottom-center { content: element(pageFooter); }
    }
    .page-header { position: running(pageHeader); }
    .page-footer { position: running(pageFooter); }
    table { border-collapse: collapse; }
    table thead { display: table-header-group; }  /* repeats on every page */
    tr { page-break-inside: avoid; }  /* no row splits */
    h2 { page-break-after: avoid; }  /* no orphaned headings */
  </style>
</head>
<body>
  <div class="page-header">Header text</div>
  <div class="page-footer">Page <span class="page-number"></span></div>
  <!-- Your content here -->
</body>
</html>

The @page and page-break-* rules are where the magic happens. Most developers skip them because screen CSS doesn't need them.

What We Tested

We sent the same template to three engines:

Engine	What it is	Entry cost	Ops burden
DocRaptor	Cloud API, Prince engine	$15–75/mo	None
PDFShift	Cloud API, Chromium	$9/mo	None
Gotenberg	Self-hosted Docker, Chromium + LibreOffice	$0 license	Small

The scorecard:

Test case	DocRaptor (Prince)	PDFShift (Chromium)	Gotenberg (Chromium)
Headers / footers	✓ Native CSS	✗ API workaround needed	✗ API workaround needed
"Page X of Y"	✓ `counter(pages)` works	✗ Harder	✗ Harder
Running table headers	✓ Perfect	✓ Works	✓ Works
Row split prevention	✓ Reliable	✓ Works	✓ Works
Glyph rendering	✓ All render	✓ All render	✓ All render
Multi-line orphan control	✓ Strong	~ Moderate	~ Moderate
File size	250 KB	280 KB	270 KB
Concurrency (50 parallel)	N/A	N/A	p95 = 2.3s

Result: Prince wins on CSS elegance (headers/footers as CSS, not API calls). Chromium ties on everything else.

The Real Decision

If you have $75/mo budget and zero ops overhead matters, Prince (DocRaptor) wins slightly.

If you have $0 tolerance for vendors, Gotenberg (free, self-hosted) is completely adequate. The header/footer API calls are a one-time 20-line setup.

If you're on a tight budget and already have a Kubernetes cluster, Gotenberg.

If you want zero infrastructure and the budget is fine, DocRaptor.

We picked DocRaptor because:

Cost is fine at scale. 200–300 offers/month = $29/mo tier. Not a blocker.
Zero ops. We don't want another container to upgrade and patch.
Security is better than DIY. Their SOC 2 audit, encryption in transit and at rest, 24-hour breach notification — better than we'd build ourselves.

But we kept Gotenberg as a fallback. If DocRaptor ever blocked us or raised prices, swapping takes one day because we wrote the templates in standard CSS, no Prince extensions.

The Security Bits (Don't Forget These)

PDF generation touches salary data. Before you pick a vendor:

Where does it run? (DocRaptor: AWS us-east-1. That's fine for India if you have a contract and disclose it in your privacy notice.)
Is content logged? (DocRaptor: no. Metadata only.)
How long is it stored? (DocRaptor: configurable, encrypted in S3, auto-deleted.)
What's their security posture? (DocRaptor: SOC 2 Type II, ISO 27001 path.)
Sub-processors? (DocRaptor: AWS, Stripe, a few analytics tools. All listed.)

Ask these in an email before you commit. Don't assume.

What We Did Wrong (So You Don't)

Tested the wrong input. Screen HTML ≠ print template. We wasted a day learning that.
Didn't specify print CSS. @page and page-break-* aren't optional. They're the entire difference between "works" and "works perfectly."
Assumed all engines were equivalent. They're not. Prince is better at paged media; Chromium is fine and cheaper.
Didn't think about fallback. We picked the vendor, then realized we had no plan B. Now we keep the Gotenberg container ready.
Didn't test glyph rendering across locales. We added that halfway through. It matters if you support multiple countries.

The Template

Here's the full test template we built. Use it:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>PDF Fidelity Test</title>
  <style>
    @page {
      size: A4;
      margin: 28mm 20mm 24mm 20mm;
      @top-center { content: element(pageHeader); }
      @bottom-center { content: element(pageFooter); }
    }
    .page-header { position: running(pageHeader); font-size: 9pt; color: #555; border-bottom: 1px solid #999; padding-bottom: 3mm; }
    .page-footer { position: running(pageFooter); font-size: 8.5pt; color: #555; border-top: 1px solid #999; padding-top: 2mm; text-align: center; }

    body { font-family: "Noto Sans", "DejaVu Sans", Arial, sans-serif; font-size: 10.5pt; line-height: 1.45; color: #111; }
    h1 { font-size: 16pt; margin: 0 0 6mm; }
    h2 { font-size: 12.5pt; margin: 8mm 0 3mm; page-break-after: avoid; }
    p { margin: 0 0 3.5mm; }

    table { width: 100%; border-collapse: collapse; margin: 3mm 0 5mm; }
    table th, table td { border: 1px solid #bbb; padding: 2mm; vertical-align: top; }
    table th { background: #eee; }
    table thead { display: table-header-group; }
    table tr { page-break-inside: avoid; }

    .glyph-test { font-size: 12pt; }
    .annexure { page-break-before: always; }
  </style>
</head>
<body>

<div class="page-header">TEST TEMPLATE — Document</div>
<div class="page-footer">Page <span class="pageNumber"></span> of <span class="totalPages"></span></div>

<h1>Document Title</h1>
<p>This is a test template for PDF rendering evaluation.</p>

<h2>Section 1: Basic content</h2>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</p>

<h2>Section 2: Glyph rendering</h2>
<table class="glyph-test">
  <tr>
    <td>Rupee (₹)</td>
    <td>₹ 1,25,000</td>
  </tr>
  <tr>
    <td>Spanish (ñ, á, ¿)</td>
    <td>Señor, Ramírez, ¿Cuándo?</td>
  </tr>
  <tr>
    <td>Euro (€)</td>
    <td>€ 1.234,56</td>
  </tr>
  <tr>
    <td>Typography</td>
    <td>"Curly quotes" — em dash … ellipsis</td>
  </tr>
</table>

<h2>Section 3: Long table (tests header repeat and row integrity)</h2>
<table>
  <thead>
    <tr><th>#</th><th>Item</th><th>Description</th></tr>
  </thead>
  <tbody>
    <tr><td>1</td><td>Row one</td><td>This row should not split.</td></tr>
    <tr><td>2</td><td>Row two</td><td>This row should not split.</td></tr>
    <tr><td>3</td><td>Row three</td><td>This row should not split.</td></tr>
    <tr><td>4</td><td>Row four</td><td>This row should not split.</td></tr>
    <tr><td>5</td><td>Row five</td><td>This row should not split.</td></tr>
    <tr><td>6</td><td>Row six</td><td>This row should not split.</td></tr>
    <tr><td>7</td><td>Row seven</td><td>This row should not split.</td></tr>
    <tr><td>8</td><td>Row eight</td><td>This row should not split.</td></tr>
    <tr><td>9</td><td>Row nine</td><td>This row should not split.</td></tr>
    <tr><td>10</td><td>Row ten</td><td>This row should not split.</td></tr>
  </tbody>
</table>

<div class="annexure">
  <h2>Annexure A</h2>
  <p>This content starts on a new page.</p>
  <p>Tables on this page should also have repeating headers.</p>
  <table>
    <thead>
      <tr><th>#</th><th>Item</th></tr>
    </thead>
    <tbody>
      <tr><td>1</td><td>Page 2 row 1</td></tr>
      <tr><td>2</td><td>Page 2 row 2</td></tr>
    </tbody>
  </table>
</div>

</body>
</html>

Save this, paste it into your vendor's try-it-out page or API, and score against the test cases above.

What We'd Do Differently

Run the test first. Don't assume all engines are the same.
Pick the simplest thing that works. If Chromium is 95% as good as Prince and costs 10x less, that's a win.
Build in fallback from day one. Lock your templates to standard CSS. That's your insurance.
Ask security questions early. Not because vendors hide things, but because your legal team needs to know the answers before you go live.
Document the decision. Six months from now you won't remember why you picked this. Write it down.

The Takeaway

PDF rendering sounds simple until it's not. The engine matters less than the input. A good template — one that declares @page, uses page-break-*, and embeds fonts — works on any modern engine. A bad template fails everywhere.

Test early, test with the right input, keep templates portable, and ask about security before you commit.

That's it. Go render PDFs.

We're using DocRaptor (for now). We haven't ruled out Gotenberg. If you've done this evaluation for your own platform, I'd love to hear what you picked and why.

Exposing a Microsoft Foundry Agent over A2A Through Azure API Management — The Click-by-Click Guide

Vignesh Athiappan — Sun, 06 Sep 2026 04:33:54 +0000

Last verified: September 2026. Foundry's incoming-A2A endpoint is still in public preview; APIM's A2A agent API support is GA.

What you will have at the end

A Foundry agent that any authorised client on your network can call through your API gateway, using the open Agent2Agent (A2A) protocol, like this:

Postman / your app / another agent
        │
        │  POST  https://api.yourcompany.com/agents/helper-agent
        │  Header: Ocp-Apim-Subscription-Key
        ▼
Azure Front Door (optional, if your APIM is private)
        ▼
Azure API Management  ──►  gets its own Entra token  ──►  Microsoft Foundry
   (rate limits, keys,      (managed identity)             (your agent runs,
    logging, policies)                                       answers, done)

You'll finish when you send a JSON-RPC message from Postman and get the agent's answer back. Nothing more. Wiring a second agent to call the first, or turning agents into MCP tools, are follow-on articles.

Why bother with the gateway at all?

Foundry agents already have an endpoint. Two reasons to put APIM in front:

Governance. Subscription keys, per-consumer quotas, content-safety policies, App Insights traces — all in one place you already operate. When Security asks "who can call this agent and how often?", the answer is an APIM product, not a wiki page.
One front door for everything. The same URL pattern works for Foundry agents today and for agents hosted anywhere else (Container Apps, other clouds) tomorrow.

Before you start — read this once

Things you need

Thing	Why
A Foundry project (New Foundry experience toggled on)	Where the agent lives
Owner or Foundry Project Manager on that project	To create agents and assign roles
An APIM instance with System-assigned managed identity turned on	APIM must prove who it is to Foundry
Contributor on the APIM instance	To add APIs and edit policies
Postman (or curl)	To test
~45 minutes	Role assignments take a few minutes to propagate; budget for waiting

Placeholders used in this article — replace every one of them

Placeholder	Meaning	Where to find yours
`contoso-agents-poc`	Foundry account name (the parent resource)	Foundry portal → Manage → Project details → Parent resource
`contoso-agents-poc`	Foundry project name	Same page → Name. Often identical to the account name for the default project.
`helper-agent`	The agent you're exposing	Whatever you named it
`contoso-apim`	Your APIM instance name	Azure portal
`https://api.yourcompany.com`	The public URL clients use to reach APIM	Either `https://contoso-apim.azure-api.net` or your custom domain / Front Door hostname

The one URL that matters

Every Foundry agent has a stable endpoint. Its A2A path is:

https://{account}.services.ai.azure.com/api/projects/{project}/agents/{agent}/endpoint/protocols/a2a

Filled in:

https://contoso-agents-poc.services.ai.azure.com/api/projects/contoso-agents-poc/agents/helper-agent/endpoint/protocols/a2a

Copy that into a notepad. You'll paste it four times.

Two things that will confuse you if nobody warns you

Foundry's A2A endpoint requires Entra authentication for everything, including reading the agent card. Anything that tries to fetch it anonymously fails. APIM's import wizard does exactly that, so it will fail, and that's expected. We handle it.
Foundry serves A2A protocol v0.3 by default unless you ask for v1.0 with a header. We'll set that header in APIM so clients don't have to think about it.

Part A — Set up the agent in Foundry

A1. Create the agent (skip if you already have one)

Open ai.azure.com. Make sure the New Foundry toggle (top right) is on.
Open your project → Build → Agents → + Create agent.
Name it helper-agent. Pick a model (any deployed chat model — gpt-4o-mini is fine).
In Instructions, write something real. A blank agent answers like a generic chatbot, which makes later testing meaningless. Example:

You answer questions about company holiday policy. Be brief. If asked anything else, say it's outside your scope.

Click Save (top right). This creates version 1. The agent's endpoint is live from this moment — there is no separate "publish" step.

A2. Create the agent card and enable A2A

The agent card is a small JSON document that tells other agents what this one can do. Creating it in the portal also turns on the A2A protocol for the agent.

Open helper-agent → Details tab.
Find the A2A / Agent card section and click Create an agent card (it's marked Preview).
Fill it in. Write it for a machine that has to decide whether to call you, not for a human:

| Field | What to put | Example |
|---|---|---|
| Name | Pre-filled | helper-agent |
| Description | 1–2 sentences: what it answers, what it doesn't | Answers questions about company holiday and leave policy. Does not handle payroll or IT. |
| Skill name | Short noun phrase | Holiday policy queries |
| Tags | Comma-separated keywords | holiday, leave, policy, hr |
| Skill description | What a request looks like, what comes back | Given a question about leave or holidays, returns the relevant policy answer. |
| Example prompts | 3–4 real ones, one per line | How many days of annual leave do I get? |

Save.

That's the Foundry side done. Your agent now accepts A2A requests at the URL you copied earlier — from anyone holding a valid Entra token with the right role. Next we make APIM that someone.

Part B — Register the agent in API Management

B1. Open the A2A wizard

Azure portal → your APIM instance (contoso-apim) → left menu APIs → + Add API.
Find the tile A2A Agent. Click it.

Don't see the tile? A2A support arrived for v2 tiers first and was extended to classic tiers in the June 2026 release. Check your tier and that the instance is updated. If it's genuinely absent, stop here — the rest of this article won't apply.

B2. Paste the agent card URL (and watch it fail on purpose)

In URL, paste the card URL — that's your A2A base URL with /agentCard/v1.0 on the end:

   https://contoso-agents-poc.services.ai.azure.com/api/projects/contoso-agents-poc/agents/helper-agent/endpoint/protocols/a2a/agentCard/v1.0

Click Next.
You'll see a red banner: "We couldn't retrieve the agent card, possibly due to a wrong url or your network configuration. Enter the agent API details manually below."

This is expected. APIM fetched the card anonymously; Foundry said no. The URL is not wrong. Carry on.

B3. Fill the form manually

Field	Value
Protocol	JSON-RPC (pre-selected, greyed out — APIM only supports this one)
Runtime URL (JSON-RPC)	Your A2A base URL — the one without `/agentCard/v1.0`
Agent ID	`helper-agent`
Display name	`Helper Agent`
Name	`helper-agent`
Description	Anything
Products	Leave empty for now
Base path	`agents/helper-agent`

As you type the base path, the two read-only URLs underneath update to:

Base URL (JSON-RPC): https://contoso-apim.azure-api.net/agents/helper-agent
Agent card URL: https://contoso-apim.azure-api.net/agents/helper-agent/agent-card.json

Click Create.

B4. Don't panic about the blank field

The Overview page shows Backend runtime base URL as empty, even though you just filled Runtime URL. Sometimes the Settings page also shows it empty on reload. This is a portal quirk. We will set the backend explicitly in the policy in Part D, which makes this field irrelevant. Move on.

Part C — Let APIM authenticate to Foundry

Foundry only accepts requests carrying an Entra token from an identity that has a Foundry role on the project. APIM will present its own managed identity. Two steps: confirm APIM has one, then grant it a role.

C1. Confirm APIM's managed identity is on

Azure portal → the APIM service (not the API you just made) → left menu Security → Managed identities → System assigned tab.
Status should be On. Note the Object (principal) ID — you'll recognise it in the next step.

If it's Off: switching it on is harmless technically, but on a shared or production instance tell whoever owns it first. Then toggle On → Save.

C2. Grant the managed identity a role on the Foundry project

Azure portal → search for your Foundry project resource (type: Foundry project, not Foundry account) → open it.
Left menu Access control (IAM) → + Add → Add role assignment.
Role tab: search Foundry User. Select it. Next.

Why Foundry User and not Foundry Agent Consumer? Agent Consumer is the least-privilege role designed for exactly this, but at the time of writing it doesn't appear in the portal's role picker. Foundry User is documented as "or higher" and works. If you need least-privilege, assign Agent Consumer via CLI with role definition ID eed3b665-ab3a-47b6-8f48-c9382fb1dad6.

Members tab: select Managed identity → + Select members.
- Subscription: the one your APIM lives in
- Managed identity dropdown: API Management service
- Pick contoso-apim → Select
Review + assign → Review + assign again.

Back on the IAM page, expand Foundry User and confirm contoso-apim is listed with type Managed identity.

Now wait 3–5 minutes. Role assignments propagate asynchronously. If you test immediately you'll get 401/403 and think something's broken.

Part D — The policy

This is where APIM is told: get a token as yourself, forward everything to the agent's A2A endpoint, and ask for protocol v1.0.

APIM → APIs → helper-agent → left menu Policies.
You'll see an XML editor with <policies>, <inbound>, <backend>, <outbound>, <on-error>.
Select all, delete, paste this, replacing the URL with yours:

<policies>
    <inbound>
        <base />
        <authentication-managed-identity resource="https://ai.azure.com" />
        <set-backend-service base-url="https://contoso-agents-poc.services.ai.azure.com/api/projects/contoso-agents-poc/agents/helper-agent/endpoint/protocols/a2a" />
        <set-header name="A2A-Version" exists-action="override">
            <value>1.0</value>
        </set-header>
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

Save.

Line by line, in plain words:

Line	What it does
`authentication-managed-identity resource="https://ai.azure.com"`	APIM gets an Entra token for itself, scoped to Foundry, and puts it in the `Authorization` header. This is the whole reason Part C existed.
`set-backend-service base-url=…`	Forward the request to the agent's A2A endpoint. Overrides whatever the Settings page did or didn't save.
`set-header A2A-Version 1.0`	Ask Foundry for protocol v1.0. Without this you silently get v0.3, which has a different message shape.

Part E — Get a subscription key

By default an APIM API requires a subscription key. Grab one:

APIM → left menu Subscriptions.
Find the row Built-in all-access subscription (or create one scoped to this API).
Click … on the right → Show/hide keys → copy Primary key.

Keep it in your notepad next to the URL.

Part F — Test it

F1. Which URL to call

If your APIM is publicly reachable: https://contoso-apim.azure-api.net
If public access is disabled and you go through Front Door or a custom domain: that hostname instead, e.g. https://api.yourcompany.com

The path is the same either way: /agents/helper-agent.

F2. Set up the request in Postman

New request. Method dropdown (left of the URL bar): POST. Yes, POST. GET will return 404 from APIM and you'll waste ten minutes.
URL:

   https://api.yourcompany.com/agents/helper-agent

Headers tab — two rows:

| Key | Value |
|---|---|
| Ocp-Apim-Subscription-Key | the key from Part E |
| Content-Type | application/json |

Body tab → raw → set the dropdown on the right to JSON → paste:

{
  "jsonrpc": "2.0",
  "id": "1",
  "method": "SendMessage",
  "params": {
    "message": {
      "messageId": "m1",
      "role": "ROLE_USER",
      "parts": [
        { "text": "Hello, what can you do?" }
      ]
    }
  }
}

Send.

F3. What success looks like

Status 200 and a body like this:

{
  "jsonrpc": "2.0",
  "id": "1",
  "result": {
    "task": {
      "id": "resp_…",
      "contextId": "ctxt_…",
      "status": {
        "state": "TASK_STATE_COMPLETED",
        "timestamp": "2026-09-06T04:18:09+00:00"
      },
      "artifacts": [
        {
          "artifactId": "msg_…",
          "parts": [
            { "text": "I answer questions about company holiday policy. …" }
          ]
        }
      ]
    }
  }
}

The agent's reply is at result.task.artifacts[0].parts[0].text. That path is what any consumer parses. contextId is the conversation — send it back in later messages to keep context.

You're done. Front Door → APIM → managed identity → Foundry → agent → back. Every hop authenticated, every call logged in APIM.

Troubleshooting — read the status code, it's telling you exactly where it broke

You see	It means	Do this
404 `{"statusCode":404,"message":"Resource not found"}`	APIM itself replied. Usually wrong HTTP method (GET instead of POST) or wrong path.	Check the method dropdown. Check the base path matches Part B3.
401 "Access denied due to missing subscription key"	APIM replied. No key or wrong header name.	Header must be exactly `Ocp-Apim-Subscription-Key`.
401 / 403 with headers like `azureml-served-by-cluster` or a body mentioning authorization	Foundry replied — APIM got through, but Foundry rejected APIM's identity.	Role hasn't propagated (wait 5 min) or Part C2 was assigned on the wrong resource (must be the project). Confirm `authentication-managed-identity` is in the policy.
405 with `allow: POST` and `azureml-served-by-cluster`	Foundry replied. You sent a GET to the base path.	This is actually good news: it proves the whole auth chain works. Switch to POST.
200 but body has `"code": -32601`	Foundry replied: method not found. You're hitting v0.3 but sent a v1.0 body (or vice versa).	Confirm the `A2A-Version` header line is in the policy and saved.
200 and the agent answers like a generic chatbot	Everything works; your agent has no instructions.	Part A1 step 4.
Front Door error page instead of any of the above	Front Door has no route for `/agents/*`.	Add a route rule or use the APIM hostname directly if reachable.

Known gap: the agent card through APIM

GET https://api.yourcompany.com/agents/helper-agent/agent-card.json returns 405 rather than the card.

Because the import wizard couldn't fetch the card (Part B2), APIM never stored one, and it simply proxies the request to the agent's base path — which only accepts POST. Rewriting the path inside the policy (rewrite-uri) didn't take effect on this API type in testing.

Does it matter? Not for calling the agent — you just proved that. It matters for discovery: a client that wants to read the card first (for example, another Foundry agent using "Connect via endpoint") will fail at that step. Two workarounds we haven't fully validated yet: serve the card from the policy using send-request + return-response, or fetch the card once with a user token and host it as a static file. Both are on the follow-up list.

Where to go from here

Let another Foundry agent call this one. In the calling agent → Tools → Add → A2A → Connect via endpoint, URL = your APIM URL, auth = custom key Ocp-Apim-Subscription-Key. Needs the card gap above closed first — or use Connect from Foundry to bypass APIM for Foundry-to-Foundry.
Expose several agents. One A2A API per agent, same policy with the name swapped. After two, script it (Bicep / apiops).
Turn agents into MCP tools. A2A APIs can't be exposed as MCP servers directly. Create one plain HTTP API in APIM with one POST operation per agent pointing at each agent's Responses endpoint (…/endpoint/protocols/openai/responses), then Expose existing API as MCP server. One MCP server, one tool per agent, fully GA end-to-end.

Cheat sheet

A2A base URL     https://{account}.services.ai.azure.com/api/projects/{project}/agents/{agent}/endpoint/protocols/a2a
Card URL (v1.0)  …/protocols/a2a/agentCard/v1.0
Token scope      https://ai.azure.com/.default
Role             Foundry User (or Foundry Agent Consumer, ID eed3b665-ab3a-47b6-8f48-c9382fb1dad6)
Version header   A2A-Version: 1.0
JSON-RPC method  SendMessage   (v1.0)   |   message/send   (v0.3)
Reply path       result.task.artifacts[0].parts[0].text

Putting Azure OpenAI Behind an API Gateway: What the Docs Don't Tell You

Vignesh Athiappan — Thu, 03 Sep 2026 08:35:16 +0000

We had a problem that will sound familiar to anyone who has let a team loose on Azure OpenAI.

One Foundry resource. One gpt-4o-mini deployment. One API key, copied into a config file, then into a second config file, then pasted into a Slack thread so someone could test something quickly. No idea who was spending what. No limits. If one app got stuck in a retry loop at 2am, everybody's model calls would start failing and we'd spend the morning working out whose fault it was.

The fix is an AI gateway. Azure API Management has a purpose-built wizard for this now, and the happy path genuinely is a wizard. But we hit roughly a dozen things that the documentation either glosses over or gets subtly wrong, and several of them are the kind of mistake you can't undo.

This is what actually happened, in order, including the parts where I was wrong.

What an AI gateway buys you

Before the mechanics, the argument, because it's easy to build this and then not enforce it.

Putting API Management in front of a model deployment gets you four things:

One key per application, revocable independently. Each consuming app gets its own gateway key. Revoke one, the others keep working. No app ever holds a credential for the model itself.

Spend caps that are actually enforced. A tokens-per-minute limit per application means one runaway loop can't drain the shared quota.

Attribution. Token counts land in Application Insights, dimensioned by which app made the call. "Who spent this" becomes a query instead of an investigation.

Caching. Semantically similar questions can be answered from a cache rather than paid for twice.

Keep that list in mind, because there's a trap at the end of this post where you build all of it and enforce none of it.

Step one: pick the right client compatibility mode

The wizard's first real decision is also its most expensive to get wrong, and it's presented as three innocuous radio buttons.

You are asked to choose between Azure OpenAI, Azure AI, and Azure OpenAI v1. The difference is where the deployment name lives.

With Azure OpenAI, the deployment name stays in the URL path:

POST {gateway}/{base-path}/openai/deployments/{deployment}/chat/completions?api-version=2024-12-01-preview

With the other two, the deployment name moves into the request body. That sounds like a trivial difference. It isn't, because the official AzureOpenAI Python SDK builds the first shape. Choose either of the other options and you are rewriting every client, not changing a URL.

Pick Azure OpenAI unless you have a specific reason not to. Your app code then changes by exactly one line:

client = AzureOpenAI(
    api_version="2024-12-01-preview",
    azure_endpoint="https://your-gateway.example.com/ai-prod",  # was the Foundry URL
    api_key=os.environ["GATEWAY_KEY"],                          # was the Foundry key
)

The SDK appends /openai/deployments/... itself.

There's a bonus hidden in this choice. Because the deployment name is a path segment, it behaves as a wildcard. Deploy a new model on the same Foundry resource, and it is immediately reachable through the gateway with no new API, no second wizard run, no config change. We only discovered this by asking; it's not obvious from the UI.

The base path is forever

The wizard asks for a base path and leaves the field blank. Everything else on that screen you can change later. This one you effectively cannot, because it becomes part of every client's URL.

Two rules. Make it lowercase and hyphenated, because base paths are case-sensitive and mixed case generates 404s that people waste an afternoon on. And make it specific rather than generic. openai feels natural until you onboard a second Foundry resource and discover the name is taken on that gateway instance.

The default token limit is a trap

Tick "manage token consumption" and the TPM field pre-fills with 1000.

Our deployment quota was 250,000 TPM. With max_tokens: 4096, a single verbose response can consume a meaningful fraction of 1000 tokens, so you'll start seeing 429s within a handful of test calls and reasonably conclude the gateway is broken.

Go and read your actual deployment quota first. Then set the gateway limit at or below it. Above is worse than useless: API Management forwards the traffic, the model throws the 429 instead, and you get the outage without the protection, with the error surfacing from the backend where it's harder to attribute. We set ours to 125,000, half the quota, leaving room to onboard a second consumer without touching the deployment.

Two settings on that screen deserve more attention than they get:

Limit by: Subscription. This makes the allowance per gateway key, so each app has its own bucket. The alternative, IP address, is wrong for most deployments because your apps will call from a small pool of shared outbound addresses and collide into a single bucket.

Estimate prompt tokens: on. With it off, the policy can only count tokens after the response returns, so a burst of concurrent requests all pass the check before any of them report. Worse: for streamed responses it can't count at all unless the client sends stream_options: {"include_usage": true}. Most client code doesn't. With estimation off, streaming calls quietly bypass your limit entirely.

Metrics dimensions: fewer than you think

The wizard offers nine dimensions to slice token metrics by, and it's tempting to take them all. We did, briefly.

Application Insights bills per unique combination of dimension values. Most of those dimensions are constants in a single-instance, single-API setup, so you pay for cardinality that tells you nothing.

Two are worth having:

Subscription ID: which application spent the tokens. This is the chargeback dimension and the whole point of the exercise.
API ID: constant today, essential the moment you add a second Foundry resource.

Skip Client IP (high cardinality, low information, since callers share outbound addresses), User ID (empty unless you're doing per-user JWT auth at the gateway), and Gateway ID / Location (constant for a single instance).

Worth knowing before you decide: Application Insights data isn't retroactive. Metrics already emitted keep their original dimensions. So adding a dimension later means old and new data can't be compared cleanly, and removing one leaves a split in your history. Get it roughly right at the start.

There are two different Application Insights connections

This one cost us a debugging session.

The token metric policy and the request telemetry logger are separate mechanisms, configured in different places, and having one does not give you the other.

The token metric policy is set up in the wizard. It emits prompt, completion, and total token counts.

The request logger is a per-API setting under Settings, Diagnostics Logs. It captures status codes, latency, which operation was called, and failures. It has its own destination field, which starts empty.

So after the wizard we could see exactly how many tokens a request consumed, but not that it had returned a 500 or taken twelve seconds. When we hit our first error, there was nothing useful to look at.

Set both. On the request logger, set sampling to 100% for a low-volume API, tick "always log errors", and switch the correlation protocol from Legacy to W3C, which is what lets you trace a request end to end across the edge, the gateway, and the backend. Leave "payload bytes to log" at 0 unless you're actively debugging, because raising it puts prompt and response content into your logs.

One more thing that surprised us: the request logger is instance-wide by default, so telemetry from unrelated APIs on the same gateway lands in the same Application Insights resource. That's not a bug. Operations are prefixed with the API ID, so filtering is trivial:

requests
| where operation_Name startswith "your-ai-api"

The only place it genuinely bites is alerting. An alert on "average duration over 5 seconds" is meaningless when one API responds in 100ms and the other takes 8 seconds. Scope alerts by operation name, not on the blended instance average.

Semantic caching is where the wizard falls apart

This is the feature with the longest prerequisite chain, and the wizard will not tell you what's missing until you're several screens in.

Your existing Redis almost certainly won't work

Semantic caching needs a vector index, which means Redis with the RediSearch module. We had a Redis instance already, running happily as a cache for another system. Its module list read RedisJSON, RedisTimeSeries, RedisBloom. No RediSearch.

Modules are fixed at creation time. You cannot add RediSearch to an existing instance. You create a new one.

RediSearch conflicts with OSS clustering

Creating that new instance, we ticked RediSearch and immediately got:

OSSCluster Cluster Policy doesn't support the selected module(s): RediSearch

The fix is one click: change Clustering Policy from OSS to Enterprise. Enterprise clustering hides sharding from the client, which is what the search module requires.

You can't set an eviction policy, and that matters

With RediSearch enabled, the eviction policy dropdown offers exactly one option: No Eviction. Azure locks it, because the search index needs its data present to stay consistent.

The consequence is easy to miss. Your cache will not self-clean. When it fills, Redis starts rejecting writes rather than dropping old entries. So the cache duration you set in the API Management policy stops being a tuning knob and becomes the only thing preventing unbounded growth. We used 3600 seconds. Size the instance with headroom.

`InsufficientCapacity` is a region problem, not a config problem

Our first three deployment attempts failed with a generic Conflict / ResourceDeploymentFailure. The portal's error summary was useless. The Raw Error tab had the real answer:

{
  "status": "Failed",
  "error": {
    "code": "InsufficientCapacity",
    "message": "Request failed due to insufficient capacity. Retry using a different Azure Managed Redis size or region."
  }
}

Always open Raw Error. The summary is a wrapper.

Two levers: change the size, or change the region. Note that changing size within the same performance tier often doesn't help, because those SKUs draw from the same capacity pool. Switching performance tier (Balanced to Memory Optimized, say) is more likely to find capacity than stepping from B0 to B1.

We eventually provisioned in a different region, which leads directly to the next mistake.

The external cache has a region field, and it's not decorative

Registering Redis as API Management's external cache asks you which gateway location "uses" this cache. Ours defaulted to the region the Redis instance was in, which was not the region the gateway was in. If no gateway location matches, the cache may never be used at all, silently. Set it to your gateway's region, or Default.

Also worth understanding: external cache registration is instance-wide, but using it is per-API. Registering the cache doesn't change the behaviour of any existing API, because nothing touches Redis unless a policy says so.

Cross-region caching may cost more than it saves

We ended up with the gateway in one continent and Redis in another. Roughly 200ms of round trip added to every request, hits and misses alike.

On a cache miss, that's ~200ms added to a call already taking 1 to 2 seconds. Annoying, survivable. On a hit, you still come out well ahead. The place it genuinely hurts is streaming, where the full round trip lands on time-to-first-token, which is the latency users actually perceive.

But the real question isn't latency. It's whether your prompts repeat at all. Semantic caching pays an embedding call on 100% of requests to get a saving on some fraction of them. If your traffic is mostly unique prompts, you're paying for nothing. Measure your hit rate before you defend the architecture.

I should also correct something I believed going in. I assumed Redis Enterprise with search would cost several hundred dollars a month, and used that to argue against the whole feature. Azure Managed Redis at the entry tier came in around $25/month. That's a materially different calculation, and it changed my recommendation. Check current pricing rather than trusting a number you remember.

The embeddings backend the wizard forgets to create

This one produced our only hard failure.

The semantic cache lookup policy references an embeddings backend by ID. That backend needs to exist as a registered API Management Backend, pointing at an embeddings deployment. The wizard normally creates it for you.

Ours didn't, because the wizard aborted partway through with a benign-looking error:

Error setting up semantic caching. The role assignment already exists.

The role assignment conflict was harmless. An earlier attempt had already granted the gateway's managed identity the right role. But the wizard treated it as fatal, stopped, and never created the backend or added the policies.

We added the policies by hand and got a clean 500 Internal Server Error on every request, in about 800ms, far too fast to have reached the model. The lookup policy was throwing because embeddings-backend didn't resolve.

The fix: create the backend manually as a Custom URL, with the name matching the policy exactly.

https://<your-resource>.openai.azure.com/openai/deployments/text-embedding-3-small/embeddings

Two lessons. First, if the wizard reports any error, go and verify what it actually created; don't assume a partial failure is a partial success. Second, and more generally: add one policy at a time and test between each. We added two policies and a backend dependency in one go, then had to bisect. Getting a single successful 200 before layering anything on would have made the failure obvious immediately.

The policy order, and what it means

The finished inbound chain, in execution order:

Validate the gateway key against the product. An unrecognised key is rejected here, before any Azure resource is touched.
Semantic cache lookup. Embed the prompt, search Redis. A match above the similarity threshold returns the stored answer and steps 3 and 4 never run.
Token rate limit. Count the call against that key's allowance.
Emit token metrics.
Route to the backend using the gateway's managed identity.

And on the way out:

Semantic cache store, writing the answer and its vector back to Redis.

The sequencing is deliberate: cheap rejections first, cache before the expensive call, and no credential anywhere in the path.

One thing worth internalising: a cache miss now costs you an embedding call plus a model call. Add content safety screening and you have three service round trips before the model sees the prompt. That's the honest price of a governed gateway. It's usually worth it. Just don't be surprised by the latency and go looking for a bug.

Similarity threshold: trust the description, not the warning

Setting the threshold, the portal showed us this:

Similarity score threshold above 0.2 may lead to cache mismatch. Consider using lower value.

That is backwards, at least as worded. The field description says similarity, where 1.0 is an exact match. A higher threshold means stricter matching and fewer false hits. A lower threshold is what causes mismatches: at 0.2 almost any two prompts would look similar and you'd cheerfully serve a user the answer to a question they didn't ask.

We started at 0.9. Start strict, watch your hit rate, and lower it deliberately if hits are too rare. This is the one setting where a wrong value produces plausible but incorrect output rather than an error, which makes it the most dangerous knob on the page.

Also worth scoping the cache per key with a vary-by on the subscription. Otherwise one application's answers can be served to another.

The header rename that saves you a rewrite

API Management expects its key in an Ocp-Apim-Subscription-Key header. The AzureOpenAI SDK sends credentials in an api-key header.

You can work around this in client code with default_headers, but there's a cleaner option: under the API's Settings, rename the subscription header to api-key. Now the SDK works unchanged and your app diff is genuinely one line.

Products and subscriptions: badly named, load-bearing

The terminology here trips people up, so plainly:

An API Management subscription has nothing to do with an Azure subscription. It's an API key.

A product is a bundle of APIs plus access rules. A subscription is a key issued against a product. So: product contains your API, you create a subscription on the product, it generates a key, the app sends the key.

This isn't optional decoration. Two of the settings you already configured point at nothing without it: Limit by: Subscription has no subscriptions to count per, and the Subscription ID metrics dimension has nothing to record. And of course without a key nobody can call the API at all.

The mistake that makes all of this decorative

Here's the one that matters most, and it's the easiest to skip because everything appears to be working.

When you finish, you have a governed gateway. You also still have the original Foundry endpoint, with its original keys, publicly reachable, bypassing every control you just built. Anyone holding a copy of that key gets no rate limit, no metrics, no cache, no attribution.

Your gateway is optional. And an optional control is not a control.

There are two ways to close it, with very different effort profiles.

Disable local key authentication. The endpoint stays publicly reachable, so the gateway keeps working unchanged (it authenticates with its managed identity, not a key), but the API keys stop working entirely.

az cognitiveservices account update \
  --name <your-foundry-resource> \
  --resource-group <your-rg> \
  --custom-domain <your-foundry-resource> \
  --api-properties disableLocalAuth=true

Or disable public network access and put a private endpoint in front of it. Stronger, because it removes the public attack surface entirely rather than just the key path. But it's a real project: private endpoint, private DNS zone linked to the VNet, and gateway VNet integration, which requires a Standard v2 or Premium v2 tier. If you're on a lower tier, this becomes a SKU migration with a cost attached.

If you go the private route, audit callers before disabling local auth, and do it last in the sequence. If some forgotten script or integration is hitting the model with a key today, it breaks the moment you flip that switch, silently and at whatever time of day you happened to run the command.

What I'd do differently

Prove the base path works before adding anything. One successful 200 through the gateway, with nothing but key validation in the chain. Then add the token limit, test. Then metrics, test. Then caching, test. We stacked four unproven dependencies and spent longer bisecting than the sequential approach would have taken.

Read the raw error, always. The portal's error summaries are wrappers around the useful message.

Verify what the wizard claims to have built. A partial failure looks a lot like success from the notifications panel. Check the policy XML. Check the backends list. Check the role assignment landed on the target resource.

Decide about direct access on day one. Not as a follow-up task. Until the bypass is closed, you have built a dashboard, not a gateway.

Ask the data residency question out loud. Semantic caching stores prompt text and responses. If your organisation deliberately chose a region for compliance reasons, a cache in a different geography is a real change to where data lives, not an implementation detail. It's the item least likely to surface on its own and most likely to become a problem later.

Was it worth it?

Yes, and not marginally.

The end state: every application holds its own revocable key, no application holds a credential for the model, spend is capped per application and attributable to the team that caused it, repeated questions are served from cache instead of being paid for twice, and there's one place to look when something breaks.

The build took an afternoon. Most of that afternoon was the dozen small things above, none of which are in the quickstart. Now that you know about them, it should take you an hour.

Promotable, Deployable, Rotatable: Making Bench Management an Actual Process

Vignesh Athiappan — Thu, 27 Aug 2026 13:57:52 +0000

Every services organisation has a bench — people between projects, waiting on a client start date, rolling off an engagement that ended early, or parked because the skill they have and the skill the market wants have drifted apart. The bench is not a defect. It's the cost of being able to say yes to work you haven't won yet.

What is a defect is not knowing anything useful about the people on it.

The usual state of affairs is a spreadsheet with a name, a skill string, a rate, and a date. That tells you who is unallocated. It tells you nothing about what to do about it. So the conversation in the weekly review becomes a series of individual negotiations, driven by whoever in the room happens to remember something about that person. Decisions don't accumulate. Six weeks later you have the same conversation about the same person, and nobody can reconstruct why the last one went the way it did.

PDR is one way out of that. It's a small idea, and most of the value is in the discipline it forces rather than the taxonomy itself.

The three dispositions

PDR classifies each person on the bench against three questions:

Promotable — is this person ready to move up a level? Not "are they good," but: is there evidence they are already operating above their current band, and would a promotion make them easier to place at a higher billing rate?

Deployable — can this person be put on billable work as-is, right now, with no intervention? This is the most misread of the three. Deployable is not a compliment. It means the skills match live demand and there is nothing blocking a start.

Rotatable — should this person move to a different technology, domain, or vertical? Rotation is the answer when someone is capable but their current specialisation has no pipeline behind it.

The point of three separate axes is that they are not mutually exclusive and they are not a ranking. Someone can be promotable and rotatable — a strong senior engineer in a dying stack. Someone can be deployable but not promotable — solid, well-matched, not ready for the next band. Someone can be none of the three, and that is the most important signal the framework produces, because it means the situation needs a decision that isn't "wait."

The moment you collapse this into a single label — a tier, a grade, an A/B/C — you lose exactly the information you built the system to capture.

The process

1. Intake

Bench entries come from allocation data, not from manual entry. If someone has to remember to add a person, the list will be wrong within a week. The system should derive the bench from the absence of active billable allocation, then let humans annotate.

Intake also needs a start date for the bench period. Aging is the single most predictive field you have, and it only works if the clock starts automatically.

2. Assessment

Someone with actual knowledge of the person marks the three flags. This is usually a delivery lead or a resource manager, not HR and not a tool.

Two design decisions matter more than they sound:

Unassessed must look different from assessed-and-negative. "Not promotable" and "nobody has looked at this yet" are completely different facts, and a UI that renders both as an empty checkbox will quietly destroy your data quality. Blank means unknown. Show it as unfinished.

Every negative or unusual combination needs a justification. Not a dropdown — free text, required, at the point of assessment. This is the gate that turns the framework from a labelling exercise into a decision record. It also slows people down, which is the intended effect. If marking someone as not deployable takes two seconds, it will be done thoughtlessly and often.

3. Action

A classification with no downstream action is theatre. Each disposition should route somewhere concrete:

Promotable → into the next promotion cycle with the justification attached as evidence
Deployable → into active matching against open demand
Rotatable → into a reskilling track with a named target skill and a timeline
None of the above → escalation, with a review date

The fourth case is the one organisations avoid building. It's also the one that determines whether the whole exercise is honest.

4. Audit

Every assessment change is appended, never overwritten. Who changed what, when, from what to what, and why. Append-only, no edits, no deletes.

This is non-negotiable for two reasons. The obvious one is that these records touch promotion, reassignment, and eventually separation decisions, and you will be asked to explain them. The less obvious one is that the history is where the actual insight lives. A person marked rotatable four quarters running, with four justifications, is telling you something about your reskilling programme, not about the person.

What actually goes wrong

Coverage gets measured instead of accuracy. "94% of the bench is assessed" is a number that goes up when people click things. It says nothing about whether the assessments are right. Coverage is a hygiene metric — worth a single indicator, not a dashboard. If your reporting makes coverage the headline, you have built an incentive to fill in boxes.

Deployable becomes the default. Marking someone deployable is the path of least resistance: it's positive, it needs no justification, it moves the problem to the matching team. Watch the distribution. If deployable is running above 70% of assessed bench while placement rates stay flat, the flag has stopped meaning anything.

Assessments go stale. A three-month-old assessment is a guess. Build in expiry — an assessment older than a defined window reverts to unassessed rather than continuing to display as fact. People will hate this. Do it anyway.

Rotation is recommended without capacity to deliver it. Marking someone rotatable is free. Actually retraining them costs money, bench time, and a mentor. If rotation recommendations exceed reskilling capacity by 5x, the flag is a way of deferring a harder conversation.

The framework gets used as a performance tool. PDR describes market fit and placement readiness. It is not a performance rating, and the two must not be joined. A high performer in a technology with no demand is not deployable, and if your system lets that read as a performance signal, you will lose good people and deserve to.

The honest framing

PDR is a decision-forcing device, not an optimisation engine. It won't tell you what to do with anyone. What it does is make the absence of a decision visible — an unassessed row, a stale assessment, a person marked rotatable for three quarters with no training assigned. That visibility is uncomfortable, which is the entire point, and it's also the reason most implementations quietly drift toward measuring completion rates instead.

If you're building something like this, the test isn't whether the dashboard looks good. It's whether the weekly bench review is shorter and produces fewer repeat conversations than it did before. If a person's situation comes up twice with no change in between, the system failed regardless of how much of the bench is "covered."

One last thing worth saying plainly: these are people, and the labels are blunt. "Not deployable" is a statement about a market, a skills pipeline, and a sales pipeline — three things the individual mostly doesn't control. Build the justification field wide, require it, and read what people write in it. That text is usually a more accurate description of your organisation's problems than of the person's.

CI/CD for Azure Logic Apps Standard on a Private (ILB) ASE — Without a VNet Agent

Vignesh Athiappan — Wed, 26 Aug 2026 04:19:58 +0000

How we wired an Azure DevOps pipeline to a Logic App Standard running on an internal App Service Environment, hit six real-world failures on the way, and ended with a deployment that needs no SAS tokens, no SCM access, and no VNet-connected build agent.

The problem

Logic Apps Standard on an ILB App Service Environment (ASE v3) has no public SCM (Kudu) endpoint. That kills the two "normal" deployment routes:

Route	Why it fails on ILB ASE
`az webapp deploy` / zip-deploy from a Microsoft-hosted agent	Talks to the SCM endpoint, which is private
VS Code / portal publish	Same SCM dependency, plus it isn't CI/CD

The usual answer is "stand up a self-hosted agent inside the VNet." That's the right long-term answer — but you can ship today without it, using Run-From-Package with a managed-identity-authenticated blob URL. The pipeline only ever talks to two public planes:

Azure Storage (upload the package)
ARM / management.azure.com (set one app setting, restart the app)

The app then pulls the package itself, from inside the VNet, using its own managed identity. SCM is never touched. No SAS token ever exists.

Git push ──▶ ADO pipeline ──▶ zip ──▶ Blob Storage
                                          ▲
                                          │ (pull via app's
                                          │  managed identity)
             ARM: set app setting ──▶ Logic App (ILB ASE)
             ARM: restart app

What you need before starting

#	Prerequisite	Notes
1	Logic App Standard resource, Started	A stopped app silently accepts settings and deploys nothing. Check the Overview blade — ours had been sitting in `Stopped` and cost us a confused hour
2	A storage account the ASE can reach	Same region as the ASE. Public network access enabled is the simple path; a private endpoint works too
3	A blob container, private access	e.g. `deployments`
4	An ADO agent (hosted or self-hosted) with Azure CLI installed	Ours was a Windows self-hosted box that did not have `az` — see failure #3
5	An ARM service connection	Any auth type works. If it's the "Managed identity (agent-assigned)" type, the identity must actually be enabled on the agent VM — see failure #2
6	Rights to create role assignments on the storage account and the Logic App's resource group	Or a friendly admin

Step 1 — Repository layout

One folder per Logic App Standard resource; one subfolder per workflow. The zip we deploy is simply the app folder's contents.

logicapps-platform/
├── apps/
│   └── la-integration-prod/          ← 1 folder = 1 Logic App Standard resource
│       ├── host.json                 ← runtime + extension bundle
│       ├── connections.json          ← connector references ({} to start)
│       ├── parameters.json           ← per-environment values ({} to start)
│       ├── .funcignore
│       └── Heartbeat-Test/           ← 1 folder = 1 workflow
│           └── workflow.json
└── pipelines/
    └── deploy-la-integration-prod.yml

host.json:

{
  "version": "2.0",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle.Workflows",
    "version": "[1.*, 2.0.0)"
  }
}

connections.json:

{
  "managedApiConnections": {},
  "serviceProviderConnections": {}
}

Heartbeat-Test/workflow.json — a deliberately trivial pilot workflow. No connectors, no secrets, nothing that can fail for reasons unrelated to the pipeline. Prove the pipe first; migrate real workflows second.

{
  "definition": {
    "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
    "contentVersion": "1.0.0.0",
    "triggers": {
      "Every_Hour": {
        "type": "Recurrence",
        "recurrence": { "frequency": "Hour", "interval": 1 }
      }
    },
    "actions": {
      "Compose_Heartbeat": {
        "type": "Compose",
        "inputs": {
          "message": "Deployed via ADO pipeline",
          "runAt": "@{utcNow()}",
          "workflow": "@{workflow().name}"
        },
        "runAfter": {}
      }
    },
    "outputs": {}
  },
  "kind": "Stateful"
}

Path gotcha: if your files live in a subfolder of the repo (e.g. myrepo/logicapps-platform/apps/...), the pipeline's sourceFolder variable and trigger paths must include that prefix. Our first run failed with Cannot find path ...\apps\ for exactly this reason.

Step 2 — Identity and role assignments

Three grants total. Two for the pipeline's identity (whatever your service connection authenticates as), one for the Logic App's own identity.

2a. Pipeline identity

Role	Scope	Purpose
`Storage Blob Data Contributor`	The storage account	Upload the package
`Website Contributor`	The Logic App's resource group	Set app settings + restart

az role assignment create \
  --role "Storage Blob Data Contributor" \
  --assignee-object-id <pipeline-identity-object-id> \
  --assignee-principal-type ServicePrincipal \
  --scope "/subscriptions/<sub-id>/resourceGroups/<storage-rg>/providers/Microsoft.Storage/storageAccounts/<storage-account>"

az role assignment create \
  --role "Website Contributor" \
  --assignee-object-id <pipeline-identity-object-id> \
  --assignee-principal-type ServicePrincipal \
  --scope "/subscriptions/<sub-id>/resourceGroups/<logicapp-rg>"

How to find the pipeline identity's object ID when nobody remembers it: run a throwaway pipeline on the target pool that asks the instance metadata service directly:

trigger: none
pool:
  name: 'SelfHosted-Windows-Pool'
steps:
- powershell: |
    $r = Invoke-RestMethod -Headers @{Metadata="true"} -Uri "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://management.azure.com/"
    $p = $r.access_token.Split('.')[1].Replace('-','+').Replace('_','/')
    switch ($p.Length % 4) { 2 {$p+='=='}; 3 {$p+='='} }
    $c = [Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($p)) | ConvertFrom-Json
    Write-Host "objectId : $($c.oid)"
    Write-Host "resource : $($c.xms_mirid)"   # full ARM ID of the VM - tells you where it lives
  displayName: Who am I

If this returns Identity not found, the agent VM's system-assigned identity is off — enable it (VM → Identity → System assigned → On) and note the object ID it produces. This exact thing happened to us: the MI-type service connection had existed for months and had never worked.

2b. Logic App identity (the key to SAS-free deployment)

Enable the app's system-assigned identity (Logic App → Identity → System assigned → On), then:

az role assignment create \
  --role "Storage Blob Data Reader" \
  --assignee-object-id <logicapp-identity-object-id> \
  --assignee-principal-type ServicePrincipal \
  --scope "/subscriptions/<sub-id>/resourceGroups/<storage-rg>/providers/Microsoft.Storage/storageAccounts/<storage-account>"

RBAC propagation is real. Allow ~5 minutes between creating a role assignment and testing it. Restarting the app 30 seconds after the grant gives a false failure.

Step 3 — The pipeline

The complete, working YAML. Written for a Windows agent (all steps PowerShell); on a Linux agent, switch scriptType: ps to bash and adjust the validation step.

# ============================================================================
# Deploy Logic App Standard via Run-From-Package (Blob, MI-authenticated URL)
# The app pulls the package with ITS OWN managed identity: no SAS, no expiry,
# and the SCM endpoint is never touched, so an ILB ASE is not a problem.
#
# PREREQS:
#   - Logic App system-assigned identity ON, with 'Storage Blob Data Reader'
#     on the storage account
#   - Pipeline identity: 'Storage Blob Data Contributor' on the storage
#     account and 'Website Contributor' on the Logic App's resource group
# ============================================================================

trigger:
  branches:
    include: [ main ]
  paths:
    include:
      - apps/la-integration-prod/**
      - pipelines/**

pool:
  name: 'SelfHosted-Windows-Pool'     # or vmImage: 'ubuntu-latest' for SP-auth connections

variables:
  serviceConnection: 'my-azure-service-connection'
  resourceGroup:     'rg-logicapps-prod'
  appName:           'la-integration-prod'
  storageAccount:    'stdeploypkgs'
  container:         'deployments'
  sourceFolder:      'apps/la-integration-prod'
  packageName:       'la-integration-prod-$(Build.BuildId).zip'

stages:

# ----------------------------------------------------------------------------
- stage: Build
  displayName: Validate & package
  jobs:
  - job: Package
    steps:

    - powershell: |
        $bad = 0
        Get-ChildItem -Path "$(sourceFolder)" -Recurse -Filter *.json | ForEach-Object {
          $raw = Get-Content $_.FullName -Raw
          try { $raw | ConvertFrom-Json | Out-Null }
          catch { Write-Host "##vso[task.logissue type=error]Invalid JSON: $($_.FullName)"; $bad++ }
          if ($raw -match '"(secret|client_secret|clientSecret|password)"\s*:\s*"(?!@)') {
            Write-Host "##vso[task.logissue type=error]Inline secret in: $($_.FullName)"; $bad++
          }
        }
        if ($bad -gt 0) { exit 1 }
      displayName: Validate JSON + block inline secrets

    - task: ArchiveFiles@2
      displayName: Zip app content
      inputs:
        rootFolderOrFile: '$(sourceFolder)'
        includeRootFolder: false          # host.json must sit at the ZIP ROOT
        archiveType: zip
        archiveFile: '$(Build.ArtifactStagingDirectory)/$(packageName)'

    - publish: '$(Build.ArtifactStagingDirectory)/$(packageName)'
      artifact: package

# ----------------------------------------------------------------------------
- stage: Deploy
  displayName: Deploy to $(appName)
  dependsOn: Build
  jobs:
  - deployment: Deploy
    environment: 'logicapps-prod'         # add an approval gate on this environment
    strategy:
      runOnce:
        deploy:
          steps:

          - download: current
            artifact: package

          - task: AzureCLI@2
            displayName: Upload package to blob
            inputs:
              azureSubscription: '$(serviceConnection)'
              scriptType: ps
              scriptLocation: inlineScript
              inlineScript: |
                az storage blob upload `
                  --account-name $(storageAccount) `
                  --container-name $(container) `
                  --name $(packageName) `
                  --file "$(Pipeline.Workspace)/package/$(packageName)" `
                  --auth-mode login --overwrite

          - task: AzureCLI@2
            displayName: Point app at package (MI-auth URL, no SAS, no SCM)
            inputs:
              azureSubscription: '$(serviceConnection)'
              scriptType: ps
              scriptLocation: inlineScript
              inlineScript: |
                $ErrorActionPreference = "Stop"
                $url = "https://$(storageAccount).blob.core.windows.net/$(container)/$(packageName)"

                az webapp config appsettings set `
                  -g "$(resourceGroup)" -n "$(appName)" `
                  --settings WEBSITE_RUN_FROM_PACKAGE="$url" WEBSITE_RUN_FROM_PACKAGE_BLOB_MI_RESOURCE_ID="SystemAssigned" `
                  -o none

                az webapp restart -g "$(resourceGroup)" -n "$(appName)"
                Write-Host "Deployed $(packageName). App restarted."

          - task: AzureCLI@2
            displayName: Verify workflow exists
            inputs:
              azureSubscription: '$(serviceConnection)'
              scriptType: ps
              scriptLocation: inlineScript
              inlineScript: |
                Start-Sleep -Seconds 60
                $sub = az account show --query id -o tsv
                az rest --method GET `
                  --uri "https://management.azure.com/subscriptions/$sub/resourceGroups/$(resourceGroup)/providers/Microsoft.Web/sites/$(appName)/workflows?api-version=2022-03-01" `
                  --query "value[].name" -o tsv

Three lines carry the whole trick:

WEBSITE_RUN_FROM_PACKAGE = the plain blob URL — no SAS query string.
WEBSITE_RUN_FROM_PACKAGE_BLOB_MI_RESOURCE_ID=SystemAssigned — tells App Service to fetch the package using the app's own managed identity. Without this setting the runtime attempts anonymous access, fails, and the app reports ServiceUnavailable from the host runtime. This is the single most-missed step.
includeRootFolder: false in the archive task — host.json must be at the root of the zip, not nested one level down.

Step 4 — Run it, and what "success" looks like

Create the pipeline from the YAML, run it. First run will pause to ask permission on the service connection / environment / pool — click Permit.
Green pipeline, and the Verify step prints your workflow name(s).
Portal → Logic App → Workflows shows the workflow, Stateful, Enabled.
Open it → Run history → confirm a run actually executed with the expected output. A workflow that loads but never runs is not success.

Everything that actually went wrong (the useful part)

Our path to green, in order. If you're debugging, scan this table first.

#	Symptom	Root cause	Fix
1	`Unable to locate executable file: 'bash'`	Assumed Linux; the self-hosted agent was Windows	All steps → PowerShell (`scriptType: ps`)
2	`Invoke-RestMethod : {"error":"invalid_request","error_description":"Identity not found"}` on the metadata endpoint	Agent VM's system-assigned managed identity was off — the MI-type service connection had never actually worked	VM → Identity → System assigned → On. If the identity was enabled after boot, a VM restart may be needed before the token endpoint responds
3	`Azure CLI 2.x is not installed on this machine`	Fresh agent VM, no `az`	Install via portal Run Command (no RDP needed), then restart the agent service so it picks up the new PATH: `Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile C:\az.msi` `Start-Process msiexec.exe -ArgumentList '/i C:\az.msi /quiet /norestart' -Wait` `Get-Service vstsagent* \
4	{% raw %}`Get-ChildItem : Cannot find path '...\apps\'`	Repo files nested one folder deeper than the YAML expected	Fix `sourceFolder` and trigger `paths` to include the prefix
5	`ERROR: incorrect usage: --expiry should be within 7 days from now`	Identity-based (user-delegation) SAS is capped at 7 days — and a 7-day SAS would be a time bomb anyway, since the app re-reads the package URL on every restart	Drop SAS entirely; switch to the MI-authenticated plain URL (this article's approach)
6	Pipeline green, but portal shows `Error retrieving workflows. Encountered an error (ServiceUnavailable) from host runtime`	App couldn't pull the package: `WEBSITE_RUN_FROM_PACKAGE_BLOB_MI_RESOURCE_ID` missing, and the app's identity had no role on storage	Add the app setting; grant `Storage Blob Data Reader` to the app's identity; wait ~5 min for RBAC; restart

A 60-second diagnostic when the runtime won't start

Run from any machine with az — checks the four usual suspects at once:

$rg="rg-logicapps-prod"; $app="la-integration-prod"
$st="stdeploypkgs"; $strg="rg-storage"; $sub="<sub-id>"

# 1. Identity on?
az webapp identity show -g $rg -n $app --query principalId -o tsv

# 2. Both settings present?
az webapp config appsettings list -g $rg -n $app `
  --query "[?starts_with(name,'WEBSITE_RUN_FROM')].{n:name,v:value}" -o table

# 3. App identity has a role on storage?
az role assignment list --assignee <principalId-from-step-1> `
  --scope "/subscriptions/$sub/resourceGroups/$strg/providers/Microsoft.Storage/storageAccounts/$st" `
  --query "[].roleDefinitionName" -o tsv

# 4. Storage reachable at all?
az storage account show -g $strg -n $st `
  --query "{publicNetworkAccess:publicNetworkAccess, defaultAction:networkRuleSet.defaultAction}" -o table

Whichever check comes back empty is your culprit.

Why this beats the alternatives

Approach	Verdict
SAS URL in `WEBSITE_RUN_FROM_PACKAGE`	Works, but user-delegation SAS caps at 7 days and account-key SAS means handling keys. Either way the app dies quietly when the token expires — on whatever future day it happens to restart
Zip-deploy via VNet self-hosted agent	The classic answer. Fine — but it makes deployment hostage to one VM's health, and you still shouldn't need SCM for a package-based app
MI-authenticated Run-From-Package	No secrets, no expiry, no SCM, works from any agent that can reach ARM + Storage. The package is also immutable per build ID, so rollback = point the setting at the previous zip and restart

Hardening checklist for after the pilot

Approval gate on the ADO Environment before the Deploy stage.
Private endpoint on the storage account once the pattern is proven (the app pulls over the VNet; the pipeline needs its own route — service endpoint or an agent with access).
Lifecycle policy on the container — build-numbered zips accumulate forever otherwise.
Keep the inline-secret linter. It costs nothing and blocks the exact class of mistake that infests legacy workflow definitions.
New workflows that replace still-running legacy ones should ship disabled and be enabled at cutover — never let two schedulers fire the same job.

Total elapsed time for us, including all six failures: one working session. Total time if you follow this article: about 30 minutes — 25 of which are waiting for an MSI installer and RBAC propagation.

# CSPM, CWPP, CIEM, CNAPP, EASM: Decoding the Cloud Security Alphabet

Vignesh Athiappan — Sat, 22 Aug 2026 04:30:21 +0000

Part 4 of 4 in the "Security Testing Landscape" series

The cloud security market has produced some of the worst acronyms in IT history. Here's the secret that makes them all click: these aren't tests — they're continuous monitoring platforms, and each one watches a different layer of your cloud.

The analogy that makes it permanent: your cloud is a house.

CSPM — Cloud Security Posture Management

Are the doors and windows locked?

CSPM watches your cloud configuration: public storage accounts, wide-open network security groups, disabled encryption, subscriptions without MFA. It continuously scans against benchmarks (CIS, well-architected frameworks) and flags drift the moment someone loosens a setting.

Answers: "Is my cloud set up wrong?"
Tools: Microsoft Defender for Cloud, Wiz, Prisma Cloud, Orca

CWPP — Cloud Workload Protection Platform

Is there an intruder inside a room?

CWPP watches the workloads themselves — VMs, containers, serverless functions: vulnerabilities inside them, malware, suspicious runtime behavior, file integrity changes. Think of it as EDR, but for cloud compute.

Answers: "Is something bad running inside my machines?"
Tools: Defender for Servers/Containers, CrowdStrike Falcon Cloud Security, Aqua, SentinelOne

CIEM — Cloud Infrastructure Entitlement Management

Who has too many keys?

CIEM watches identities and permissions: which users, service principals, and managed identities can do what. It hunts over-permissioned accounts, roles nobody has used in months, and toxic combinations like "this identity can read the secrets vault and is reachable from the internet."

Answers: "Who can do too much?"
Tools: Microsoft Entra Permissions Management, Wiz, Sonrai, Tenable (Ermetic)

CNAPP — Cloud-Native Application Protection Platform

The whole smart-home security system.

Here's the trick: CNAPP is not a new capability. It's the umbrella that bundles CSPM + CWPP + CIEM (plus IaC scanning and container scanning) into one platform with one dashboard. Its real power is chaining findings into attack paths:

vulnerable container + public exposure + admin identity = critical attack path, fix this first

That prioritization — attack paths instead of ten thousand isolated alerts — is why the entire market converged on CNAPP.

Answers: "Show me the actual attack paths across everything."
Tools: Wiz, Prisma Cloud, Orca, Defender for Cloud (full plans) — the same vendors, selling the full suite

EASM — External Attack Surface Management

Walking around the street outside, checking what a burglar sees.

EASM flips the camera: it scans your organization from the public internet, like an attacker with a browser and patience. It routinely finds assets you forgot existed — abandoned subdomains, exposed test APIs, expired certificates, shadow IT spun up by a team three years ago.

The uncomfortable, recurring outcome: EASM discovers things that aren't in your asset inventory at all.

Answers: "What can the internet see of mine?"
Tools: Microsoft Defender EASM, Censys, Palo Alto Cortex Xpanse — and Shodan for the manual version

The House, One Last Time

Acronym	Watches	House analogy
CSPM	Settings/config	Doors and windows locked?
CWPP	Workloads	Intruder inside a room?
CIEM	Identities/permissions	Who has too many keys?
CNAPP	All of the above	The full smart-home system
EASM	The outside view	What the burglar sees from the street

One final classification note: CSPM, CWPP, CIEM, and CNAPP are all white box — they read your subscriptions with granted access. EASM is the lone black box of the group, and that outsider's perspective is exactly what makes it valuable.

Series Wrap-Up

Across four posts, the entire landscape reduces to four questions:

Is my application code safe? → App Security Testing (SAST, DAST, SCA…)
Can someone actually break in? → Offensive assessments (VAPT, red teams, bug bounties)
Are we designing, building, and proving it right? → Reviews and audits
Is my cloud continuously watched? → CSPM/CNAPP family

Master which question each acronym answers, and you'll never nod along blankly in a security meeting again.

The Security Work That Never Attacks Anything: Reviews, Audits & Threat Modeling

Vignesh Athiappan — Sat, 22 Aug 2026 04:30:00 +0000

Part 3 of 4 in the "Security Testing Landscape" series

The previous two posts covered tools and humans that attack systems. This group is different: nobody attacks anything. These are humans (and a few platforms) inspecting your designs, code, configurations, and paperwork — with full access. Every single item here is white box; that's the group's signature.

The best way to remember them is by the question each one answers on the project timeline.

"Are we designing it right?" — Before You Build

Threat Modeling

A structured brainstorm at design time: for each component, how would an attacker abuse it? The output is a prioritized list of threats and mitigations — before a single line of code exists, when fixes are cheapest.

The frameworks you'll hear:

STRIDE — the classic threat checklist: Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, Elevation of privilege.
PASTA — a risk-centric, seven-stage methodology that ties threats to business impact.
DREAD — a scoring model for ranking threats: Damage, Reproducibility, Exploitability, Affected users, Discoverability.

Tools: Microsoft Threat Modeling Tool, OWASP Threat Dragon, IriusRisk

Architecture / Design Review

Security engineers walk the diagrams and data flows: where are the trust boundaries, is authentication centralized, are secrets in a vault, does a frontend talk directly to a backend when it shouldn't. If you've ever defended an architecture diagram in front of a senior reviewer, you've lived this one.

References: cloud well-architected security pillars, OWASP ASVS

"Did we build it right?" — During and After the Build

Secure Code Review (Manual)

A human reads the code — especially auth logic, crypto usage, and input handling — catching design-level flaws that automated SAST cannot, like "this check can be bypassed by calling the API endpoints in a different order." Tooling assists (Semgrep, CodeQL, PR review workflows), but the value is the brain.

Configuration Review / Hardening Assessment

Compare actual server, database, and cloud settings against a hardened baseline: TLS versions, open ports, default accounts, logging enabled. The gold-standard baselines are the CIS Benchmarks and DISA STIGs.

Tools: CIS-CAT, Nessus compliance scans, Azure Policy

"Are we running it right?" — Continuously

Cloud Security Posture Assessment

The configuration review idea, applied to your entire cloud estate, continuously: public storage, over-permissive IAM, missing encryption, exposed endpoints — all scored against benchmarks in near-real time.

Tools: Microsoft Defender for Cloud (secure score), Prisma Cloud, Wiz, Orca

"Can we prove it to outsiders?" — When Auditors Arrive

Security Audit / Compliance Assessment

An external auditor checks whether you actually follow a standard — through evidence, policies, interviews, and screenshots. The output is a certification or findings report, never an exploit.

The standards you'll meet:

ISO 27001 — certifies your information security management system
SOC 2 — a trust-criteria report, the staple for SaaS vendors
PCI DSS — mandatory if you touch card data
HIPAA / GDPR — health data and personal data regulations

Tools: GRC platforms like Vanta, Drata, ServiceNow GRC — but mostly document review.

"Are we spending money right?" — Above Everything

Risk Assessment

The business-level exercise that sits on top of all of it: list assets → identify threats → score likelihood × impact → prioritize. This decides where the security budget goes and feeds every other activity on this page.

Frameworks: ISO 27005, NIST 800-30, and FAIR if you want risk quantified in actual currency.
Tools: honestly, spreadsheets — or a GRC platform if you're fancy.

The Timeline Recap

Question	Activity	When
Designing it right?	Threat Modeling, Architecture Review	Before build
Built it right?	Code Review, Config Review	During/after build
Running it right?	Cloud Posture Assessment	Continuously
Proving it?	Compliance Audit	Audit season
Spending right?	Risk Assessment	Always, above all

Remember: the testing groups attack; this group inspects with full access. Both are necessary — inspection finds what attacks miss, and attacks prove what inspections suspect.

Final post in the series: the cloud alphabet soup — CSPM, CWPP, CIEM, CNAPP, and EASM.

From Vulnerability Scans to Red Teams: The Offensive Security Ladder

Vignesh Athiappan — Sat, 22 Aug 2026 04:29:14 +0000

Part 2 of 4 in the "Security Testing Landscape" series

Every company eventually asks: "are we actually hackable?" There's an entire industry of ways to answer that question, and they form a ladder of realism and cost. This post climbs it rung by rung.

Rung 1: VA — Vulnerability Assessment

An automated scanner sweeps your servers, VMs, and network for known CVEs and misconfigurations. It's breadth over depth: it finds and lists problems, but never exploits them. Think of it as a metal detector — it beeps, it doesn't dig.

Tools: Nessus, Qualys, OpenVAS, Rapid7 InsightVM
Box: Usually grey (the scanner gets network access or credentials); can run uncredentialed as black box.

Rung 2: PT — Penetration Testing

Now a human takes the scanner's findings and actually exploits them — chaining vulnerabilities, escalating privileges, reaching real data — to prove genuine business impact. Depth over breadth, scoped and time-boxed.

Tools: Burp Suite, Metasploit, Nmap, SQLMap, BloodHound, Cobalt Strike

This is where the black/grey/white box terminology originates:

Black box: the tester gets nothing but a URL or IP — simulates an external attacker.
Grey box: the tester gets a normal user account and basic docs — simulates an insider or a compromised user. This is the most common flavor in practice, and the best value.
White box: the tester gets source code, architecture diagrams, and admin credentials — maximum coverage per paid day.

Rung 3: VAPT — The Compliance Staple

VAPT is simply VA + PT packaged together: scan everything automatically, then let a human manually exploit the interesting bits. This is the standard annual engagement companies buy for compliance, and the term you'll see on every audit checklist in this part of the world.

Rung 4: Red Teaming

A red team gets a goal, not a scope: "reach the HR database without being detected." Weeks long, stealthy, no holds barred — phishing, physical entry, custom malware, whatever works. Crucially, it tests your defenders and detection capability, not just your systems. The blue team usually isn't told it's happening.

Tools: Cobalt Strike, Sliver, Mythic, GoPhish
Box: Black by design.

Rung 5: Purple Teaming

Red and blue sit at the same table. Red runs an attack technique, blue immediately checks "did our SIEM catch that?", they tune the detection, and repeat. Collaborative rather than adversarial — the fastest way to actually improve detection coverage.

Tools: MITRE ATT&CK framework, Atomic Red Team, VECTR
Box: White — full transparency is the point.

Rung 6: BAS — Breach & Attack Simulation

Software that continuously and automatically replays known attack techniques inside your environment to verify your controls (EDR, SIEM, WAF) still detect them — an always-on mini red team that never sleeps or invoices per day.

Tools: AttackIQ, SafeBreach, Cymulate, Picus
Box: Grey/white — agents installed inside, techniques known.

Rung 7: Bug Bounty

Open the doors and invite researchers worldwide to attack your production app, paying per valid bug instead of per day. Continuous coverage, pay-for-results economics.

Platforms: HackerOne, Bugcrowd, Intigriti
Box: Black — researchers get exactly what the public gets.

The Human-Focused Rungs

Three assessments attack things other than servers:

Social Engineering / Phishing Simulation targets your people — fake phishing emails, phone pretexting (vishing), USB drops — measuring click and credential-submission rates. Tools: GoPhish, KnowBe4, Microsoft Attack Simulator.
Wireless Assessment targets your airwaves — rogue access points, WPA2 handshake cracking, evil-twin attacks. Tools: Aircrack-ng, Kismet, WiFi Pineapple.
Physical Security Testing targets your building — tailgating, badge cloning, lock picking, plugging into exposed ports. Tools: Proxmark, LAN Turtle.

All three are black box: the targets don't know.

The Ladder, One Line

VA (scanner lists bugs) → PT (human proves bugs) → VAPT (both — the compliance staple) → Red Team (full attack, tests your people and detection) → Purple (red and blue learn together) → BAS (automate it forever) → Bug Bounty (outsource it to the world).

Each rung up costs more and simulates reality more faithfully. Most organizations start with VAPT and grow upward.

Next in the series: the quiet group that never attacks anything — reviews, audits, and threat modeling.

The Alphabet Soup of Application Security Testing, Explained With a Car

Vignesh Athiappan — Sat, 22 Aug 2026 04:28:48 +0000

Part 1 of 4 in the "Security Testing Landscape" series

If you've ever sat in a security review meeting and heard SAST, DAST, IAST, and SCA thrown around in the same sentence, you know the feeling: everyone nods, half the room is quietly Googling. This post fixes that permanently, using one analogy — your application is a car you're building and driving.

The Core Four

SAST — Static Application Security Testing

SAST is reading the car's blueprint before it's ever built. You spot that the brakes are drawn wrong without starting an engine.

In IT terms: SAST scans your source code for insecure patterns — SQL injection, hardcoded credentials, weak crypto — without running the application. It lives in your CI pipeline and fires on every commit or pull request.

Tools: SonarQube, Checkmarx, Fortify, Semgrep, GitHub CodeQL
Box: White box — it has full access to your code.

DAST — Dynamic Application Security Testing

DAST is test-driving the finished car and deliberately trying to crash it. It never looks at the blueprint.

In IT terms: DAST attacks your running, deployed application over HTTP exactly like an external hacker would — malicious payloads into URLs, forms, and headers. It has zero knowledge of your code.

Tools: OWASP ZAP, Burp Suite, Acunetix, Invicti
Box: Black box — it only sees what an attacker sees.

IAST — Interactive Application Security Testing

IAST is a sensor inside the car during the test drive. When the brakes get slammed, it feels exactly which part shook.

In IT terms: an agent sits inside the running app (in the JVM or .NET runtime) and watches code execute while your QA or DAST tests run. Result: it pinpoints the exact vulnerable line, combining the strengths of SAST and DAST.

Tools: Contrast Security, Synopsys Seeker, HCL AppScan
Box: Grey box — the app is running, but the agent sees internals.

SCA — Software Composition Analysis

Your build might be flawless, but the tires you bought from another shop have a recall notice. SCA checks the parts you didn't make.

In IT terms: SCA scans your dependency manifests — package.json, pom.xml, .csproj — against CVE databases for vulnerable open-source libraries (think Log4j) and license risks. Given that most modern codebases are 70–90% third-party code, this one is not optional.

Tools: Snyk, Mend, OWASP Dependency-Check, GitHub Dependabot
Box: White box.

The Supporting Cast

RASP (Runtime Application Self-Protection) is the odd one out — it's not a test at all. It's the airbag: an agent embedded in your production app that blocks real attacks as they happen, like killing a SQL injection mid-execution. Tools: Contrast Protect, Imperva RASP.

MAST (Mobile AST) is the same testing story for a motorbike — SAST and DAST applied to APK/IPA binaries, hunting mobile-specific issues like insecure local storage and reverse-engineering exposure. Tools: MobSF, NowSecure, Appknox.

API Security Testing ignores the car body and inspects the fuel pipe connections. Doors can be locked while a leaky pipe sinks you: it tests APIs directly for auth bypass, BOLA/IDOR, and mass assignment, usually driven by the OpenAPI spec. Tools: Postman, Burp Suite, 42Crunch, Salt Security. Typically grey box.

Fuzzing pours random junk into the fuel tank — sand, juice, bolts — until the engine chokes. Massive volumes of malformed input fired at an app or API until it crashes or misbehaves. Tools: AFL++, libFuzzer, Burp Intruder, RESTler. Typically black box.

Secret Scanning checks whether someone left the car keys taped to the windshield — leaked API keys, tokens, and connection strings in repos and git history. Tools: GitLeaks, TruffleHog, GitHub Secret Scanning. White box.

Container Scanning inspects the shipping crate the car arrives in — Docker images scanned layer by layer for OS-level CVEs before deployment. Tools: Trivy, Grype, Aqua. White box.

IaC Scanning audits the factory setup instructions. If the manual says "leave the door open," every car built there is unsafe: Terraform, Bicep, ARM, and Kubernetes YAML scanned for misconfigurations before anything is provisioned. Tools: Checkov, tfsec, Terrascan, KICS. White box.

The Timeline That Makes It Stick

Before the app runs (paperwork and parts): SAST, SCA, Secret Scanning, Container Scanning, IaC Scanning
While it runs in testing (test drives): DAST, IAST, Fuzzing, API Testing
While it runs in production (the airbag): RASP

And the box rule: anything that reads your files is white box; anything that attacks the running app from outside is black box; the agent that does both is grey.

Next in the series: the offensive world — VA, PT, VAPT, red teams and bug bounties.

The Product Maturity Matrix: How We Measure Whether Our Internal Products Are Actually "Grown Up"

Vignesh Athiappan — Sat, 22 Aug 2026 04:07:50 +0000

Every engineering team ships features. Very few teams can answer a harder question: is this product mature?

Not "does it work" — mature. Can it survive an audit? Can it survive the person who built it leaving? Can it survive an AI prompt injection attack at 2 AM on a weekend?

Recently, I was asked to review and refine our product maturity matrix — the framework we use to assess every internal product across our portfolio. What started as a review exercise turned into a genuinely useful lesson in what "maturity" actually means for modern products, especially ones with AI baked in. Here's what I learned.

Why a maturity matrix at all?

When you're running a portfolio of internal products — a PSA platform, an ATS, an AI assistant, dozens of integrations — you can't rely on gut feel. Each product has different teams, different tech debt, different risk profiles. A maturity matrix gives you one honest lens across all of them.

The version I reviewed covered 12 areas. By the time I was done, it was 10 — and sharper for it. The final structure:

Access Management
Security Monitoring
Encryption & Data Protection
AI Security & Risk Management
Architecture & Design
Data Quality
Integration Management
DevOps & Engineering Automation
Observability
Engineering Excellence

Each area has five concrete criteria. Not aspirations — checkable statements. "Rollback procedure is defined and tested." "Access reviews are done periodically with business owners." The difference between a checklist and a wishlist is whether you can fail it.

Lesson 1: Kill the duplicates, or the matrix eats itself

The first thing I found when reviewing was overlap. "Security attacks" appeared under Observability. "AI red-teaming" appeared in two places. Credential expiry monitoring was floating around with no clear home.

This matters more than it sounds. When the same control lives in three rows, three teams either all claim it or all assume someone else owns it. Both outcomes are bad.

The fix was a simple ownership rule:

Observability = visibility. Can you see what's happening — logs, dashboards, traces, latency, failures?
Security Monitoring = detection and response. Attacks, network traffic, alerts, SLAs.
AI Security = assessment. Red-teaming, risk reviews, guardrail evaluations.
Integration Management = operational hygiene. Job monitoring, credential and token expiry.

One control, one home. Everything got easier after that.

Lesson 2: "100% accuracy" is not a criterion — it's a trap

The original data quality row demanded "100% accuracy of all business reports." Sounds rigorous. It's actually the opposite.

You cannot measure 100%. You cannot prove it. Every audit against it either fails or gets hand-waved — and hand-waving is precisely what a maturity matrix exists to eliminate.

We replaced it with something you can actually run:

Business reports are validated periodically by the tech team against source data, and reported data issues are resolved within a defined SLA.

Measurable. Auditable. Honest. If your maturity criteria require perfection, teams will quietly stop taking the whole matrix seriously. Write criteria a good team can pass and a struggling team will visibly fail.

Lesson 3: AI changed what "mature" means

Five years ago, a maturity matrix stopped at DevOps and monitoring. Today, if your products have AI agents in them — and ours increasingly do — you need two new muscles:

AI Security & Risk Management as a first-class area. Prompt injection, data leakage, model misuse — these aren't hypotheticals. In our own security assessments we've caught things like PII slipping out through encoded payloads that a naive filter would never flag. Red-teaming your own AI features before someone else does it for you is now table stakes.

AI observability inside the observability row. Traditional APM tells you the request took 800ms. It doesn't tell you what the user asked, what the model answered, how many tokens it burned, or where in the agent chain the latency lives. Mature AI products trace the full conversation path: query → layers → response, with token usage and guardrail outcomes visible.

If your maturity framework predates your AI features, it's assessing a product that no longer exists.

Lesson 4: The most embarrassing finding is the most valuable one

One row in the matrix — Encryption & Data Protection — contained criteria that had clearly been copy-pasted from a completely different section. It described automation workflows. Nothing about encryption at all.

Nobody had noticed, because nobody had read it. It had been reviewed in meetings, circulated in emails, and nodded at repeatedly.

That's the real argument for periodic deep reviews of governance documents: not to add more rows, but to check that the existing ones still say what everyone assumes they say. We rewrote it properly — encryption at rest and in transit, secrets in a managed vault with no hardcoded credentials, PII classification and masking, retention and disposal policies.

Lesson 5: Merge what belongs together

The original matrix had separate rows for "Coding & Secure Practices" and "Release Notes," plus an empty placeholder called "Engineering Excellence." We merged all three.

Code review, static analysis, dependency scanning, release notes, updated documentation — these aren't separate disciplines. They're one discipline: the craft of shipping responsibly. Splitting them across rows just multiplied the paperwork without adding rigor.

A shorter matrix that people actually use beats a longer one they skim.

The takeaway

A product maturity matrix isn't a compliance artifact. Done well, it's a mirror. It tells you which of your products would survive scrutiny and which are running on luck and tribal knowledge.

Three principles carried the whole review:

Every criterion must be checkable. If you can't fail it, delete it.
Every control needs exactly one home. Overlap is unowned risk.
The matrix must evolve with the stack. AI features demand AI-shaped criteria.

If your team has a maturity framework gathering dust, open it up and actually read every row. You might find automation criteria hiding in your encryption section too.

What does product maturity look like in your organization? I'd genuinely like to hear how other teams are handling AI security and observability in their governance frameworks.

Serving Millions of Users Without Melting the Database (Explained with Swiggy)

Vignesh Athiappan — Thu, 20 Aug 2026 04:08:33 +0000

At 8:30 PM on a Friday, millions of people open a food app at once. How does it stay fast — and how does the database survive? Not with one trick, but with five layers, each removing load before it reaches the expensive part. Here they are, using Swiggy.

The 5 techniques

#	Technique	One-line
1	Caching	Keep hot data close, skip the slow DB
2	Load Balancing	Spread traffic across many servers
3	CDN	Serve content from near the user
4	Partitioning / Sharding	Split one huge DB into pieces
5	Autoscaling	Add/remove servers automatically

1. Caching

Keep frequently-used data somewhere fast so you don't hit the slow database every time.

Where caches live:

Level	Where	Swiggy example
Client	On your phone	Your cart, cached locally
CDN	Edge servers worldwide	Restaurant images
Distributed	A shared cache cluster	Menu, sessions (Redis)

How writes interact with the cache:

Strategy	How it works	Trade-off
Cache-aside	App fills the cache on a miss	Simple; first read is slow
Write-through	Write to cache + DB together	Always fresh; writes slower
Write-back	Write to cache now, DB later	Fast writes; risk of loss if cache dies

Swiggy: menu in a distributed cache (Redis), images on a CDN, cart on the client. Different data, different cache home.
Azure: Azure Cache for Redis.

The golden rule: cache things where "a few seconds stale is fine" (menu, restaurant list). Never cache things where stale = wrong (wallet balance, payment status).

2. Load Balancing

Spread incoming traffic across many identical servers so no single one drowns.

              ┌→ Server 1
Users → [ LB ]┼→ Server 2
              └→ Server 3   (LB picks which one)

Two types — L4 vs L7:

	L4 (Transport)	L7 (Application)
Sees	IP + port only	Full HTTP (URL, headers)
Speed	Faster, dumber	Smarter, slightly slower
Can do	Raw distribution	Route `/search` vs `/pay` to different pools

How it picks a server:

Algorithm	Rule
Round-robin	Next server in turn
Least connections	The least-busy server
IP hash	Same user → same server (sticky)

Swiggy: millions of requests spread across hundreds of servers. An L7 balancer routes /search to search servers and /payment to payment servers.
Azure: Load Balancer (L4), Application Gateway / Front Door (L7).

3. CDN — Content Delivery Network

Copy static content to servers around the world, so users get it from nearby.

❌ No CDN:  user in Chennai → fetches image from a US server (slow, far)
✅ CDN:     user in Chennai → fetches from a Chennai edge server (fast, near)

How content actually gets there — it's pull-based, not push:

First user in a region requests logo.png → the edge has nothing → it fetches from your origin, stores a copy, serves it.
Next users in that region → served from the edge copy; origin untouched.
Each edge manages its own cache, so another region still pulls from origin until its edge caches it too.

You never "upload" files to a CDN. The first request in each region fills the cache — which is why it needs almost no setup.

Three things beginners get wrong:

Myth	Reality
"The CDN caches automatically"	On modern services (like Azure Front Door) caching is opt-in per route — you switch it on
"It's safe to cache everything"	Never cache authenticated or per-user pages — a shared edge cache can serve one user's data to another
"Staleness is controlled"	If your origin sends no `Cache-Control`, the default TTL can be days — set your own

Swiggy: restaurant photos, app JS/CSS, menu images — all static, all cached at the nearest edge. Your biryani photo loads instantly because it's cached 20km away, not 12,000km. But the logged-in order page is never cached — that stays dynamic.
Key: a CDN is caching for location. Distance = latency, so serve from close by.

4. Partitioning / Sharding

Split one giant database into smaller pieces (shards), each holding part of the data.

The make-or-break decision is the partition key — how you split.

Strategy	How	Watch out
Hash	`hash(key)` decides the shard	Even spread, but no range queries
Range	A–M here, N–Z there	Simple, but can create hotspots
Directory	A lookup table maps key → shard	Flexible, but the lookup is a dependency

✅ Good	❌ Bad
Scales beyond one machine	Cross-shard queries are painful
Each shard is smaller/faster	The partition key is ~irreversible

Swiggy: orders partitioned by cityId or userId — Bengaluru's orders on one shard, Delhi's on another.
Azure: this is exactly the Cosmos DB partition key — the single most important, effectively permanent design choice you make up front.

⚠️ The trap: a bad key creates a hot partition — one shard takes all the traffic while the others sit idle. Choose a key that spreads load evenly.

5. Autoscaling

Automatically add servers when busy, remove them when quiet.

3 PM (quiet):   ▪▪         (3 servers)
8:30 PM (peak): ▪▪▪▪▪▪▪▪▪▪ (auto-scaled to 20)
2 AM (dead):    ▪          (scaled back to 1)

Type	Rule
Reactive	Scale when CPU/queue crosses a threshold
Predictive	Scale ahead of a known pattern (the dinner rush)

✅ Good	❌ Bad
Pay only for what you use	Scaling isn't instant (there's lag)
Handles spikes automatically	Cold starts on new instances

Swiggy: scale up before the 8 PM rush, scale down after midnight. Never pay for 500 servers at 3 AM.
Azure: VM Scale Sets, AKS autoscaler, App Service autoscale.

How they connect

CDN            → serves static content from the edge (never hits your servers)
Load Balancer  → spreads the dynamic traffic that remains
Caching        → cuts DB reads for that traffic
Partitioning   → splits the DB when caching isn't enough
Autoscaling    → adjusts server count to match live demand

Each layer removes load before it reaches the expensive part: CDN catches the easy stuff → the load balancer spreads the rest → the cache skips the DB → partitioning shares the DB load → autoscaling flexes capacity.

The whole thing in one line

Scaling isn't one big trick — it's five layers peeling load away before it reaches the database. Serve static from the edge, spread the rest, cache what you can, split what you can't, and flex capacity to match the crowd.

How Services Send Messages Without Losing Them (Explained with Swiggy)

Vignesh Athiappan — Wed, 19 Aug 2026 04:05:51 +0000

Instead of calling each other directly, services often drop messages and move on. It's more resilient — but it raises real questions. What if the receiver is down? What if a message gets processed twice and a customer is charged double? These 5 patterns are the answers.

The 5 patterns

#	Pattern	One-line
1	Message Queue	A buffer between sender and receiver
2	Publish-Subscribe	One message, many independent receivers
3	Competing Consumers	Many workers share one queue to scale
4	Dead-Letter Queue	Where failing messages go to be inspected
5	Idempotent Consumer	Processing twice = same result as once

1. Message Queue

A buffer that holds messages between sender and receiver. One message → one consumer.

Order svc → [ ▪▪▪ QUEUE ▪▪▪ ] → Payment svc
            (holds messages safely)

✅ Good	❌ Bad
Sender & receiver decoupled	Adds a component to manage
Receiver can be down — the message waits	Not instant (it's async)
Absorbs traffic spikes	Ordering can be tricky

Swiggy: "charge this order" is dropped on a queue. If Payment is briefly down, the message waits — nothing is lost.
Azure: Service Bus Queue.
Key idea: each message is consumed once, by one worker.

2. Publish-Subscribe (Pub-Sub)

One message published → delivered to MANY subscribers, each independently.

The difference from a queue: a queue delivers to one. Pub-sub delivers a copy to everyone who subscribed.

"Order placed" → [ TOPIC ] →→ Payment    (own copy)
                            →→ Restaurant (own copy)
                            →→ Analytics  (own copy)

✅ Good	❌ Bad
Add new subscribers without touching the sender	Harder to trace who got what
Fully decoupled	Eventual consistency
One event, many reactions	—

Swiggy: "order placed" → payment, restaurant, and analytics each get their own copy and react independently.
Azure: Service Bus Topic (with subscriptions), or Event Grid.

Queue vs Topic — lock this in:

Queue	Topic
One message → one consumer	One message → many subscribers
"Assign this task"	"Announce this happened"

3. Competing Consumers

Multiple workers pull from the SAME queue to process faster. They compete for messages.

             ┌→ Worker 1
[ QUEUE ] ───┼→ Worker 2   (whoever's free grabs the next message)
             └→ Worker 3

✅ Good	❌ Bad
Scale by adding workers	Message order isn't guaranteed
Auto load-balances	Workers must be stateless
Drains faster at peak	—

Swiggy: at dinner peak, 50,000 "charge order" messages pile up. Run 100 payment workers off one queue → it drains fast. Each message still goes to exactly one worker (no double-processing).
This is how you scale a queue consumer.

4. Dead-Letter Queue (DLQ)

A separate queue for messages that keep failing — so one bad message can't block everything.

The problem it solves is a poison message: one malformed message that fails, retries, fails, retries… forever, jamming the queue.

[ QUEUE ] → try process → fail → retry → fail (3x) → move to [ DLQ ]
                                                       ↑ inspect later; queue keeps flowing

✅ Good	❌ Bad
One bad message can't block the line	Needs monitoring
Failed messages saved for inspection	Manual cleanup

Swiggy: a corrupt order message fails 3 times → it's shoved into the DLQ → the main queue keeps flowing for everyone else. An engineer inspects the DLQ later.
Azure: Service Bus has a DLQ built-in on every queue and subscription.

⚠️ The trap: teams set up a DLQ and never watch it. Messages die silently. Always alert on DLQ depth.

5. Idempotent Consumer

Processing the same message twice produces the SAME result as processing it once.

Why you need it: queues can deliver a message twice (a network hiccup, a retry). Without protection, "charge ₹450" runs twice → the customer is charged ₹900. 💥

Message "charge order #123" arrives twice:
  ❌ Not idempotent: charge ₹450 + charge ₹450 = ₹900 (disaster)
  ✅ Idempotent:     "already processed #123?" → skip the 2nd → ₹450 ✅

How it works: track a unique message/operation ID; if you've seen it, skip it.

✅ Good	❌ Bad
Safe against duplicate delivery	Must store processed IDs
Retries become harmless	Adds a check on every message

Swiggy: every payment carries an idempotency key. A duplicate "charge" message is detected and skipped. Charged once.
This is non-negotiable in money flows — "at-least-once delivery" means duplicates will happen, so consumers must be idempotent.

How it all connects

Message Queue        → the basic buffer (1 → 1)
Publish-Subscribe    → the broadcast version (1 → many)
Competing Consumers  → scale the consumer side (many workers, 1 queue)
Dead-Letter Queue    → catch the failures
Idempotent Consumer  → survive duplicate deliveries

The whole thing in one line

A queue assigns one task; a topic announces to all. Then you scale it (competing consumers), protect it (dead-letter queue), and make it safe to retry (idempotent consumer). Get those right and messaging becomes the most reliable way for services to talk.

DEV Community: Vignesh Athiappan

PDF Rendering for HR at Scale: Why Your Simple Test Fails (and How to Fix It)

PDF Rendering for HR at Scale: Why Your Simple Test Fails (and How to Fix It)

The Problem

How We Broke Our Own Test

The Right Test

What We Tested

The Real Decision

The Security Bits (Don't Forget These)

What We Did Wrong (So You Don't)

The Template

What We'd Do Differently

The Takeaway

Exposing a Microsoft Foundry Agent over A2A Through Azure API Management — The Click-by-Click Guide

What you will have at the end

Why bother with the gateway at all?

Before you start — read this once

Part A — Set up the agent in Foundry

A1. Create the agent (skip if you already have one)

A2. Create the agent card and enable A2A

Part B — Register the agent in API Management

B1. Open the A2A wizard

B2. Paste the agent card URL (and watch it fail on purpose)

B3. Fill the form manually

B4. Don't panic about the blank field

Part C — Let APIM authenticate to Foundry

C1. Confirm APIM's managed identity is on

C2. Grant the managed identity a role on the Foundry project

Part D — The policy

Part E — Get a subscription key

Part F — Test it

F1. Which URL to call

F2. Set up the request in Postman

F3. What success looks like

Troubleshooting — read the status code, it's telling you exactly where it broke

Known gap: the agent card through APIM

Where to go from here

Cheat sheet

Putting Azure OpenAI Behind an API Gateway: What the Docs Don't Tell You

What an AI gateway buys you

Step one: pick the right client compatibility mode

The base path is forever

The default token limit is a trap

Metrics dimensions: fewer than you think

There are two different Application Insights connections

Semantic caching is where the wizard falls apart

Your existing Redis almost certainly won't work

RediSearch conflicts with OSS clustering

You can't set an eviction policy, and that matters

InsufficientCapacity is a region problem, not a config problem

The external cache has a region field, and it's not decorative

Cross-region caching may cost more than it saves

The embeddings backend the wizard forgets to create

The policy order, and what it means

Similarity threshold: trust the description, not the warning

The header rename that saves you a rewrite

Products and subscriptions: badly named, load-bearing

The mistake that makes all of this decorative

What I'd do differently

Was it worth it?

Promotable, Deployable, Rotatable: Making Bench Management an Actual Process

The three dispositions

The process

1. Intake

2. Assessment

3. Action

4. Audit

What actually goes wrong

The honest framing

CI/CD for Azure Logic Apps Standard on a Private (ILB) ASE — Without a VNet Agent

The problem

What you need before starting

Step 1 — Repository layout

Step 2 — Identity and role assignments

2a. Pipeline identity

2b. Logic App identity (the key to SAS-free deployment)

Step 3 — The pipeline

Step 4 — Run it, and what "success" looks like

Everything that actually went wrong (the useful part)

A 60-second diagnostic when the runtime won't start

`InsufficientCapacity` is a region problem, not a config problem