DEV Community

Cover image for Azure API Management CI/CD: APIOps Pipelines with GitHub Actions
Martin Oehlert
Martin Oehlert

Posted on AI-assisted

Azure API Management CI/CD: APIOps Pipelines with GitHub Actions

What has to pass before a merged pull request may change the production API Management instance without anyone opening the portal? Part 5's publish workflow already ran a dry run on every pull request and published on merge, but nothing required the dry run to pass, it signed in with the same identity that published to dev, and nothing read the spec or the policies before the CLI did. A pipeline you can trust with prod needs four gates, each catching something the others can't: checks that need no Azure at all, a dry run that can only read, a publish to dev that API Management itself validates, and a person who approves prod.

The companion repo keeps Part 5's layout: apiops/ holds the artifact tree, the extract filter and configuration.prod.yaml, and the APIOps CLI is still pinned at @azure-tools/apiops-cli@1.0.5. The publish workflow is what changes. The dry run becomes a required check that runs on every pull request and can only read, dev gets the merged commit, prod gets it after an approval, and a manual run publishes the full tree when prod has fallen behind. Each stage signs in through OIDC with its own user-assigned managed identity, so the repository holds no Azure secret. As in Part 5, nothing is deployed. Both workflows pass actionlint, the three validate steps ran locally against the sample, the claim that the dry run only reads comes from the CLI's 1.0.5 source, and no workflow has run on GitHub or exchanged a token with Entra ID. Where a step depends on how GitHub or Azure behaves at run time, I link the documentation it rests on.

From a local change to production

The APIOps pipeline for the order API: pull request checks, publish to dev on merge, approval, publish to prod

A change starts on a branch that edits something under ApimOrdersDemo/apiops/. Its pull request gets two checks. validate lints the specs, checks the policy files and compares each spec with the base branch, without signing in to Azure. dry-run signs in as id-apiops-plan, which can only read, and plans the change against dev, then against prod with the prod overrides. Both are required, so a merge waits for them. The merge lands one commit on main. publish-dev publishes that commit to dev as id-apiops-dev, then publish-prod waits in the prod environment until a reviewer approves and publishes the same commit to prod as id-apiops-prod. The nightly extract from Part 5 keeps running on the side and turns portal edits into drift pull requests, which pass through the same two checks as every other change.

The rule behind the identities: only a job running on main can reach one that writes.

The workflow is about 200 lines, most of them checkout, Node setup and CLI flags. Here is its job graph, with steps, runners, permissions and concurrency left out:

jobs:
  changes:
    if: github.event_name == 'pull_request'

  validate:      # required check, no Azure sign-in
    needs: changes
    if: needs.changes.outputs.validate == 'true'

  dry-run:       # required check, id-apiops-plan (Reader), no environment
    needs: changes
    if: needs.changes.outputs.apiops == 'true'

  publish-dev:   # id-apiops-dev
    if: github.event_name != 'pull_request'
    environment: dev

  publish-prod:  # id-apiops-prod, after a reviewer approves
    needs: publish-dev
    environment: prod
Enter fullscreen mode Exit fullscreen mode

On a pull request, the run takes changes and then validate and dry-run side by side. On a push to main or a manual run, it takes publish-dev and then publish-prod. Every job has an if: or a needs: that ties it to one of the two paths, so no run does both. The sections below show the parts of each job that matter.

Triggers: pull requests, pushes to main, and a manual full publish

on:
  # No paths filter on pull_request: a required check from a path-filtered workflow stays Pending
  # on every PR that doesn't match. The changes job decides instead, and a skipped job counts as passed.
  pull_request:
  push:
    branches: [main]
    paths: ['ApimOrdersDemo/apiops/**']
  # Full publish without --commit-id, to reconcile prod after a rejected or cancelled run.
  workflow_dispatch:
Enter fullscreen mode Exit fullscreen mode

pull_request has no paths filter, on purpose. GitHub's workflow syntax reference spells out what a path filter does to a required check: when a workflow is skipped by path filtering, its checks "will remain in a 'Pending' state", and a pull request that requires them "will be blocked from merging." The samples repo has two dozen folders next to ApimOrdersDemo. With Part 5's paths: ['ApimOrdersDemo/apiops/**'] and dry-run marked as required, every pull request that touched only a Functions demo would wait forever for a check that never starts.

So the workflow starts on every pull request and decides inside. A skipped job reports Success, even when it is a required check, which makes a job-level if: the answer: green for pull requests that don't touch apiops/, a real dry run for those that do. The changes job computes it:

  changes:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    outputs:
      apiops: ${{ steps.diff.outputs.apiops }}
      validate: ${{ steps.diff.outputs.validate }}
    steps:
      # A change to the policy script alone runs validate against the unchanged artifacts. It plans
      # nothing, so dry-run stays skipped.
      - id: diff
        run: |
          changed() { if git diff --quiet HEAD~1 HEAD -- "$@"; then echo false; else echo true; fi; }
          echo "apiops=$(changed ApimOrdersDemo/apiops)" >> "$GITHUB_OUTPUT"
          echo "validate=$(changed ApimOrdersDemo/apiops ApimOrdersDemo/scripts/check-policies.mjs)" >> "$GITHUB_OUTPUT"
Enter fullscreen mode Exit fullscreen mode

On a pull request, actions/checkout checks out the merge commit GitHub builds for it, and that commit's first parent is the tip of the base branch. HEAD~1..HEAD is everything the pull request changes. It's also the range --commit-id diffs in the dry run (Part 5: <sha>~1 against <sha>), so the two can't disagree about whether apiops/ changed, and a checkout with fetch-depth: 2 is all the history either needs. A paths-filter action from the Marketplace does the same with more options, but a git diff --quiet doesn't add a third-party action to a workflow that signs in to Azure. The price is a runner start, a few seconds, on every pull request in the repository.

There are two outputs because the two checks read different files. dry-run plans the artifacts, so apiops/ is all it cares about. validate also runs scripts/check-policies.mjs, which lives outside apiops/, and with one output a pull request that changed only the script would merge without the new script running once. That pull request now gets validate against the unchanged artifacts and a skipped dry-run, which would have nothing to plan.

push keeps its path filter. Nothing requires a check on a push, so a push that doesn't touch apiops/ can skip the whole workflow. Per-commit publishing still assumes one commit per merge: a rebase merge of three commits lands three commits on main, and the run publishes only the last one. Allow merge commits or squash merges in the repository settings and turn rebase merging off.

workflow_dispatch takes no inputs. A manual run follows the same publish-dev and publish-prod path as a push, with one difference in the publish command:

          ${{ github.event_name == 'push' && format('--commit-id {0}', github.sha) || '' }}
Enter fullscreen mode Exit fullscreen mode

On a push, the expression becomes --commit-id <sha>. On a manual run it's empty, and the CLI publishes the full tree. That's the reconcile path. When a reviewer rejects a prod approval or a waiting run gets cancelled, that commit's change never reaches prod, and the next per-commit run only publishes its own diff. A full publish PUTs every artifact (Part 5: safe to re-run, not free) and brings prod back to what main says. Start it from main: the dev and prod environments only accept the main branch, so a manual run from anywhere else fails at publish-dev.

Validation that needs no Azure

validate has no permissions block, so it gets the workflow's contents: read and no id-token. It couldn't sign in to Azure if it tried, which also makes it the one check that passes on a fork's pull request. It needs changes like dry-run does but reads its own output, so both start when apiops/ changed, validate alone starts when only the policy script did, and both count as passed when neither changed. After checkout (fetch-depth: 2, which gives it the base version of each spec) and Node setup, it runs three steps from ApimOrdersDemo/apiops, like every job that touches the artifacts. Each one reads what is inside a file, which the dry run never does: it plans calls, and a valid policy plans the same PUT as a broken one.

Linting the OpenAPI specs

      - name: Lint the OpenAPI specs
        run: npx --yes @stoplight/spectral-cli@6.17.1 lint --ruleset .spectral.yaml 'apim-artifacts/apis/*/specification.yaml'
Enter fullscreen mode Exit fullscreen mode

Spectral is pinned like the CLI, and the glob is quoted so Spectral expands it instead of the shell. The ruleset sits next to the artifacts in apiops/.spectral.yaml, so editing it is a change under apiops/ and runs the check that uses it:

extends: [[spectral:oas, recommended]]
rules:
  # API Management names each operation after its operationId, and operations/<name>/policy.xml hangs off
  # that name. Without one, the import generates a name from method and path.
  operation-operationId: error
  # Partners find the contact in the developer portal, not in the spec.
  info-contact: off
Enter fullscreen mode Exit fullscreen mode

Two edits fit spectral:oas to API Management. operation-operationId goes from warning to error, because the import names each operation after its operationId and the operation policies in this repo hang off that name: apis/orders/operations/get-order/policy.xml belongs to whatever operation is called get-order. On the sample the step reports 0 errors and 8 warnings (six missing tags, and get-order has no description in either version), and warnings don't fail it. Delete operationId: get-order from the v1 spec, and it does:

  59:9    error  operation-operationId  Operation must have "operationId".                             paths./orders/{orderId}.get
✖ 5 problems (1 error, 4 warnings, 0 infos, 0 hints)
Enter fullscreen mode Exit fullscreen mode

Gotcha: the linter has to know where the gateway enforces security. Redocly CLI 2.76.0 with its recommended config fails both specs with six errors, one security-defined per operation, because neither spec declares a security scheme. That's deliberate: the subscription key and the Entra token are checked in policy (Parts 2 and 3), not described in the spec. Turn the rule off in redocly.yaml, or declare the key header and the bearer scheme in the spec so the developer portal shows them too. Spectral's oas rules only check that a scheme an operation references exists, so they stay quiet.

Checking policies when they aren't quite XML

apis/orders/policy.xml:27: parser error : attributes construct error
      <when condition="@(context.Request.Method == "POST")">
Enter fullscreen mode Exit fullscreen mode

That's xmllint --noout on a policy API Management accepts. For XML, the quote before POST ends the attribute. API Management doesn't mind, because the CLI sends policies as rawxml, which the ARM API calls "a non XML encoded policy document". The next tool to reach for would be a schema, and there isn't one: I found no official XSD for policies on Learn, in the VS Code extension or in the policy snippets repo. So the check is a script of about 140 lines, Node built-ins only. It's JavaScript because the job already has Node for npx. The script never runs the C# in the expressions; it only finds where each one starts and ends.

      - name: Check the policies
        run: node ../scripts/check-policies.mjs apim-artifacts
Enter fullscreen mode Exit fullscreen mode

It blanks every policy expression and parses what's left as XML. It then requires one inbound, backend, outbound and on-error section in every API, operation and product policy, each with a <base /> directly inside, and compares apis/orders/policy.xml with apis/orders-v2/policy.xml byte for byte. The blanking is what makes the parse possible:

// Expressions are C#, and policies travel as rawxml, so condition="@(x == "POST")" is a valid policy
// and invalid XML. Replace each @(...) and @{...} with spaces, keeping line breaks for line numbers.
function blankExpressions(src) {
  let out = ''
  for (let i = 0; i < src.length; i++) {
    const close = src[i] === '@' && { '(': ')', '{': '}' }[src[i + 1]]
    if (!close) {
      out += src[i]
      continue
    }
    let depth = 0
    let end = -1
    for (let j = i + 1; j < src.length && end < 0; j++) {
      if (src[j] === '"' || src[j] === "'") j = skipLiteral(src, j)
      else if (src[j] === src[i + 1]) depth++
      else if (src[j] === close && --depth === 0) end = j
    }
    if (end < 0) throw new PolicyError(src, i, 'unclosed policy expression')
    out += src.slice(i, end + 1).replace(/[^\n]/g, ' ')
    i = end
  }
  return out
}
Enter fullscreen mode Exit fullscreen mode

The scan counts brackets and skips C# string and char literals, so a ) inside "..." doesn't end the expression. Spaces instead of an empty string keep every tag on its original line, which is the line the error message reports.

On the sample's six policy files it prints 6 policy files OK. On a copy with three typical mistakes, it exits 1: the get-order operation policy without <base /> in inbound, an unquoted attribute in the partner product's policy, and Part 5's Orders.Read role added to v1 only:

apis/orders/operations/get-order/policy.xml: <inbound> has no <base />, so the broader scope's inbound section never runs
products/orders-partners/policy.xml: line 4: malformed markup (unquoted attribute, or < in an attribute value?)
apis/orders/policy.xml: differs from apis/orders-v2/policy.xml; change both
Enter fullscreen mode Exit fullscreen mode

The <base /> rule is the reason the script exists. An operation policy without it skips the product rate limit, the token check and the backend route, and nothing reports an error. In this sample the call reaches the function without the managed identity token or the function key and gets a 401 from the backend instead of the gateway. The comparison is the sample-specific part: replace the pair in MUST_MATCH with yours, or delete it.

What the script can't know is which policy elements exist. <rate-limt> parses and passes, and so does a {{named-value}} nobody created. Without a schema, only API Management can reject those, and its first chance is the PUT in publish-dev.

Catching breaking changes before they become a new version

Copy the v2 spec over the v1 spec, commit, and the third step fails:

1 changes: 1 error, 0 warning, 0 info
error   [response-body-type-changed] at /head/apim-artifacts/apis/orders/specification.yaml
    in API GET /orders
        the response's body `type` changed from `array<object>` to `object` for status `200`
Enter fullscreen mode Exit fullscreen mode

That's Part 4's change: GET /orders going from a bare array to a page with items and nextLink. Part 4 made it a new version for exactly this reason, and now a check says so when someone forgets.

      # A spec the base branch doesn't have is a new API or version: nothing to break.
      - name: Breaking changes against the base branch
        run: |
          for spec in apim-artifacts/apis/*/specification.yaml; do
            if ! git cat-file -e "HEAD~1:./$spec" 2>/dev/null; then
              echo "$spec: new, skipped"
              continue
            fi
            git show "HEAD~1:./$spec" > "$RUNNER_TEMP/base.yaml"
            docker run --rm -v "$PWD:/head:ro" -v "$RUNNER_TEMP:/base:ro" tufin/oasdiff:v1.35.0 \
              breaking /base/base.yaml "/head/$spec" --fail-on ERR
          done
Enter fullscreen mode Exit fullscreen mode

oasdiff v1.35.0 runs from its Docker image, pinned to the release, and compares each spec with its version on HEAD~1, the base branch tip, the same parent changes diffs against. --fail-on ERR fails on breaking changes and prints warnings without failing. There's an oasdiff/oasdiff-action too; when its breaking action finds changes, it uploads the comparison (encrypted) to oasdiff.com unless you set review: false. Running the image directly keeps the specs on the runner.

Skipping new specs is deliberate. A breaking change to a spec that exists on the base branch fails the check, because existing callers depend on it. A spec the base branch doesn't have passes unchecked, because nobody calls it yet: apis/orders-v3/ can take any shape it likes. That's Part 4's rule as a required check: the old contract doesn't change, the new one gets a new version. When breaking v1 really is the plan, because it's past its sunset and the logs show no callers, the check stays red and someone on the ruleset's bypass list merges it, with the reason in the pull request.

The loop only looks at specs that exist on the pull request's side, so deleting a version's folder passes. Part 5 showed that the publish doesn't delete the API either, so that one is for the reviewer.

The dry run as a required check

  # Required check, next to validate. No environment, so the token subject is pull_request and the identity
  # has only Reader. With dev and prod limited to main, a PR that edits this file can't publish before review.
  dry-run:
    needs: changes
    if: needs.changes.outputs.apiops == 'true'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write
Enter fullscreen mode Exit fullscreen mode

The job names no environment, and that missing line decides which identity it can get. GitHub writes the trigger into the subject claim of the OIDC token, and for a pull request run without an environment the subject is repo:<owner>/<repo>:pull_request. The only identity whose federated credential trusts that subject is id-apiops-plan, with Reader on both instances and nothing else.

Part 5's dry run had environment: dev and the dev publisher's client ID. Its token carried environment:dev, the same subject publish-dev presents, and a pull_request run uses the workflow file from the pull request's merge commit, not the one on main. Anyone who could push a branch could open a pull request that deleted --dry-run from that step and published to dev before a reviewer had opened the diff. Without the environment, the same edit gets a Reader token, and Reader has no write actions, so Azure should refuse the PUT with a 403; I haven't run that edit against an instance. Putting environment: dev back into the pull request's copy of the workflow doesn't get around it, as long as the dev environment's deployment branch rule allows only main: a job running for refs/pull/<n>/merge can't enter it. You need both settings. The workflow file alone proves nothing, because the pull request can change it.

Reader is enough because of how 1.0.5 implements --dry-run. Before the dry-run branch returns, publish GETs the resource group and the instance (a pre-flight check) and each resource it plans to write, the last only to mark new ones as (new). Every PUT, PATCH and DELETE comes after that branch. That's from reading the CLI's publish-service.js, dry-run-reporter.js and apim-client.js in the published 1.0.5 package; no dry run has run against a live instance with a Reader token, so the first real pull request is the test.

The steps after checkout and Node setup sign in and plan twice. $APIOPS_CLI is a workflow-level variable set to @azure-tools/apiops-cli@1.0.5. The dev plan is the same command with the dev instance and without --overrides:

      - uses: azure/login@v3
        with:
          client-id: ${{ vars.APIOPS_PLAN_CLIENT_ID }}
          tenant-id: ${{ vars.AZURE_TENANT_ID }}
          subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}

      - name: Dry run against prod with the prod overrides
        run: >
          npx --yes "$APIOPS_CLI" publish
          --subscription-id ${{ vars.AZURE_SUBSCRIPTION_ID }}
          --resource-group ${{ vars.APIM_PROD_RESOURCE_GROUP }}
          --service-name ${{ vars.APIM_PROD_SERVICE_NAME }}
          --source apim-artifacts
          --overrides configuration.prod.yaml
          --commit-id ${{ github.sha }}
          --dry-run
Enter fullscreen mode Exit fullscreen mode

The prod plan is new. It runs the plan against prod with configuration.prod.yaml, the file Part 5 called the change dev never tested. An override that names a product the artifacts don't have now fails on the pull request instead of in publish-prod after someone approved it. Both plans sit in one job so the ruleset has a single check name to require.

A job without an environment only sees repository variables, so APIOPS_PLAN_CLIENT_ID and both instances' names and resource groups are repository variables. That isn't a leak, for reasons the sign-in section below covers.

Required means a branch ruleset on main (Settings, Rules, Rulesets) with "Require status checks to pass" and the job names dry-run and validate as the checks; available rules for rulesets lists the options. Add "Require a pull request before merging" in the same ruleset, or the checks only guard the people who chose to open one, and give it at least one required approval. The checks run from the pull request's copy of this file, so a pull request that sets if: false on both jobs gets two skipped checks, which count as passed. Only a reviewer reading the workflow diff stops that.

Gotcha: pull requests from forks fail the dry run. For pull_request runs from forks, the permissions key can't grant write access unless an admin turned on "Send write tokens to workflows from pull requests", so id-token: write isn't granted and azure/login has nothing to exchange. The check goes red, and it should: a fork has no business signing in to your subscription. A maintainer reads the diff, pushes the branch to the repository, and the check runs there. Don't "fix" it with pull_request_target: that event runs with the base repository's permissions, and checking out the fork's code in it hands your identity to whoever opened the pull request. I haven't watched a fork run fail on the samples repo; this follows from GitHub's documentation.

What the dry run can't tell you is whether API Management accepts the policies. It lists PUT ApiPolicy/orders whether the XML is valid or not, because the instance only parses a policy when the real PUT arrives. A policy it rejects fails in publish-dev, after the merge. The policy check in validate catches part of that before the merge; the rest is why dev always comes before prod.

Publishing to dev on merge

  publish-dev:
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest
    environment: dev
    concurrency:
      group: apiops-publish-dev
      cancel-in-progress: false
    permissions:
      contents: read
      id-token: write
Enter fullscreen mode Exit fullscreen mode

This job names environment: dev, so its token's subject is repo:<owner>/<repo>:environment:dev, and vars.AZURE_CLIENT_ID in its azure/login step resolves to the one variable the dev environment holds, the client ID of id-apiops-dev. Set the environment's deployment branches to main only. That rule is what stops a pull request from borrowing this job's identity, as described above, and it costs nothing, because nothing but main should publish to dev.

The publish step is Part 5's, with the expression from the triggers section in place of a fixed --commit-id. Part 5's prod job ran a dry run right before the publish. The plan now runs on the pull request against both instances, so the publish jobs go straight to publish. If dev fails, publish-prod never starts, because it needs: publish-dev. Exit code 1 means a partial publish; rerunning the job PUTs the same commit's artifacts again, which Part 5 showed is safe.

Dev is the first place anything validates a policy for real. API Management parses the XML on the PUT, and a policy that references a named value the instance doesn't have should fail there; Learn doesn't spell that out, and this pipeline hasn't published yet. On the order API those references (tenant-id and the three app IDs) are all Bicep-owned, so a policy that mentions a named value nobody created yet passes the dry run, passes review, and fails here, on an instance where failing is cheap.

The concurrency group is new too. Part 5's publish had none, so two merges a minute apart published to dev in parallel, and the second PUT of a shared file could land before the first. With cancel-in-progress: false, the second run waits for the first. It's not a queue, though. By default, GitHub's concurrency docs allow one running and one pending run per group, and a third arrival cancels the pending one. Three merges inside one publish and the middle commit's change never reaches dev. In dev, the next commit to the same file or a manual full publish repairs that. In prod, the same rule meets an approval that can take hours, and that's a problem for the approval gate section.

Signing in without secrets: OIDC and a user-assigned managed identity

The three APIOps identities: id-apiops-plan with Reader for the PR dry run and nightly prod extract, id-apiops-dev and id-apiops-prod with API Management Service Contributor on their instance, each trusting only its own subject

Each identity is a user-assigned managed identity with one federated credential per subject it trusts. The azure/login step trades GitHub's OIDC token for an Entra ID token for that identity, and client-id is the managed identity's client ID; there's no client secret and no creds JSON anywhere. The CLI then signs in through DefaultAzureCredential, which picks up the Azure CLI session azure/login left behind, so nothing in the npx steps knows about OIDC. Learn's OIDC page for azure/login documents both options, an app registration or a user-assigned managed identity, and the workflow is identical for either. The apiops-cli docs assume an app registration throughout.

The managed identity wins on one practical point: it's an Azure resource, created by anyone with Contributor on the resource group. App registrations are a tenant-wide object, and the tenant this sample was built in doesn't let users create them. If yours does the same, a managed identity is the only way OIDC works at all.

Creating the plan identity's pull request credential (the sample's README has the other credentials, the role assignments and the GitHub settings):

az identity federated-credential create -g <rg> --identity-name id-apiops-plan -n github-pull-request \
  --issuer https://token.actions.githubusercontent.com \
  --subject repo:<owner>/<repo>:pull_request --audiences api://AzureADTokenExchange
Enter fullscreen mode Exit fullscreen mode

The GitHub OIDC reference decides which subject a job presents, and one rule shapes the whole table: the environment wins over the branch. A job on main that names environment: dev presents environment:dev, not ref:refs/heads/main. A federated credential for the main branch would never match publish-dev, which is a common reason an OIDC login that "should work" fails with AADSTS70021. The branch subject only applies to jobs with no environment that aren't pull request runs. Microsoft Entra adds its own rules: the subject must match exactly, wildcards aren't supported, and one identity holds at most 20 federated credentials. Create them one at a time; creating several on the same identity concurrently fails with a 409, and a new credential takes a moment to propagate before the first login works.

The permissions block sits on each job that signs in, not on the workflow. id-token: write is what lets a job request the OIDC token at all, and changes has no reason to. The workflow-level default is contents: read.

The variables split along the same line. Repository variables hold what every job may know: tenant, subscription, both instances' names and resource groups, and the plan identity's client ID. Each environment holds one variable, AZURE_CLIENT_ID, and an environment variable overrides a repository variable of the same name inside that environment's jobs. None of these values is a secret, the client IDs included. A pull request can paste id-apiops-dev's client ID into a job without an environment, and Entra ID will refuse the exchange, because that identity only trusts environment:dev. The protection is the subject on the credential, not the secrecy of the ID.

The extract workflow needed only the follow-up changes: the same action versions, instance names from the repository variables, and prod-read now pointing at id-apiops-plan. That leaves one Reader identity for everything that only reads prod, the pull request plan and the nightly drift check, and two writers that only main can reach.

Prod overrides, briefly

Nothing about overrides changed since Part 5's environments section. The artifacts describe dev, configuration.prod.yaml sets subscriptionsLimit: 1 on orders-partners, and only the prod commands pass --overrides. What's different is when you find out the file is wrong. In Part 5, the first command that applied it to prod ran inside publish-prod, after the approval. Now the prod dry run reads it on every pull request that touches apiops/, so a broken override is a red check next to the diff that introduced it, and the reviewer of a change to configuration.prod.yaml sees the prod plan in the same pull request.

The prod approval gate

  # The prod environment has required reviewers, so this job waits for an approval after dev succeeded.
  publish-prod:
    needs: publish-dev
    runs-on: ubuntu-latest
    environment: prod
Enter fullscreen mode Exit fullscreen mode

The gate isn't in the YAML. environment: prod only names it; the prod environment's settings (Settings, Environments, prod) decide what it does, and a pull request can't change them. The sample uses two: required reviewers and deployment branches. GitHub's environments reference allows up to six users or teams as reviewers, and one approval releases the job. The branch rule is main only, as on dev, so the job can't run for any other ref, approval or not.

A third setting is worth turning on in a team and is off in the sample: prevent self-review. With it, "users who initiate a deployment cannot approve the deployment job, even if they are a required reviewer." On a push to main, the one who initiated the deployment is whoever merged (the push's actor), so the merger can't wave their own merge into prod; the pull request's author can, if someone else merged, and so can any other reviewer on the list. That turns "a second person approved prod" from a habit into a rule. The samples repo has one maintainer, who would have to bypass the rule on every run, and a rule that's bypassed every time records nothing.

Bypass is the fourth setting. By default, administrators can push a waiting job past its protection rules, with a comment; "Allow administrators to bypass configured protection rules" turns that off. The emergency section below is about when you'd want it.

None of this exists everywhere. On GitHub Free, Pro and Team, required reviewers and the bypass setting are only available for public repositories, and a private repository on Free gets no environments at all. The samples repo is public. On a private repository on Team, environment: prod still resolves its variables and its OIDC subject, and waits for nobody. Open the environment's settings once and check that the reviewer list is there.

A job waits up to 30 days for an approval and then fails; a rejection fails it at once. Either way, that commit's change isn't in prod, and the next merge's run only publishes its own diff. The concurrency rule from the dev section makes it worse, because a third arrival cancels the pending run. Whether a job waiting for approval holds the group or is the pending run isn't documented, and I haven't tested it, but the worst case is the same: a cancelled run whose commit never reaches prod. queue: max on the group would keep up to 100 pending runs instead; I left it out, because a line of unapproved prod runs that each publish one old diff is its own problem, and I haven't seen how it behaves behind an approval. The repair is the manual run from the triggers section: start it from main, approve it, and prod gets the full tree.

What a reviewer reads before approving is the pull request, not the approval dialog. The dialog shows the run and its commit. The plan against prod with the prod overrides is on the pull request that produced that commit, in the dry-run check, and that's the thing to open.

Gotcha: every job that names prod waits for its reviewers. A job that only reads, a job with deployment: false, a nightly schedule: required reviewers apply to all of them. That's why Part 5's extract signs in to prod through a separate prod-read environment with no reviewers and the Reader identity. Add a job that reads prod, a smoke test or a report, and give it prod-read, or it waits for an approval every time it runs.

When production is on fire

Part 5 started with an incident: the biggest partner gets 429s, and the fix is 60 to 300 in products/orders-partners/policy.xml. There are four ways to get that into prod, best first.

The first is a pull request. One line, on a branch, through the same pipeline: validate, dry-run, a review, a merge, publish-dev, an approval, publish-prod. The checks run without a person; what takes time is the two people, one to review and one to approve (the same one, if you left prevent self-review off). Practice that path on quiet days, so the only new thing on the bad day is the pressure.

The second is git revert of the merge commit, on a branch, through the same pull request. The per-commit publish then PUTs the files as they were before. It undoes changes, not additions: reverting a commit that added an API removes its folder, and Part 5 showed that the CLI without --delete-unmatched leaves that API live in API Management.

Third, an administrator skips the gates. Someone on the ruleset's bypass list merges without waiting for the checks, and the prod job gets pushed past its reviewers with "Start all waiting jobs" and a comment. It's still the pipeline: the change is a commit, id-apiops-prod publishes it, and the run shows who bypassed and why. Decide whether you keep this before you need it. Turning bypass off means an incident waits for a reviewer. Leaving it on means an administrator can skip the gate on any ordinary Tuesday, too.

The fourth is the portal, and someone will use it. id-apiops-prod is the only identity the pipeline writes with, but people with API Management Service Contributor on prod can still click save, and the 300 is live within a minute. The pipeline can't prevent that; it notices. The next nightly extract reads prod through prod-read, finds calls="300", and puts that line into the apiops-extract/prod pull request (Part 5). That pull request also always shows the prod override, so it isn't the one to merge: copy the 300 into a branch of its own and send it through the pipeline, which publishes it to dev and then, after an approval, to prod, where it's already live. Do it within a day. Until then, two things put the 60 back: the next per-commit publish that touches that policy file, and any manual full publish. The second is the trap, because it's exactly what someone starts on the quiet morning after an incident to reconcile a rejected run.

One change end to end

Part 5's role change, carried all the way to prod. A partner wants a read-only client, so someone adds <value>Orders.Read</value> to the token check.

  1. The branch edits apis/orders/policy.xml and opens a pull request. changes sees apiops/ and starts both checks. validate fails in the policy step with differs from apis/orders-v2/policy.xml; change both: the file Part 5 found missing by reading the file list, now found without anyone reading.
  2. A second commit adds the line to apis/orders-v2/policy.xml. validate passes: the specs didn't change, so Spectral prints the same 8 warnings and oasdiff reports no changes. dry-run plans ApiPolicy/orders and ApiPolicy/orders-v2 against dev, then the same two against prod.
  3. The reviewer reads the diff and asks Part 5's question. With match="any", a token with only Orders.Read now passes the token check for POST /orders too. No check here can answer that, because it's a design decision, not a syntax error. The author adds a create-order operation policy to both versions that requires Orders.ReadWrite. validate holds the new files to a <base /> in every section, so they can't drop the API's token check by accident, and the dry run lists them as (new).
  4. The pull request is squash-merged, one commit on main. publish-dev publishes its four policy files to dev as id-apiops-dev. API Management parses them for the first time here.
  5. publish-prod waits. A second person opens the run, checks it's the commit whose prod plan they read on the pull request, and approves. id-apiops-prod publishes the same four files with the prod overrides.
  6. The next nightly prod extract finds nothing beyond the standing override line.

Every check passed, and the change still doesn't work. Orders.Read has to exist as an app role on the API's app registration, which is infra/entra.bicep, and that file only defines Orders.ReadWrite. Nobody can assign a role that doesn't exist, and with appRoleAssignmentRequired set (Part 3), Entra ID won't issue the read-only client a token for the API at all. Bicep owns that half of the change, and it has to ship first.

What each stage catches, and what it can't

What each stage catches and can't see: validate, dry-run, publish-dev, the prod approval, publish-prod and the nightly extract

The misses have one cause. Nothing in this pipeline sends a request through the gateway, so everything it can't see shows up at run time: the role nobody defined, the limit that's too tight, the response a client didn't expect. A smoke test after publish-dev would be the next gate; scripts/verify.sh in the sample is most of one, and it needs a client secret the pipeline doesn't have.

Conclusion

Part 1 put a gateway in front of one Azure Function. Six parts later, the same order API has policies at four scopes, an Entra token check, two versions, two products, and one way to change any of it: a commit that passed two checks, ran on dev, and got a second person's approval for prod. When someone uses the portal anyway, the next morning's pull request says so.

Keeping that costs something: three identities with exact-match subjects, three environments whose reviewers and branch rules live outside the YAML, a ruleset, a policy script with a list of files that must match, and pinned versions that each need a bump eventually. For an API with one policy and one caller, it's too much. For an API that partners depend on, it's what lets you answer "who changed the rate limit, and who approved it" from Git instead of from memory.

Does a pull request in your API repo need a passing dry run before it can merge: yes or no?

Top comments (0)