DEV Community

Cover image for Automate Azure Load Testing Using GitHub Actions
Raphael Gab-Momoh
Raphael Gab-Momoh

Posted on Originally published at raphaelgmomoh.pages.dev

Automate Azure Load Testing Using GitHub Actions

#azureloadtesting #githubactions #cicd #performanceengineering

Why Performance Testing Belongs in the Pipeline, Not Before Release

Most teams that have load testing at all run it as a pre-release ritual: someone spins up a JMeter run a day or two before a big deployment, eyeballs the dashboard, ships. The problem isn't that the test happens — it's when. A regression introduced three sprints ago doesn't surface until that pre-release run, by which point tracing it back to the specific commit means bisecting through dozens of merges.

This guide follows the same shape as Microsoft's Automate Azure Load Testing by using GitHub Actions Applied Skills module: wire Azure Load Testing into GitHub Actions so every pull request carries its own load test result, with pass/fail criteria that gate the merge exactly like a failing unit test would.

1. Azure Load Testing, Briefly

Azure Load Testing is a fully managed load-testing service built on Apache JMeter — you either author a JMeter test plan (.jmx) for complex multi-step scenarios, or use a URL-based test for the common case of hammering a single endpoint with configurable virtual users, ramp-up time, and duration. Both approaches accept the same pass/fail criteria and both integrate identically into a pipeline.

For a CI/CD gate, the URL-based test is usually the right starting point — it's a YAML config, not a JMeter plan, so it lives comfortably next to application code and is trivial for any engineer to read and modify in a pull request.

2. Authentication: Federated Credentials, Not Secrets

Before the how, the what — three terms this section leans on:

  • Client secret — a password-like string issued to an application (here, the GitHub Actions pipeline acting as an Azure "service principal") so it can prove its identity. It's just a string, valid until it's rotated or revoked, stored wherever you put it — meaning if that storage location is ever misconfigured (printed to a log, committed by accident, exposed by a bug), whoever gets a copy can use it exactly like the real pipeline can, for as long as it's valid.

  • OIDC (OpenID Connect) — a standard way for one service to prove its identity to another without handing over a stored password. GitHub Actions runs its own OIDC "token issuer": at the start of a workflow run, GitHub can mint a short-lived, cryptographically signed token that says, in effect, "this run, from this exact repository and branch, is genuinely happening right now." That token expires in minutes and is useless outside the run that requested it — there's nothing long-lived to leak.

  • Federated credential (Workload Identity Federation) — the trust relationship that lets Azure accept GitHub's OIDC token as proof of identity, instead of a client secret. You configure it once, on the Azure side: "trust OIDC tokens from GitHub Actions, but only ones claiming to be this specific repo/branch." From then on, Azure checks each token's signature and claims against that rule and issues a normal Azure access token in exchange — no password ever changes hands, because none exists.

Put together: the pipeline needs to authenticate to Azure to provision and run the load test. The wrong way is a client secret stored in GitHub Secrets — a long-lived credential that has to be rotated, and is a real liability if a workflow file is ever misconfigured to leak it in logs. The right way is Microsoft Entra Workload ID federation: GitHub's OIDC token issuer is registered as a federated credential on an Entra app registration scoped to just the Load Testing resource. At runtime, the pipeline presents its short-lived OIDC token, Azure verifies it against that trust rule, and hands back an access token — the whole exchange happens without a secret existing anywhere to begin with.

permissions:
  id-token: write   # required for OIDC — the workflow can't request a token without this
  contents: read

steps:
  - name: Azure login (OIDC, no secrets)
    uses: azure/login@v2
    with:
      client-id: ${{ vars.AZURE_CLIENT_ID }}
      tenant-id: ${{ vars.AZURE_TENANT_ID }}
      subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}
Enter fullscreen mode Exit fullscreen mode

Note these are vars, not secrets — a client ID, tenant ID, and subscription ID aren't secret values; only the trust relationship (configured once, in Azure, between the Entra app and this specific GitHub repo/branch) makes the authentication work. There's no credential in this file to leak.

Setting it up, end to end, with az and gh:

# 0. Confirm both CLIs are installed and you're logged in
gh --version || brew install gh
az --version || brew install azure-cli
gh auth login
az login

# 1. Variables for this setup
export APP_NAME="payments-api-loadtest-gha"
export RG_NAME="rg-payments-perf"
export REPO="raphgm/YOUR_REPO_NAME"
export SUBSCRIPTION_ID=$(az account show --query id -o tsv)
export TENANT_ID=$(az account show --query tenantId -o tsv)

# 2. Create the app registration + service principal
az ad app create --display-name "$APP_NAME"
export APP_ID=$(az ad app list --display-name "$APP_NAME" --query "[0].appId" -o tsv)
az ad sp create --id "$APP_ID"

# 3. Scope it to only the resource group the load test runs in — never subscription-wide
az role assignment create \
  --assignee "$APP_ID" \
  --role "Load Test Contributor" \
  --scope "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RG_NAME"

# 4. The federated credential is what makes OIDC work — this is the actual trust relationship,
#    not a secret. Match the subject to how the workflow triggers.
az ad app federated-credential create \
  --id "$APP_ID" \
  --parameters '{
    "name": "github-actions-main",
    "issuer": "https://token.actions.githubusercontent.com",
    "subject": "repo:'"$REPO"':ref:refs/heads/main",
    "audiences": ["api://AzureADTokenExchange"]
  }'

# add a second one if pull_request-triggered runs also need it
az ad app federated-credential create \
  --id "$APP_ID" \
  --parameters '{
    "name": "github-actions-pr",
    "issuer": "https://token.actions.githubusercontent.com",
    "subject": "repo:'"$REPO"':pull_request",
    "audiences": ["api://AzureADTokenExchange"]
  }'

# 5. Store the three values as repo VARIABLES (not secrets — none of these three are sensitive)
gh variable set AZURE_CLIENT_ID --repo "$REPO" --body "$APP_ID"
gh variable set AZURE_TENANT_ID --repo "$REPO" --body "$TENANT_ID"
gh variable set AZURE_SUBSCRIPTION_ID --repo "$REPO" --body "$SUBSCRIPTION_ID"
Enter fullscreen mode Exit fullscreen mode

2b. The Alternative: Client-Secret Authentication

Not every setup can use OIDC on day one — some teams already have an existing service principal, or a compliance process built around a stored credential. azure/login@v1 (the predecessor to the v2 OIDC-first version) authenticates with a single JSON blob instead:

{
  "clientId": "...",
  "clientSecret": "...",
  "subscriptionId": "...",
  "tenantId": "..."
}
Enter fullscreen mode Exit fullscreen mode

Generating it, in one command:

az ad sp create-for-rbac \
  --name "payments-api-loadtest-gha" \
  --role "Load Test Contributor" \
  --scopes "/subscriptions/$(az account show --query id -o tsv)/resourceGroups/rg-payments-perf" \
  --sdk-auth
Enter fullscreen mode Exit fullscreen mode

--sdk-auth is what produces exactly that JSON shape. The clientSecret in the output is shown once — Azure doesn't store it in retrievable form, so copy it immediately or you'll have to reset it.

If the app registration already exists (say, from the OIDC setup above) and you just need to add a secret credential to it without disturbing the federated credential:

export APP_ID="<existing clientId>"
NEW_SECRET=$(az ad app credential reset --id "$APP_ID" --append --query password -o tsv)
Enter fullscreen mode Exit fullscreen mode

--append adds a new secret alongside any existing federated credential rather than replacing it — useful if you're running both auth methods side by side during a migration.

Storing it as a GitHub *secret* (unlike the OIDC values, this one is genuinely sensitive):

AZURE_CREDENTIALS=$(az ad sp create-for-rbac \
  --name "payments-api-loadtest-gha" \
  --role "Load Test Contributor" \
  --scopes "/subscriptions/$(az account show --query id -o tsv)/resourceGroups/rg-payments-perf" \
  --sdk-auth)

echo "$AZURE_CREDENTIALS" | gh secret set AZURE_CREDENTIALS --repo raphgm/YOUR_REPO_NAME
gh secret list --repo raphgm/YOUR_REPO_NAME   # confirms it exists; value itself is never shown back
Enter fullscreen mode Exit fullscreen mode 
- name: Azure login (client secret)
  uses: azure/login@v1
  with:
    creds: ${{ secrets.AZURE_CREDENTIALS }}
Enter fullscreen mode Exit fullscreen mode

The tradeoff, plainly: this works, and it's still common in the wild — but it's exactly the long-lived, rotatable, leak-if-a-log-is-misconfigured credential the OIDC section above was built to eliminate. Reach for it when a specific tool or compliance requirement demands the JSON-blob format; default to OIDC otherwise.

3. The Test Configuration 

# loadtest-config.yaml
version: v0.1
testId: payments-api-pr-check
testPlan: null              # omit for a URL-based test
engineInstances: 1
failureCriteria:
  - avg(response_time_ms) > 800
  - percentage(error) > 1
  - avg(latency) > 500
requestUrls:
  - url: https://payments-api-staging.example.com/health
    method: GET
    headers:
      Content-Type: application/json
Enter fullscreen mode Exit fullscreen mode

failureCriteria is the entire point — these thresholds are what turn a load test from "a graph someone looks at" into "an automated gate." Set them from actual SLOs, not guesses: if the payment API's real production SLO is p95 < 800ms, that's the number that belongs here, not an arbitrary round figure.

4. The GitHub Actions Workflow 

name: PR Load Test

on:
  pull_request:
    branches: [ "main" ]

permissions:
  id-token: write
  contents: read

jobs:
  load-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Azure login (OIDC)
        uses: azure/login@v2
        with:
          client-id: ${{ vars.AZURE_CLIENT_ID }}
          tenant-id: ${{ vars.AZURE_TENANT_ID }}
          subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}

      - name: Run Azure Load Test
        uses: azure/load-testing@v1
        with:
          loadTestConfigFile: 'loadtest-config.yaml'
          loadTestResource: 'payments-api-loadtest'
          resourceGroup: 'rg-payments-perf'

      - name: Upload test results
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: load-test-results
          path: ${{ github.workspace }}/loadTest
Enter fullscreen mode Exit fullscreen mode

The official azure/load-testing Action handles provisioning the test run, polling for completion, and — critically — failing the job if the configured criteria aren't met. No custom polling script, no manual "did it pass" judgment call.

5. Wiring In Server-Side Diagnostics

A failed load test that just says "error rate exceeded 1%" isn't enough to act on. Connect the target environment's Application Insights so a failure comes with the actual server-side trace attached — slow downstream dependency calls, database connection pool exhaustion, GC pauses — in the same pull request, not a follow-up investigation days later.

az monitor app-insights component connect-webapp \
  --app payments-api-insights \
  --web-app payments-api-staging \
  --resource-group rg-payments-perf
Enter fullscreen mode Exit fullscreen mode

With this wired in, a failed avg(response_time_ms) > 800 criterion in the load test result links directly to the Application Insights trace for the slowest requests in that same time window — the engineer who introduced the regression gets the root cause, not just the symptom.

6. What This Actually Buys You

  • Regressions caught at the commit that introduced them, not days later in a pre-release scramble — the same shift-left principle that made automated unit testing standard practice now applied to performance.
  • No manual load-testing ritual to remember — the test runs on every pull request automatically, the same way linting and unit tests do.
  • Zero stored Azure credentials in the pipeline — OIDC federation means there's nothing to rotate, audit, or worry about leaking in a misconfigured log line.
  • A merge gate a reviewer can trust without personally re-running anything — a green check means the performance thresholds were actually verified against real traffic simulation, not just "someone probably tested this."

Closing Thoughts

The pattern here isn't specific to Azure Load Testing — it's the same "make the expensive-to-remember manual check into a cheap, automatic pipeline gate" idea that CI/CD applies to tests, linting, and security scanning. Performance regressions are just another category of bug; the fix is treating them like one, with a pipeline that catches them before they ship rather than after.

GitHub Repository: azure-load-testing-github-actions-lab — the full working lab: OIDC-authenticated workflow, Bicep IaC for the Load Testing resource, and a Python verification script.

Azure Load Testing · GitHub Actions · CI/CD · Performance Engineering · OIDC · Application Insights

Originally published on my portfolio.

Top comments (0)