Prateek Singh for Cloudeval AI

Posted on Oct 13 Originally published at cloudeval.ai

GitHub Actions architecture review gate for Azure pull requests

#github #githubactions #devops #azure

TL;DR: An architecture review gate is a GitHub Actions job that checks every pull request against thresholds you set for Well-Architected score, high-risk findings, validation and monthly cost. Run ganakailabs/cloudeval-action with mode: review, start with enforcement: comment_only, then switch to block_pull_request and make the job a required status check.

By Prateek Singh, founder of Ganak AI Labs. Updated 13 October 2026.

Your infrastructure pipeline is green. That still doesn't tell you whether the change is a good idea.

Does it push the bill past budget? Does it open a high-risk exposure? Does it drag the project's Well-Architected score down?

Usually a busy reviewer answers those questions, if they have time. This guide turns them into a CI step that runs on every pull request, posts one comment and fails when your thresholds aren't met.

Disclosure: I'm the founder of Cloudeval.

The example uses ganakailabs/cloudeval-action. The rollout pattern (comment first, block later, require the check) works with any review tool.

What will you build?

A linked repository that syncs every pushed commit into a review project.
A workflow that reviews each pull request and posts one result comment.
Gates for Well-Architected score, high-risk findings, validation and monthly cost.
A downloadable review artifact for every run, pass or fail.
A required status check that blocks merges once you trust the thresholds.

Which terms do you need to know?

Term	Meaning
GitHub Actions	GitHub's built-in CI/CD. A workflow job can run on every pull request.
Required status check	A branch protection or ruleset setting that blocks a merge until a named job passes.
Well-Architected score	A project score against Azure Well-Architected Framework pillars such as security and reliability.

Here's the whole flow in under a minute:

How does an architecture review gate work in GitHub?

Two separate GitHub integrations do the work. Mixing them up is the most common setup mistake.

Piece	What it does	Where it runs
Cloudeval GitHub App	Imports a branch into a project, records the commit SHA and refreshes on push	Cloudeval
GitHub Action	Runs the CI review with a scoped access key, applies gates, posts the comment and uploads artifacts	Your runner

On each pull request push, the sequence is:

The GitHub App syncs the commit.
The project refreshes its graph, cost, Well-Architected and validation results.
The workflow runs cloudeval-action in mode: review and waits for that commit.
The gates run, then the action posts the PR comment and uploads artifacts.

Coverage is Azure, plus AWS CloudFormation (beta). On Azure it reads ARM JSON (including nested templates through templateLink) and Bicep compiled to ARM JSON. CloudFormation JSON or YAML is reviewed as a static beta.

Tip: If you write Bicep, commit the compiled main.json and fail CI when it's stale. The Bicep pre-deployment review answer shows the full pattern.

How do I add an architecture review gate to GitHub pull requests?

Link the repository with the Cloudeval GitHub App.
Create a project-scoped CI access key and add it as repository secrets.
Add a workflow that runs ganakailabs/cloudeval-action in mode: review.
Define gates in .cloudeval/config.yaml, starting with comment_only.
Read the PR comment and keep the review artifact.
Switch to block_pull_request and make the job a required status check.

Step 1: Link the repository

In Cloudeval, create an Infrastructure-as-code project from GitHub. Install the Cloudeval GitHub App on the repo and pick a branch.

If your templates live in a subfolder, set a source root such as infra. Then infra/azuredeploy.json appears as azuredeploy.json in the project.

Put .cloudeval/config.yaml in the source root. This names the entry template:

version: 1
stacks:
  - id: primary-architecture
    name: Primary architecture
    entry: azuredeploy.json
    parameters: azuredeploy.parameters.json
resolve:
  linked_templates: true   # follow relative ARM templateLink files

After import, the project should show the repo, branch, commit SHA and source root. Fix that first if it doesn't.

After import, files, validation, cost and Well-Architected status share one project.

Step 2: Create a CI access key and add secrets

In the app, open Developer, then API & CLI access keys. Create a GitHub Actions CI key scoped to that one project. Include:

Project and report review access (part of the template).
review:summary, if you want the AI-written summary in the comment.
github:comment, if the comment should come from the Cloudeval GitHub App. Without it, the action falls back to github-actions[bot].

Or create one from your terminal and store it in GitHub with this script:

cloudeval credentials create \
  --name "GitHub Actions CI - owner/repo" \
  --template ci \
  --project "$CLOUDEVAL_PROJECT_ID" \
  --expires 90d \
  --show-secret \
  --format json

gh secret set CLOUDEVAL_ACCESS_KEY --repo owner/repo
gh secret set CLOUDEVAL_PROJECT_ID --repo owner/repo

Warning: Run that on your own machine, never in CI logs.

Keys expire and aren't renewed in place, so put the expiry date on the team calendar.

Step 3: Add the workflow

Create .github/workflows/cloudeval-review.yml with this content:

name: Cloudeval review
on:
  pull_request:

permissions:
  contents: read
  pull-requests: write   # PR comment
  issues: write          # PR lifecycle reactions and the bot-comment fallback

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: ganakailabs/cloudeval-action@v1   # or pin a full commit SHA
        with:
          access_key: ${{ secrets.CLOUDEVAL_ACCESS_KEY }}
          project_id: ${{ secrets.CLOUDEVAL_PROJECT_ID }}
          mode: review
          post_pr_comment: true
          upload_artifacts: true

Note: Set those three inputs explicitly. By default mode is ask, and post_pr_comment and upload_artifacts are false. Leave them out and the workflow runs but gates nothing.

The review waits for the GitHub sync of that commit, then writes review.json and review.md. It reviews pushed commits only. If an earlier step changes the working tree, it stops with this message:

Reviews pushed commits only. Add --ignore-dirty to review HEAD anyway.

So put steps that generate files, like az bicep build, in a separate job.

Two optional inputs are worth knowing:

ai_summary: "false" gives you gate and report data with no model-written prose.
sarif: true writes review.sarif.json from source-mapped findings, ready for GitHub's SARIF upload action.

The optional AI summary. Turn it off with ai_summary: "false".

Step 4: Define the gates, in comment-only mode first

Gates live in .cloudeval/config.yaml under ci.gates. Without that block, review mode warns instead of failing, so a fresh setup never blocks anyone by surprise.

Once the block exists, keys you leave out take defaults: a minimum Well-Architected score of 80, with the high-risk and validation checks on. Set each one on purpose.

This starter block publishes results without failing anything yet:

ci:
  gates:
    enforcement: comment_only          # publish results, don't fail the job (yet)

    minimum_well_architected_score: 85 # overall, out of 100
    minimum_pillar_score: 80           # default floor for every pillar
    pillars:
      security: 90
      reliability: 85

    fail_when_high_risk_findings_exist: true
    fail_when_validation_fails: true   # policy checks or unit tests
    max_monthly_cost_usd: 500          # omit to report cost without gating it

  review:
    outputs:
      pdf:
        enabled: true                  # attach review/review.pdf to the artifact
        report_type: all
        verbosity: evidence
        fail_on_error: false

Tip: Don't copy these numbers blindly. Run a week of PRs in comment_only, see where your projects score, and set thresholds just below that.

A gate that fails on day one gets disabled on day two.

Step 5: Read the PR comment and keep the evidence

With post_pr_comment: true, the action reacts when the review starts and finishes. It keeps one result comment per run instead of adding one on every push.

A typical comment starts like this:

Overall: PASS
Well-Architected posture: 23.1/100 (critical)
Validation: 3 unit tests failed
Policy checks: good
Cost: 143.81 USD/mo (under 100K budget)

Read it as two signals. Overall is your gate: did the PR meet your thresholds? The posture line is what the review observed.

A PR can pass a loose gate and still show critical posture. That isn't a bug. It means your thresholds are looser than your architecture deserves.

The gate result sits above what the review observed.

Below that sit the source details and expandable sections for pillar scores, cost by service, test failures and the AI summary.

Pillar scores and the monthly cost breakdown expand in place.

With upload_artifacts: true, each run uploads a cloudeval-output artifact, kept for 14 days by default (artifact_retention_days). It holds the review JSON, Markdown and, if enabled, PDF.

It uploads even when the gate fails, which is exactly when you need it.

The run summary carries the result and a downloadable artifact.

Step 6: Turn it into a real gate

Once the thresholds stop surprising people:

Change enforcement: comment_only to enforcement: block_pull_request.
In branch protection or a ruleset, mark the review job's check as required.

Important: A workflow can't stop someone clicking Approve. The required status check is what blocks the merge.

Should you use comment_only or block_pull_request?

	`comment_only`	`block_pull_request`
Job result when a gate fails	Passes, results are published	Fails
Blocks the merge	No	Yes, once the job is a required status check
Best for	The first week, while you calibrate thresholds	Steady state, once thresholds stop surprising people
PR comment and artifact	Yes	Yes

Why is my review gate failing?

Match the symptom to the fix:

Symptom	Likely cause	Fix
`credential_expired`	The access key expired	Create a new GitHub Actions CI key and replace `CLOUDEVAL_ACCESS_KEY`
"Missing project" style failure	`CLOUDEVAL_PROJECT_ID` not set	Add the secret and pass `project_id`
Review can't match repo or ref	Project not linked to this repo	Reconnect through GitHub repository sync
Stops with "Reviews pushed commits only"	An earlier step changed files	Move generation to another job, or set `ignore_dirty: "true"`
Comment is from `github-actions[bot]`	Key lacks `github:comment`, or the project isn't App-linked	Add the capability or relink

What should you check before you enforce the gate?

Copy this checklist into the rollout PR:

- [ ] GitHub App installed, project shows repo, branch, commit SHA, source root
- [ ] .cloudeval/config.yaml has the correct stack entry (relative to the source root)
- [ ] GitHub Actions CI key scoped to one project, expiry on the team calendar
- [ ] Workflow sets mode: review, post_pr_comment: true, upload_artifacts: true explicitly
- [ ] One week in comment_only, thresholds set from observed scores
- [ ] enforcement: block_pull_request and a required status check in branch protection

Note: PR reviews refresh reports and, by default, generate an AI summary. Both use Cloudeval credits, so budget for them on busy repos.

Frequently asked questions

How do I review infrastructure changes in a GitHub pull request?

Run a review job on the pull_request event and post the result to the PR. With Cloudeval, the GitHub App syncs the commit, and ganakailabs/cloudeval-action in mode: review applies your gates and posts one result comment.

How do I make a GitHub Actions job block a merge?

Make the job fail when the gate isn't met (enforcement: block_pull_request), then mark it as a required status check in branch protection or a ruleset.

Does the review gate work with Bicep?

Yes, through compiled ARM JSON. Commit the main.json that az bicep build produces and fail CI when it's stale. Cloudeval doesn't parse .bicep files directly.

Why is the PR comment posted by github-actions[bot]?

The comment uses the Cloudeval GitHub App identity only when the project is linked to the same repo, the key has github:comment, and the job has pull-requests: write and issues: write. Otherwise it falls back to github-actions[bot].

Does it comment on pull requests from forks?

Often not. GitHub usually gives fork PRs a read-only workflow token, so the comment is skipped. That's GitHub's restriction, not the action's.

What is the takeaway?

A review gate is a policy decision more than a tooling one. Decide what "safe to merge" means, write it down as thresholds, and publish results before you enforce them.

Want to see it first? The public Azure ARM review example has demo PRs that pass, fail on cost and fail on risk.

For the overview, read Cloud infrastructure review in GitHub pull requests. The full input reference is in the GitHub Actions guide.

Related on cloudeval.ai

Next step: Ready to gate your own pull requests? Open the Cloudeval app and follow the GitHub Actions guide.

About the author

Prateek Singh is the founder of Ganak AI Labs and builds Cloudeval AI.

cloudeval.ai | Docs | YouTube | GitHub | LinkedIn

DEV Community