Azure Repos Git does not run a pipeline on a pull request because of a pr: block in YAML. PR validation is implemented through branch policies, so a PR-Agent setup copied from the GitHub workflow does nothing until someone adds a Build Validation policy on the target branch. The PR-Agent Azure DevOps page says this outright, and Microsoft's pipeline documentation for Azure Repos Git states the same rule: to enable PR validation, configure the Build Validation policy for that branch.
That difference decides the rest of the rollout, because the trigger and the identity the agent authenticates as both branch from it, and so does whether the bot can answer a comment at all.
Build Validation replaces the PR trigger
The policy lives under Project Settings, Repositories, Branches. Select the target branch, open Branch Policies, and add the PR-Agent pipeline as a Build Validation policy marked Required. Then delete the pr: section from azure-pipelines.yml, since Azure Repos Git ignores it.
Two conditions are easy to miss. You need to be a project administrator of the project that owns the repository to configure validation builds for an Azure Repos Git repo. And draft pull requests do not trigger a pipeline even when a branch policy is configured, so a team that opens PRs as drafts will not see a review until the PR is marked ready.
There is a difference in what gets built, too. Validation pipelines run against the merge commit of the source and target branches. A push to the source branch carrying [skip ci] in a commit message suppresses the pipelines triggered by that branch, while the Build Validation pipeline still runs on the merge commit. If you use [skip ci] to control CI cost, it will not control this.
The pipeline itself is short. It runs the prebuilt pragent/pr-agent:latest image as a container with the entrypoint cleared, builds a PR URL out of the variables Azure Pipelines injects (System.CollectionUri, System.TeamProject, Build.Repository.Name, System.PullRequest.PullRequestId), sets config__git_provider=azure, and calls the CLI three times for describe, review and improve. The PAT and the model key come from a variable group, and the pipeline needs explicit permission to read that group or the job fails on an empty secret.
Pipeline mode cannot answer comments
Azure Pipelines has no support for triggering a workflow from a pull request comment, and the PR-Agent docs say so plainly. A pure pipeline setup reviews every new pull request and cannot respond to /review typed into a thread. That is the same shape as the self-managed GitLab placement, where pipeline mode fires automatically and comment commands require the webhook server instead.
On Azure Repos the webhook route is a manual service hook. You create it under Project Settings, Service hooks, with the trigger set to Pull request created for a review or Pull request commented on for a supported slash command, and API v2.0 is required for the comment event. The hook posts to your PR-Agent endpoint with a sporadic username and password pair you configure on both sides, sent as basic auth, which is why the endpoint has to be HTTPS.
So the split matches the other forges: pipeline mode when you want describe, review and improve to fire automatically with no service to run, webhook when your team drives review from comments.
PAT or DefaultAzureCredential
The Azure DevOps provider takes either a personal access token or DefaultAzureCredential. The PAT is quick to create, carries an expiration date, and API calls run under the identity of the user who created it. DefaultAzureCredential uses a managed identity or a service principal, and creates a separate Azure DevOps identity for the agent through Microsoft Entra ID.
The org value has to be set in .secrets.toml under [azure_devops] either way. The PAT, when you use one, goes in the same file. With DefaultAzureCredential you supply AZURE_CLIENT_SECRET or rely on the managed identity and the Azure CLI for local runs. The service principal path is the one that keeps the reviewer's identity separate from a person's account, which matters the month that person leaves or their token expires.
How the agent finds its own comments
Once PR-Agent has posted in a thread it answers there, and it works out which comments are its own by reading its earlier ones. A generic mention such as hi agent does not trigger a response. Before the first post on a pull request, or when the Azure DevOps identity changes, you set a stable identity in the [azure_devops_server] section using a GUID or a unique name. During an identity transition the setting accepts a list.
That section name is a trap of its own. azure_devops_server is where the agent identity and webhook credentials are configured, and it is used for the provider regardless of which edition you run, so a team on the hosted service will still find itself editing a section labelled for the server edition.
What the support matrix says works on Azure DevOps
The supported platforms matrix is worth reading before you promise anything to a team. Describe, Review, Improve, Ask, Add Docs and Help are supported on Azure DevOps. Generate Labels is applied, since the provider can set labels. Update CHANGELOG is posted as a pull request comment instead of being applied, because the provider cannot push files. Ask on code lines, Similar Issues and the tagging bot are not supported on Azure DevOps at all.
What else runs on Azure Repos
PR-Agent is not the only option, and the choice comes down to which identity the tool authenticates as and how the incoming webhook is secured.
Kodus connects to Azure DevOps through a PAT with a defined scope set: Analytics Read, Code Read & Write, Graph Read, Identities and Groups Read, Project and Team Read, User Profile Read. Code Write is the one that matters for posting review comments, since without it the connection looks healthy and no comments appear on the pull request. On a self-hosted install the Azure Repos webhook needs a signed token query parameter, an AES-256-CBC value derived from CODE_MANAGEMENT_SECRET and CODE_MANAGEMENT_WEBHOOK_TOKEN, and requests without it get a 403. That is a step GitHub, GitLab, Bitbucket and Forgejo connections do not need, and it is the kind of detail that stalls a self-hosted rollout on Azure Repos specifically. Kodus subscribes to four events: pull request created, updated, merge attempted, and commented.
| PR-Agent | Kodus | |
|---|---|---|
| Trigger | Build Validation policy, or a manual Azure service hook | Azure service hook, created by the integration or registered by hand |
| Auth | PAT, or DefaultAzureCredential via managed identity or service principal | PAT with a fixed scope list |
| Comment commands | Webhook route only, API v2.0 required | Pull request commented event |
| Self-hosted | Docker image in a pipeline or a webhook server | Self-hosted deployment supported |
| Webhook security | Basic auth username and password | Signed token query parameter, 403 without it |
The edition question decides more than the tool
None of this changes the fact that Azure DevOps Services and Azure DevOps Server are not the same target. Which AI review routes actually run on each edition is the part that decides whether a hosted reviewer is in scope at all, since vendor pages that document a PAT and a service hook against dev.azure.com are describing the hosted service. A Server team that reads those pages and expects the same setup will spend a while chasing permissions that do not exist in the same form.
The wider comparison of what each self-hosted reviewer supports across all three forges is in PR-Agent on GitLab, Azure DevOps and Bitbucket.
What the docs do not establish
Cost per pull request for the pipeline route is not stated on the Azure DevOps page, and neither is whether the webhook route works against Azure DevOps Server behind an air gap. Both questions come up during planning, and neither has a number to cite from the vendor page. For those, the docs point at the general configuration reference and at the issue tracker for comment-trigger support.
Top comments (0)