DEV Community

Cover image for Run UCP + ACP Validation in Your CI/CD Pipeline
Peter
Peter

Posted on

Run UCP + ACP Validation in Your CI/CD Pipeline

#devops #githubactions #ecommerce #webdev

You deploy a change to your storefront. The JSON still parses. The linter is happy. Tests pass.

Two weeks later you notice AI agents have been silently skipping your store, because a capability schema endpoint started returning a 404 and nothing anywhere told you.

This is the failure mode of agentic commerce readiness: it degrades quietly. There is no 500 page, no error rate spike, no angry customer email. Agents just move on to a merchant whose profile resolves.

The fix is the same as for every other class of quality problem in software. Stop checking it by hand. Put it in CI.

What actually breaks

UCP profiles are not simple documents. They carry namespace bindings, HTTPS requirements, signing keys, and extension chains that reference capabilities elsewhere in the file. A profile can be perfectly valid JSON and still be broken in ways that matter:

  • UCP_NS_ORIGIN_MISMATCH - your declared namespace does not match the origin serving it
  • UCP_ENDPOINT_NOT_HTTPS - agents require HTTPS, without exception
  • UCP_MISSING_SIGNING_KEYS - required before order and fulfillment capabilities will be trusted
  • UCP_SCHEMA_FETCH_FAILED - a capability schema URL returns an error
  • UCP_ORPHANED_EXTENSION - an extension points at a capability that no longer exists

Every one of these survives JSON.parse(). Every one of these makes you invisible to some agent. And the last two are exactly the kind of thing a refactor breaks without touching the profile file at all, because the breakage lives at the other end of a URL.

That is the argument for CI. Not "validate once at launch" - validate on every push, against the environment you are about to ship.

The GitHub Action

If you are on GitHub, this is four lines of YAML:

- uses: Nolpak14/ucp-validate-action@v1
  with:
    domain: 'yourstore.com'
    fail-on-grade: 'C'
Enter fullscreen mode Exit fullscreen mode

The action runs four-level validation (structural, rules, network, SDK compliance) against the live domain and scores AI readiness. fail-on-grade fails the build at that grade or worse; fail-on-score does the same against a 0-100 number. Use whichever matches how your team talks about quality.

It exposes score, grade, ucp-found, passed, and result-json as outputs, so you can branch on the result rather than just pass or fail:

- uses: Nolpak14/ucp-validate-action@v1
  id: ucp
  with:
    domain: 'yourstore.com'

- run: echo "UCP score: ${{ steps.ucp.outputs.score }}"

- if: steps.ucp.outputs.ucp-found == 'false'
  run: echo "::warning::No UCP profile at /.well-known/ucp"
Enter fullscreen mode Exit fullscreen mode

On pull requests it posts a comment with the score, a category breakdown, and any validation issues with their error codes, updating in place rather than piling up duplicates. Set comment: 'false' if you would rather have step summaries only.

The pattern that actually pays: gate the deploy

Validating on push is useful. Gating the deploy on it is what stops a broken profile reaching production:

name: Deploy
on:
  push:
    branches: [main]

jobs:
  validate-ucp:
    runs-on: ubuntu-latest
    steps:
      - uses: Nolpak14/ucp-validate-action@v1
        with:
          domain: 'staging.yourstore.com'
          fail-on-grade: 'C'

  deploy:
    needs: validate-ucp   # deploy only runs if validation passed
    runs-on: ubuntu-latest
    steps:
      - run: echo "Deploying to production..."
Enter fullscreen mode Exit fullscreen mode

Point it at staging. If staging's profile does not clear your bar, the deploy job never runs. You catch the regression before customers see it, and before agents quietly reroute around you.

Not on GitHub? Use the CLI

The validator ships as an npm package, so it drops into any runner with Node.js 20+. It exits non-zero when validation fails, which is all a CI system needs.

GitLab CI:

ucp-validation:
  image: node:20-alpine
  script:
    - npx -p @ucptools/validator ucp-validate validate --remote yourstore.com
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
Enter fullscreen mode Exit fullscreen mode

CircleCI:

jobs:
  ucp-validation:
    docker:
      - image: cimg/node:20.11
    steps:
      - run:
          name: Validate UCP profile
          command: npx -p @ucptools/validator ucp-validate validate --remote yourstore.com
Enter fullscreen mode Exit fullscreen mode

Or wire it into your test suite so it runs wherever your tests already run - Jenkins, Bitbucket Pipelines, a Makefile, a pre-commit hook:

npm install --save-dev @ucptools/validator
Enter fullscreen mode Exit fullscreen mode 
{
  "scripts": {
    "test:ucp": "ucp-validate validate --remote yourstore.com"
  }
}
Enter fullscreen mode Exit fullscreen mode

You can also validate the file before it ever ships, which is faster and catches structural errors without a network round trip:

npx -p @ucptools/validator ucp-validate validate --file ./public/.well-known/ucp
Enter fullscreen mode Exit fullscreen mode

The other protocol: ACP readiness

UCP is not the only spec agents care about. The Agentic Commerce Protocol - OpenAI and Stripe's standard, the one behind checkout inside ChatGPT - has its own readiness surface: platform detection, payment integration, policy endpoints, security posture.

Worth being precise about how this runs, because it is different: the GitHub Action and the CLI check UCP. ACP readiness is a public REST endpoint. No key, no install, so it works in any CI that has curl and jq.

curl -sX POST https://ucptools.dev/api/acp-check \
  -H 'Content-Type: application/json' \
  -d '{"domain":"yourstore.com"}' > acp.json

SCORE=$(jq -r '.score' acp.json)
GRADE=$(jq -r '.grade' acp.json)
echo "ACP readiness: $GRADE ($SCORE/100)"

[ "$SCORE" -ge 70 ] || {
  jq -r '.recommendations[]? | "[\(.priority)] \(.title) - \(.action)"' acp.json
  exit 1
}
Enter fullscreen mode Exit fullscreen mode

One trap worth knowing: the endpoint answers 200 OK even when a domain fails, with "ok": false in the body. A reflexive curl --fail gate would pass every grade, including an F. Read .score or .grade from the response, not the HTTP status.

The response also carries score_breakdown across platform, payment, policies, and security, plus recommendations as structured objects with priority, title, description, and action - which is why the snippet above prints something a developer can act on rather than a wall of JSON.

Where to set the bar

Resist the urge to start at fail-on-grade: 'A'. You will spend a week fighting your own pipeline and then disable the check, which is worse than never adding it.

Start where you are. Run the validator once, take the grade it gives you, and set the threshold one notch below. Now the build fails only on regression, which is the thing you actually care about. Ratchet the bar upward as you fix the backlog. A gate that fires on real regressions and stays quiet otherwise is a gate people keep.

For ACP, 70 is a reasonable opening threshold. For UCP, fail-on-grade: 'C' catches the profile-level breakage without blocking on Schema.org polish.

Getting started

  1. Add the action, or the npx one-liner, to a workflow you already run
  2. Point it at staging, not production
  3. Set a threshold one notch below your current grade
  4. Add the ACP curl step if you care about ChatGPT checkout
  5. Make the deploy job needs: the validation job

Copy-paste recipes for GitHub Actions, GitLab CI, CircleCI, npx, and the ACP endpoint live at ucptools.dev/ci. If you would rather call validation from your own code than from a CI step, the API and npm reference covers the public REST endpoints and the typed package.

The point is not the score. The point is that a profile which silently stopped resolving should break your build, not your revenue.

UCPtools is an independent community tool. UCP is an open standard by Google and Shopify; ACP is an open standard by OpenAI and Stripe. We are not affiliated with any of them.

Top comments (0)