DEV Community

JinHyuk Sung
JinHyuk Sung

Posted on

I made Agent Gate installable in 30 seconds for AI PR checks

#ai #github #security #showdev

One problem with security-ish developer tools is that the install path can ask for trust before it has earned any.

Agent Gate is a GitHub Action for AI-generated pull requests. It does not review code with an LLM. It checks repeatable CI evidence such as workflow permission changes, agent control-plane drift, package lifecycle script drift, and missing test-file evidence.

The important constraint is that the Action should be safe to try first:

  • no checkout of PR code
  • no runtime LLM calls
  • no repository script execution
  • no policy loaded from the PR head branch
  • warn mode by default for first runs

I recently changed the onboarding flow so the first install is basically one pinned workflow download.

mkdir -p .github/workflows \
  && curl -fsSL https://raw.githubusercontent.com/sjh9714/Agent-Gate/v0.2.5/templates/agent-gate-observe.yml \
  -o .github/workflows/agent-gate.yml
Enter fullscreen mode Exit fullscreen mode

This downloads a tag-pinned GitHub Actions workflow YAML file. It is not curl | bash, and it does not execute a remote script.

The downloaded workflow uses:

uses: sjh9714/Agent-Gate@v0.2.5
with:
  mode: warn
  fail-on-block: false
Enter fullscreen mode Exit fullscreen mode

It also uses only:

permissions:
  contents: read
  pull-requests: read
Enter fullscreen mode Exit fullscreen mode

The other change is that first runs no longer require agent-gate.yml.

If the default config file is confirmed missing on the PR base branch, Agent Gate uses its built-in default policy and records:

configSource: default
Enter fullscreen mode Exit fullscreen mode

That means a maintainer can install the workflow first, see what the default warnings look like, and only add repo-specific policy later.

I also verified the README install path in a sandbox PR. The run:

  • loaded sjh9714/Agent-Gate@v0.2.5
  • used no actions/checkout
  • had no agent-gate.yml
  • fell back to the built-in default policy
  • finished successfully
  • produced a warn decision for package lifecycle script drift

The compact log looked like this:

Agent Gate: NEEDS HUMAN DECISION
Decision: warn
Why: preinstall script added in package.json.
Path: package.json
Policy status: warning today; eligible to become a merge gate after tuning.

- warn agf_2ac4687b2f8f712a dependency/lifecycle-script-added package.json
Enter fullscreen mode Exit fullscreen mode

This is the first-run model I want:

install quickly
observe warnings
understand the report
then tune policy
Enter fullscreen mode Exit fullscreen mode

It still does not prove semantic correctness. A deterministic CI gate should not pretend to know whether a PR is “good.” The goal is narrower: surface repeatable evidence that a maintainer can inspect before merge.

If you maintain a GitHub Actions-heavy repo or use coding agents to open PRs, I would like feedback on one thing:

Does this first-run path feel safe and clear enough to try in a real repo?

Repo:

https://github.com/sjh9714/Agent-Gate

Top comments (0)