DEV Community

Cover image for I built a GitHub Action that scores pull requests and never auto-closes them
Solo Developer
Solo Developer

Posted on Originally published at github.com

I built a GitHub Action that scores pull requests and never auto-closes them

Maintainers are spending more and more time on low-effort pull requests: vague titles, empty descriptions, no tests, no context. I built PRFloodGuard, a GitHub Action that scores each PR and explains the score, so you can decide faster. It never closes or merges anything on its own.

How the scoring works

Every PR starts at 100. Each triggered rule subtracts a documented number of points, and the final score maps to a risk level: low, medium, high or critical. The comment on the PR lists every rule that fired, so nothing is a black box.

The rules look at the PR title, description, linked issues, commit messages, change size, whether tests or docs changed, and the author's public PR history in the repository.

Real results from my own tests

Same rules, same repository, three different pull requests.

High risk, 31/100. Title "Update", empty description, a new code file and no tests:

PRFloodGuard comment on a pull request showing High risk, score 31 out of 100, with the list of signals

Signals: generic title, weak description, no linked issue, tiny diff with a weak description, no tests changed, no docs changed, low public merge rate.

Medium risk, 59/100. Same title and empty description, but a README-only change, so the code-related rules did not fire:

PRFloodGuard comment on a pull request showing Medium risk, score 59 out of 100

Low risk, 78/100. A small typo fix with a short description:

PRFloodGuard comment on a pull request showing Low risk, score 78 out of 100

This one still lost points for the commit message, which I consider a false positive. I'm tuning that rule, and I'm showing it here on purpose so you can see how the tool behaves.

Reducing false positives

My first version was too harsh. A one-line docs typo fix got penalized as "suspicious", and a generic title plus a generic commit message stacked two big penalties. I fixed this by capping the combined title and commit penalty and by not penalizing tiny docs-only changes. It still isn't perfect, so scores are signals, not verdicts.

Install in three steps

  1. Add .github/workflows/prfloodguard.yml to your repository:

yaml
name: PRFloodGuard
on:
  pull_request_target:
    types: [opened, synchronize, reopened, ready_for_review]
permissions:
  contents: read
  pull-requests: write
  issues: write
  checks: write
jobs:
  analyze:
    runs-on: ubuntu-latest
    steps:
      - uses: solodevelopersai/PR-Flood-Guard/action@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
solo_developer_ai profile image
Solo Developer •