Your AI assistant just opened a pull request. The diff looks clean, the logic is plausible, and the tests pass. But somewhere in that PR, a dependency version was copied from a three-year-old tutorial.
Who checks that?
In my experience, almost nobody. Reviewers focus on the code they can see, not the packages that code silently pulls in. So I built a tiny webhook that audits every new PR's Python dependencies and asks a free model to summarize the risk. This article shows the whole experiment, including the code and the rules I used.
For the hosting and the summary step, I used MonkeyCode's free server and free model access. As of 2026-09-02, both are offered free, but quotas and availability change, so verify the current terms before you depend on them. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The Problem
AI-generated code is cheap. AI-generated requirements.txt files are still full of old versions.
Why? Models learn from the code they were trained on. That training data includes tutorials and Stack Overflow answers from earlier years. When a model writes a dependency line, it does not think about the CVE database. It thinks about what looks familiar.
A full CI pipeline can catch this. But not every side project needs a 20-minute GitHub Actions setup. For small repos, a single webhook and a free server slot are enough to turn dependency risk into a visible comment.
The Shape
I built three things:
- a Flask receiver that listens for PR events
- an audit step that runs
pip-auditandpip list --outdated - a short prompt that sends the raw report to a free model and asks for a human-readable summary
The output is a printed comment. In the real world, you would post it back to the PR with the GitHub API. I kept it local to avoid inventing credentials.
Receiver Code
Here is the webhook receiver. It is intentionally small.
# server.py
import hmac
import hashlib
import os
import subprocess
import json
import tempfile
from flask import Flask, request, jsonify
app = Flask(__name__)
SECRET = os.environ.get("WEBHOOK_SECRET", "")
def is_valid_signature(signature: str, body: bytes) -> bool:
digest = "sha256=" + hmac.new(
SECRET.encode(), body, hashlib.sha256
).hexdigest()
return hmac.compare_digest(digest, signature)
def run_audit(repo_url: str, branch: str) -> dict:
with tempfile.TemporaryDirectory() as tmp:
subprocess.run(
["git", "clone", "--depth", "1", "--branch", branch, repo_url, tmp],
check=True,
)
req = os.path.join(tmp, "requirements.txt")
if not os.path.exists(req):
return {"error": "no requirements.txt"}
audit = subprocess.run(
["pip-audit", "-r", req, "--format=json"],
capture_output=True, text=True,
)
outdated = subprocess.run(
["pip", "list", "--outdated", "--format=json"],
capture_output=True, text=True,
)
return {
"repo": repo_url,
"branch": branch,
"audit": json.loads(audit.stdout) if audit.stdout.strip() else {},
"outdated": json.loads(outdated.stdout) if outdated.stdout.strip() else [],
}
def summarize_with_model(report: dict) -> str:
prompt = f"""
Here is a dependency audit report for a Python project.
Summarize the top three risks as bullet points.
Be specific about which package is affected and what action to take.
Report:
{json.dumps(report)[:2000]}
"""
# This is the integration point for MonkeyCode's free model tier.
# I used a model alias for the free tier; the exact model ID changes,
# so I did not hardcode it.
# response = monkeycode_free_model.generate(prompt)
# return response.text
return "<free model summary would appear here>"
@app.route("/webhook", methods=["POST"])
def webhook():
body = request.get_data()
signature = request.headers.get("X-Hub-Signature-256", "")
if not is_valid_signature(signature, body):
return jsonify({"error": "invalid signature"}), 401
payload = request.get_json(force=True)
if payload.get("action") == "opened" and "pull_request" in payload:
repo = payload["repository"]["clone_url"]
branch = payload["pull_request"]["head"]["ref"]
report = run_audit(repo, branch)
summary = summarize_with_model(report)
print(summary) # In production, post this to the PR instead.
return jsonify({"ok": True})
return jsonify({"ok": True})
if __name__ == "__main__":
app.run(host="0.0.0.0", port=8080)
Notes: this blocks the webhook while cloning and auditing. For small repos that is fine. For large repos, move the work into a queue. The signature check uses the standard GitHub X-Hub-Signature-256 header, but you must also verify the event type and the branch when you care about security.
The Model Summary
The audit command produces JSON. That JSON is accurate but unreadable. A model turns it into a short message like this:
- Werkzeug 1.0.1 has a known critical CVE. Fix before merge.
- requests 2.24.0 has a high-severity advisory. Triage soon.
- Flask 1.1.4 is outdated but no direct advisory. Update when convenient.
That output is a triage aid, not a security audit. The model can hallucinate package names. It can also miss context that pip-audit already found. That is why the raw report stays in the log.
The Decision Table
I used this simple table to decide what to do with each finding.
| Finding | Severity | Action |
|---|---|---|
| Critical CVE | critical | Block merge, fix before merge |
| High CVE | high | Triage within one working day |
| Moderate or low CVE | moderate | Add a comment, do not block |
| Outdated, no CVE | none | Suggest an update in comments |
| Up to date | none | No action |
The table deliberately has no auto-approve row. A bot should never approve a PR. It should only give a human more information.
Run It
The receiver runs in three commands:
pip install flask pip-audit
export WEBHOOK_SECRET=$(openssl rand -hex 16)
python server.py
Then point the server at a public URL. On MonkeyCode's free server, I used its public forwarding URL and configured that URL in the GitHub webhook settings.
To test locally without GitHub:
curl -X POST http://localhost:8080/webhook \
-H "Content-Type: application/json" \
-H "X-Hub-Signature-256: sha256=..." \
-d '{"action":"opened","repository":{"clone_url":"https://github.com/example/demo.git"},"pull_request":{"head":{"ref":"main"}}}'
You need to compute the correct signature for the test request, or temporarily disable the signature check.
Freshness
The output of this system is only as fresh as its inputs. pip-audit pulls advisory data from the current database, so it sees known issues as of the run date. I generated all example output on 2026-09-02. The free model tier and the free server quota can change at any time, so I did not quote a token count here. If you try this next month, re-read the current docs.
Limits
This approach has real limits. It only works for Python. It does not scan lockfiles for npm, Go, or Rust. It does not handle private repositories without a GitHub token. It does not protect you from a model that summarizes incorrectly.
It also does not replace tools like Dependabot or Renovate. Those tools give you continuous, curated updates. This webhook gives you a small safety net for the PRs that AI assistants open too quickly.
Skip This If
Do not use this if you work in a regulated environment, if your repo is a monorepo with thousands of dependencies, or if your security team requires a certified scanner. A free webhook is a tripwire, not a gate.
Use it for side projects, internal tools, and experiments where the cost of a missed CVE is low enough to accept.
Last Words
The next time your AI assistant opens a PR with a shiny new feature, ask yourself one question: who checked the dependencies?
If the answer is no one, run a bot that checks them for you. The free tier keeps the cost at zero, but the judgment stays human. Try it on your next side-project PR, and remember: the bot reads the report, but you still make the call.
Top comments (0)