Stop writing || true before your SARIF upload

When a scanner exits 0 for a clean scan and non-zero for findings, that convention works for a gate. It needs more care when you also want the results in GitHub Code Scanning: the upload is a separate step, and later steps are skipped after a failed step unless their if: condition opts in.

So you get this, in workflow after workflow:

- run: scanner --sarif results.sarif src/ || true
- uses: github/codeql-action/upload-sarif@v4
  with:
    sarif_file: results.sarif

This lets the upload run after findings. It also lets the upload run when the scanner itself fails.

You can instead put if: ${{ !cancelled() }} on the upload step. That lets it run after scan failure, but a scanner error still needs to be distinguished from findings before you try to upload. if: always() also runs after cancellation, which is usually unnecessary for an upload
(GitHub workflow troubleshooting).

What || true actually swallows

|| true does not mean "ignore findings". It means "ignore every non-zero exit". Findings are one reason a scanner exits non-zero. So are:

• a typo in the source path
• a config file that no longer parses after someone edited it
• a Python version the scanner does not support yet
• the scanner crashing on a file it cannot handle
• pip install having pulled a version with a different CLI

I maintain pyobfus, a Python obfuscator with a pre-flight risk scan that writes SARIF, so the reproducible check below uses its public 0.10.0 release. It uses three exit codes: 0 clean, 1 high-severity findings, 2 the scan itself failed. With a mistyped path:

$ pyobfus --check srcc/ --sarif results.sarif --offline
Error: Invalid value for '[INPUT_PATH]': Path 'srcc/' does not exist.
$ echo $?
2
$ ls results.sarif
ls: cannot access 'results.sarif': No such file or directory

Add || true and the step reports exit 0. The scan never ran, no SARIF file exists, and the step is green. The upload step then fails because the report is missing, so the red X lands one step later than the actual problem.

The quiet version is worse. Some scanners exit 0 when they had nothing to scan and still write a perfectly valid SARIF file with zero results. Bandit 1.9.4 does this with a missing path, no || true needed:

$ bandit -r nonexist/ -f sarif -o results.sarif
$ echo $?
0

A zero-result report does not prove that any source was scanned. If this report is accepted by the upload step, the absence of alerts can look like a clean codebase.

In the first case, || true discards the scanner's distinction between findings and errors. In the Bandit case, the scanner exits 0 despite the missing path. An exit code alone cannot prove that the intended files were scanned.

Keep the exit code, decide later

Record the exit code, reject tool errors, let the upload run for completed scans, and gate on findings at the end with the information you kept. Here, scanner stands in for your tool's real command.

- id: scan
  shell: bash
  run: |
    report_dir=$(mktemp -d "${RUNNER_TEMP:-/tmp}/scanner.XXXXXX")
    sarif_file="$report_dir/results.sarif"
    set +e
    scanner --sarif "$sarif_file" src/
    code=$?
    set -e
    echo "code=$code" >> "$GITHUB_OUTPUT"
    # Anything other than "clean" or "findings" is the tool failing. Fail now.
    if [ "$code" -ne 0 ] && [ "$code" -ne 1 ]; then
      echo "::error::scanner failed with exit $code"
      exit "$code"
    fi
    test -s "$sarif_file" || { echo "::error::no SARIF written"; exit 1; }
    echo "sarif_file=$sarif_file" >> "$GITHUB_OUTPUT"

- uses: github/codeql-action/upload-sarif@v4
  with:
    sarif_file: ${{ steps.scan.outputs.sarif_file }}
    category: my-scanner

- name: Gate on findings after uploading
  if: steps.scan.outputs.code == '1'
  run: exit 1

Three things happen here:

1. set +e turns off exit-on-error just for the scanner call, so the script can read $? instead of dying on it.
2. Tool errors still fail immediately, and so does a missing or empty SARIF file. A fresh directory prevents a stale report from passing the check, especially on self-hosted runners or when one job runs several scans.
3. Findings get uploaded first and fail the job afterwards, so the results are in Code Scanning whether or not the gate trips.

Check your scanner's documentation for its exit codes before copying the 0 and 1. Not every tool separates "findings" from "error". If yours uses 1 for both, a crash looks like findings; the non-empty report check only stops errors that leave no report. It won't catch the Bandit case above, where the file exists and says nothing; if your tool reports how many files it scanned, check that number is above zero too.

Reproduce all three outcomes locally

Use a temporary directory outside your project and the public release:

work=$(mktemp -d)
cd "$work"
python3 -m venv venv
venv/bin/python -m pip install 'pyobfus==0.10.0'
printf 'def total(value):\n    return value + 2\n' > clean.py
printf 'result = eval("1 + 2")\n' > high.py
for name in clean high missing; do
  report_dir=$(mktemp -d "$work/report.XXXXXX")
  if venv/bin/python -m pyobfus --check "$name.py" --offline \
      --sarif "$report_dir/results.sarif" --json; then
    code=0
  else
    code=$?
  fi
  if [ -s "$report_dir/results.sarif" ]; then report=present; else report=absent; fi
  echo "$name: exit=$code report=$report"
done

Expected: clean exits 0 with a zero-result report; high exits 1 with one finding; missing exits 2 without a report. The JSON output also reports files_scanned: 1 for both existing files; check that field when it matters whether the intended input was actually scanned. The eval call is a small synthetic risk fixture, not a recommendation for application code. I also ran the Bash scan step locally against these three cases (harness not shown): clean and high reached the upload point, only high tripped the simulated final gate, and the missing path stopped at the scan step. These checks do not exercise GitHub ingestion.

Two smaller traps nearby

Pipes. Pipe the scanner into tee to keep a log and the step's exit status becomes tee's, which is almost always 0. When you leave shell unset on Linux, GitHub documents bash -e {0}, with no pipefail (shell reference). Writing shell: bash explicitly gets you bash --noprofile --norc -eo pipefail {0}.
The example above sets it for that reason even though it doesn't pipe.

Exit-zero flags. Ruff and Bandit both have --exit-zero. It's narrower than || true, but not by as much as I expected. I assumed a broken run would exit non-zero anyway. In the versions I tested, a missing input doesn't: Ruff 0.17.1 turns a missing path into an E902 diagnostic and exits 0 under --exit-zero, and Bandit 1.9.4 exits 0 on a missing path with or without the flag. Ruff documents the distinction in its exit codes. What still fails is the hard stuff, like an invalid Ruff config (exit 2) or a Bandit -c file that isn't there. So the flag is fine for the scan step, but it doesn't replace checking the output. It also means the final gate must count findings in the report, because the exit code no longer carries that information.

If you use pyobfus

I got tired of writing the block above, so pyobfus has a composite action that does the same thing. For a completed scan, it writes SARIF and JSON before applying fail-on, and a tool error fails the step even with fail-on: never.

- id: scan
  uses: zhurong2020/pyobfus-action@v1
  with:
    source: src/
    pyobfus-version: "0.10.0"
    offline: true
    fail-on: never

- uses: github/codeql-action/upload-sarif@v4
  with:
    sarif_file: ${{ steps.scan.outputs.sarif-file }}
    category: pyobfus-preflight

- if: steps.scan.outputs.findings-high != '0'
  run: exit 1

The composite action does not upload SARIF. The workflow containing the uploader needs contents: read and security-events: write, plus actions: read for private repositories and Code Scanning availability for the repository (GitHub's upload guide).
These examples are steps inside a job after checkout and Python setup. In a production workflow, pin third-party actions to full commit SHAs. The public v1 action's CI has a job asserting that fail-on: never still fails on a bad path, which is the exact case || true gets wrong.

The action is a convenience for one tool. The pattern is the part worth keeping: let the scan step record, let the upload step upload, and make the decision last.

Have you seen a scanner report success after scanning no files? I'd be interested in the exit code and report it produced.

---

AI assisted with drafting and local command checks. The examples were verified on Linux with Python 3.12; no GitHub SARIF upload was tested.