Four supply-chain badges for a one-person open-source project, in a day

I maintain schemagate, an open-source library and MCP server that stops an AI agent from seeing database tables the user isn't allowed to read. It's a security tool, so "trust me" isn't good enough. A security team looking at it reasonably asks: is the project run properly, is the licensing clean, and is the package on PyPI really built from this repository?

There are free, public answers to all three questions. This week I went through them in one sitting. Here's what each one took, including the parts that didn't go as expected.

1. OpenSSF Best Practices: Passing

bestpractices.dev is the Linux Foundation's self-certification: about 60 criteria covering how you take bug reports, how you handle vulnerabilities, whether you test, whether you use static analysis. You sign in with GitHub and it pre-fills what it can detect (licence, HTTPS, public repo).

The honest part is that every answer needs a justification, and most of mine were links to files that already existed: SECURITY.md for private vulnerability reporting, CONTRIBUTING.md for the test policy, CodeQL for static analysis. Before answering "no open static-analysis findings", I checked the code-scanning API instead of assuming. CodeQL had zero alerts. The 19 open alerts were all Scorecard repository-settings findings, which is a different claim.

One criterion I answered "Unmet": dynamic analysis. There's no fuzzer yet. It's only suggested at this level, and a badge with an honest "no" in it is worth more than one without.

Result: passing, 100%.

2. OpenSSF Baseline Level 1

Baseline is the newer, shorter OpenSSF checklist, aimed at security controls. Almost everything was already true. The one gap was branch protection: main required CI to pass but didn't stop a direct push. Requiring a pull request (zero approvals, so a solo maintainer can still merge) and turning off the admin bypass closed it.

Result: baseline-1, 100%.

3. REUSE compliance (FSFE)

REUSE checks that every file in the repository has machine-readable copyright and licence information. That matters to anyone whose legal team scans dependencies. You don't have to add a header to every file: one REUSE.toml can annotate whole globs.

Running reuse lint was the useful part, because it made me find the third-party code I'd forgotten about:

• a bundled BLAKE2b implementation (blakejs, CC0-1.0), inlined into three files, not the one I remembered;
• a test fixture generated from the MCP registry's schema, a project that's partway through relicensing from MIT to Apache-2.0, so I declared both.

Then you register at api.reuse.software, confirm by email, and their checker runs against the public repo.

Result: compliant, 238 of 238 files.

4. SLSA Build Level 3 provenance

This is the one that answers "is the file on PyPI really built from this repo". I used the official slsa-github-generator. Its generic generator runs as an isolated reusable workflow and signs a provenance statement for whatever hashes you give it.

The design choice that matters is which hashes. I didn't rebuild the package in the provenance job. After the publish workflow finishes, a separate workflow downloads the exact wheel and sdist PyPI serves, checks them against PyPI's own sha256 digests, and attests those. The provenance then names the files people actually install.

Then I verified it the way a user would, with the official slsa-verifier, after checking the verifier binary's own hash against its published checksums. Both files from PyPI passed. A deliberately tampered file failed, and so did a wrong source repository.

The gotcha: my docs told users to verify with --source-tag v1.3.1, and that fails. A workflow triggered by workflow_run runs on the default branch, so the provenance records refs/heads/main and the commit, not the tag. --source-branch main passes, and the commit it names is the one the tag points to. If you trigger provenance after publish, test the verification command you document, not just the generation.

What I'd tell another solo maintainer

• Do them in this order. Passing makes you write the policies, Baseline makes you lock down the repository, REUSE makes you find code you didn't write, and SLSA proves the build. Each step makes the next one easier.
• Check every claim against the system, not your memory. Two of my assumptions were wrong: the third-party file count, and the verification flag.
• Answer "Unmet" when it's unmet. I'm still at 98% on OpenSSF Silver because it requires a second maintainer, and I'd rather show that than fake it.
• A security process gets used. While writing the reply to a review comment I found a real bug, handled it through a private GitHub security advisory, and shipped the fix in a release with an advisory. One practical note: an advisory's temporary private fork doesn't run your CI, so run the full suite locally before merging from it.

The badges are on the README: https://github.com/ashishsinha1602/schemagate. Everything above is reproducible with free tools, and a weekend is plenty.

Comments

[Reid Marlow]

The workflow_run ref gotcha catches almost everyone who decouples provenance from the release job. Because workflow_run executes in the context of the default branch, the runner records refs/heads/main in the predicate even if the triggering event was a release tag. If you ever need --source-tag to pass for downstream auditors, passing github.event.workflow_run.head_sha into the generator payload lets you pin the exact immutable commit, though you still lose the signed tag string unless the generator runs directly on the release: published trigger.

The note on private advisory forks skipping CI is worth highlighting too. GitHub disables Actions in advisory forks by design to keep untrusted PR code away from repo secrets, so running the full local test suite before merging the advisory PR is the only defense against shipping broken wheels on release day.

[Ashish sinha]

You've put your finger on the part I glossed over. Recording refs/heads/main isn't only cosmetic: the commit in the predicate is whatever main pointed to when the provenance job ran. It matched the release only because nothing merged in between. If a PR had landed in that window, the provenance would name a commit the wheel wasn't built from.

So I'm moving it onto the release itself: the job triggers on release: published and polls PyPI until the files appear. Then the predicate carries the tag ref and the tagged commit, and --source-tag passes as people expect. Thanks for spelling out the advisory-fork reason too. Keeping untrusted PR code away from secrets is the right default, and it means the local suite is the only gate on that path.

[DEV SUPPORTS]

You need to verify your account.

tr.ee/dev-to