Run a no-token CI/CD preflight before your next release

vasiliy0 — Fri, 22 May 2026 15:10:35 +0000

Most release checklists focus on the code that ships. But a lot of release pain comes from the repository around the code:

a GitHub Actions workflow still using an old action major,
test reports disappearing when CI fails,
a release job with broad write permissions,
missing SECURITY.md or CODEOWNERS,
no dependency update config,
publish workflows that are hard to audit before a tag is pushed.

These are usually small fixes, but they are easiest to catch before a repo is public, before a package release, or before a CI change becomes urgent.

That is the use case for Repository Hygiene / CI Risk Preflight: a no-token GitHub Action and local CLI for repository hygiene and CI/CD release-readiness signals.

It reads files from a checked-out repository and produces a report. It does not call the GitHub API, require a token, upload source code, or need a SaaS account.

What it checks

The scanner is intentionally conservative. It looks for practical signals that maintainers often review by hand:

CI deprecations: old GitHub Actions majors such as actions/upload-artifact@v3, actions/download-artifact@v3, actions/cache@v3, actions/checkout@v3, and actions/setup-node@v3.
CI runtime risk: local JavaScript actions using obsolete Node runtimes.
Workflow permissions: broad write-all, contents: write, and pull_request_target review prompts.
Repo hygiene: missing CODEOWNERS, SECURITY.md, CONTRIBUTING.md, changelog/release history.
Dependency hygiene: missing Dependabot/Renovate config.
CI observability: missing report artifacts or uploads that do not run with if: always().
Release safety: publish/release workflows without visible guardrails.
CI cost/safety: jobs without visible timeout-minutes.

It is not a compliance scanner and it is not a replacement for security review. The goal is a fast preflight report you can run before a release or use as a report-only CI job.

Use it from GitHub Actions

name: repo-hygiene-preflight
on:
  pull_request:
  workflow_dispatch:
permissions:
  contents: read
jobs:
  hygiene:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: vasiliy0/repo-hygiene-ci-risk-preflight@v0.1.1
        with:
          format: markdown
          output: repo-hygiene-report.md

For gradual rollout, keep it report-only at first. After the team agrees on the baseline, turn on a gate only for findings that should block a merge or release.

Marketplace listing: https://github.com/marketplace/actions/repository-hygiene-ci-risk-preflight

Or run it locally

python3 -m pip install repo-hygiene-ci-risk-preflight
repo-hygiene-preflight . --format markdown

Generate JSON for automation:

repo-hygiene-preflight . --format json --output repo-hygiene-report.json

Use a severity gate after reviewing the first report:

repo-hygiene-preflight . --fail-on-severity high

Example findings

A report may include findings like:

- high workflow-write-all-permissions
  Why: Broad workflow permissions increase blast radius if a workflow is abused.
  Fix: Prefer least-privilege permissions at workflow/job scope.

- medium missing-security-policy
  Why: Public repos should tell users how to report vulnerabilities.
  Fix: Add SECURITY.md with supported versions and a private reporting path.

- medium missing-ci-report-artifact
  Why: Test-heavy CI is harder to debug when reports disappear after failed runs.
  Fix: Upload test reports/logs with actions/upload-artifact@v4 or write a concise step summary.

Baselines for existing repos

Existing repositories often have known issues that should not block every PR immediately. The scanner supports config and baselines so you can adopt it incrementally:

repo-hygiene-preflight . --format json --output report.json --write-baseline repo-hygiene-baseline.json
repo-hygiene-preflight . --baseline repo-hygiene-baseline.json --fail-on-severity high

Example config:

{
  "ignore_rules": ["workflow-without-timeout"],
  "ignore_paths": ["docs/generated/**"],
  "severity_overrides": {"missing-contributing": "info"},
  "baseline_fingerprints": []
}

When this is useful

This kind of preflight is useful before:

publishing a new open-source repo,
cutting a package release,
adding CI as a required branch check,
migrating old GitHub Actions workflows,
preparing a repository for external contributors,
reviewing release workflows that publish packages, containers, docs, or artifacts.

The best first run is usually report-only. If the report finds nothing, great. If it finds issues, most are small checklist items that are easier to fix before they become release blockers.

Triage Playwright flakes from CI logs before opening traces

vasiliy0 — Wed, 13 May 2026 15:54:12 +0000

Flaky Playwright tests usually do not start as a clean debugging session. They start as a red CI job, a rerun button, and a long trace or log that someone has to interpret under time pressure.

I built Playwright Flake Triage Toolkit as a small local CLI for the first pass: scan Playwright JSON reports, JUnit XML, and CI logs, then produce a Markdown or JSON checklist of likely causes.

GitHub: https://github.com/vasiliy0/playwright-flake-triage

PyPI: https://pypi.org/project/playwright-flake-triage/

What it tries to answer

Instead of replacing the Playwright trace viewer, the tool answers a narrower question:

What kind of flake is this likely to be, and what should I check first?

Current categories include:

ambiguous/brittle selectors
auth/session state mismatch
timeout or readiness instability
network/backend dependency flakes
browser/context/page lifecycle races
navigation/frame detachment races
visual snapshot instability
parallel/shared-state collisions
repeated failure fingerprints across retries/log files

Example

pip install playwright-flake-triage
pw-flake-triage playwright-report.json junit.xml ci.log

For CI usage, the tool can write a GitHub Actions step summary:

pw-flake-triage test-results/ --github-step-summary --fail-on-severity high

It is read-only and local: no service account, no token, no upload of private logs.

Why this is useful before deep debugging

A failed Playwright trace is still the source of truth, but teams often lose time by treating every flake as a generic timeout. A first-pass classifier helps split failures into different queues:

selector work: improve locators and avoid stale element handles;
product/test-state work: verify auth, seeded data, and permissions;
infrastructure work: separate backend/network failures from browser timing;
CI policy work: fail only on selected severity or known categories.

What the tool does not do

It does not claim to prove root cause. The output is a triage checklist, not an automatic fix. It also intentionally avoids uploading logs to a hosted service.

Feedback I am looking for

If you run Playwright in CI, the most useful feedback would be:

Which failure wording is missing from the current rules?
Which categories are too broad or too noisy?
Would a CI summary / fail-on-severity mode fit your workflow?

Repo issues are the best place for examples, with sensitive logs sanitized first.

DEV Community: vasiliy0