A repository with a dozen red CI runs raises one question before any other: is this one problem or twelve? GitHub gives you two views. gh run list shows a column of statuses, and gh run view --log-failed shows the failed log of one run. Comparing the two dozen log lines that matter is left to you.
I wrote a small command line tool for that comparison. It is called gh-failmap.
Try it
pipx install git+https://github.com/Arthur031221/gh-failmap
gh-failmap --repo denoland/deno --limit 100
It uses your existing gh login. Inside a clone, gh-failmap alone reads the current repository.
What it does
For the newest failed runs it fetches the logs of the failed jobs and picks one error line from each: an explicit ##[error] annotation first, then the nearest error looking line before the failure, then the bare exit code. It removes timestamps, ANSI colour, commit hashes, addresses, durations and line numbers, and keeps file names and the message. Runs are grouped inside the same workflow and job, and a matrix suffix such as (ubuntu, 3.12) is ignored.
Each group prints a count, the first and last date, and a link to every run. In the example above, six runs of one nightly job share a single runner shutdown message, which reads very differently from six unrelated failures.
What it refuses to guess
Some runs cannot be read: startup failures, expired logs, missing permission, runs with no failed job. They are listed under a separate heading. When the same job passed in another sampled run of the same workflow and commit, the row is marked as an intermittent candidate. That is a hint, not a verdict.
What is rough
The error line is a heuristic. A test summary with several failures shows the last one, so two runs of one cause can split into two rows. Matching is on text, so the same cause worded two ways counts twice. It reads at most 15 failed runs and 5 jobs per run, and only the latest attempt of each run.
It is read only, sends logs nowhere except GitHub, and needs no change to your workflows or any API key.
Feedback
The most useful report is the two error lines that landed in the wrong group, with private parts removed. The source, tests and contribution notes are at https://github.com/Arthur031221/gh-failmap.
Top comments (0)