Your First PR Needs Two Test Runs, Not One

Your first PR will fail tests you did not break. Most newcomers cannot tell their breakage from pre-existing debt. The fix is boring: record a baseline before you edit, then diff against it.

This drill takes about 40 minutes on a repo whose suite runs in under ten. You need git, a clean checkout, and one throwaway machine. You do not need permission to touch CI.

Why pass/fail counts mislead a newcomer

A red suite tells you nothing about ownership. The count blends your change with old debt, flaky tests, and services that never started. Two runs of the same commit can also disagree.

So stop counting failures. Compare sets of normalized failure lines instead. That comparison answers one question: did my edit create this?

Step 1: Freeze a baseline commit

Check out the commit your branch starts from. Never benchmark on your dirty working tree. A stale virtualenv or a half-installed dependency will manufacture failures that belong to nobody.

git fetch origin
git switch -c probe/baseline <base-sha>
git status --short   # must be empty

If git status prints anything, stash it before you continue. A clean tree is the whole point of a baseline.

This is also where a remote sandbox earns its keep. Your laptop has a history; a fresh box does not. MonkeyCode, an open-source coding agent project, currently offers a free server option that can hold this throwaway baseline box. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Verify current terms and quota on the project page before you rely on them.

# on the sandbox, not your laptop
git clone --depth 50 <repo-url> probe-baseline
cd probe-baseline
git checkout <base-sha>

Run the suite once before your change and once after. Same machine, same env, same commit history. That control is what makes the diff readable.

Step 2: Turn noise into fingerprints

Raw logs are unusable for comparison. Timings move, paths move, IDs move. Normalize them into a sorted set of stable lines.

Save this as fp.sh and make it executable.

#!/usr/bin/env bash
set -euo pipefail
LOG=${1:?usage: fp.sh test-output.log}
grep -E '^(FAIL|ERROR|not ok|✗)' $LOG | sed -E 's/[0-9]+([.][0-9]+)?(ms|s)//g' | sed -E 's/[[:space:]]+/ /g' | sort -u

Three things happen here. It keeps only failure-shaped lines. It strips durations and collapses whitespace. It sorts and deduplicates, so ordering cannot create fake diffs.

The pattern is runner-specific. Go's --- FAIL and pytest's FAILED need their own grep branches. Write the rule once for your repo's runner and commit the script next to the tests.

Step 3: Record the baseline set

Run the suite and capture output even when it fails. || true is not sloppiness here; a red baseline is expected input.

make test > baseline.log 2>&1 || true
./fp.sh baseline.log > fp.before
wc -l fp.before

Run it twice if you can afford the minutes. A fingerprint that appears in one run and not the other is flaky, and flaky noise will wreck your later diff.

Step 4: Make the smallest possible edit

Now change one thing. One function, one guard, one test. Not a refactor plus a fix.

git switch -c fix/one-thing
# edit
make test > after.log 2>&1 || true
./fp.sh after.log > fp.after

Small diffs keep the fingerprint delta small. If your edit touches ten files, you will not know which one moved the set.

Step 5: Read the delta, not the totals

comm -13 fp.before fp.after   # new failures: likely yours
comm -23 fp.before fp.after   # vanished failures: you fixed something
comm -12 fp.before fp.after   # shared failures: pre-existing, leave them

Sort first if your comm complains. sort -u in the script already guarantees sorted input.

Observed delta	Likely meaning	Your move
New fingerprints only	Your edit broke behavior	Fix your edit; do not fix the test
No delta, same failures	Your edit is invisible so far	Add the missing assertion or wiring
Vanished fingerprints	You changed unrelated behavior	Revert the extra change before review
New fingerprints matching a flaky test	Noise, not regression	Rerun baseline; note the flake in the PR

Write the three comm lines into the PR description. Reviewers read a delta in seconds. They will not read a 400-line log.

Step 6: Use a model for intent, not for a patch

When a fingerprint is yours, the useful question is what the test was protecting. Ask that before you ask for code. A model reading the test file plus the failing assertion can name the behavior and the files involved.

A prompt that works, kept narrow:

Here is one failing test and its assertion output.
Do not write a patch.
Answer: which behavior does this test assert,
and which source files does it exercise?

Then read those files yourself. Patch generation is the easy part and the least trustworthy. A newcomer who can explain why a test exists reviews far better than one who accepts a diff.

This is where free model access lowers the entry cost. MonkeyCode's outreach materials state a free token allowance, listed as 10M tokens as of November 2026, alongside the free server option. I have not independently verified limits, duration, or model selection. Treat those as current claims, not guarantees.

Limits you should respect

Fingerprint normalization is language-specific and needs maintenance. A suite that takes 25 minutes breaks the two-run loop. A permanently red baseline means the method cannot isolate your change, so escalate instead.

Flaky tests will still lie to you. Rerun before you believe any new fingerprint.

Do not paste proprietary code or secrets into a hosted model without written approval. If your team bans third-party tools, run the whole drill locally and skip Step 6.

Who should not use this

Skip it during a live incident. You need a revert, not a baseline. Skip it on repos with no runnable local tests; fix the test command first.

Also skip it if you are tempted to treat the delta as a substitute for reading code. The fingerprints tell you where to look. They do not tell you what the code should do.

The habit worth keeping

Two runs, one diff, one question. That habit survives your first week, your first review, and your first agent-assisted PR.

If you want to rehearse it without provisioning a machine, the sandbox route is the cheapest way to get a clean baseline box. Run the drill once on a repo you have not touched, and you will stop guessing which failures are yours.