FAQ: A Green Free Shell Is Not a Retry Rule

Did one free shell pass, so you widened retry?
I keep that question on a worksheet before any edit.
A green shell is a clue, not a finished rule.

The debate I am trying to slow down

Retry talk gets sloppy after a cheap reproduction.
Someone pastes a later pass and calls the flake gone.
I want a class for the failure before YAML changes.

I am not publishing a fleet study or a private metric.
I am handing you a worksheet you can run on one log.
Treat every result as local notes, not as merge approval.

Myth one: a later pass means it was a blip

The thin story

People treat one later pass as proof of a blip.
That story skips the failure class, by rush or by habit.
A blip, a dirty cache, and a bad script can all recover.

What would actually support a transient platform class?
You need the first failure text, not only the second pass.
You also need the same image, script, and inputs.

What I require instead

Would you widen retry from a story that thin?
I would not, and the worksheet exists to block me.
Corrected model: classify the miss, then pick a when list.

A pass without a class is only a second sample.
Two samples still do not author a policy commit.
If the class is unknown, the edit waits for quotes.

Myth two: the free shell is the runner

Questions the shell cannot answer

A free interactive box can feel like the job.
You see the same repo, commands, and green text.
The runner still owns image, tags, cache, and secrets.

Did you pin the digest the failed job actually pulled?
Did your shell add tools the CI image never had?
Did the cache key differ between those two attempts?

I do not assume hardware, region, duration, or permanence.
Those facts move, and this FAQ will not freeze them.
Parity is a checklist you can fail, not a mood.

Myth three: the model should choose retry.when

Draft label, not a commit

A model can read a log and sound completely sure.
Sure text in a chat is not a policy commit.
A wide retry can hide a script bug for days.

Have you watched a summary agree with your hunch?
I treat that agreement as a draft, then I discard it.
Agreement is cheap when a bad when list is not.

I use model help only to draft a bucket label.
I still map that draft onto documented when values.
If your version docs disagree, the docs win the edit.

Myth four: cheap exploration means cheap retries

Two different bills

A free shell makes a second run feel almost free.
That feeling mixes exploration cost with production damage.
They are different bills, paid by different people.

A wrong retry spends shared runner time and reviewer trust.
It can also mask a step that is not idempotent.
Cheap exploring does not make a guessed rule safe.

Would you retry a migrate because the shell felt cheap?
I hope the answer is no, and the table agrees.
Cost of curiosity is not cost of a bad merge.

Four buckets before anyone touches YAML

I split one failed job into four buckets first.
This table is a proposal, not a measured study.
Fill one row from a real log, then stop writing.

Bucket	Evidence required	Retry stance	Do not claim
Transient platform	Runner system text, timeout, or API failure lines	Narrow when list only	That every timeout is safe
Dirty cache	Outcome changes when the cache key is cleaned	Fix the key first	That retry heals a bad key
Script defect	Same miss on a clean shell and in CI	Do not add retry	That a later pass deletes it
Unknown	Missing job id, image line, or command exit	No policy change	That a summary closed the gap

Commands that collect facts, not vibes

Run these in a clone, not inside a generated summary.
They print local git facts and a retry search.
They do not call an API, and they need no token.

git rev-parse --short HEAD
git status --porcelain
test -f .gitlab-ci.yml && echo "ci file present"
grep -n "retry:" .gitlab-ci.yml || echo "no retry key found"

Save the failed job log outside the chat window.
Then point a small search at that file only.
If the search is empty, you are stuck in unknown.

JOB_LOG="${1:?pass a saved job log}"
grep -nE "ERROR:|exit code|timed out|system failure" "$JOB_LOG" | head -n 40
grep -nE "Using docker image|Pulling docker image" "$JOB_LOG" | head -n 20

Silence from either search is not a clean bill.
Silence means the bucket stays unknown for this change.
Unknown means you do not edit retry in that request.

A cache check that retry loves to skip

People blame the runner when the cache key is the bug.
I want that possibility visible before I blame the platform.
A retry will happily replay a poisoned cache for you.

grep -nE "cache:|key:|paths:" .gitlab-ci.yml
git log -1 --oneline -- .gitlab-ci.yml

Ask whether the key includes the lockfile that changed.
Ask whether two jobs share a key but not paths.
If you cannot answer, mark dirty cache or unknown.

A heuristic script you must not trust blindly

I also keep a tiny classifier next to the worksheet.
It is an unexecuted proposal with obvious over-match risk.
It must never open a merge request by itself.

#!/usr/bin/env bash
# Proposal only. Heuristic. Do not pipe this into git commit.
set -eu
log="${1:?pass a saved job log}"
if grep -E -q "system failure|stuck_or_timeout|API failure|execution took longer" "$log"; then
  echo "bucket=transient_platform_candidate"
elif grep -E -q "script failure|exit code 1" "$log"; then
  echo "bucket=script_defect_candidate"
else
  echo "bucket=unknown"
fi
echo "next=quote lines and confirm against current GitLab docs"

Why does the script print candidate instead of verdict?
Because a keyword can appear in a red herring line.
You still owe a human quote before any YAML edit.

The first matching branch wins, even when both patterns hit.
That is why the label says candidate, not verdict.
A timeout line plus a script error still needs your eyes.

Feed it a synthetic fixture before you trust the shape.
This fixture is fake text I wrote for the demo.
It is not a log from a real pipeline run.

ERROR: Job failed (system failure): execution took longer than expected
Using docker image sha256:example-not-a-real-digest

Expected output is a candidate line, plus the next hint.
If your real log prints unknown, believe that unknown.
Do not massage the regex until it flatters your plan.

A YAML shape you still have to verify

This fragment is an unexecuted example for discussion.
Copy the when names only after your docs confirm them.
I am not claiming this list stays complete forever.

lint_job:
  script:
    - sh ci/lint.sh
  retry:
    max: 2
    when:
      - runner_system_failure
      - stuck_or_timeout_failure
      - api_failure

Why is script_failure missing from that when list?
A script failure is often your bug, not the platform.
Retrying it can turn a fast red into a slow lie.

Confirm the allowed max for your version before you copy two.
Some versions cap retries below what a chat will suggest.
A copied max that the linter rejects is not a clever rule.

Set the image to the digest from the failed job.
A floating tag can change between the fail and the pass.
If you cannot pin it, stay in the unknown bucket.

Read the current retry keyword before you copy names.
Start from the GitLab CI YAML reference, then search retry.
Your version page beats this snippet if they disagree.

Check the primary YAML reference before you trust my names.
I linked the docs root so a moved anchor cannot trick you.

https://docs.gitlab.com/ci/yaml/

Where free model access and a free server fit

Here is the only role I give MonkeyCode in this worksheet.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Use its free server to rerun the failing command cleanly.

Use free model access to draft a bucket from the saved log.
I am not naming models, quotas, hardware, or time limits.
If your account screen disagrees, trust that screen over me.

The model may misread a wrapped line as a platform fault.
The free server may differ in tools, architecture, and clock.
Neither gap is fixed by a confident paragraph in chat.

So I require three saved facts before any YAML edit.
Keep the job id, the image line, and the exit.
If one of those three is missing, stay unknown.

Use this request beside the log, and strip secrets first.
A token in the prompt is a leak, not context.
I stop the draft if redaction is still incomplete.

Classify this saved job log into one bucket only.
Choose transient platform, dirty cache, script defect, or unknown.
Quote the lines you used for that bucket label.
Do not suggest a retry max or a when list.
Do not invent a missing image, job id, or exit code.

That request refuses a policy recommendation on purpose.
I want a clerk for quotes, not a release manager.
If the draft bucket says unknown, I stop the edit.

What this worksheet cannot prove

It cannot prove parity across runner tags or executors.
It cannot prove a job stays safe under concurrency.
It cannot prove a free offer remains available later.

It also cannot replace a flaky test quarantine process.
Quarantine needs a named owner and a review date.
A tidy table is not that owner, even on paper.

Do not use this flow on deploys that mutate shared state.
A retried deploy can double-write or stop halfway through.
Those jobs need idempotency tests, not a wider when list.

Who should skip it

Skip this FAQ if you cannot download the job log.
Skip it if the image floats and you refuse to pin it.
Skip it if you wanted a model to approve the merge.

Also skip it when the log holds secrets or customer data.
Do not paste tokens into a model just to gain speed.
Redaction comes before any classification prompt, every time.

A pass I can defend in review

I timebox the pass so the debate cannot sprawl.
Seven checks are enough for one failed job.
More checks usually mean I am avoiding the unknown bucket.

1. Save the failed job log outside the chat pane.
2. Run the git commands and keep their raw output.
3. Run the heuristic and treat the bucket as a candidate.
4. Fill one table row with quoted lines from that log.
5. Ask for a label only, and ban retry advice.
6. Edit YAML only for a confirmed transient platform case.
7. Link the current GitLab retry docs in the merge request.

If step four has an empty evidence cell, stop there.
Empty cells are the useful part of this worksheet.
They block a confident story you cannot quote yet.

Close the myth without a new one

A free shell can make a second run cheaper to try.
It still cannot sign a retry rule for an unmatched runner.
I would rather ship no retry than ship a guessed when.

If you already have MonkeyCode's free server, use it here.
Run this worksheet on one saved log before you edit YAML.