FAQ: Green Pytest Is Not the GitLab MR Widget

Your pytest log looks perfect. The merge request widget still looks empty. Why would GitLab ignore a green terminal?

I see this mix-up in review comments every week. People paste a local run and call the pipeline done. GitLab never promised that contract.

This FAQ is a corrected mental model. It is not a product tour. I walk a small, reproducible check you can run before you merge.

What problem are we actually solving?

GitLab shows test summaries on merge requests. It does not read your laptop. It does not read a chat transcript either.

It reads job artifacts from that pipeline. The job must upload a real JUnit file. The YAML must name the report type. Miss either piece and the widget stays blank.

So the question is not "did tests pass somewhere?" The question is "did this MR pipeline publish a report GitLab can parse?"

Myth 1: A green console is the test widget

Claim: 41 passed in 3.02s means the MR is green.

Evidence you can check: Open the merge request. Look at the tests tab. If no JUnit report arrived, GitLab has nothing to render.

Corrected model: Terminal output is for humans. The widget is for XML GitLab ingested. Those are different channels.

Ask yourself one blunt question. Which job ID uploaded report.xml?

If you cannot name the job, the widget cannot either.

Myth 2: artifacts:paths publishes the report

Claim: Listing the XML under paths is enough.

Evidence: paths only keeps files for download. The tests widget keys off artifacts:reports:junit.

Here is a proposed job, not a production template:

# example: junit report contract, not a full pipeline
unit:
  stage: test
  image: python:3.12-slim
  script:
    - pip install pytest
    - pytest --junitxml=report.xml
  artifacts:
    when: always
    paths:
      - report.xml
    reports:
      junit: report.xml
    expire_in: 7 days

See the two keys? paths is the zip. reports:junit is the widget.

Drop reports and reviewers download XML by hand. GitLab still shows no summary.

Myth 3: A free server run is the GitLab job

Claim: I ran pytest on a spare host, so the pipeline is proven.

Evidence: That host never received CI_PIPELINE_ID. It never posted artifacts to GitLab. The MR has no report attachment.

I still use a throwaway box for drafting. I do not confuse draft with ingest.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I treat MonkeyCode's free model access and free server option as a draft bench only. They help me sketch YAML and sample XML. They do not upload GitLab artifacts.

Corrected model: draft elsewhere, publish inside GitLab. The widget only trusts the latter.

Myth 4: CI Lint validates JUnit XML

Claim: The lint API said the YAML is valid, so reports will parse.

Evidence: Lint checks configuration structure. It does not open report.xml. It does not run pytest. It does not schema-check JUnit.

Proposed check against the project lint endpoint:

# example only: lint the YAML, not the XML
curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  --header "Content-Type: application/json" \
  "https://gitlab.example.com/api/v4/projects/$PROJECT_ID/ci/lint" \
  --data @- <<'JSON'
{"content": "$(sed 's/\\/\\\\/g;s/"/\\"/g;s/$/\\n/' .gitlab-ci.yml | tr -d '\n')"}
JSON

Green lint means GitLab accepted the CI file. It does not mean the report file exists. It does not mean the XML is well formed.

Myth 5: Failed jobs cannot publish reports

Claim: The job failed, so GitLab skipped the XML.

Evidence: Default artifact upload often runs on success only. Failed pytest still produced XML. You never kept it.

That is why the example uses when: always. I want the widget on red jobs too. Reviewers need the failing cases, not only the green ones.

Corrected model: failure output is the report you needed. Do not hide it behind success-only artifacts.

Myth 6: A coverage number in chat is the badge

Claim: The model printed coverage: 81%, so the MR badge is 81.

Evidence: GitLab coverage uses the job coverage regex. It scans that job's log in GitLab. A chat percentage never matches the regex.

Proposed fragment:

# example: coverage regex reads the GitLab job log
unit:
  coverage: '/TOTAL.*?([0-9]{1,3})%/'
  script:
    - pytest --cov=app --cov-report=term-missing --junitxml=report.xml

No log line, no badge. Wrong regex, no badge. Coverage in a sidebar chat still no badge.

Artifact: a report-contract checklist

I use this table before I argue about "tests passed." Treat it as a review card, not a benchmark.

Claim you hear	What GitLab actually used	Check this
Terminal said passed	Job log only	Job page, not your laptop
XML sits on disk	Uploaded artifact	Job artifacts list
paths includes XML	reports:junit	YAML under artifacts.reports
Lint is green	YAML syntax	Lint never opened XML
Chat said 81% coverage	coverage regex on job log	Regex match on GitLab log
Failed job hid results	artifacts:when	Set always if you need red reports

If any row fails, the MR widget is not in evidence. Stop quoting the chat.

Artifact: a tiny JUnit shape check

This is a proposed local checker. I have not claimed production numbers. Run it on the XML you think GitLab will ingest.

# example: validate JUnit-ish shape before you push
# python check_junit.py report.xml
from pathlib import Path
import sys
import xml.etree.ElementTree as ET

def main(path: str) -> int:
    raw = Path(path).read_bytes()
    if not raw.strip():
        print("empty report")
        return 2
    try:
        root = ET.fromstring(raw)
    except ET.ParseError as exc:
        print(f"not xml: {exc}")
        return 2

    tag = root.tag.split("}")[-1]
    if tag not in {"testsuite", "testsuites"}:
        print(f"root tag {tag!r} is not a testsuite")
        return 2

    cases = [el for el in root.iter() if el.tag.split("}")[-1] == "testcase"]
    if not cases:
        print("no testcase nodes")
        return 2

    failed = [
        el for el in cases
        if any(c.tag.split("}")[-1] in {"failure", "error"} for c in list(el))
    ]n    print(f"cases={len(cases)} failed_or_error={len(failed)} root={tag}")
    return 0

if __name__ == "__main__":
    raise SystemExit(main(sys.argv[1]))

What this script does not do:

• It does not call GitLab.
• It does not prove the runner uploaded the file.
• It does not prove the MR widget will render.

It only blocks the embarrassing case. Broken XML never becomes a widget.

A workflow I actually follow

I keep the draft lane and the GitLab lane apart. Mixing them is how the myths start.

1. Draft the job YAML with a model if you want speed.
2. Generate report.xml on any throwaway server.
3. Run the shape checker on that file.
4. Commit YAML plus the test command, not the chat log.
5. Push a branch and open the merge request.
6. Confirm the tests widget against the GitLab job, not the draft host.

Commands I keep in the MR description as a reminder:

# example local draft, still not GitLab ingest
python -m pytest --junitxml=report.xml
python check_junit.py report.xml

After push, I open the job page. I look for report.xml in artifacts. Then I look at the MR tests widget. Two screens. Same pipeline. That is the contract.

Need a regex debug? I copy the GitLab job log, not a local reprint. The coverage keyword never sees your laptop.

What this approach will not save

Do not use this checklist as a release signature. A parsed JUnit file is not provenance. It is not a signed build. It is not an environment stop job.

Skip this draft-server path when:

• The job needs protected variables.
• The job deploys to production.
• The job must run a specific GitLab runner tag.
• You need compliance evidence from GitLab itself.

A free draft host cannot fake those constraints. I will not pretend it can.

Also skip it if your suite emits only custom JSON. GitLab's test widget wants JUnit. Convert or do not expect the tab.

Limitations you should say out loud

Short list. No slogans.

• Lint does not parse reports.
• paths is not reports.
• Local pytest is not a GitLab job.
• Coverage regex reads GitLab logs only.
• Artifact expiry can delete the XML later.
• Child pipelines may hide reports from the parent MR view.

That last one bites teams using triggers. Check where the test job actually ran. Parent widgets do not always inherit child reports the way people assume. Verify on your GitLab version. Do not trust a remembered blog from 2022.

Corrected model in one pass

GitLab merge request test UI is an ingest pipeline. YAML names the report. The runner uploads XML. GitLab parses that XML. The widget is a view over that parse.

Everything else is commentary. Commentary can still be useful. Commentary is not evidence.

So I ask reviewers a narrower question now. Not "did you run tests?" I ask "which job uploaded JUnit for this SHA?"

If they paste a chat log, I send them the table. If they paste lint JSON, I send them the checker. If they paste a free-server screenshot, I ask for the GitLab artifact URL.

That is the whole FAQ. Green pytest is necessary. It is not the widget.

If you already draft YAML on a free model host, keep that host on the checklist's first two rows. Let GitLab own the ingest.