You cloned an agent repo earlier this week. Did that green demo become a real review app? I keep seeing that leap in pipeline reviews.
A README and a free route get mashed together. A free server option gets pulled into that same mash. That mash is a story, not pipeline evidence.
This FAQ splits those claims into separate layers. I will not invent quotas, hardware, or uptime numbers. If today's docs omit a figure, I omit it too.
Why this mix-up shows up now
Agent demos are fast, and review apps are slow. A teammate pastes a chat screenshot into the thread. Someone then asks why the merge request lacks a URL.
Those two systems do not share a clock. Why would one green chat prove a deployment? I am not filing a lab report with fake timings.
The claims I keep hearing
People repeat four claims that sound practical enough. Each claim skips a layer you can actually check. I listed them so the review can point at one.
- The open-source checkout is already our CI include.
- Free model access means the job image is chosen.
- A free server option is already our review app.
- A local demo is the same run as CI.
Here is the corrected model I want teams to use. You hold a repo, a route, a server, and a contract. Proof does not cross those layers unless you captured it.
What each layer may prove
A repo proves you can read source, not that CI ran. A route proves a call only when the log shows it. A server proves a host only when you name the owner.
Where this example sits
MonkeyCode is the open-source project used in this example. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Operator notes say free model access is currently offered.
Those same notes also mention a free server option. I am not naming models or stating a token grant. I am not stating machine size or a time limit.
Those details live in current docs, and docs change. Remove the product name and this checklist still works. You can point the same questions at any agent route.
What did this job actually prove, line by line? Why would a product page choose your image tag? I would not let a homepage answer either question.
Myth 1: The README is a CI include
A README can describe a workflow you might want. It cannot include itself into your pipeline file. Did you commit a config that really calls the tool?
Did that file pin a ref you can reproduce later? If the answer is no, you only have a clone. A clone is not a pipeline step you can rerun.
# Proposal only. Not a live include from this article.
agent_check:
stage: test
image: alpine:3 # placeholder tag; pin a digest before real use
script:
- test -n "$CI_COMMIT_SHA"
- echo "commit=$CI_COMMIT_SHA" | tee agent-evidence.txt
artifacts:
paths:
- agent-evidence.txt
That snippet only records the commit under test. It does not call a model or start a server. Why add it before the demo talk starts?
Myth 2: Free access picks the job image
Free model access means a route may be offered. It does not mean your job selected that route. GitLab will not infer a model from a product page.
You pass variables on purpose, and you mask secrets. A missing variable is a missing pin in that job. A present variable is still not a successful call.
# Unexecuted example. Use a masked variable, never a pasted key.
test -n "$MODEL_ROUTE" || echo "MODEL_ROUTE unset"
if test -n "$MODEL_TOKEN"; then echo "token=present"; else echo "token=absent"; fi
Did the job log show a response id you can cite? If not, stop claiming the model ran inside CI. I will not list model names, because names go stale.
Myth 3: A free server is the review app
A free server option is only a product choice. A review app is a GitLab environment you control. A vendor button does not create your environment URL.
GitLab review apps need a job and a recorded URL. A shared free host is not your environment url field. If the host lives outside the job, say that plainly.
# Proposal only. Replace the host with a URL you actually own.
review:
stage: deploy
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://$CI_COMMIT_REF_SLUG.example.test
script:
- echo "url=$CI_ENVIRONMENT_URL" | tee review-evidence.txt
artifacts:
paths:
- review-evidence.txt
Ask whether GitLab actually recorded an environment name. Ask whether that URL resolves to your own deployment. Ask whether a job you own can stop that environment.
Call an outside host an external demo server instead. Do not call that host the review app in review. Words in the thread become words in the incident.
Myth 4: The laptop demo is the CI run
You ran a prompt on a laptop at your desk. The pipeline ran on a runner you may not own. Those two clocks do not match, and they should not.
A chat transcript is not a CI job URL. A screenshot has no job id you can click. Why argue from a picture when the job page exists?
# Run against a redacted log you saved. Do not upload secrets.
job_log="${1:-job.log}"
grep -E 'CI_JOB_URL|CI_COMMIT_SHA|CI_ENVIRONMENT_URL' "$job_log" || true
grep -E 'model|route|server' "$job_log" || true
No match means you do not have that claim yet. A match means you can quote that exact line. It still may not prove a model response happened.
Look for an id your provider actually documents. I am not inventing that field name in this FAQ. Adjust the grep after you read the current docs.
A table for the review thread
I use a small table when the thread gets loud. Each row says what you hold and what it proves. Paste it into the merge request, then ask for gaps.
| Evidence you hold | Layer it proves | What it does not prove |
|---|---|---|
| A git clone or a fork | You have source | A pipeline ran |
| A README mention of free access | A claim exists in text | Your job used it |
| A set CI variable name | The name was injected | A call succeeded |
| A job log line with a response id | That job saw a response | Cost, quota, or latency |
| An environment URL in GitLab | A review URL was recorded | The free server is that URL |
| A screenshot of a chat | Someone saw a reply | The pipeline produced it |
Reviews get shorter when the missing row is visible. I would rather see a blank cell than a vague claim. Which row are you actually holding right now?
A local checker, labeled unexecuted
This script is a proposal, not a measured benchmark. I have not executed it against your private logs. It only reads files you already saved on disk.
It does not call a network or need a token. Run it on redacted files before you share output. A printed missing line is a boundary, not an insult.
#!/usr/bin/env bash
# evidence-layers.sh - local classifier, unexecuted in this article.
set -euo pipefail
log="${1:-job.log}"
envf="${2:-agent-evidence.txt}"
prove() { printf 'PROVEN %s\n' "$1"; }
miss() { printf 'MISSING %s\n' "$1"; }
test -f "$log" && prove "job log file" || miss "job log file"
test -f "$envf" && prove "evidence file" || miss "evidence file"
if grep -q 'CI_COMMIT_SHA=' "$envf" 2>/dev/null; then
prove "commit was recorded"
else
miss "commit was recorded"
fi
if grep -q 'CI_ENVIRONMENT_URL=' "$envf" 2>/dev/null; then
prove "review URL was recorded"
else
miss "review URL was recorded"
fi
if grep -Eq 'response_id|request_id' "$log" 2>/dev/null; then
prove "log mentions a response id pattern"
else
miss "log mentions a response id pattern"
fi
echo "Done. Missing rows are not failures of the product."
echo "They are claims you should not make yet."
How I would comment
I would not write that the model failed the deploy. I would write which evidence row is still empty. Here is a comment shape you can paste and edit.
Layer check for this job:
- clone: was CI_COMMIT_SHA recorded in artifacts?
- route: is a response id present in the log?
- server: external demo only, or our environment?
- review app: is CI_ENVIRONMENT_URL present?
Claim I will not make yet: <fill this blank>
Fill the last line with the claim you almost made. If you cannot fill a row, delete that claim. Would you approve a merge that skips this note?
What I refuse to treat as proof
I will not treat a round token number as your balance. People repeat figures copied from older posts. Today's grant can differ, so check the current docs.
Then check your own usage page before you budget CI. A blog figure is not a GitLab variable you can spend. Why pin a number you did not read this morning?
I will not treat a free server as a runner tag. Runner tags belong to a runner you administer. A hosted demo is another machine with another owner.
Read the runner docs before you mix those words. I will not treat a green chat as a release receipt. A release needs an artifact, a digest, and a job URL.
A reply in a browser is none of those three. If you need that receipt, build the artifact in CI. Do not borrow it from a chat window.
Limits of this FAQ
This checklist does not measure latency or audit quotas. It does not scan prompts for secrets you pasted. It will not tell you if a free tier stays free.
The sample YAML uses Alpine only to show a write. It is not a recommended build image for your app. The grep patterns are hints, not a provider contract.
Your provider may use another id field entirely. Adjust the pattern after you read their current docs. Do not send production source to a free route for this.
Customer data and credentials stay out of that trial. The checklist reads local files you already redacted. The model call is a separate decision with separate risk.
Who should skip this approach
Skip this approach if you need a contractual SLA. A free option is not that paper, and it may change. Skip it if the pipeline handles regulated data too.
Free access does not rewrite your data handling rules. Skip it if you wanted a benchmark with timed numbers. I did not run a timed suite, and I will not fake one.
Skip it if you only want a sticker that says passed. A badge without the four layers is just decoration. Reviewers who want a cleaner argument get the value.
Release managers who need a receipt should look elsewhere. Students exploring a demo can use the table as a brake. Anyone who cannot redact logs should wait as well.
A public paste of a raw job log is a leak. I would rather delay the post than leak a token.
A closing check
Open the failed job and copy the job URL first. Then fill one row of the table with real evidence. Which layer failed: the clone, the route, the server, or the environment?
I use that order so the thread stops guessing. It stops people blaming a model for a missing key. If you try a free route, keep the docs tab open.
Read the current limits before you quote a grant. One natural next step is enough for a lab branch. Run the checklist on one failed job, then stop.
Primary docs I am pointing at
These links are GitLab docs, not product claims from me. I am not asserting a MonkeyCode quota from these pages. Check the product docs yourself before you repeat a number.
- Predefined CI variables: https://docs.gitlab.com/ci/variables/predefined_variables/
- Environments and URLs: https://docs.gitlab.com/ci/environments/
- Runner tags and registration: https://docs.gitlab.com/runner/
Top comments (0)