FAQ: Who Owns the Assertions the Agent Just Wrote?

Did your agent just invent a green suite?
I see that claim in almost every PR.
A scratch box ran the suite to green.

What did you actually measure after that?
This is not a rant about tools.
It is a FAQ about assertion ownership.

Who owns the assertions when a model wrote them?
I treat free model access as a drafting aid.
I treat a free server as a scratch runner.

MonkeyCode is one drafting surface for that pair.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
You still owe the repo a real contract.

Why this FAQ exists

Teams treat a generated suite like a measurement system.
They paste coverage numbers into the ticket.
They skip the written plan on every ticket.

A free model can draft tests quickly.
A free server can execute that draft quickly.
Neither one named your invariants for you.

Myth 1: Generated tests equal honest coverage

Here is the claim I keep hearing weekly.
"The agent wrote tests, so coverage is real."
Coverage only counts lines that the runner touched.

Touched is not asserted in any useful way.
Asserted is not specified in your language.
Specified is not owned by you yet.

I want a file that lies less than coverage.html.
I want assertion density, not line theater.
Run this auditor before you merge anything.

#!/usr/bin/env python3
# Proposal: audit agent-drafted tests. Run locally, then in CI.
from pathlib import Path
import re
import sys

ASSERT_RE = re.compile(r"(assert|expect|should)")
SKIP_RE = re.compile(r"(skip|xfail|todo)", re.I)
WEAK_RE = re.compile(r"(toBeTruthy|toBeDefined|assert True)")


def scan(root: Path):
    rows = []
    for p in root.rglob("*"):
        if p.suffix not in {".py", ".js", ".ts", ".go"}:
            continue
        name = p.name.lower()
        if "test" not in name and "spec" not in name:
            continue
        text = p.read_text(encoding="utf-8", errors="ignore")
        lines = text.splitlines()
        rows.append({
            "file": str(p),
            "asserts": len(ASSERT_RE.findall(text)),
            "skips": len(SKIP_RE.findall(text)),
            "weak": len(WEAK_RE.findall(text)),
            "loc": len(lines),
        })
    return rows


if __name__ == "__main__":
    root = Path(sys.argv[1] if len(sys.argv) > 1 else "tests")
    rows = scan(root)
    if not rows:
        print("No test files found. That is also a finding.")
        sys.exit(2)
    print("asserts skips weak loc file")
    for r in rows:
        print(f"{r['asserts']:8d} {r['skips']:6d} {r['weak']:6d} {r['loc']:6d}  {r['file']}")
    weak_files = [r for r in rows if r["asserts"] == 0 or r["weak"] > r["asserts"]]
    if weak_files:
        print("Reject: empty or weak assertions:")
        for r in weak_files:
            print(" -", r["file"])
        sys.exit(1)
    print("Pass: every test file asserts something specific.")

Treat that script as an unexecuted example.
You should run it on your tree.
Then read every file that it flags.

Corrected model for myth 1

Coverage numbers are a hint, nothing more.
Assertions are the contract that you publish.
You own both after the agent logs off.

Myth 2: Green on a scratch server means the contract holds

Did it pass on the free box today?
Good, but that box is not your topology.
Did you pin runtime, fixtures, and network rules?

A scratch server is a drafting surface only.
It is not an environment contract by itself.
Your CI still has to say the same thing.

I use a tiny parity checklist on every PR.
I refuse to merge without those boxes checked.
Copy this block into your PR template.

## Scratch-box parity (fill before merge)
- [ ] Same language version as CI
- [ ] Same dependency lockfile hash
- [ ] Same fixture seed and clock
- [ ] Network calls stubbed, not live
- [ ] Secrets never present on the box
- [ ] Auditor script exit code is 0

If any checklist box stays unchecked, stop merging.
The green check is local folklore after that.
Folklore does not belong on the main branch.

Ask the agent to explain each check.
Then you should verify the answers yourself.
Do not accept a summary as evidence.

Myth 3: A generated suite replaces a test plan

Who wrote the risks in this change?
If the answer is the model, you should stop.
You do not have a test plan yet.

A plan names failures you refuse to ship.
A suite without that list is decoration.
Decoration will not save you in an incident review.

I keep a four-row table in the ticket.
It forces a human to pick the job.
Use it before you trust a free box.

Job you want	Free model plus free server?	What you still owe
Draft happy-path tests from your spec	Yes, as a first pass	You edit names, data, and assertions
Explore a bug with a throwaway repro	Yes, then throw it away	You rewrite or delete before merge
Prove an API contract across environments	No	CI, staging, and a human-owned spec
Gate a release on generated coverage	No	A review, a plan, and pinned runtimes

Do you see the pattern in those rows?
Drafting is allowed on a scratch server.
Gating a release is not allowed on that box.

Myth 4: Scratch-box timing is your performance story

People paste durations from a free session.
Then they call the paste a benchmark.
Against what baseline, and on whose topology?

I will not invent numbers for this article.
I do not have your traces or hosts.
A duration without topology is only a vibe.

If you care about API realism, write the scenario first.
Name the payload, the auth, and the error path.
Do that before you ask any model for tests.

# Proposal: record topology next to any timing you keep.
# Do not treat this as a vendor benchmark.
uname -a
python --version
echo "lockfile=$(sha256sum package-lock.json 2>/dev/null || sha256sum poetry.lock)"
echo "runner=${CI_RUNNER_DESCRIPTION:-local-scratch}"
echo "note=scratch timings are not SLOs"

Store that blob beside any number you keep.
If you cannot store it, drop the number.
A myth dies when the metadata is missing.

Myth 5: More agent tests mean less review

Why would extra volume reduce your review load?
Volume increases the surface of bad asserts.
Weak truthy checks multiply in silence overnight.

I review agent tests in three passes.
Pass one checks names against the spec I wrote.
Pass two proves each test can fail on purpose.

# Fail-on-purpose pass. Proposal only. You pick the runner.
# Break one assertion. Confirm the runner goes red.
# If it stays green, the test never measured that path.
git stash push -m "good suite"
# edit one expect(...) or assert to a wrong value
npm test -- --testPathPattern="$FILE"
# or: pytest "$FILE" -q
# you must see a failure here
git checkout -- "$FILE"
git stash pop

Pass three deletes a duplicate test file.
If the story still holds, keep the deletion.
More files are not the same as more safety.

A fail-on-purpose snippet

Here is a tiny pytest sketch for ownership.
It is a proposal, not a captured run.
Replace compute_total with your real function.

# proposal: one owned assertion, one named failure
def test_invoice_total_rejects_negative_tax():
    """Incident we refuse: negative tax silently ships."""
    invoice = {"net": 1000, "tax": -50}
    try:
        total = compute_total(invoice)
    except ValueError:
        return
    raise AssertionError(f"negative tax produced {total}")

If the agent rewrites this into a tautology, reject it.
A tautology always passes and it never teaches.
You should feel the test bite you.

A workflow that keeps you honest

Start from a written spec, not a chat.
List three failures you refuse to ship.
Then let a free model draft tests on a free server.

Pull the files onto your laptop next.
Run the auditor and fill the parity checklist.
Break one assertion and confirm the runner goes red.

Then review the names against your spec.
Only then should you open the pull request.
The agent session is only a drafting step.

It is not evidence that the contract exists.
Keep the three failures in the PR body.
Future you will need that failure list.

Limitations

This auditor is a heuristic, not a proof.
It will miss custom matchers and helper wrappers.
It will flag some valid tests as weak.

The decision table is not a compliance program.
It will not satisfy a regulated release process.
It will not replace mutation or contract tests.

A free server can vanish out from under you.
Do not store secrets on that disk.
Do not treat its workspace as an artifact store.

Who should not use this approach

Do not use it as a release gate.
Do not use it on security-sensitive code paths.
Do not use it if you cannot read the tests.

Skip it if your org already has a test guild.
Skip it if you need signed provenance per run.
Skip it if the spec still lives only in chat.

The mental model I want you to keep

The model only drafts text for you.
The scratch server only executes that draft.
You publish the contract, or nobody did.

Ask one hard question on every PR.
Which assertion would I defend in an incident?
If you cannot name it, the suite is still a myth.

I keep the auditor beside agent-drafted tests.
Run it on any scratch box you already use.
A free MonkeyCode server is one drafting surface, nothing more.