DEV Community

Dakota Liu
Dakota Liu

Posted on

Plant the Canary Before You Spend a Free Model Call

If the prompt still holds a credential, do not call the model. Scan the fixture on your laptop, prove the scanner can fail, and only then spend a free model call or a free server run.

I keep seeing the same shortcut in agent writeups. Someone wants a quick demo, so they paste a working tree. The tree still has a .env.

Sometimes it has a cloud key. Sometimes it has a customer id that never belonged in a prompt. Why hand that to a model you are only trying?

People are arguing, loudly, about what a model does when the task looks sensitive. I am not recycling a score from a thread I cannot re-run. I want a gate I can fail on my own laptop before any free call.

This is a from-zero tutorial for that preflight. It is not a benchmark. I am not claiming a measured latency, a saved dollar figure, or a quota. The scripts are unexecuted examples. Run them yourself.

What I will and will not claim

Two availability claims drive the later steps: free model access, and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

That sentence is the relationship. I am not auditing the service in this draft. I will not name a model, invent a token allotment, describe hardware, or promise how long a free server stays up.

Those facts go stale. On the day you run this, read the page you can reload. If it disagrees with an old post, believe the page.

Pull the product out and the gates still stand. They are local on purpose.

What you need before step 1

You need Python 3, a shell, and a directory that is not your production checkout. You also need a rule you can say out loud. The model may propose. You apply.

Can you keep those roles apart when you are in a hurry? If the answer is no, skip any remote run. A toy fixture under your home directory is enough. Your real repo is not the demo.

Step 1 — Create a fixture the model is allowed to see

Start empty. Add one function and one test. Add nothing else.

mkdir -p $HOME/prompt-gate-demo/src
cd $HOME/prompt-gate-demo
printf 'def add(a, b):\n    return a + b\n' > src/add.py
printf 'from src.add import add\n\ndef test_add():\n    assert add(2, 3) == 5\n' > test_add.py
python3 -m py_compile src/add.py test_add.py && echo STEP1_PASS
find . -name .env -o -name '*.pem' -o -name '*.key'
Enter fullscreen mode Exit fullscreen mode

Verification is two checks, not one. STEP1_PASS must print. The find must print nothing. If either check fails, delete the directory and start again. Cleaning it up later is how a key survives into the prompt.

Step 2 — Plant a canary and prove the scanner fails closed

A scanner that has never seen a secret is theater. Why trust a green result you have not tried to break?

Save the following as scan_prompt.py next to the demo, not inside it. It is a teaching scanner. It is not a replacement for a dedicated secret-scanning tool your team already trusts.

#!/usr/bin/env python3
# Unexecuted example. Fail closed if a denylisted pattern appears.
import pathlib
import re
import sys

PATTERNS = [
    ('aws_access_key', re.compile(r'AKIA[0-9A-Z]{16}')),
    ('private_key', re.compile(r'-----BEGIN (?:RSA |OPENSSH |EC )?PRIVATE KEY-----')),
    ('assignment', re.compile(r'(?i)\b(api_key|secret|token|password)\b\s*[:=]\s*\S+')),
    ('canary', re.compile(r'CANARY_SECRET_DO_NOT_SEND')),
]
SKIP = {'.git', '.venv', 'node_modules', '__pycache__'}
ALLOW = {'.py', '.md', '.txt', '.toml', '.json'}

def files(root: pathlib.Path):
    for path in root.rglob('*'):
        if not path.is_file():
            continue
        if any(part in SKIP for part in path.parts):
            continue
        if path.suffix.lower() not in ALLOW:
            continue
        yield path

def main() -> int:
    root = pathlib.Path(sys.argv[1] if len(sys.argv) > 1 else '.')
    hits = 0
    for path in files(root):
        text = path.read_text(encoding='utf-8', errors='replace')
        for name, cre in PATTERNS:
            if cre.search(text):
                hits += 1
                print(f'HIT {name} {path}')
    print('PASS' if hits == 0 else 'FAIL')
    return 0 if hits == 0 else 2

if __name__ == '__main__':
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

Run the boring side first, then the mean side.

python3 scan_prompt.py $HOME/prompt-gate-demo
# expect: PASS and exit 0

printf '\nAPI_KEY=CANARY_SECRET_DO_NOT_SEND\n' >> $HOME/prompt-gate-demo/src/add.py
python3 scan_prompt.py $HOME/prompt-gate-demo
echo exit=$?
# expect: HIT canary, then FAIL and exit=2
Enter fullscreen mode Exit fullscreen mode

If the canary run exits 0, stop. The rest of this tutorial is invalid until the scanner fails closed. Delete the canary line, scan again, and demand PASS before you continue. I would rather lose ten minutes here than explain a leaked key later.

Step 3 — Copy an allowlisted bundle

A passing scan is not permission to zip the tree. Hidden files stay behind. So does anything you did not name.

rm -rf $HOME/prompt-gate-bundle
mkdir -p $HOME/prompt-gate-bundle
cp $HOME/prompt-gate-demo/src/add.py $HOME/prompt-gate-bundle/add.py
cp $HOME/prompt-gate-demo/test_add.py $HOME/prompt-gate-bundle/test_add.py
find $HOME/prompt-gate-bundle -type f -print
python3 scan_prompt.py $HOME/prompt-gate-bundle
Enter fullscreen mode Exit fullscreen mode

Verification: the find prints exactly two paths, then the scanner prints PASS. Three files means you copied a surprise. Stop and delete the bundle. Do not debug the surprise by uploading it.

Step 4 — Spend the free model call on that bundle only

This is the first moment free model access belongs in the workflow. I paste the two files and one ask: make add reject non-integers, and keep the existing test green. I do not paste the parent directory. I do not paste shell history. I do not paste a dashboard screenshot for context.

What do I check in the reply before I touch a server?

  1. It names files I already have.
  2. It does not ask me to export a secret or read a credential file.
  3. It does not add a network call the test never required.
  4. I can save the text without executing it.

I write that reply to proposal.md outside the bundle. The proposal is untrusted input. It is not a shell script, and it is not a merge.

If the surface in front of you no longer offers a free call, stop. Do not upload the real repository somewhere else just to finish the demo.

Step 5 — Apply only after a test you wrote

A free server is optional, and it comes last. It runs a command you already typed, against a directory you already scanned. It does not get to invent the command.

cp -R $HOME/prompt-gate-bundle $HOME/prompt-gate-apply
# Apply the proposal by editing add.py yourself.
python3 -m pytest $HOME/prompt-gate-apply/test_add.py
echo pytest_exit=$?
python3 scan_prompt.py $HOME/prompt-gate-apply
Enter fullscreen mode Exit fullscreen mode

Verification: pytest_exit=0, and the second scan prints PASS. If pytest is missing, install it in a local virtualenv or rewrite the check as a plain assert runner. A missing tool is not a reason to skip the check.

Only after that local pass do I consider a free server run. Upload the apply directory, not your home directory. Run the same test command because you typed it. If the current free-server page lists extra limits, those limits win over this article.

The stop/go table

I keep this next to the terminal. A demo that is almost done does not get a special row.

Scan Canary proven this session Bundle Model call Server run
FAIL yes or no any no no
PASS no any no no
PASS yes extra files no no
PASS yes exact allowlist yes, bundle only only after local tests pass

Pick the row that matches the terminal, not the row you wish you were on.

What this does not do

Regexes miss secrets in images, archives, and encoded blobs. A canary in a .py file does not prove you would catch a key inside notebook output. Encoding a secret so the pattern misses is not a fix. It is how people fool their own gate.

Do not use this approach as a workplace data-loss program. If your team already mandates a scanner, use that one. Do not use it when the task truly needs a live environment file.

Redesign the task until a fake value is enough. Do not treat a free server as a long-lived host, a customer-data store, or a way around terms you have not read.

I also will not pretend a free option is permanent. Availability you saw today can disappear. That is why the gates live on your machine, in files you can re-run without an account.

A five-minute re-run the next morning

Tomorrow the demo will feel familiar, and that is when people skip the canary. I do not.

I delete prompt-gate-apply, plant the canary again, and require exit=2 before I remove it. Then I rebuild the bundle from the two cp lines, not from memory. Then I ask the model the same question and diff the new proposal against proposal.md.

Did the reply grow a request for environment variables? Throw it away. Did the file list change? Throw it away. Did your local pytest fail? The server stays off.

That re-run is the whole method. The product is only the last hop, and only when the page in front of you still offers the free path.

Where I would spend the trial

If the fixture scans clean and the test already passes locally, the remaining choice is where to spend the trial. I would open the current MonkeyCode docs, confirm that free model access and the free server option are still listed, and only then send the two-file bundle. If the page does not list them, I keep the local gates and I skip the upload.

Top comments (0)