Test Coding-Agent Admission in a Free Sandbox Before the Queue Eats the SLO

At 09:12 the coding-agent queue looked healthy: CPU 41%, workers idle-ish, no provider alarm. At 09:19 the p95 completion time crossed the internal SLO. The contradiction was the signal: utilization was low while deadline slack was already negative for a burst of jobs that had been admitted too optimistically.

The operational question is not 'are the workers busy?' It is 'which jobs should never have entered the queue, and can we prove that locally before production?'

This drill uses a small admission gate in front of coding-agent workers. MonkeyCode currently advertises free model access and a free server option; I treat that strictly as a sandbox for rehearsing the gate, not as capacity, durability, latency, or permanence. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The useful part below still works if you point the runner at any mock model endpoint.

Topology

loadgen -> admission gate -> Redis list queue -> worker pool -> model endpoint
              |                    |                  |
              v                    v                  v
        reject counter      queue age gauge      latency histogram

Minimum local setup:

# compose.yaml
services:
  redis:
    image: redis:7-alpine
    ports: ['6379:6379']
  gate:
    build: .
    environment:
      REDIS_URL: redis://redis:6379
      QUEUE: agent_jobs
      MAX_DEADLINE_SLACK_MS: '1500'
      MARGIN_MS: '250'
      MODEL_URL: ${MODEL_URL:-http://mock-model:8000}
    depends_on: [redis, mock-model]
  worker:
    build: .
    command: python worker.py
    environment:
      REDIS_URL: redis://redis:6379
      QUEUE: agent_jobs
      MODEL_URL: ${MODEL_URL:-http://mock-model:8000}
      MODEL_LATENCY_MS: ${MODEL_LATENCY_MS:-350}
      FAIL_RATE: ${FAIL_RATE:-0}
    deploy:
      replicas: 3
    depends_on: [redis, mock-model]
  mock-model:
    build: .
    command: python mock_model.py
    ports: ['8000:8000']

Declared workload for the drill: fixed seed, 240 jobs, 40 jobs/s offered for 6 seconds, per-job deadline uniformly 900-2600 ms, three workers, mock service time 350 ms unless injected. This is not a benchmark of any hosted model; it is a controlled way to make admission decisions observable.

Admission rule

Reject early when the job cannot plausibly finish before its deadline:

est_service_ms = max(recent_p95_service_ms, injected_floor_ms)
deadline_slack_ms = deadline_ms - now_ms
admit only if deadline_slack_ms >= est_service_ms + MARGIN_MS + queue_wait_estimate_ms

queue_wait_estimate_ms is deliberately crude: queued_jobs / workers * est_service_ms. Crude is fine for a drill because the failure mode is visible: under estimate wait and the gate admits work that later misses; over estimate and you reject useful work. The threshold debate should be explicit.

# gate.py - runnable artifact, standard library + redis-py
import json, os, time, statistics
import redis

r = redis.Redis.from_url(os.environ['REDIS_URL'])
QUEUE = os.environ.get('QUEUE', 'agent_jobs')
MAX_SLACK = int(os.environ.get('MAX_DEADLINE_SLACK_MS', '1500'))
MARGIN = int(os.environ.get('MARGIN_MS', '250'))
HIST = 'svc_ms'

def metrics(name, value, **labels):
    lab = ','.join(f'{k}={v}' for k, v in sorted(labels.items()))
    print(f'{name}{{{lab}}} {value}', flush=True)

def service_floor():
    vals = [float(x) for x in r.lrange(HIST, 0, 199)]
    if len(vals) < 10:
        return 350.0
    return max(statistics.quantiles(vals, n=100)[94], 350.0)

def admit(job):
    now = time.time() * 1000
    slack = job['deadline_ms'] - now
    queued = r.llen(QUEUE)
    workers = int(r.get('workers') or 3)
    est = service_floor()
    wait = (queued / max(workers, 1)) * est
    decision = slack >= est + MARGIN + wait and slack <= MAX_SLACK
    metrics('agent_admission_decision', 1, result='admit' if decision else 'reject')
    metrics('agent_deadline_slack_ms', round(slack, 1))
    metrics('agent_queue_wait_estimate_ms', round(wait, 1))
    if decision:
        r.rpush(QUEUE, json.dumps(job))
    return decision

# load generator: deterministic for review
if __name__ == '__main__':
    import random
    random.seed(7)
    base = time.time() * 1000
    for i in range(240):
        job = {
          'id': i,
          'created_ms': base + i * 25,
          'deadline_ms': base + i * 25 + random.randint(900, 2600),
          'repo': 'sandbox',
          'prompt_bytes': 1200,
        }
        admit(job)

Worker sketch:

# worker.py
import json, os, random, time, urllib.request
import redis
r = redis.Redis.from_url(os.environ['REDIS_URL'])
QUEUE = os.environ.get('QUEUE', 'agent_jobs')
LAT = int(os.environ.get('MODEL_LATENCY_MS', '350'))
FAIL = float(os.environ.get('FAIL_RATE', '0'))
r.set('workers', 3)
while True:
    item = r.blpop(QUEUE, timeout=5)
    if not item:
        continue
    job = json.loads(item[1])
    start = time.time() * 1000
    time.sleep(LAT / 1000)
    ok = random.random() >= FAIL
    svc = time.time() * 1000 - start
    r.lpush('svc_ms', svc)
    r.ltrim('svc_ms', 0, 199)
    late = svc > (job['deadline_ms'] - job['created_ms'])
    print(f'agent_job_result{{ok={str(ok).lower()},late={str(late).lower()}}} 1', flush=True)
    if late:
        print(f'agent_deadline_miss_ms {round(svc,1)}', flush=True)

Expected output shape for the clean run, labeled expected not measured on your machine:

agent_admission_decision{result=admit} 173
agent_admission_decision{result=reject} 67
agent_job_result{ok=true,late=false} 1
agent_queue_wait_estimate_ms 610.0

The exact counts depend on runtime jitter, docker scheduling, and clock resolution. The invariants matter: rejects rise before late results rise; deadline_slack_ms near zero is a leading indicator; utilization can remain moderate while misses increase.

Failure injection

Run three local scenarios:

Scenario	Env change	Expected operational read
Baseline	MODEL_LATENCY_MS=350 FAIL_RATE=0	Admitted jobs finish; rejects are mostly impossible deadlines
Slow provider	MODEL_LATENCY_MS=1200 FAIL_RATE=0	Gate rejects more; queue age should not grow unbounded
Flaky provider	MODEL_LATENCY_MS=350 FAIL_RATE=0.35	Late=false but ok=false rises; separate retry budget from admission

Prometheus-style fields to keep: agent_admission_decision, agent_deadline_slack_ms, agent_queue_wait_estimate_ms, agent_job_result, agent_deadline_miss_ms, plus redis_llen and worker heartbeat age. Alert on rate(agent_deadline_miss_ms[5m]) > 0 only after you also chart reject reason; a miss alert without admission context sends humans to the wrong lever.

Where the free sandbox helps, and where it does not

A free model/server sandbox is useful for rehearsing the control loop: admit, reject, drain, measure, roll back. It is useful for catching schema drift in job payloads, proving that workers stop pulling during degradation, and testing whether your gate fails closed when the model endpoint is slow. It is not evidence for production capacity, provider availability, cost ceilings, model quality, or latency SLOs. Do not send proprietary code, secrets, customer data, or regulated workloads to any free tier. Do not publish benchmark claims from an uncontrolled free endpoint; label them sandbox observations.

Rollback path for this drill: set MAX_DEADLINE_SLACK_MS high enough to admit everything, scale workers to one, FLUSHALL the drill Redis only if it is dedicated, then restore the previous gate config from version control. Cleanup: docker compose down -v, delete local Redis volume, rotate any sandbox credentials, and remove test repos or branches created by agents.

Threshold I would start with: reject when deadline slack is below estimated service plus one p95 queue wait, and page only when misses persist after rejects have already increased. If you run agent workers, try this drill against a sandbox and share which signal predicted the miss first: queue age, deadline slack, or provider latency.