Workshop: Cap Agent Tool Fan-Out on Shared Inference in 90 Minutes

Unbounded tool fan-out, not weak prompting, is what usually knocks a shared classroom inference endpoint offline. A ninety-minute lab makes that failure visible, then contains it with a permit file, a call budget, and a replay harness. The method stays useful if the remote model changes, because admission lives on the student machine. Instructors should grade the admission log rather than the fluency of whatever the planner happened to emit.

This outline is a teaching proposal, not a production runbook, and it assumes a toy agent with two or three tools. Times below fit a cohort that already serializes tool calls as JSON and can run pytest locally. Nothing in the lab requires a GPU on the laptop or a paid inference contract. The live model stays optional until the stub exercise, and even then it remains behind the budget.

Who this lab is for

Use this workshop when students share one inference endpoint and their agents may call tools in a loop. The usual damage pattern is a recursive self-repair step that fans out search, HTTP, and retry tools until the shared queue stalls. Instructors get a fair-share rule they can mark in minutes. Students get a deterministic fixture instead of an outage that depends on whoever ran first.

Skip the session if agents never call tools, or if every laptop already talks to an isolated endpoint with hard vendor caps. Also skip it when the assignment executes arbitrary student code on the same host as the model process. Those cases need isolation and quotas at the runtime, not a JSON permit beside the client.

Timing board (90 minutes)

• 0:00–0:12 — Reproduce a fan-out storm from a fixture, not from a live completion.
• 0:12–0:28 — Read the permit schema and fail closed when required keys are missing.
• 0:28–0:50 — Implement ToolBudget.admit() and unit-test three distinct denial reasons.
• 0:50–1:10 — Wire the budget in front of a stub planner and a stub tool runner.
• 1:10–1:25 — Replay the same transcript twice and diff the decision lists.
• 1:25–1:30 — Debrief what the budget cannot see, including raw HTTP bypasses.

Materials. Python 3.11 or newer, pytest, one JSON permit, one JSON trace, and any OpenAI-compatible chat client. Students may point that client at a shared free server if the instructor supplies a base URL. No laptop GPU is required, and wall-clock retries should stay disabled during the stub segment.

The failure you will reproduce

A naive agent treats every tool error as a reason to attempt more tools in the same turn. One failed search becomes three retries, each retry plans extra http_get calls, and a cohort of twenty laptops multiplies that tree together. Model quality is irrelevant once the endpoint queue is full of duplicate planning prompts. Starting from a recorded plan keeps the outage reproducible on a quiet machine.

The lab therefore injects a fixture that already contains an unbounded repair loop. Students prove the budget would have stopped it before anyone touches the shared endpoint. The fixture below is synthetic teaching data, not a capture from a production agent, and it should be checked into the lab repository beside the permit.

{
  "trace_id": "lab-fanout-01",
  "steps": [
    {"tool": "search", "args": {"q": "retry"}},
    {"tool": "search", "args": {"q": "retry site:example"}},
    {"tool": "http_get", "args": {"url": "https://example.invalid/a"}},
    {"tool": "http_get", "args": {"url": "https://example.invalid/b"}},
    {"tool": "http_get", "args": {"url": "https://example.invalid/c"}}
  ]
}

Count the planned calls on paper before writing code: two searches and three fetches. Under the permit in the next section, the third fetch must deny. If a student implementation admits all five steps, the later stub exercise will still be able to stampede the shared endpoint.

Permit file: fail closed

The permit is a small JSON document in the lab repo, not a comment in a prompt. Missing keys deny the call. Unknown tools deny the call. Depth and parallelism are first-class fields because retries hide inside both of those axes. Host allowlists belong here as well, because a shared planner may still emit absolute URLs outside the assignment.

{
  "max_total_calls": 6,
  "max_depth": 2,
  "max_parallel": 1,
  "tools": {
    "search": {"max_calls": 2, "max_args_bytes": 200},
    "http_get": {"max_calls": 2, "allowed_hosts": ["example.edu"]},
    "shell": {"max_calls": 0}
  }
}

shell is listed with max_calls set to zero so a hallucinated shell tool fails closed instead of falling through. That zero is a policy bit, not a seccomp jail, and the debrief should say so out loud. Students who delete the shell entry should watch the unknown-tool path deny the same call, which is the second fail-closed check.

Worked example students can rerun

The skeleton below is meant to be typed, tested, and extended during the lab. It is not a framework, and several branches are deliberately harsh. Parallelism other than one call is denied so the course does not pretend to schedule concurrent tool I/O. Label any extra fields you add as local extensions, then keep the four required permit keys stable so fixtures stay replayable.

from __future__ import annotations

import json
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from urllib.parse import urlparse


class BudgetDenied(Exception):
    """Raised when a tool call would exceed the local permit."""


@dataclass
class ToolBudget:
    permit: dict[str, Any]
    total_calls: int = 0
    depth: int = 0
    per_tool: dict[str, int] = field(default_factory=dict)
    decisions: list[dict[str, Any]] = field(default_factory=list)

    @classmethod
    def from_path(cls, path: Path) -> "ToolBudget":
        permit = json.loads(path.read_text())
        required = {"max_total_calls", "max_depth", "max_parallel", "tools"}
        missing = required - set(permit)
        if missing:
            raise BudgetDenied(f"permit missing keys: {sorted(missing)}")
        return cls(permit=permit)

    def admit(self, tool: str, args: dict[str, Any], *, depth: int) -> None:
        tools = self.permit["tools"]
        if tool not in tools:
            self._deny(tool, "unknown_tool")
        spec = tools[tool]
        used = self.per_tool.get(tool, 0)
        if used >= spec["max_calls"]:
            self._deny(tool, "per_tool_cap")
        if self.total_calls >= self.permit["max_total_calls"]:
            self._deny(tool, "total_cap")
        if depth > self.permit["max_depth"]:
            self._deny(tool, "depth_cap")
        if self.permit["max_parallel"] != 1:
            self._deny(tool, "parallel_not_implemented")
        encoded = json.dumps(args, sort_keys=True)
        max_bytes = spec.get("max_args_bytes", 4096)
        if len(encoded.encode("utf-8")) > max_bytes:
            self._deny(tool, "args_too_large")
        if tool == "http_get":
            host = urlparse(str(args.get("url", ""))).hostname or ""
            allowed = spec.get("allowed_hosts", [])
            if host not in allowed:
                self._deny(tool, "host_not_allowed")
        self.total_calls += 1
        self.per_tool[tool] = used + 1
        self.depth = max(self.depth, depth)
        self.decisions.append({"tool": tool, "ok": True, "reason": "admit"})

    def _deny(self, tool: str, reason: str) -> None:
        self.decisions.append({"tool": tool, "ok": False, "reason": reason})
        raise BudgetDenied(reason)


def replay(trace_path: Path, permit_path: Path) -> list[dict[str, Any]]:
    budget = ToolBudget.from_path(permit_path)
    trace = json.loads(trace_path.read_text())
    for depth, step in enumerate(trace["steps"], start=1):
        try:
            budget.admit(step["tool"], step.get("args", {}), depth=depth)
        except BudgetDenied:
            break
    return budget.decisions

Students replay lab-fanout-01 against the permit and stop at the first denial. The intended teaching result is a denial on the third http_get with reason per_tool_cap, after search has already used its cap of two. Host denial may fire first if a student leaves example.invalid in the fixture while the allowlist only contains example.edu; that ordering is worth a two-minute discussion, not a silent patch.

# test_budget.py
from pathlib import Path

from budget import BudgetDenied, ToolBudget, replay


def test_missing_depth_fails_closed(tmp_path: Path) -> None:
    payload = {
        "max_total_calls": 6,
        "max_parallel": 1,
        "tools": {"search": {"max_calls": 1}},
    }
    path = tmp_path / "permit.json"
    path.write_text(json_dumps(payload))
    try:
        ToolBudget.from_path(path)
        raise AssertionError("missing max_depth must deny")
    except BudgetDenied as exc:
        assert "max_depth" in str(exc)


def test_fixture_stops_on_http_or_host_cap() -> None:
    decisions = replay(Path("fixtures/lab-fanout-01.json"), Path("permit.json"))
    denied = [row for row in decisions if not row["ok"]]
    assert denied, "unbounded fixture must produce at least one denial"
    assert denied[0]["reason"] in {"per_tool_cap", "total_cap", "host_not_allowed"}


def json_dumps(payload: dict) -> str:
    import json

    return json.dumps(payload)

Commands the cohort should run without hitting a remote model:

python -m pytest test_budget.py -q
python -c "from pathlib import Path; from budget import replay; print(replay(Path('fixtures/lab-fanout-01.json'), Path('permit.json')))"

Exercise sequence

1. Reproduce (12 minutes). Import the fixture and do not call a live model. Confirm five planned tool calls exist. Write a pytest that still fails while admit() returns unconditionally, so the later caps have a red bar to turn green.
2. Fail closed (16 minutes). Load the permit and assert that missing max_depth raises BudgetDenied. Assert shell with max_calls of zero never admits. Assert an unknown tool name is denied even when total capacity remains.
3. Cap fan-out (22 minutes). Implement per-tool and total caps, then replay lab-fanout-01. Record the first denial reason in a markdown table on the lab sheet, including whether host policy fired before count policy.
4. Stub the planner (20 minutes). Wrap a chat client so each model-proposed tool call hits admit() before any I/O. If the instructor points students at a shared endpoint, keep the client timeout short and the retry count at zero. The budget, not the HTTP adapter, is the backpressure mechanism.
5. Replay twice (15 minutes). Run the same fixture through the stub twice and diff the decisions lists. They must match byte-for-byte on tool, ok, and reason. A mismatch usually means a wall-clock retry is still hiding in the client.

Proposed stub shape, unlabeled as production code:

import json
import os
import urllib.request

LAB_BASE_URL = os.environ.get("LAB_BASE_URL", "http://127.0.0.1:8080/v1")
LAB_TIMEOUT_SEC = float(os.environ.get("LAB_TIMEOUT_SEC", "20"))


def plan_once(messages: list[dict], budget) -> dict:
    req = urllib.request.Request(
        f"{LAB_BASE_URL}/chat/completions",
        data=json.dumps({
            "messages": messages,
            "tools": [{"type": "function", "function": {"name": "search"}}],
        }).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    with urllib.request.urlopen(req, timeout=LAB_TIMEOUT_SEC) as resp:
        payload = json.loads(resp.read().decode("utf-8"))
    tool = payload["choices"][0]["message"].get("tool_call", {})
    if tool:
        budget.admit(tool["name"], json.loads(tool.get("arguments") or "{}"), depth=1)
    return payload

Set LAB_BASE_URL only after local replay is green. Leave retries at zero so a slow shared server cannot multiply fan-out through the HTTP layer. If the stub cannot parse a tool call, deny by default rather than executing a guessed name.

Instructor decision table

Symptom in the room	First check	Permit field to tighten
Endpoint returns 429 or queue timeouts	Concurrent student jobs and retry settings	max_total_calls
Duplicate web fetches in traces	Repeated http_get argument objects	tools.http_get.max_calls
Recursive “try again” plans	Depth of tool-calling messages	max_depth
Surprise shell execution	Tool name allowlist	tools.shell.max_calls = 0
Prompt injection through huge arguments	Byte length of serialized args	max_args_bytes
Hosts outside the assignment	URL hostname after urlparse	allowed_hosts

Where a free shared server fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The budget object does not require a particular vendor, and it only requires that tool calls pass through admit() before I/O. If the course needs a shared chat endpoint without each student provisioning hardware, MonkeyCode's free model access and free server option can sit behind that same client stub. Treat the remote model as an untrusted planner. Treat the permit file as the trusted policy, and grade the decision log rather than the prose.

Do not send secrets in tool arguments or prompts to any shared endpoint, including a classroom server. Redact tokens before writing decisions to disk. The permit is not an isolation boundary for untrusted code execution, and a free shared planner does not change that rule. If you want that shared endpoint for the stub segment, point LAB_BASE_URL at the server the course already approved, then collect the same fixtures you collected on localhost.

Limitations

This lab does not measure model quality, latency, or token cost, and it should not be cited as a benchmark. It does not implement true parallelism; max_parallel other than 1 is denied on purpose so students cannot hide fan-out in threads. It does not sandbox shell, so a zero cap is policy, not a jail. It cannot stop a student from bypassing admit() with a raw HTTP client, which is why the debrief asks for the decision file rather than a screenshot of model text.

The approach also assumes JSON tool calls that are complete before admission. Streaming function-call deltas need a join step this skeleton omits. Multi-agent graphs need one budget per agent plus a course-level job queue, which is a different workshop. A proposed extension, not implemented here, is a fair-share token on the server that rejects a student id after N admitted calls per hour; without that, a determined client can still starve peers.

Who should not use this approach

• Teams that already enforce tool policy inside a managed agent runtime with audited caps.
• Assignments that run arbitrary student code on the same host as the model process.
• Production incident response, where you need distributed tracing rather than a JSON permit.
• Courses that cannot fail closed, because a denied tool call would strand a graded live demo.

If those constraints apply, teach vendor quotas and request tracing instead of this local budget. The permit pattern is for shared teaching endpoints where the cheapest control plane is a file students can diff.

What to collect before dismissing the cohort

Ask each pair for three artifacts: the permit they used, the decisions list from lab-fanout-01, and a one-line statement of the first denial reason. Grade those artifacts, not the fluency of the planner output, and keep the live endpoint off until replay is stable. Shared inference survives a tool-using agent lab only when fan-out is admitted locally, with fixtures students can rerun without touching the server.