Part 2: Pinning the Use Case and Writing Tool Contracts Like Specs

Part 2 of a series building a support-ticket agent with no framework. Part 1 covered why. This part covers Steps 1–2 of the build order: pinning the use case, and writing tool contracts. Repo: github.com/akash-pal/agent-from-scratch

Before any code, two documents: docs/use-case.md and docs/tool-contracts.md. Skipping this step is the single most common reason teams end up with an agent nobody trusts — not because the idea was bad, but because nothing downstream (evals, prompts, memory) had a fixed target to hit.

Step 1: pin the use case

Four gates, filled in before writing a line of code:

Gate	Definition
Bounded input	One support ticket: { subject, body, customer_id, order_id? }
Bounded output	Exactly one of: resolved, refund_proposed (pending approval), escalated (with a reason)
Tool count	5
Success metric	Resolution rate > 85% without escalation; escalation rate < 10%

The tool count cap matters more than it looks. An agent given 10+ tools starts hallucinating tool names and picking the wrong one — a cognitive load problem, not a dependency problem. Keeping this agent to 5 tools, covering exactly three request types (order status, refunds, KB lookups), keeps every run in the healthy 3–8 tool-call range instead of ballooning into a system that needs to be split into multiple specialist agents. (Part 7 covers the actual cost math for when a split is worth it.)

Bounded output matters too: resolved / refund_proposed / escalated isn't just documentation — it becomes a literal parseable prefix (RESOLVED:, REFUND_PROPOSED:, ESCALATED:) that the agent's final message must start with. Part 4 shows exactly how that gets parsed, and Part 5 shows why trusting that string alone turned out to be a real bug.

Step 2: tool contracts are a schema, not a docstring

This is the part that's easy to under-invest in. A tool's description field isn't a comment for future developers — it's the only thing the LLM reads to decide when to call the tool. Treat it as a specification.

Here's the actual refund_eligibility definition from src/tools/index.ts:

{
  name: "refund_eligibility",
  description:
    "Check whether an order is eligible for a refund BEFORE ever proposing one. " +
    "Orders are eligible only if delivered and within a 30-day window of order_date. " +
    "Cancelled orders are already auto-refunded and are never eligible for a manual " +
    "refund. Returns eligible, max_amount_usd, and a policy_ref explaining the decision.",
  gated: false,
  input_schema: {
    type: "object",
    properties: {
      order_id: { type: "string" },
      reason: { type: "string", description: "Customer's stated reason for the refund request" },
    },
    required: ["order_id", "reason"],
  },
}

Notice the 30-day window is stated explicitly in the tool description, not just mentioned once in the system prompt. That's not redundancy — it's a finding from later iteration (full story in Part 6): policy logic that lives only in the system prompt gets missed under load. It has to be load-bearing in the tool itself.

The full contract table, matching each tool to its schema and failure mode:

Tool	Input	Output	Failure mode
order_lookup	{ order_id }	{ status, items, total_usd, order_date }	order not found → structured error, not a guess
refund_eligibility	{ order_id, reason }	{ eligible, max_amount_usd, policy_ref }	cancelled/undelivered → eligible=false with a reason; ambiguous → escalate, never assume true
issue_refund (gated)	{ order_id, amount_usd, idempotency_key }	{ confirmation_id }	idempotency_key required — prevents double-refund on retry
kb_search	{ query }	{ articles: [...], relevance_scores }	no match → explicit empty result; agent must not fabricate an answer
send_email (gated)	{ to, subject, body, ticket_id }	{ sent, message_id }	gated behind pre-action approval (Part 5)

Three details worth calling out specifically:

Idempotency keys aren't optional. issue_refund requires one because retries happen — a flaky network call, a model retrying after a transient error — and a refund tool without idempotency protection means a retry can double-refund a customer. This is enforced in the tool executor itself, not assumed from good behavior:

// src/tools/issueRefund.ts
const issuedRefunds = new Map<string, string>(); // idempotency_key -> confirmation_id

export async function issueRefund(args: Record<string, unknown>) {
  const idempotencyKey = args.idempotency_key ? String(args.idempotency_key) : "";
  if (!idempotencyKey) return { error: "idempotency_key_required" };

  const existing = issuedRefunds.get(idempotencyKey);
  if (existing) return { confirmation_id: existing, replayed: true };
  // ... issue the refund, store it under idempotencyKey
}

Failure modes are structured, not guessed. order_lookup on a missing order returns { error: "order_not_found", order_id }, not an empty object the model has to interpret. kb_search on no matches returns an explicit empty articles: [], specifically so the system prompt can say "if this comes back empty, escalate — don't fabricate an answer."

Gated tools are marked at the contract level. issue_refund and send_email carry gated: true right in their definition. That flag is read by the policy layer (Part 5) to decide which tool calls need a human in the loop before they execute — the gating decision starts here, at the contract, not somewhere downstream.

Data source: mock, but shaped like something real

All five tools read from an in-memory mock store — src/data/mockData.ts: five customers, ten orders spanning inside/outside the refund window, five KB articles. It's deliberately fake data (this repo is a reference build, not a production system — more on that in the repo's own README), but the schemas are shaped to mirror a real commerce API's order/return/line-item structure, so swapping in a real backend later is a data-layer change, not a tool-contract rewrite.

What's next

Part 3: Build the Eval Set Before the Agent Exists → covers building the eval set — 21 cases written before the agent loop existed, and why that ordering is the actual point, not a nice-to-have.

Repo: github.com/akash-pal/agent-from-scratch