Define Success Before You Write a Single Line: The Pre-Code Contract That Changes Everything

You're shipping an LLM agent tomorrow. The logic looks sound. But here's the question nobody asks until something breaks: what does success actually mean to this agent, and how will you know when it fails silently?

Most builders think observability is something you bolt on after shipping. You log the tokens, you watch the latency, you set up alerts on execution errors. But that's reactive. By then, the silent failures are already in production—the agent called the model successfully, got a valid response back, and did absolutely nothing with it.

The insight that changes this is simple: define your success contract before you code. Not after. Not when something breaks. Now.

What's a Success Contract?

A success contract answers three specific questions about your agent:

1. What counts as a successful execution? Not "the model returned a response"—that's framework success. What did your business need to happen? An email sent? A document filed? A decision made and handed off to a human? That's your real success criteria.

2. How does this agent fail silently? Every agent has failure modes that don't trigger an error. The model returns valid output but it's hallucinated nonsense. The agent loops endlessly on retries. The agent makes a call to an external service that succeeds (HTTP 200) but the actual consequence never lands. The agent returns early and skips a critical step. List them.

3. What side effects must you verify? Beyond "did the code execute," what real-world consequences must actually land? If your agent publishes a document, can you read it back from the target system to confirm it really published? If it sends a notification, can you query the inbox to verify it arrived? These verifications are your safety net.

Most builders skip all three and wonder why their agents produce silent zeros in production.

The Difference This Makes in Your Code

Here's where it gets practical. Let's say you're building an agent that files customer support tickets:

Without a success contract, you might write:

def file_ticket(customer_id, issue):
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": f"File a ticket for {customer_id}: {issue}"}]
    )
    return response.choices[0].message.content

You call the model, you get output back, you return it. Framework success. Ship it.

Three weeks later, the agent is filing tickets that don't actually land in your ticket system. It reports success. The logs show a response. But the ticket creation failed silently downstream, and your customers are waiting for help that never arrives.

With a success contract, you'd ask three things during design:

• What's success? A ticket filed AND an ID returned from our ticketing API AND that ID queryable immediately after.
• How does it fail silently? The agent hallucinates a ticket ID that doesn't exist. The API accepts the request but doesn't actually create the ticket. The agent loops indefinitely trying to format the ticket. The agent gets interrupted mid-execution.
• What must we verify? After the agent reports success, we read the ticket ID back from the API. If we can't find it, the execution failed—even though the agent said it succeeded.

Your code changes:

def file_ticket(customer_id, issue):
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": f"File a ticket for {customer_id}: {issue}"}],
    )
    content = response.choices[0].message.content

    # Extract the ticket ID the agent claims to have created
    ticket_id = extract_ticket_id(content)

    # Verify it actually exists in our system
    try:
        ticket = verify_ticket_exists(ticket_id)
        return {"success": True, "ticket_id": ticket_id, "verified": True}
    except TicketNotFound:
        # Agent reported success, but the ticket doesn't exist
        # This is a silent failure caught at design time
        return {"success": False, "ticket_id": ticket_id, "verified": False, "error": "ticket_not_found"}

The contract is now explicit in your code. Observability isn't a separate thing you add later—it's baked into how you think about the agent's contract with reality.

Why This Changes Your Architecture

When you define your success contract upfront, three things happen:

1. You catch design flaws before shipping. If you can't actually verify that a consequence landed, you discover that problem during design, not during the post-incident retrospective. Maybe the target system doesn't expose verification endpoints. Maybe there's a race condition in the handoff. You have time to fix it.

2. You instrument for the right things. Instead of generic "did the model run" metrics, you're now tracking what actually matters: verification successes vs. failures, the latency of the verification step, which failure modes you're hitting in production. Your observability answers the question that matters—"Is this agent doing what it's supposed to?"—not just "Is this agent running?"

3. You document the failure modes your team actually cares about. When your agent starts failing in production, you're not hunting through logs trying to guess what "success" was supposed to mean. You have a written contract that says "success = execution returns AND verification confirms AND no silent zeros in the output." Everyone on the team knows what they're looking for.

Making It Observable

Once your contract is defined, the observability piece becomes straightforward. You're not trying to invent signals—you're instrumenting the contract you already wrote.

With AI Agents Control Tower, you'd wire up checks for exactly these cases:

• Silent Failure detection — when the agent returns output but verification fails.
• Correctness Check — optionally, a rubric where you define what "correct output" looks like for this agent, and an AI judge grades every execution against it.
• Action Confirmation — when a real-world consequence must land (file created, email sent, system updated), your code verifies it and reports back whether the consequence actually materialized.

But here's the key: you're not relying on the tool to invent these checks. You defined the contract. The tool just helps you observe it.

The Template: Three Questions to Answer Now

Before you write your next agent, answer these:

1. Success = ? (Not "the model ran"—what real consequence must land?)
2. Silent failures = ? (List the ways it could look successful but do nothing.)
3. Verify = ? (What can you query or check to confirm the consequence actually happened?)

Write those down. Design your agent around them. Wire up the verification into your code. Then ship, and watch the right signals.

The builders who do this catch their silent failures during design, not during the post-incident call. They're not guessing what went wrong—they built visibility into the contract from the start. Their agents fail loudly, or they don't fail at all.

You can integrate these checks into your observability pipeline with something like https://agents.opsveritas.com, which lets you define exactly these kinds of success contracts and catch when your agents produce silent zeros. But the contract itself? That's your job as the builder. Define it now. The observability tool just watches it for you.

---

Start with the contract. Everything else follows.

Comments

[Kent Bodrov]

The ticket readback needs to check more than existence: the right customer, issue and operation reference must match this request. A hallucinated ID might resolve to an older ticket, and an immediately missing ticket might just be eventually consistent. I'd keep an unconfirmed state with bounded reconciliation rather than treating every failed read as a failed creation. Retrying the creation at that point could produce the duplicate the success check was supposed to prevent.