How to Build an AI Agent: From One Prompt to a Production Setup

You have had the moment. You write one prompt, the model nails it, and you think: I could turn this into a product. Then you try, and the gap between a clever prompt and a system you can actually run in production turns out to be almost all of the work.

This post is that gap, walked end to end.

What an agent actually is

Strip away the hype and an agent is three things: a model, instructions, and tools. OpenAI's practical guide to building agents defines one as a system that independently accomplishes tasks on your behalf, using an LLM to manage the workflow and tools to interact with external systems.

Anthropic's Building effective agents adds the distinction that saves you the most pain:

• Workflows are LLMs and tools orchestrated through predefined code paths.
• Agents are LLMs directing their own process and tool usage, deciding their own route through the task.

Their rule of thumb: find the simplest solution possible and only add agentic complexity when it earns its keep. Plenty of things people call agents are really a workflow with one LLM step, and that is fine. It is also cheaper and much easier to debug.

For this post, one example all the way through: a GitHub issue triage agent. It reads new issues on your repo, applies labels, and drafts a first reply for a maintainer to approve.

Step 1: Start with one prompt

Before any loop exists, write the system prompt for the job and run it as a single, tool-less call:

You are a triage assistant for an open source repo.
Read the issue the user pastes in and return JSON with:
- "labels": up to 3 labels from this list only: bug, feature,
  docs, question, duplicate, good-first-issue
- "priority": one of low, medium, high
- "draft_reply": a short, friendly first response
Never invent a label that is not on the list.
If the issue is unclear, say what is missing instead of guessing.

Test it against twenty old issues by hand. If this single call does not label well, an agent will not fix it. It will just fail in more interesting ways.

Step 2: Give it hands (the loop)

Anthropic describes agents as "typically just LLMs using tools based on environmental feedback in a loop." The whole architecture fits in a paragraph: the model asks for a tool, your code runs it, the result goes back into the conversation, and you repeat until the model says it is done or hits a stopping condition.

For the triage agent, the tools are small: get_open_issues, add_labels, save_draft_reply. Here is the shape of the loop as a minimal sketch (whatever SDK or model you use, the shape is the same):

def run_agent(client, system_prompt, task, tools, max_steps=10):
    messages = [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": task},
    ]
    for _ in range(max_steps):
        response = client.chat(messages, tools=tool_schemas)
        if response.tool_calls:
            for call in response.tool_calls:
                result = tools[call.name](**call.arguments)
                messages.append(tool_message(call, result))
        else:
            return response.text  # the agent decided it is done
    raise RuntimeError("hit max_steps without finishing")

Keep max_steps. That line is not decoration. It is the difference between a bug and a bug that bills you.

Step 3: Check you actually need an agent

Before building anything, OpenAI's guide suggests checking the work against three cases where agents earn their complexity:

1. Complex decision-making with nuanced judgment, like refund approvals.
2. Rules that got too tangled to maintain, like vendor security reviews.
3. Heavy unstructured data, like interpreting documents or free text.

Issue triage qualifies on the first and third: the right label depends on context, and the input is free text written by strangers. If your use case fails all three checks, a deterministic script will beat an agent on cost, speed, and reliability, and it will not hallucinate.

Step 4: Guardrails before autonomy

This is the part demo videos skip. Both guides land on the same ideas:

• Least-privilege tools. Start read-only. Let the agent draft replies, not post them. Write access is something an agent earns after it has behaved.
• Human checkpoints. Anthropic notes that agents can pause for human feedback at checkpoints or when blocked. For triage, that means drafts go to a queue and a human hits approve.
• Stopping conditions. Max iterations, a cost ceiling per run, and timeouts. Anthropic is blunt about why: autonomy compounds errors and costs, so they recommend extensive testing in sandboxed environments before agents touch anything real.
• Layered defenses. OpenAI frames guardrails as layers: input classifiers, tool risk ratings, rules-based protections like blocklists, and output validation, rather than one big filter.

Step 5: The production part nobody posts about

The loop is maybe ten percent of the work. Production is the other ninety:

• Evals. Build a set of past issues with their correct labels and run the agent against it after every prompt change. Version your prompts like code, because they are.
• Traces. Log every model call and every tool result. When the agent mislabels something at 3am, the trace is the only way to find out why.
• Cost and latency budgets. Forty cents per triage run is fine for one repo. It is not fine for four hundred.
• Deployment. Webhook to queue to worker, with results stored for review. Do not run the agent inside the web request; LLM latency will eat your timeouts.

The honest version

Nobody's first agent is a good agent. The order that works is prompt, then tools, then guardrails, then evals, with autonomy last. Autonomy is the reward for the boring parts being solid.