DEV Community

Cover image for How to Build Production-Ready AI Agents with LangGraph
Ciphernutz
Ciphernutz

Posted on

How to Build Production-Ready AI Agents with LangGraph

AI agents are easy to demonstrate and yet surprisingly difficult to productionize for consistent value.

A basic AI agent can receive a prompt, call an LLM, use a tool, and return a response. That is enough for a prototype.

Production systems are different.

A production AI agent needs to:

  • Manage state
  • Make controlled decisions
  • Call tools reliably
  • Handle failures
  • Maintain context
  • Support human intervention
  • Provide observability into what happened during execution

This is where LangGraph becomes useful.

This article explores how to design production-ready AI agents with LangGraph, including architecture, state management, tool execution, conditional workflows, error handling, and deployment considerations.

Why Evolve Beyond a Basic AI Agent?

A simple agent typically performs like this:

User
  ↓
LLM
  ↓
Tool
  ↓
LLM
  ↓
Response
Enter fullscreen mode Exit fullscreen mode

This works well for simple tasks.

Real-world applications often require a more controlled workflow:

User Request
     ↓
Input Validation
     ↓
Intent Detection
     ↓
State Management
     ↓
Tool Selection
     ↓
Tool Execution
     ↓
Result Validation
     ↓
Decision
   ↙     ↘
Retry   Human Review
   ↓
Final Response
Enter fullscreen mode Exit fullscreen mode

For example, an AI agent responsible for handling customer support requests may need to:

  1. Understand the user's request.
  2. Identify the customer.
  3. Retrieve account information.
  4. Check previous interactions.
  5. Decide which tool to use.
  6. Execute the tool.
  7. Validate the result.
  8. Ask for human approval for sensitive actions.
  9. Update the system.
  10. Respond to the customer.

Managing everything inside one LLM prompt quickly becomes difficult to maintain.

A graph-based architecture makes the workflow explicit and easier to control.

Understanding State in LangGraph

State is one of the most important concepts when building a production agent.

Instead of passing every piece of information manually between functions, the workflow maintains a shared state object.

A simplified state could contain:

from typing import TypedDict

class AgentState(TypedDict):
    user_input: str
    intent: str
    tool_result: str
    response: str
Enter fullscreen mode Exit fullscreen mode

Each node can read information from the state and return updates to it.

For example:

def analyze_request(state: AgentState):
    user_input = state["user_input"]

    intent = classify_intent(user_input)

    return {
        "intent": intent
    }
Enter fullscreen mode Exit fullscreen mode

Another node can consume that information:

def generate_response(state: AgentState):
    intent = state["intent"]
    tool_result = state.get("tool_result", "")

    response = generate_answer(intent, tool_result)

    return {
        "response": response
    }
Enter fullscreen mode Exit fullscreen mode

This separation makes complex workflows easier to reason about and maintain.

Designing the Agent as Nodes

A common mistake is creating one enormous agent function:

def agent():
    # classify request
    # call LLM
    # search database
    # call API
    # validate response
    # send email
    # handle errors
    # generate final response
Enter fullscreen mode Exit fullscreen mode

As the application grows, this becomes difficult to test and modify.

Instead, separate responsibilities into individual nodes:

START
  ↓
classify_request
  ↓
retrieve_context
  ↓
select_tool
  ↓
execute_tool
  ↓
validate_result
  ↓
generate_response
  ↓
END
Enter fullscreen mode Exit fullscreen mode

Each node should ideally have one clear responsibility.

For example:

def retrieve_context(state):
    context = search_knowledge_base(
        state["user_input"]
    )

    return {
        "context": context
    }
Enter fullscreen mode Exit fullscreen mode

This architecture allows individual components to be tested independently and modified more easily.

Connecting Nodes with Edges

Once the nodes are defined, the graph controls how execution moves between them.

A simple workflow can be created using StateGraph:

from langgraph.graph import StateGraph, START, END

builder = StateGraph(AgentState)

builder.add_node("analyze", analyze_request)
builder.add_node("retrieve", retrieve_context)
builder.add_node("respond", generate_response)

builder.add_edge(START, "analyze")
builder.add_edge("analyze", "retrieve")
builder.add_edge("retrieve", "respond")
builder.add_edge("respond", END)

graph = builder.compile()
Enter fullscreen mode Exit fullscreen mode

The resulting workflow is:

START
  ↓
Analyze
  ↓
Retrieve
  ↓
Respond
  ↓
END
Enter fullscreen mode Exit fullscreen mode

The benefit is that developers can see exactly how the agent is expected to execute.

Conditional Routing

Production agents rarely follow only one path.

The next step may depend on the current state or detected intent.

For example:

              Analyze Request
                     ↓
              Determine Intent
                ↙         ↘
        Knowledge         API Tool
          Search          Execution
                ↘         ↙
                  Validate
                     ↓
                  Respond
Enter fullscreen mode Exit fullscreen mode

A routing function can determine where the workflow should go next:

def route_request(state):
    intent = state["intent"]

    if intent == "knowledge":
        return "retrieve"

    if intent == "account":
        return "account_tool"

    return "respond"
Enter fullscreen mode Exit fullscreen mode

The graph can then use that decision to select the next node.

This is more predictable than asking an LLM to control every part of the application's execution.

Tool Calling in Production Agents

Tools allow an agent to interact with external systems.

Common examples include:

  • REST APIs
  • Databases
  • Search engines
  • CRMs
  • Payment systems
  • Internal services
  • File storage
  • Business applications

A production agent should not blindly execute every tool requested by an LLM.

Instead, introduce validation around tool execution.

A safer flow is:

LLM Decision
     ↓
Tool Validation
     ↓
Permission Check
     ↓
Tool Execution
     ↓
Result Validation
     ↓
Update State
Enter fullscreen mode Exit fullscreen mode

For example:

def execute_tool(state):
    tool_name = state["selected_tool"]

    if not is_allowed_tool(tool_name):
        return {
            "error": "Tool execution not permitted"
        }

    result = tools[tool_name].invoke(
        state["tool_input"]
    )

    return {
        "tool_result": result
    }
Enter fullscreen mode Exit fullscreen mode

The important principle is:

The LLM should make decisions only within boundaries defined by the application.

Handling Errors and Retries

LLM applications can fail for many reasons:

  • A tool may return an error.
  • An API may time out.
  • A model may generate invalid structured output.
  • A database may temporarily become unavailable.

A production workflow needs to account for these cases.

Instead of:

Tool
 ↓
Failure
 ↓
Agent stops
Enter fullscreen mode Exit fullscreen mode

Use a recovery flow:

Tool
 ↓
Validate
 ↓
Success?
 ↙       ↘
Yes       No
 ↓        ↓
Continue  Retry / Recover
              ↓
          Still failing?
              ↓
       Human Review /
        Error Response
Enter fullscreen mode Exit fullscreen mode

The state can contain error and retry information:

class AgentState(TypedDict):
    user_input: str
    tool_result: str
    error: str
    retry_count: int
Enter fullscreen mode Exit fullscreen mode

A routing function can determine whether another attempt should be made:

def handle_tool_result(state):
    if not state.get("error"):
        return "respond"

    if state["retry_count"] < 2:
        return "retry"

    return "human_review"
Enter fullscreen mode Exit fullscreen mode

This prevents the agent from entering an uncontrolled retry loop.

Human-in-the-Loop

Not every decision should be fully autonomous.

For sensitive operations, a human approval step may be required.

Examples include:

  • Sending high-value transactions
  • Changing customer account information
  • Approving refunds
  • Updating sensitive records
  • Sending legal or compliance-related communications

A production architecture can include:

Agent Decision
      ↓
Sensitive Action?
   ↙          ↘
 No           Yes
 ↓             ↓
Execute    Human Approval
               ↓
           Approved?
           ↙      ↘
         Yes       No
          ↓         ↓
       Execute     Stop
Enter fullscreen mode Exit fullscreen mode

LangGraph can therefore provide a controlled boundary between autonomous reasoning and business-critical actions.

Persistence and Long-Running Workflows

Some agents complete their work in a few seconds.

Others may require minutes, hours, or human intervention.

For example:

Customer Request
       ↓
Agent Analysis
       ↓
Document Review
       ↓
Human Approval
       ↓
External API
       ↓
Final Response
Enter fullscreen mode Exit fullscreen mode

In these cases, the application needs to preserve relevant state throughout the workflow.

This is one reason stateful agent architectures are important for production systems.

Instead of thinking only about:

"What should the LLM answer?"

Developers also need to think about:

"What state does the application need to preserve while the workflow executes?"

Observability: Know What the Agent Is Doing

One of the biggest differences between a demo and a production AI system is observability.

When a traditional API fails, developers can inspect logs to identify the request, service, response, and error.

Agentic systems introduce additional execution steps:

User Input
    ↓
LLM Decision
    ↓
Tool Selection
    ↓
Tool Input
    ↓
Tool Response
    ↓
Conditional Decision
    ↓
Final Output
Enter fullscreen mode Exit fullscreen mode

Every important step should be observable.

Useful information to capture includes:

  • Request ID
  • Workflow ID
  • Node execution
  • Model used
  • Tool called
  • Tool arguments
  • Execution duration
  • Errors
  • Retry count
  • Token usage
  • Final outcome

Without this information, debugging an agent can become extremely difficult and time-consuming.

Guardrails Matter More Than Prompts

A strong system prompt is useful, but it should not be the only control mechanism.

For production agents, combine model instructions with application-level controls:

LLM
 ↓
Output Validation
 ↓
Business Rules
 ↓
Permission Check
 ↓
Tool Execution
Enter fullscreen mode Exit fullscreen mode

Suppose an agent is allowed to issue refunds based on certain parameters.

Instead of allowing the LLM to directly execute:

refund(amount)
Enter fullscreen mode Exit fullscreen mode

the application can enforce a rule:

if amount > MAX_REFUND:
    require_human_approval()
Enter fullscreen mode Exit fullscreen mode

This creates a stronger safety boundary because the rule exists outside the model.

Testing a LangGraph Agent

Testing an agent requires more than checking whether the final response looks correct.

Test individual nodes as well as complete workflows.

Unit Tests

Test functions such as:

classify_request()
retrieve_context()
validate_tool_input()
route_request()
Enter fullscreen mode Exit fullscreen mode

Workflow Tests

Test complete execution paths:

Normal Request
     ↓
Expected Nodes
     ↓
Expected Final State
Enter fullscreen mode Exit fullscreen mode

Failure Tests

Simulate:

  • API failures
  • Invalid model output
  • Missing data
  • Timeouts
  • Tool errors
  • Duplicate requests

Human-Approval Tests

Verify that sensitive operations cannot bypass the approval step.

The goal is to test not only what the agent does when everything works, but also what happens when things go wrong.

A Practical Production Architecture

A production LangGraph application can be structured into several layers:

┌─────────────────────────────┐
│         API / UI Layer      │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│       Agent Entry Point     │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│          LangGraph          │
│                             │
│ Analyze → Retrieve → Tool   │
│      ↓          ↓           │
│    Route ← Validate         │
│             ↓               │
│          Response           │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│     Tools / APIs / DBs      │
└──────────────┬──────────────┘
               ↓
┌─────────────────────────────┐
│ Observability / Persistence │
└─────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Keeping these responsibilities separated makes the system easier to scale and maintain.

Final Thoughts

Building a basic AI agent is not difficult.

Building an AI agent that can reliably operate inside a real production environment is a different engineering problem.

The important shift is from:

Prompt → LLM → Response
Enter fullscreen mode Exit fullscreen mode

to:

State
  ↓
Decision
  ↓
Controlled Action
  ↓
Validation
  ↓
Recovery
  ↓
Human Intervention
  ↓
Final Outcome
Enter fullscreen mode Exit fullscreen mode

LangGraph provides a useful architecture for making these workflows explicit.

The real value is not simply adding an LLM to an application.

It is designing a system where:

  • LLMs can reason
  • Tools can act
  • Application code can enforce rules
  • Workflows can recover when something goes wrong

That is the foundation of a production-ready AI agent.

Looking to build a production-ready AI agent for your business? Explore Ciphernutz AI Agent Development to learn more.

Top comments (0)