Building a Med-Tech Powerhouse: Creating an Autonomous Health Agent with LangGraph and Playwright

In the rapidly evolving landscape of Autonomous Agents, the intersection of healthcare and AI is where things get truly life-changing. We've moved beyond simple chatbots; we are now building systems capable of "reasoning" through medical data and taking real-world actions. Today, we are diving deep into building a Health Agent that doesn't just read lab reports but acts on them using a sophisticated LangGraph workflow, OpenAI Tool Calling, and Playwright automation.

If you've been looking for a way to master LLM orchestration and agentic workflows, this guide is for you. We will build a pipeline that detects abnormalities in a liver function test and automatically searches for a specialist to book an appointment. For those looking to scale these patterns into enterprise-grade systems, I highly recommend checking out the advanced production-ready examples over at WellAlly Tech Blog, which served as a major inspiration for this architecture. 🚀

The Architecture: Reasoning with State

Traditional linear chains fail when logic requires loops or conditional branching based on tool outputs. That’s where LangGraph shines. It allows us to define a state machine where nodes represent functions and edges represent the transition logic.

graph TD
    A[User Uploads Lab Report] --> B{Analyze Report}
    B -- Normal --> C[Notify User: All Good]
    B -- Abnormal Detected --> D[Search for Specialist]
    D --> E{Specialist Found?}
    E -- Yes --> F[Execute Booking via Playwright]
    E -- No --> G[Notify User: Manual Action Required]
    F --> H[Final Confirmation]
    G --> H

Prerequisites

To follow this tutorial, you'll need the following stack:

• LangGraph: For the agentic state machine.
• OpenAI GPT-4o: For high-accuracy tool calling.
• Playwright: To handle the browser automation for the booking process.
• FastAPI: To expose our agent as a modern web service.
• Pydantic: For strict data validation.

Step 1: Defining the Medical Schema

First, we need to ensure the LLM understands exactly what it's looking for in a liver function test (LFT). We use Pydantic to enforce this structure.

from pydantic import BaseModel, Field
from typing import List, Optional

class LabIndicator(BaseModel):
    name: str = Field(description="Name of the indicator, e.g., ALT, AST, Bilirubin")
    value: float
    unit: str
    is_abnormal: bool
    reference_range: str

class LabReport(BaseModel):
    patient_name: str
    indicators: List[LabIndicator]
    summary: str
    requires_followup: bool

Step 2: Crafting the Tools

The Agent needs "hands" to interact with the world. We'll define two tools: search_specialist and book_appointment.

from langchain_core.tools import tool

@tool
def search_specialist(department: str, location: str):
    """Search for top-rated doctors in a specific department and location."""
    # Logic to query a medical database or API
    return [
        {"id": "doc_101", "name": "Dr. Smith", "specialty": "Hepatology", "available": "2023-11-25T10:00:00"},
        {"id": "doc_102", "name": "Dr. Wong", "specialty": "Gastroenterology", "available": "2023-11-26T14:30:00"}
    ]

@tool
async def book_appointment(doctor_id: str, appointment_time: str):
    """Executes the booking on the hospital portal using Playwright."""
    from playwright.async_api import async_playwright

    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=True)
        page = await browser.new_page()
        # Simulated booking flow
        await page.goto("https://mock-hospital-system.com/book")
        await page.fill("#doctor-id", doctor_id)
        await page.fill("#time", appointment_time)
        await page.click("#confirm-btn")
        screenshot_path = f"confirmation_{doctor_id}.png"
        await page.screenshot(path=screenshot_path)
        await browser.close()
        return f"Successfully booked. Confirmation saved to {screenshot_path}"

Step 3: Orchestrating the Graph

Now, we define the LangGraph logic. We create a node for the LLM and a "conditional edge" that decides whether to call a tool or finish.

from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
from typing import TypedDict, Annotated, Sequence
from langchain_openai import ChatOpenAI

class AgentState(TypedDict):
    messages: Annotated[Sequence[dict], "The sequence of messages in the conversation"]

# Define the Model with Tool Binding
model = ChatOpenAI(model="gpt-4o", temperature=0).bind_tools([search_specialist, book_appointment])

def call_model(state):
    response = model.invoke(state["messages"])
    return {"messages": [response]}

# Define the Graph
workflow = StateGraph(AgentState)

workflow.add_node("agent", call_model)
workflow.add_node("action", ToolNode([search_specialist, book_appointment]))

workflow.set_entry_point("agent")

# Logic: If model calls a tool, go to 'action', otherwise end
def should_continue(state):
    last_message = state["messages"][-1]
    if not last_message.tool_calls:
        return END
    return "action"

workflow.add_conditional_edges("agent", should_continue)
workflow.add_edge("action", "agent")

app = workflow.compile()

The "Official" Way: Best Practices for Health Agents

When building agents that handle sensitive medical data or automated actions, error handling and "Human-in-the-loop" (HITL) checkpoints are non-negotiable. While the code above is a functional prototype, production systems require robust audit logs and retry mechanisms.

For a deeper dive into production-grade agentic design patterns, including how to implement secure HITL with LangGraph, check out the specialized guides at WellAlly Tech Blog. They provide extensive documentation on securing LLM outputs and managing complex state in regulated environments. 🥑

Step 4: Serving with FastAPI

Finally, we wrap everything in a FastAPI endpoint to allow users to submit their lab data.

from fastapi import FastAPI, BackgroundTasks

api = FastAPI()

@api.post("/process-report")
async def process_report(report_text: str):
    initial_state = {
        "messages": [
            {"role": "system", "content": "You are a health assistant. Analyze this report. If abnormalities exist in liver metrics, find a hepatologist and book an appointment."},
            {"role": "user", "content": report_text}
        ]
    }

    final_output = await app.ainvoke(initial_state)
    return {"status": "Complete", "history": final_output["messages"][-1].content}

Conclusion: The Future is Agentic

By combining LangGraph for decision logic, OpenAI for medical interpretation, and Playwright for real-world execution, we've created a prototype that demonstrates the power of autonomous health systems. No more manual searching, no more "Dr. Google" anxiety—just a streamlined path from diagnosis to treatment.

What's next for your Agent?

• [ ] Add a "Human-in-the-loop" step to verify the appointment before booking.
• [ ] Integrate Twilio to SMS the confirmation to the user.
• [ ] Check out WellAlly Tech for more advanced AI tutorials!

If you enjoyed this build, drop a comment below and let me know what agent you want to see next! 💻🔥