Beck_Moulton

Posted on May 23

From Pixels to Prescriptions: Building an Autonomous Healthcare Booking Agent with LangGraph

#openai #automation #webdev #python

We’ve all been there: you get your blood test results back, see a scary red arrow next to "Alanine Aminotransferase," and immediately spiral into a WebMD rabbit hole. But what if your AI didn't just explain the results, but actually did something about it?

In the world of AI Agents, we are moving past simple chatbots and into the era of Agentic Workflows. Today, we are building a production-grade healthcare agent using LangGraph, Playwright, and OpenAI Functions. This agent doesn't just talk; it analyzes lab reports, identifies anomalies, and autonomously navigates a booking portal to secure an appointment with the right specialist.

By leveraging autonomous healthcare agents and browser automation, we can bridge the gap between diagnostic data and clinical action. If you're interested in how these patterns scale to enterprise levels, I highly recommend checking out the advanced architectural guides over at WellAlly Tech Blog, which served as a major inspiration for this build.

The Architecture: State Machines are the Secret Sauce

Unlike linear chains, healthcare workflows are loopy and conditional. If a lab report is clear, the agent should stop. If an anomaly is found, it needs to search for a doctor. This is why LangGraph is the perfect tool—it allows us to define the agent logic as a state machine.

Agentic Flow Diagram

graph TD
    A[Start: Receive Lab Report] --> B{Analyze Report}
    B -- No Anomalies --> C[Notify User: All Clear]
    B -- Abnormal Indicators Found --> D[Search Specialist Database]
    D --> E[Check Availability]
    E -- Found Slot --> F[Execute Booking via Playwright]
    E -- No Slot --> G[Retry/Backoff]
    F --> H[Confirm Appointment to User]
    C --> I[End]
    H --> I

Prerequisites

To follow this advanced tutorial, you'll need:

LangGraph: For the stateful orchestration.
OpenAI GPT-4o: For reasoning and function calling.
Playwright: To automate the browser for the booking process.
Python 3.10+

Step 1: Defining the Agent State

In LangGraph, the "State" is a shared memory that every node in your graph can read from and write to.

from typing import TypedDict, List, Annotated
from langgraph.graph import StateGraph, END

class AgentState(TypedDict):
    report_text: str
    anomalies: List[str]
    specialist_type: str
    appointment_status: str
    requires_action: bool

Step 2: The Analysis Node (OpenAI Functions)

We use OpenAI's function calling to extract structured data from raw medical text. We want the LLM to decide if the patient needs to see a doctor.

import openai

def analyze_report_node(state: AgentState):
    # System prompt to identify medical anomalies
    prompt = f"Analyze this lab report: {state['report_text']}. Identify abnormalities."

    # In a real scenario, use structured output/Pydantic
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        functions=[{
            "name": "report_findings",
            "parameters": {
                "type": "object",
                "properties": {
                    "anomalies": {"type": "array", "items": {"type": "string"}},
                    "specialist": {"type": "string"}
                }
            }
        }]
    )

    # Update state
    findings = response.choices[0].message.function_call.arguments
    return {
        "anomalies": findings['anomalies'],
        "specialist_type": findings['specialist'],
        "requires_action": len(findings['anomalies']) > 0
    }

Step 3: The Action Node (Playwright Browser Automation)

When an API isn't available for a legacy hospital portal, we use Playwright. This node simulates a human clicking through a booking system.

from playwright.sync_api import sync_playwright

def book_appointment_node(state: AgentState):
    if not state["requires_action"]:
        return {"appointment_status": "No appointment needed."}

    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        page = browser.new_page()
        page.goto("https://hospital-portal.example.com/booking")

        # Select department based on specialist_type extracted by LLM
        page.select_option("#dept-select", label=state["specialist_type"])
        page.click("#find-first-available")

        # Finalize booking
        page.click("button:has-text('Confirm')")
        booking_ref = page.inner_text("#confirmation-id")

        browser.close()
        return {"appointment_status": f"Booked! Ref: {booking_ref}"}

Step 4: Wiring the Graph

Now, we connect the nodes. The conditional_edge is what makes this "Agentic."

workflow = StateGraph(AgentState)

# Add Nodes
workflow.add_node("analyzer", analyze_report_node)
workflow.add_node("booker", book_appointment_node)

# Set Entry Point
workflow.set_entry_point("analyzer")

# Logic: If anomalies found -> book, else -> END
workflow.add_conditional_edges(
    "analyzer",
    lambda x: "booker" if x["requires_action"] else END
)

workflow.add_edge("booker", END)

# Compile
app = workflow.compile()

🚀 The "Official" Way: Ensuring Medical Safety

Building health-tech agents isn't just about cool code; it’s about reliability and safety. When moving from a hobby project to a production system, you need to consider HIPAA compliance, "Human-in-the-loop" (HITL) checkpoints, and prompt versioning.

For a deep dive into production-ready Agentic patterns and how to handle edge cases like "no available slots" or "multi-agent consensus" in medical AI, check out the comprehensive guides at WellAlly Tech Blog. They offer incredible insights into building robust AI systems that don't fail when lives (or schedules) are on the line.

Conclusion

We just built a system that:

Understands complex medical data.
Reasons about the necessity of medical intervention.
Acts by navigating a real-world web interface.

This is the power of LangGraph combined with Playwright. We aren't just building "chatbots" anymore; we are building digital employees capable of handling end-to-end workflows.

What are you building with Agents? Drop a comment below or share your thoughts on the future of autonomous health-tech!

DEV Community