🚀 Technical Briefing: This tutorial is part of our deep-dive series on Agentic Workflows at Gate of AI. For the full technical breakdown, interactive code sandbox, and the native Arabic translation, visit the original article here.

Tutorial · Advanced

Build a LangGraph API with FastAPI

Learn how to plan an OpenAI-compatible API boundary around a LangGraph workflow. This guide focuses on the verified architectural distinction between an application’s workflow and its public HTTP contract, and on the compatibility checks required for ordinary responses and server-sent events (SSE). It does not present an unverified, copy-and-run RAG implementation.

What this tutorial covers

A client may expect an OpenAI-shaped Chat Completions request and response, while the application behind the endpoint is a LangGraph workflow. Those are different concerns. The request and response format belongs to the public interface; state, routing, tools, model calls, streaming, persistence, and human-in-the-loop execution belong to the workflow implementation.

This separation is the central design point for a LangGraph service exposed through FastAPI. A compatibility adapter receives and validates the public request, invokes the graph using an application-owned representation, and assembles a response in the format expected by the client. If streaming is offered, a separate SSE serializer must emit events with the intended semantics. Keeping those responsibilities distinct makes it possible to reason about each boundary instead of assuming that a graph’s internal events automatically constitute a compatible public API.

The verified material supports this architecture and highlights a critical testing lesson: a successful demonstration of one request proves only a narrow form of transport compatibility. It does not show that the endpoint honors the full request surface, preserves expected response fields, streams valid semantics, or behaves correctly through a public gateway. The steps below therefore focus on defining and validating that contract rather than asserting a particular unverified codebase, model version, database, or approval policy.

Prerequisites and scope

• Familiarity with HTTP request and response formats.
• Basic understanding of FastAPI routes and LangGraph workflows.
• A clearly documented client-facing request and response contract.
• A test plan for both non-streaming responses and SSE, if streaming is part of the interface.

The available sources do not specify a current package matrix, model availability, dependency pins, a vector database, or a particular RAG design. This tutorial intentionally does not invent them. Select versions and infrastructure for your deployment and verify them against the applicable official documentation before implementation.

Step 1: Define the public contract before building the graph adapter

Write down what clients are allowed to send and what they are entitled to receive. For an OpenAI-compatible endpoint, begin with the request shape your intended clients use, including the fields your service will support. Then define the response shape, including which response fields the adapter will return and how it represents the final graph output.

Do not equate a familiar-looking JSON body with complete compatibility. The verified compatibility analysis makes this distinction explicit: accepting an OpenAI-shaped Chat Completions request, invoking a graph, and returning a choices array demonstrates a happy path for one request. It does not, by itself, prove that the endpoint honors the broader request surface or preserves response fields clients rely on.

Create a contract checklist that answers these questions before implementation:

• Which request fields will the endpoint accept?
• Which fields will be validated, used, ignored, or rejected?
• How will the adapter translate an accepted request into graph input?
• Which response fields will be returned, and how will the final graph message map to them?
• What should a client observe when the request is invalid or the graph cannot provide a final result?
• Will the endpoint support streaming, non-streaming responses, or both?

These are design and test questions, not claims that a particular implementation automatically answers them. Record the intended behavior and test it. Avoid silently claiming support for fields or behaviors that the adapter does not implement.

Step 2: Keep the API adapter separate from LangGraph state

Use a boundary layer to translate between the public protocol and the application workflow. Conceptually, the service has five parts: an OpenAI client sends a request; a FastAPI route receives it; a validator and compatibility adapter check and translate it; the adapter invokes LangGraph; and a response assembler serializes the result. The response assembler can then select a non-streaming serializer or an SSE serializer, depending on the request and the service’s documented behavior.

Think of this as a responsibility map rather than an implementation recipe:

1. Request validation: Check that the incoming public request matches the contract the endpoint actually supports.
2. Request translation: Convert accepted public fields into the application input expected by the graph.
3. Workflow execution: Let LangGraph represent application orchestration, rather than forcing internal graph state to mimic an external wire format.
4. Response assembly: Convert the graph’s final result into the documented public response.
5. Protocol serialization: Use the appropriate serializer for an ordinary response or SSE stream.

This division preserves an important architectural boundary: LangGraph’s state and event models do not need to look like the OpenAI wire protocol. The application can represent its own workflow, while the adapter owns the public contract. That is useful when the workflow changes without requiring every internal graph detail to become part of the API.

When reviewing an implementation, trace one request through each boundary. Verify what the route receives, what the validator accepts, what input reaches the graph, and what the response assembler returns. Do not assume that because the graph produced a final message, the serialized HTTP response necessarily meets the client contract.

Step 3: Handle ordinary responses as a distinct compatibility case

Start with non-streaming requests if that is part of the intended service. Exercise the complete path: submit a representative request, validate it, invoke the graph, obtain its final result, and serialize that result into the documented response shape. Confirm that the response contains the fields your client contract promises—not merely a field that makes one basic client example appear to work.

Test more than the simplest successful request. Include requests that vary the supported fields and requests that should fail validation. For every case, compare the actual response against an explicit expectation. The objective is to discover whether the adapter implements the contract, not merely whether a client can connect to the route.

Keep the graph’s internal representation out of the public response unless the API contract deliberately exposes it. The verified architecture recommends that the graph represent the application and that the adapter own the public contract. This keeps compatibility decisions in one place and prevents changes in internal state or graph events from accidentally changing the external response.

The sources do not specify a complete list of OpenAI request fields or response fields that every service must support. Do not imply universal compatibility based on an assumed list. Define the profile your endpoint implements, document its limits, and test those specific promises with the clients and gateway that will use it.

Step 4: Treat SSE streaming as its own protocol implementation

Streaming is not simply the non-streaming response split into arbitrary pieces. The verified analysis notes that a FastAPI route can add server-sent events and that an official SDK may consume them, but also warns that this observation is not proof of valid streaming semantics. The public stream must be tested as a sequence and as a whole.

A graph may hold, transform, or combine its events before the adapter emits OpenAI-shaped chunks. A gateway may also inject usage events or normalize deltas. For these reasons, validating each event in isolation is necessary but insufficient. A stream can consist of individually plausible events and still produce an invalid or misleading sequence when a client consumes the complete response.

Build a streaming test that records the exact events the client receives. Reconstruct the stream from those events and assess the complete sequence against the behavior the endpoint promises. Check, at minimum, that the sequence can be consumed as intended by the actual client, that the response ends according to the documented convention, and that any gateway in the deployed path does not change the resulting semantics.

Do not claim that a stream is compatible because it returns an SSE content type or emits chunks. Transport delivery and protocol behavior are related but distinct. The verified source explicitly cautions that transport compatibility for one request does not establish valid streaming semantics or resilience through a public gateway. Treat each of those as a separate test objective.

Step 5: Test the complete path, including the gateway

Organize compatibility tests around what the client observes. For ordinary responses, compare submitted requests and returned fields with the endpoint contract. For streaming, capture and reconstruct the full sequence. Where the service is deployed behind a gateway, run the same tests through that gateway rather than relying only on direct access to the FastAPI route.

A useful test matrix has independent cases for:

• Accepted requests using the fields the endpoint documents.
• Requests with invalid or unsupported input.
• Non-streaming responses and their documented fields.
• SSE event delivery and the meaning of the reconstructed sequence.
• Behavior when the request passes through the intended gateway.

Keep the public contract and test expectations together. When the adapter or graph changes, rerun the tests that cover the affected boundary. When a gateway changes, test the externally visible behavior again. This does not establish a particular performance, availability, or security guarantee; it provides a disciplined way to check that the observable interface continues to meet its documented behavior.

The Hugging Face analysis provides a practical warning for teams building compatibility endpoints: a successful demo is easy to produce, while a precise compatibility specification is harder. Treat the demo as an initial connectivity check, not as evidence that all client interactions, response fields, stream sequences, or gateway paths have been validated.

Where persistence and human review fit

LangGraph is described in the verified context as an orchestration runtime for stateful workflows, streaming, persistence, and human-in-the-loop execution. Those capabilities concern application workflow behavior. They do not remove the need for a well-defined external API adapter, and they do not establish a particular persistence backend or approval design.

If a workflow uses state, persistence, or human review, decide how those application concepts should be represented to the caller. Do not automatically expose internal graph events as public protocol events. Instead, decide what the API contract promises and translate the relevant application result at the boundary. The supplied sources support the architectural separation, but do not validate any particular approval endpoint, database schema, access-control policy, or audit-log implementation.

A research publication in the supplied context describes a backend implemented with FastAPI, PostgreSQL for session management, LangGraph for workflow automation, and OpenAI GPT-3.5-turbo. That is an example reported in a publication, not a general requirement or proof that the draft’s SQLite design, model choice, or workflow code is correct. Treat it as evidence that these technologies have been used together in one described system—not as a deployment recipe.

Common mistakes to avoid

• Calling one successful request full compatibility. A happy-path response proves only that one path worked. Define and test the request and response contract more broadly.
• Mixing workflow state with the public wire format. Let LangGraph represent the application and keep protocol-specific behavior at the adapter boundary.
• Assuming SSE chunks are correct because they arrive. Validate the reconstructed sequence and its meaning to the client.
• Testing only without the gateway. A gateway can affect events or response behavior, so include the actual public path in compatibility checks.
• Claiming support the adapter has not implemented. Document the endpoint’s supported profile and test each promise.
• Copying unverified example code into production. Confirm library APIs, versions, persistence behavior, and deployment requirements against current documentation.

Key takeaways

Building an OpenAI-compatible API around LangGraph is an exercise in boundary design as much as workflow orchestration. FastAPI can serve as the HTTP layer, but the endpoint needs a clearly specified compatibility adapter between the client-facing contract and the graph. Keep graph state and events in the application layer; validate and serialize the public protocol at the edge.

For non-streaming requests, test the supported request surface and the response fields clients receive. For SSE, validate the stream as a complete sequence, not only event by event. Finally, test through the intended gateway. A working demo is a useful first check, but the verified sources do not support treating it as proof of full request, response, streaming, or gateway compatibility.

Sources

• Measuring OpenAI Compatibility for LangGraph Endpoints — compatibility boundaries, request and response handling, SSE validation, and gateway considerations.
• On-Device PHI Anonymization, Medical Report Retrieval … — source summary describing one FastAPI, PostgreSQL, LangGraph, and OpenAI GPT-3.5-turbo backend.
• An Agentic AI Approach with MARARE — source summary describing a Python and FastAPI backend using LangGraph and the OpenAI Realtime SDK.

Comments

[hilde43]

how do you manage stateful node outputs across requests in the FastAPI router, do you rely on in‑memory caches or external storage? also, have you benchmarked the latency impact of graph depth on uvicorn workers