Shipping an Agentic RAG Assistant on AWS Lambda with LangGraph and Terraform

Ved Suthar — Fri, 30 Oct 2026 19:45:38 +0000

How I built a tool-calling ReAct agent over a private knowledge base, packaged it as a Lambda container, and provisioned all of it as code.

Most retrieval-augmented generation (RAG) demos follow one fixed path: embed the question, fetch the top-k chunks, put them in a prompt, and return whatever the model says. That works until a question needs more than one lookup, or needs arithmetic the model shouldn't guess at, or doesn't need retrieval at all.

I wanted an assistant that could decide what to do: search the knowledge base, search again with a better query, call a calculator, or just answer. I also wanted to deploy it with no hand-configured infrastructure. This post walks through the design: a LangGraph ReAct agent, FAISS for vector search, a FastAPI interface, a provider-agnostic LLM layer, and an AWS Lambda container image provisioned end-to-end with Terraform.

Why "agentic" RAG?

In classic RAG, retrieval is a step that always runs once. In agentic RAG, retrieval is a tool. The model works in a loop (the ReAct pattern: reason, act, observe) and picks which tool to call based on what it has learned so far.

That gets you three things:

Multi-hop questions. If the first search returns half the answer, the agent can write a sharper second query.
Exact computation. "What's the combined budget of projects A and B?" should go to a calculator, not to next-token prediction.
Skipping retrieval. A greeting or a follow-up about the previous answer doesn't need a vector search.

The cost is more LLM calls per question and a bit of extra complexity. LangGraph keeps that complexity manageable.

Architecture at a glance

Client ──HTTPS──▶ Lambda (container image)
                     │
                     ├─ FastAPI app (via ASGI adapter)
                     │
                     └─ LangGraph agent
                          ├─ LLM node (OpenAI or Anthropic)
                          └─ Tool node
                               ├─ retriever → FAISS index
                               └─ calculator

I split the codebase into small modules so each part could be tested on its own:

config holds typed settings (model provider, model name, top-k, index path), loaded from environment variables.
retriever loads the FAISS index and returns ranked chunks with their source metadata.
tools defines the retriever and calculator as tools the LLM can call.
llm is a thin, provider-agnostic wrapper.
graph wires the agent.
api is the FastAPI layer.

The agent graph

LangGraph models an agent as a state machine. The state is mainly the list of messages; nodes read the state and add to it; edges decide what runs next.

The ReAct loop needs two nodes and one conditional edge:

from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.prebuilt import ToolNode

tools = [retrieve_docs, calculator]
llm_with_tools = llm.bind_tools(tools)

def call_model(state: MessagesState):
    return {"messages": [llm_with_tools.invoke(state["messages"])]}

def should_continue(state: MessagesState):
    last = state["messages"][-1]
    return "tools" if last.tool_calls else END

graph = StateGraph(MessagesState)
graph.add_node("agent", call_model)
graph.add_node("tools", ToolNode(tools))
graph.add_edge(START, "agent")
graph.add_conditional_edges("agent", should_continue, ["tools", END])
graph.add_edge("tools", "agent")
app_graph = graph.compile()

The model either asks for tool calls or produces a final answer. If it asks for tools, ToolNode runs them, appends the results as tool messages, and control goes back to the model. I also set a recursion limit when invoking the graph so a confused model can't loop forever.

Writing the loop as an explicit graph, rather than hiding it inside an agent class, paid off quickly. I could see every transition, test should_continue with plain unit tests, and add nodes later (a query-rewriting step, for example) without touching the rest.

Tools: retriever and calculator

The retriever tool's docstring matters as much as its code, because the model reads it to decide when to call it:

@tool
def retrieve_docs(query: str) -> str:
    """Search the private knowledge base. Use for any question about
    internal documents. Returns the most relevant passages with sources."""
    hits = retriever.search(query, k=settings.top_k)
    return "\n\n".join(f"[{h.source}] {h.text}" for h in hits)

Returning the source with each passage lets the model cite where an answer came from, and makes bad retrievals easy to spot while debugging.

The calculator is deliberately narrow. It evaluates arithmetic expressions with a small safe parser rather than Python's eval, because tool inputs are model-generated text and should be treated as untrusted.

FAISS: simple, local, fast to load

For a private knowledge base of modest size, FAISS is a good fit: no separate database to run, and the index is just a file. I build the index offline (chunk documents, embed them, write the index plus a metadata file mapping vector IDs to text and source), then ship it with the application.

On Lambda that has a nice property: the index loads from the container's filesystem at cold start and stays in memory for warm invocations. I load it once at module level, not per request, so warm requests skip that cost.

The trade-off is that updating the knowledge base means rebuilding the index and redeploying. For a knowledge base that changes rarely, I'll take that over running a vector database.

A provider-agnostic LLM layer

I didn't want the agent tied to one vendor. The llm module exposes one factory that returns a chat model supporting tool binding, chosen by config:

def get_llm(settings: Settings):
    if settings.provider == "openai":
        return ChatOpenAI(model=settings.model, temperature=0)
    if settings.provider == "anthropic":
        return ChatAnthropic(model=settings.model, temperature=0)
    raise ValueError(f"Unknown provider: {settings.provider}")

The rest of the code depends only on this interface. Switching providers is a config change, which also makes it easy to compare how different models behave with the same tools and prompts. API keys come from the environment and never live in the code or the image.

FastAPI on Lambda

FastAPI gives a typed request and response schema and automatic docs during local development. To run the same app on Lambda, I wrap it with an ASGI-to-Lambda adapter such as Mangum, so one codebase serves both uvicorn locally and Lambda in production.

app = FastAPI()

class Ask(BaseModel):
    question: str

@app.post("/ask")
def ask(body: Ask):
    result = app_graph.invoke(
        {"messages": [("user", body.question)]},
        config={"recursion_limit": 10},
    )
    return {"answer": result["messages"][-1].content}

handler = Mangum(app)

Packaging as a Lambda container image

Lambda's zip packages have tight size limits, and FAISS plus the LLM SDKs add up. A container image avoids that. The Dockerfile starts from the AWS Lambda Python base image, installs pinned dependencies, copies the code and the FAISS index, and points the command at the handler:

FROM public.ecr.aws/lambda/python:3.11
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app/ ${LAMBDA_TASK_ROOT}/app/
COPY index/ ${LAMBDA_TASK_ROOT}/index/
CMD ["app.api.handler"]

I can run the same image locally with Lambda's runtime interface emulator, which caught packaging mistakes before they reached AWS.

Terraform: zero manual setup

The goal was that a fresh AWS account could go from nothing to a working HTTPS endpoint with terraform apply. The configuration provisions:

an ECR repository for the image;
an IAM role for the function with only the permissions it needs, including logging;
the Lambda function using the image, with memory, timeout, and environment variables (provider, model, top-k) set in code;
an HTTPS endpoint in front of the function.

resource "aws_lambda_function" "agent" {
  function_name = "agentic-rag"
  package_type  = "Image"
  image_uri     = "${aws_ecr_repository.agent.repository_url}:${var.image_tag}"
  role          = aws_iam_role.lambda.arn
  timeout       = 60
  memory_size   = 1024
  environment {
    variables = { LLM_PROVIDER = var.llm_provider }
  }
}

Two settings deserve attention. The timeout has to cover several LLM round trips, because an agentic question can take more than one model call. Memory on Lambda also scales CPU, which affects how quickly the FAISS index loads. I treat both as variables and tune them rather than hard-coding guesses. Secrets such as API keys should come from a secrets service, not plain Terraform variables in version control.

Having everything in Terraform also makes teardown trivial, which matters for a side project you don't want billing you quietly.

Testing

Because each component is separate, most tests don't need an LLM or AWS:

the retriever is tested against a tiny fixture index;
the calculator is tested on valid and malicious inputs;
the routing function is tested with hand-built messages that do and don't contain tool calls;
the graph is tested end-to-end with a fake LLM that returns scripted tool calls.

That last test is the most useful: it checks the wiring of the loop deterministically, without paying for or depending on a real model.

What I learned

Tool descriptions are prompts. Vague docstrings led to the model skipping retrieval or over-calling it. Clear "use this when" language fixed most routing problems.
Make the loop explicit. An explicit graph is easier to debug and extend than an opaque agent executor.
Bound everything. Recursion limits, Lambda timeouts, and a narrow calculator keep an autonomous loop predictable.
Infrastructure as code from day one. It was more work up front than clicking through the console, and it was worth it the first time I rebuilt the stack.

What's next

I'd like to add a query-rewriting node before retrieval, evaluation on a fixed question set to compare providers and chunking strategies, and response streaming. If you're building something similar, I'd be glad to compare notes.

Code and setup details: github.com/Ved210105

DEV Community: Ved Suthar