The cost of a serverless agent isn't the Lambda. It's the loop.

When I moved Wingman onto Lambda, I assumed the thing I'd have to watch was compute duration. That's the serverless mental model everyone starts with: you pay per millisecond, so keep the function fast and cheap.

That model was wrong for an agent. The Lambda bill turned out to be the boring, predictable part. The two things that actually move cost and latency are the cold start and the retry loop, and neither of them shows up where you'd look first.

Here's what the code taught me.

The setup: what Wingman actually is

Wingman is a LangGraph worker-evaluator agent. A worker node (running gpt-4o-mini) does the task. An evaluator node (also gpt-4o-mini) grades the answer against a success criteria. If the criteria isn't met, control routes back to the worker for another attempt, up to five times.

It runs as a container image on Lambda, fronted by API Gateway and CloudFront, with session history in DynamoDB. The compute is stateless: every request rebuilds the conversation from DynamoDB, so there's no persistent checkpointer to worry about across cold and warm starts. I wrote about that state design in an earlier post. This one is about what it costs to run.

Cold start #1: the image

A Lambda container cold start begins by unpacking the image. Wingman's image carries langchain, langgraph, langchain-community, langchain-experimental, boto3, plus reportlab and sendgrid for the agent's tools. That's not a small pile of Python.

The one decision I'm glad I made is in the Dockerfile. The project has a local dependency group with playwright, gradio, and uvicorn, and a dev group on top of that. None of those belong in Lambda. So the build exports only the core deps and drops the rest:

RUN uv export \
      --no-dev \
      --no-group dev \
      --no-group local \
      --format requirements-txt \
      -o /tmp/requirements.txt \
    && pip install -r /tmp/requirements.txt --no-cache-dir \
      --target "${LAMBDA_TASK_ROOT}"

Playwright alone would have pulled a browser runtime into the image. Keeping it in the local group means it exists for running the agent on my laptop and never ships to Lambda. Smaller image, faster unpack, faster cold start. This is the kind of thing you only notice you did right when you compare it to the version where you didn't.

Cold start #2: the init you pay for on import

The bigger cold-start cost is in the handler, and it's easy to miss because it happens at import time:

# backend/main.py — runs once per new execution environment
wingman = Wingman()
wingman.setup()

_dynamodb = boto3.resource("dynamodb", region_name=os.getenv("AWS_REGION", "us-east-1"))
_table = _dynamodb.Table(_dynamodb_table_name)

setup() isn't cheap. It builds the tool list, constructs two ChatOpenAI clients, binds tools to the worker, wraps the evaluator in structured output, and compiles the LangGraph:

def setup(self):
    self.tools = get_tools()
    worker_llm = ChatOpenAI(model="gpt-4o-mini")
    self.worker_llm_with_tools = worker_llm.bind_tools(self.tools)
    evaluator_llm = ChatOpenAI(model="gpt-4o-mini")
    self.evaluator_llm_with_output = evaluator_llm.with_structured_output(EvaluatorOutput)
    self._build_graph()

Putting this at module level is the right call. It runs once when a new execution environment spins up, then every warm invocation reuses the same compiled graph and the same clients. If I'd built the graph inside the request handler instead, I'd be paying graph compilation on every single /api/chat call, warm or not. Module-level init trades a heavier cold start for near-zero per-request setup, which is the trade you want when warm invocations vastly outnumber cold ones.

The lever for cold-start speed is memory:

resource "aws_lambda_function" "backend" {
  timeout     = 300   # 5 minutes
  memory_size = 1024
}

On Lambda, CPU scales with memory. At 1024MB you get roughly a full vCPU, so bumping memory doesn't just buy RAM headroom, it makes that import-time init finish faster. Memory is a latency knob here, not only a capacity one.

The real cost: the retry loop, not the duration

Now the part that surprised me.

Look at the timeout: 300 seconds. Lambda's max is 900. That's a generous ceiling for a function whose real job takes a few seconds. So why so high? Because the retry loop can run the worker up to five times, and each turn is a full round of model calls plus whatever tools the worker decides to invoke.

Here's the accounting that matters. turn_count only increments in the evaluator:

def evaluator(self, state: State) -> Dict[str, Any]:
    ...
    return {
        ...
        "turn_count": state.get("turn_count", 0) + 1,
    }

def route_based_on_evaluation(self, state: State) -> str:
    if state["success_criteria_met"] or state["user_input_needed"]:
        return "END"
    if state.get("turn_count", 0) >= self.MAX_TURNS:
        return "END"
    return "worker"

So a single /api/chat request is not one model call. In the worst case it's five worker calls, five evaluator calls, and every tool call the worker makes in between. The Lambda duration for that is trivial to reason about and cheap. The token cost is the thing that swings, and it swings by a factor of five depending on how many times the evaluator sends the work back.

That reframed the whole cost model for me. Moving to stateless compute on Lambda made the infrastructure cost flat and predictable. What it did was push all the cost variance out of the infrastructure and into the model loop. The bill I have to watch isn't Lambda-seconds. It's tokens times retries.

The 29-second cliff

One more thing the config quietly documents. API Gateway has a hard 29-second integration timeout. A five-turn loop with tool calls can blow past that even though the Lambda itself is happily configured for 300 seconds. So the request dies at the gateway while the function is still working.

The fix is in the Terraform: a Lambda Function URL that bypasses API Gateway entirely for long tasks.

resource "aws_lambda_function_url" "backend" {
  function_name      = aws_lambda_function.backend.function_name
  authorization_type = "NONE"
}

Short chats go through CloudFront and API Gateway. Long agent runs go straight to the Function URL and get the full timeout budget. It works, but authorization_type = "NONE" and allow_origins = ["*"] are both flagged in the code as things to tighten after the first deploy, and they're still open. An unauthenticated public URL that invokes a model loop is a bill waiting to happen if someone finds it. That one's on my list.

What I'd do differently

The retry loop has no token budget. It caps at five turns, but a "turn" can be an expensive worker call with several tool invocations, and nothing stops a run from burning far more tokens than the answer is worth. A turn cap is a crude proxy for a cost cap. If I rebuilt this, the evaluator would carry a token budget alongside turn_count, and the loop would stop on whichever it hit first.

Takeaway

For a serverless agent, Lambda duration is the cheap, predictable line on the bill. The cost that actually moves lives in the cold-start init you pay on import and the retry loop you pay per request, so budget those, not the milliseconds.