AWS Lambda Web Functions vs EC2 Auto Scaling: What Changes for LLM Agents and AI Microservices?

AWS has released a very interesting feature which allows lambda to run like a web function as per https://aws.amazon.com/blogs/aws/announcing-aws-lambda-web-functions-for-fast-consistent-web-app-performance-anywhere/. This introduces another design option for us to host agent & microservices in AWS.

Consider a production LLM agent or a set of AI microservices running on Node.js 24 or Python using FastAPI. The application receives user requests, orchestrates LLM calls, invokes tools, retrieves context from vector databases or other data sources, and streams responses back to the client.

A typical AWS deployment might look like this:

Client → API Gateway → Application Load Balancer → EC2 Auto Scaling Group → Node.js 24 / FastAPI services

This is a proven architecture. We control the runtime, configure compute capacity, tune application servers, and scale the infrastructure based on demand.

Now AWS has introduced Lambda Web Functions, a new way to run standard Node.js HTTP applications without managing an EC2 fleet.

Does this change how we should architect our LLM agents and AI microservices?

I believe it is worth examining—not because EC2 Auto Scaling is obsolete, but because we now have another execution model to evaluate.

1. Understanding the two architectures

Option A: Traditional EC2-based architecture

Client → API Gateway → ALB → EC2 Auto Scaling Group → Application services

Each layer has a distinct responsibility:

• API Gateway: Centralised API management, authentication and authorisation, throttling, request validation, and other API controls.
• Application Load Balancer: Routes traffic to healthy application instances.
• EC2 Auto Scaling Group: Adjusts compute capacity according to configured scaling policies.
• Application services: Node.js 24 agents, Python/FastAPI APIs, and other microservices that implement business logic and AI orchestration.

We have considerable control over instance sizing, application-server configuration, connection pools, networking, and deployment strategies.

The trade-off is operational responsibility: managing instances, patching, scaling policies, capacity planning, and infrastructure health.

Option B: Lambda Web Functions for Node.js applications

Client → API Gateway → Lambda Web Function → Node.js 24 HTTP application

The intended architectural benefit is to simplify the compute layer. Instead of maintaining EC2 instances and an ALB to distribute traffic across those instances, we can run a supported Node.js HTTP application on Lambda Web Functions.

AWS manages execution environments and scales them according to incoming traffic. Each environment can handle multiple concurrent HTTP requests, and the service can scale down to zero when idle.

API Gateway remains relevant to our design because we require centralised API management, authentication, throttling, request validation, and governance.

One implementation detail matters: Lambda Web Functions expose their own HTTPS endpoints. API Gateway integration should be validated for the chosen integration type, authorisation flow, request and response behaviour, and streaming requirements. Do not assume that a Web Function can be configured exactly like a conventional Lambda event function.

2. What changes when the application is an LLM agent?

Consider an agent that receives a question, retrieves relevant documents, invokes an LLM, calls external tools, and streams the response to the user.

The workload is often I/O-heavy. Much of the elapsed time may be spent waiting for the LLM provider, a database, a vector store, or an external API.

This is where Lambda Web Functions become interesting.

Concurrency and resource utilisation

On EC2, we configure the instance fleet and application-server concurrency. During a traffic spike, Auto Scaling adds capacity, but new instances take time to launch and become healthy.

Lambda Web Functions manage the number of execution environments and the concurrent requests handled by each environment. This can improve resource utilisation for suitable I/O-heavy workloads.

However, the agent's dependencies still impose limits. A rapidly scaling agent can overwhelm an LLM provider quota, database connection pool, or downstream API unless we apply appropriate concurrency controls.

Streaming LLM responses

Both architectures can support streaming.

On EC2, the application server, ALB, API Gateway, and client connection must be configured to support the intended streaming behaviour.

Lambda Web Functions support native HTTP response streaming, which makes them worth evaluating for interactive AI applications. The complete request path still needs to be tested to ensure that the gateway and client preserve streaming correctly.

Cold starts and idle traffic

EC2 instances can remain warm and ready to serve requests, but we pay for that provisioned capacity.

Lambda Web Functions can scale to zero, reducing idle compute expenditure. A request arriving after inactivity may, however, encounter environment initialisation latency.

For an interactive AI assistant, we should measure time to first token and end-to-end response latency—not just the time taken to execute the application code.

Agent memory and session state

A production agent should not depend on the local memory of one execution environment for durable conversation history, long-term memory, or workflow state.

Store durable state in appropriate external systems and design for requests to be served by different environments.

This is equally important when horizontally scaling EC2 services. The deployment model does not remove the need for deliberate state-management architecture.

3. What about Python/FastAPI-based AI microservices?

This is the critical distinction.

Lambda Web Functions currently support Node.js runtimes only. They are not a direct deployment target for Python applications using FastAPI.

If our AI platform has Python services for retrieval, reranking, model inference, data processing, or specialised ML libraries, we cannot simply move those services to Lambda Web Functions.

We have several practical choices:

• Keep FastAPI on EC2: Retain the existing architecture when runtime control and the current operating model make sense.
• Use ECS with AWS Fargate: Run containerised FastAPI services without directly managing EC2 instances.
• Use conventional Lambda event functions: Suitable for particular Python workloads that fit the event-driven execution model and its limits.
• Use a hybrid architecture: Run supported Node.js HTTP services on Lambda Web Functions while keeping Python/FastAPI microservices on an appropriate container or compute platform.

For many AI platforms, the hybrid approach is worth considering. Agent orchestration, retrieval, inference, and background processing do not necessarily need the same hosting model.

4. Comparing the deployment options

Dimension	EC2 Auto Scaling	Lambda Web Functions	ECS with Fargate
Node.js 24 HTTP services	Yes	Yes	Yes
Python/FastAPI	Yes	Not directly supported	Yes
Compute management	We manage EC2 capacity	AWS manages execution environments	AWS manages underlying compute
Scaling	Configured Auto Scaling policies	Managed HTTP concurrency and environment scaling	Service autoscaling policies
Scale to zero	Not with a running EC2 fleet	Yes	Possible with suitable configuration
HTTP response streaming	Yes, with suitable configuration	Native HTTP streaming	Yes, with suitable configuration
Runtime and OS control	Extensive	Managed environment	Container-level control
Best fit	Workloads needing infrastructure control	Supported Node.js HTTP applications with variable traffic	Containerised services needing runtime flexibility

These are architectural trade-offs, not performance rankings. Actual latency and cost depend on the workload, scaling configuration, traffic profile, and downstream services.

5. Does serverless automatically reduce cost?

Not necessarily.

With EC2, the cost model includes running instances, the ALB, storage, networking, monitoring, and idle capacity maintained for availability and traffic spikes.

With Lambda Web Functions, the model includes requests, active CPU time, memory consumed over the lifetime of execution environments, data transfer, and any retained API Gateway or other services.

For an LLM agent, remember that the LLM provider's inference charges may dominate the infrastructure bill. Moving the agent's HTTP server to serverless does not eliminate those costs.

The meaningful comparison is the total cost of ownership at realistic traffic levels: low traffic, normal utilisation, sudden spikes, and sustained high demand.

6. My approach to a production agentic AI platform

I would not migrate an established EC2 architecture simply because Lambda Web Functions are new.

I would start by measuring:

• Request volume, concurrency, and traffic variability.
• Time to first token and end-to-end response latency.
• CPU and memory utilisation while the agent waits on external services.
• Cold-start sensitivity and the need for warm capacity.
• LLM provider quotas, database limits, and downstream service capacity.
• Total infrastructure and operational costs.
• Security, API governance, observability, and deployment requirements.

Then I would benchmark the current deployment against suitable alternatives using representative workloads.

For a Node.js agent API with variable traffic and streaming requirements, Lambda Web Functions deserve evaluation.

For Python/FastAPI microservices, EC2 or ECS with Fargate may remain more appropriate. For long-running, durable agent workflows, I would separately design orchestration, persistence, retries, and recovery rather than relying on the HTTP execution environment to provide those capabilities.

Final takeaway

AWS Lambda Web Functions introduce another option for hosting HTTP applications. They can simplify the compute layer for supported Node.js applications, including LLM-facing APIs and microservices.

But they do not make EC2 Auto Scaling obsolete, nor do they directly replace Python/FastAPI hosting.

The real architectural opportunity is to choose the right execution model for each workload while retaining the security, API management, and governance capabilities the platform requires.

For agentic AI platforms, the best design may not be entirely EC2-based or entirely serverless. It may be a hybrid architecture in which each service runs where its language, workload, performance requirements, and economics make the most sense.

How are you approaching this decision for production AI systems: EC2 Auto Scaling, Lambda Web Functions, ECS with Fargate, or a hybrid architecture?