Migrating an Agentic RAG App to AWS Serverless

A previous post covered building an agentic RAG router over **LLM Zoomcamp* capstone submissions. This series is about moving it off a single EC2 box and onto AWS serverless, where nearly every obvious choice turned out to be wrong, and the most valuable decision was deleting a database rather than migrating it. This part covers the starting point and the resulting architecture;

---

The course on designing and creating a RAG project has concluded. The final project is working as expected. Now it's time to optimize and use the developed concept as a production system. In the previous article, I described creating an agent-based RAG router for processing LLM theses submitted at Zoomcamp. I decided to write a series of articles dedicated to migrating the project from a single EC2 server to a serverless AWS environment. This section examines the starting point and the resulting architecture.

Starting point and its drawbacks

The initial assistant is working. It answers fundamentally different types of questions about these repositories.
For example, "What does this project do?" using a hybrid search of README files, and "various questions about performance statistics and libraries used in projects?" using a precise parameterized SQL query. The agent router chooses between them.

In my capstone, it runs as six Docker Compose containers on a single t4g.small instance, deployed via SSH, with a 2GB swap file as a cheap safety net so that Postgres, Streamlit (which contains the Torch embedder and Torch cross-encoder in RAM), and Grafana can coexist on a 2GB RAM instance.

This isn't bad for a capstone project. But it has properties I'd like to eliminate for a real system.

• The corpus - 24,775 fragments, 115MB of JSON-is loaded and rebuilt into the process's RAM on each run.
• Two local machine learning models require a 1.7GB virtual environment and an image. Several gigabytes in size.
• There's no infrastructure as code at all. This approach is completely unsuitable for a real system. Deployment is done via a bash script.
• The cost of downtime is about $15 per month, regardless of whether anyone visits it.

So: rebuild it as a fully serverless AWS application using real infrastructure as code (IaC) as a learning path. All development is done using Amazon's own resources—Nova for output, Titan for embeddings, Amazon Rerank for reranking, and boto3—the only SDK. No OpenAI, no Anthropic.

Here is what actually happened.

Two architectures:

Before — everything on one box, everything in RAM:

flowchart LR
    User["User"] --> ST["Streamlit :8501<br/>torch embedder + reranker<br/>24.8k chunks in RAM"]
    ST --> KW[("minsearch.Index<br/>keyword, in-process")]
    ST --> Vec[("minsearch.VectorSearch<br/>MiniLM, in-process")]
    ST --> PG[("Postgres 17<br/>projects · conversations<br/>feedback · budget")]
    ST --> OAI["OpenAI<br/>Responses + Batch API"]
    PG --> Graf["Grafana :3000"]
    Cache[("vector_index_cache.npz<br/>37MB on disk")] -.-> Vec

    subgraph EC2 ["one t4g.small — 6 containers, ~$15/mo idle"]
        ST
        KW
        Vec
        PG
        Graf
        Cache
    end

    style PG fill:#336791,color:#fff
    style EC2 fill:#f6f8fa,stroke:#8b949e

After — nothing idling, and the corpus is an artifact rather than a runtime cost:

flowchart LR
    User["User"] --> CF["CloudFront<br/>+ edge origin check"]
    CF --> S3W[("S3: static SPA<br/>3 files, no build step")]
    CF --> L["api Lambda<br/>~7MB zip · no VPC"]

    L --> BR["Bedrock<br/>Nova Pro / Nova Lite<br/>Titan · Rerank"]
    L --> SV[("S3 Vectors<br/>25,482 vectors<br/>identity + filters")]
    L --> SQ[("S3 artifact → /tmp<br/>projects.sqlite<br/>projects · chunks · FTS5")]
    L --> DDB[("DynamoDB<br/>conversations · feedback<br/>budget/day · rate limits")]

    SFN["Step Functions<br/>scrape · Map(crawl + Wait)<br/>embed · build artifact"] --> SV
    SFN --> SQ

    style BR fill:#232F3E,color:#fff
    style DDB fill:#4053D6,color:#fff
    style L fill:#0C8C7D,color:#fff

---