DEV Community

Cover image for RAG Powered Apps with Amazon Bedrock, Part 2: Automating the RAG Pipeline with Terraform
Chigozirim Eke
Chigozirim Eke

Posted on

RAG Powered Apps with Amazon Bedrock, Part 2: Automating the RAG Pipeline with Terraform

#aws #bedrock #terraform #rag

Before you start: This picks up where Part 1 left off. From part 1, you would've learned how to setup a Bedrock Knowledge Base in the console. In addition to that, you should have a general understanding of how the ingestion and query pipeline works.

Introduction & Motivation

I started this project with a singular goal: to build a comprehensive Terraform module that allows developers to deploy the entire infrastructure for a "Chat with PDF" application faster.

When Amazon Bedrock was first unveiled in April 2023, I jumped in immediately. Like many of you, I built several proof-of-concepts (PoCs) through the AWS Console. The UI is amazing for building quick pocs, but once I moved into experimentation, I realized it would be best to quickly setup and tear down the infra.

An example use case was testing if there were any cost savings in using S3 Vectors vs OpenSearch and how much cost savings exactly.

None of the Terraform modules I found on GitHub (at the time) seemed to cover the end-to-end pipeline I was looking for, so I decided to build mine. I'm also big on learning so why not.

What Are We Building?

A couple of terraform modules to automate everything we clicked through manually in Part 1. One terraform apply brings up the full stack:

  • S3 Bucket: your document store. Encrypted at rest, versioning on, zero public access.
  • OpenSearch Serverless: the vector database. Stores the embeddings Bedrock generates during ingestion.
  • Bedrock Knowledge Base: orchestrates the chunking, embedding, and storage of documents, and retrieval at query time.
  • Ingestion Lambda: triggered automatically when you upload a file to S3. Starts a Bedrock ingestion job so documents are chunked, embedded, and indexed without ClickOps.
  • Query Lambda: accepts a natural language question, calls RetrieveAndGenerate, and returns an answer with source citations.

Full source code + ReadMe: Bedrock Project. If you run into issues or want to extend the module, feel free to open an issue.

Architecture 

sequenceDiagram
    participant User
    participant S3
    participant IngestionLambda as Ingestion Lambda
    participant Bedrock
    participant Titan as Titan (Embeddings)
    participant OSS as OpenSearch Serverless
    participant QueryLambda as Query Lambda
    participant Claude

    Note over S3,OSS: Ingestion Phase
    User->>S3: Upload document
    S3->>IngestionLambda: S3 ObjectCreated event
    IngestionLambda->>Bedrock: StartIngestionJob
    Bedrock->>S3: Fetch document
    Bedrock->>Titan: Chunk + embed text
    Titan-->>Bedrock: Vectors
    Bedrock->>OSS: Store vectors + metadata

    Note over QueryLambda,Claude: Query Phase
    User->>QueryLambda: Invoke with question
    QueryLambda->>Bedrock: RetrieveAndGenerate
    Bedrock->>Titan: Embed query
    Titan-->>Bedrock: Query vector
    Bedrock->>OSS: Search for similar vectors
    OSS-->>Bedrock: Top matching chunks
    Bedrock->>Claude: Query + chunks
    Claude-->>QueryLambda: Answer + citations
    QueryLambda-->>User: Answer + source citations
Enter fullscreen mode Exit fullscreen mode

In Part 3 we'll put an API Gateway in front of the query Lambda. For now we're invoking it directly from the CLI.

Project Structure 

rag-bedrock-project/
├── main.tf
├── variables.tf
├── outputs.tf
├── backend.tf
├── terraform.tfvars.example
├── bootstrap/
└── modules/
    ├── storage/
    ├── opensearch/
    ├── bedrock/
    └── lambda/
Enter fullscreen mode Exit fullscreen mode

Each module owns one piece of the infrastructure and exposes what other modules need through outputs. The root main.tf wires them together by passing outputs from one module as inputs to another.

To be honest, I went back and forth on this architecture, and granted having multiple modules might be overkill. But designing this took me back to my Node.js applications days where I would put everything in a single server.js which made it difficult to debug errors. I learned about MVC which changed the way I build software. Terraform modules clicked the same way for me. One module per function. The Lambda module does not need how OpenSearch is set up. It just gets the IDs it needs through variables. As the architect, you know how each module communicates with the others.

Implementation

Step 1: Bootstrap Remote State First

Before running terraform apply on anything, you need somewhere to store your Terraform state.

Hold up? State what? Terraform state essentially tells Terraform what infrastructure already exists. Every resource it creates gets recorded in a terraform.tfstate file. Without it, Terraform can't tell what's already deployed.

If your local state file is ever lost or corrupted, Terraform loses track of everything it deployed. Storing it in S3 keeps it versioned and safe. Terraform 1.10 introduced native S3 state locking so you don't need a seperate DynamoDB table. You can read more here

The bootstrap/ directory sets this up. Run it once before anything else

All commands use aws-vault, which stores AWS credentials in your OS keychain and injects temporary credentials at runtime. The --no-session flag skips STS session tokens, which some IAM operations reject. If you're not using aws-vault, replace aws-vault exec YOUR_PROFILE --no-session -- with your usual credential method.

aws-vault exec YOUR_PROFILE --no-session -- \
  terraform -chdir=bootstrap init

aws-vault exec YOUR_PROFILE --no-session -- \
  terraform -chdir=bootstrap apply \
  -var="project_name=my-rag" -var="environment=dev"
Enter fullscreen mode Exit fullscreen mode

Bootstrap creates two things: the S3 state bucket, and a scoped deployer IAM policy. You need AdministratorAccess to run bootstrap itself, because you can't use a scoped policy to create the scoped policy. Once it's done, you attach the scoped policy to your IAM user, detach AdministratorAccess, and every deploy from here runs least-privilege.

After it finishes, two outputs matter:

backend_config = <<EOT
  terraform {
    backend "s3" {
      bucket       = "my-rag-dev-terraform-state-123456789012"
      key          = "dev/terraform.tfstate"
      region       = "us-east-1"
      use_lockfile = true
      encrypt      = true
    }
  }
EOT

deployer_policy_arn = "arn:aws:iam::123456789012:policy/my-rag-dev-terraform-deployer"
Enter fullscreen mode Exit fullscreen mode

Copy the backend_config block into backend.tf in the root directory, then run terraform init again. It'll migrate your local state to S3. Do this once and forget about it.

aws-vault exec YOUR_PROFILE --no-session -- terraform init
Enter fullscreen mode Exit fullscreen mode

Then swap to the scoped policy:

# Attach the deployer policy
aws-vault exec YOUR_PROFILE --no-session -- aws iam attach-user-policy \
  --user-name YOUR_IAM_USER \
  --policy-arn YOUR_DEPLOYER_POLICY_ARN

# Drop AdministratorAccess
aws-vault exec YOUR_PROFILE --no-session -- aws iam detach-user-policy \
  --user-name YOUR_IAM_USER \
  --policy-arn arn:aws:iam::aws:policy/AdministratorAccess
Enter fullscreen mode Exit fullscreen mode

Step 2: Configure Your Variables 

cp terraform.tfvars.example terraform.tfvars
Enter fullscreen mode Exit fullscreen mode

Four variables, that's it:

# terraform.tfvars
project_name         = "my-rag"
environment          = "dev"
aws_region           = "us-east-1"
embedding_dimensions = 512  # 256 or 512, half the storage cost vs 1024
Enter fullscreen mode Exit fullscreen mode

Note: One thing about Bedrock model access: AWS now enables foundation models automatically on first invocation. The one exception is Anthropic models (including Claude 3 Haiku), which may prompt first-time users to submit brief use case details before the first request goes through. If your first query returns an access error, check the AWS Console under Bedrock → Model access and complete the form. This is a one-time step per AWS account.

Step 3: Walking Through the Modules

This section walks through the key pieces of each module and why they're built that way.

IAM: Inline Where It Belongs

So, IAM matters here because three different principals need to talk to each other: Bedrock during ingestion, the ingestion Lambda when it starts a job, and the query Lambda at query time. You need separate roles for each.

Please don't be that person who slaps AdministratorAccess on everything just to stop the 403s. Each role lives in the module that owns it. The Bedrock KB role is in modules/bedrock/ because that module creates the Knowledge Base. The Lambda execution roles live in modules/lambda/ for the same reason.

# modules/bedrock/main.tf

resource "aws_iam_role" "kb" {
  name = "${var.config.environment}-${var.config.project_name}-bedrock-kb"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect    = "Allow"
        Principal = { Service = "bedrock.amazonaws.com" }
        Action    = "sts:AssumeRole"
        Condition = {
          StringEquals = {
            "aws:SourceAccount" = local.account_id
          }
        }
      }
    ]
  })
}
Enter fullscreen mode Exit fullscreen mode

The Condition block scopes the trust to your account. Bedrock can assume this role, but only for a Knowledge Base in your account, not for any other service or account.

The role gets three inline policies: s3:GetObject and s3:ListBucket on the document bucket, aoss:APIAccessAll on the AOSS collection (required by Bedrock to write embeddings), and bedrock:InvokeModel scoped to the Titan embedding model ARN. Nothing else. The query Lambda can only call bedrock:RetrieveAndGenerate. It cannot read S3, touch AOSS, or invoke models directly. Bedrock handles all of that through the KB role.

Vector Store: OpenSearch Serverless

OpenSearch Module

Alright, this is the main event!

OpenSearch Serverless is where the embeddings live. It's not a persistent cluster so you pay per OCU (OpenSearch Compute Unit) when the collection is active. Which essentially means 🤑

AOSS requires three security policies before the collection will create:

  1. Encryption policy: which key encrypts data at rest
  2. Network policy: who can reach the endpoint
  3. Data access policy: which IAM principals can read/write 
# modules/opensearch/main.tf

resource "aws_opensearchserverless_collection" "vectors" {
  name = local.collection_name
  type = "VECTORSEARCH"

  depends_on = [
    aws_opensearchserverless_security_policy.encryption,
    aws_opensearchserverless_security_policy.network,
    aws_opensearchserverless_access_policy.bedrock
  ]
}
Enter fullscreen mode Exit fullscreen mode

Without the depends_on, Terraform might try creating the collection before the policies exist, and AWS will reject it with an unhelpful error.

Security: Data at rest is encrypted with an AWS-owned KMS key by default. You can bring your own key if your compliance requirements call for it. All traffic to and from AOSS goes over TLS.

One thing I deliberately left out of this module: manually creating the OpenSearch index. In a previous version of this project, I used the opensearch Terraform provider to create the vector index directly. This included the right field mappings, knn settings, and all that. It worked, but it added two pain points: a cold-start timing issue where the data access policy hadn't propagated before the index creation tried to run, and an extra provider dependency.

Bedrock manages the index automatically when you create the Knowledge Base. The field names, dimensions, and knn configuration all get handled at the KB level. One less thing to provision manually, one less thing that can go wrong.

We also cap capacity at the account level because we are cost-conscious💅

resource "aws_opensearchserverless_account_settings" "capacity" {
  capacity_limits {
    max_indexing_capacity_in_ocu = 1
    max_search_capacity_in_ocu  = 1
  }
}
Enter fullscreen mode Exit fullscreen mode

This keeps costs predictable. AOSS bills per OCU-hour (~$0.24/hour per OCU), and the minimum is 0.5 OCU per type. That's roughly $350/month just for the collection to exist😿

Knowledge Base: The Brain

Bedrock Module

The KB ties everything together. It knows where to find documents (S3), where to store vectors (AOSS), which embedding model to use, and how to name the index fields:

# modules/bedrock/main.tf

resource "aws_bedrockagent_knowledge_base" "main" {
  name     = local.kb_name
  role_arn = aws_iam_role.kb.arn

  knowledge_base_configuration {
    type = "VECTOR"
    vector_knowledge_base_configuration {
      embedding_model_arn = local.embedding_model_arn
      embedding_model_configuration {
        bedrock_embedding_model_configuration {
          dimensions = var.config.embedding_dimensions
        }
      }
    }
  }

  storage_configuration {
    type = "OPENSEARCH_SERVERLESS"
    opensearch_serverless_configuration {
      collection_arn    = var.collection_arn
      vector_index_name = "bedrock-knowledge-base-default-index"
      field_mapping {
        vector_field   = "bedrock-knowledge-base-default-vector"
        text_field     = "AMAZON_BEDROCK_TEXT_CHUNK"
        metadata_field = "AMAZON_BEDROCK_METADATA"
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Why 512 dimensions instead of the full 1024? At dev volume, the accuracy difference is negligible, but storage costs drop by half. You can always re-ingest at higher dimensionality later. The embedding_dimensions variable enforces this because it only accepts 256 or 512, so you can't accidentally provision at full dimensions and then wonder why your bill is larger than expected.

Lambda: The Query Handler and the Trigger

Lambda Module

Two Lambda functions, both Node.js 20 on ARM64. ARM64 (Graviton) is cheaper per millisecond than x86 for the same memory allocation, and these functions aren't doing anything CPU-intensive enough to justify the extra cost.

The ingestion handler (ingest.mjs) is triggered automatically when you upload a file to S3. It calls StartIngestionJob so Bedrock picks up the new document and indexes it:

// lambda/src/ingest.mjs

export async function handler(event) {
  const command = new StartIngestionJobCommand({
    knowledgeBaseId: process.env.KNOWLEDGE_BASE_ID,
    dataSourceId: process.env.DATA_SOURCE_ID,
  });

  try {
    const response = await client.send(command);
    return { statusCode: 202, body: JSON.stringify({ jobId: response.ingestionJob.ingestionJobId }) };
  } catch (error) {
    if (error.name === "ConflictException") {
      // An ingestion job is already running. That's fine, the new file will be picked up.
      return { statusCode: 202, body: JSON.stringify({ message: "Ingestion already in progress" }) };
    }
    throw error;
  }
}
Enter fullscreen mode Exit fullscreen mode

Why ConflictException? If someone uploads several files in quick succession, multiple S3 events fire and multiple Lambda invocations try to start ingestion jobs. Bedrock only allows one at a time so, the second request throws a ConflictException. We catch it and return 202 anyway, because the running job will pick up all the new files.

The query handler (query.mjs) accepts a natural language question and returns an answer with citations:

// lambda/src/query.mjs

export async function handler(event, context) {
  // Parses event.body.query (API Gateway proxy format)
  const query = parseQuery(event);

  const command = new RetrieveAndGenerateCommand({
    input: { text: query.trim() },
    retrieveAndGenerateConfiguration: {
      type: "KNOWLEDGE_BASE",
      knowledgeBaseConfiguration: {
        knowledgeBaseId: KNOWLEDGE_BASE_ID,
        modelArn: MODEL_ARN,
      },
    },
  });

  // Retries once on throttle/5xx if there's > 15 seconds left in the timeout budget
  let response;
  try {
    response = await client.send(command);
  } catch (error) {
    if (isRetryable(error) && hasTimeForRetry(context)) {
      response = await client.send(command);
    } else {
      return buildResponse(503, { error: "Service temporarily unavailable" });
    }
  }

  return buildResponse(200, {
    answer: response.output?.text || "",
    citations,
  });
}
Enter fullscreen mode Exit fullscreen mode

The retry logic checks context.getRemainingTimeInMillis() before retrying. If there's less than 15 seconds left, we fail fast instead of starting a request we can't finish. Better a clean 503 than a Lambda timeout.

Logging follows a strict rule: structured JSON, no PII. We log request IDs, durations, and error types, but never the actual query or response content:

function log(level, requestId, message) {
  console.log(JSON.stringify({
    level,
    timestamp: new Date().toISOString(),
    requestId: requestId || undefined,
    message,
  }));
}
Enter fullscreen mode Exit fullscreen mode

Security: Lambda dependencies are locked in package-lock.json. No floating version ranges that could pull in a compromised package on the next deploy.

Step 4: Deploy

The Bedrock module needs the AOSS collection endpoint. The OpenSearch module needs the Bedrock KB role ARN. Both modules need something the other creates. On a fresh deploy, Terraform can't resolve both at once. Fix: Create the collection first in a separate targeted apply, so the endpoint exists by the time the full apply runs.

The README has the complete deployment reference. The short version:

# Step 1: create the OSS collection first
aws-vault exec YOUR_PROFILE --no-session -- terraform apply -target=module.opensearch

# Step 2: deploy everything else
aws-vault exec YOUR_PROFILE --no-session -- terraform apply
Enter fullscreen mode Exit fullscreen mode

Go make coffee. AOSS takes about 10 minutes to spin up.

You only need to do this once. After that first run, the endpoint is stored in state and every subsequent deploy is just terraform apply.

One thing worth knowing if you're using aws-vault: some IAM operations reject session tokens that aws-vault generates by default. If you hit an InvalidClientTokenId error mid-apply, make sure you're using --no-session.

Step 5: Test It

Upload, check status, and invoke

Upload a document. The ingestion Lambda fires automatically on upload. No manual trigger needed:

aws-vault exec YOUR_PROFILE --no-session -- \
  aws s3 cp ./my-document.pdf \
  s3://$(terraform output -raw document_bucket_name)/
Enter fullscreen mode Exit fullscreen mode

Check ingestion status and wait for COMPLETE before querying:

aws-vault exec YOUR_PROFILE --no-session -- \
  aws bedrock-agent list-ingestion-jobs \
  --knowledge-base-id $(terraform output -raw knowledge_base_id) \
  --region us-east-1
Enter fullscreen mode Exit fullscreen mode

Then invoke the query Lambda:

aws-vault exec YOUR_PROFILE --no-session -- \
  aws lambda invoke \
  --function-name $(terraform output -raw query_function_name) \
  --payload '{"body":"{\"query\":\"What is the document about?\"}","requestContext":{"requestId":"test-1"},"headers":{}}' \
  --cli-binary-format raw-in-base64-out \
  /tmp/response.json &amp;&amp; cat /tmp/response.json
Enter fullscreen mode Exit fullscreen mode

The payload wraps the query in a body field because the Lambda parses event.body.query.

A successful response looks like:

{
  "statusCode": 200,
  "headers": { "Content-Type": "application/json" },
  "body": "{\"answer\":\"The document covers...\",\"citations\":[{\"text\":\"...\",\"sources\":[{\"uri\":\"s3://my-rag-dev-documents/my-document.pdf\"}]}]}"
}
Enter fullscreen mode Exit fullscreen mode

If you're getting empty answers, ingestion likely isn't done yet. Wait for the status to show COMPLETE and try again.

Step 6: Cleanup

Before you run terraform destroy, there's one thing to sort out. The data source has a data_deletion_policy that defaults to DELETE. When Terraform tears down the stack, it tries to clean up vectors from OpenSearch as part of deleting the data source. If the collection is also being destroyed in the same apply, Bedrock can't reach it and the deletion gets stuck.

Set it to RETAIN first, apply, then destroy:

# modules/bedrock/main.tf

resource "aws_bedrockagent_data_source" "s3" {
  name                 = "${var.config.environment}-${var.config.project_name}-s3-source"
  knowledge_base_id    = aws_bedrockagent_knowledge_base.main.id
  data_deletion_policy = "RETAIN"  # set this before destroying

  # ... rest of config
}
Enter fullscreen mode Exit fullscreen mode

You could also use the console to do this

aws-vault exec YOUR_PROFILE --no-session -- terraform apply
Enter fullscreen mode Exit fullscreen mode

Full cleanup: tear down the stack

aws-vault exec YOUR_PROFILE --no-session -- terraform destroy
Enter fullscreen mode Exit fullscreen mode

To also clean up the bootstrap resources, first empty the versioned state bucket (versioning means objects have to be deleted explicitly), then destroy:

# Remove all object versions from the state bucket
aws-vault exec YOUR_PROFILE --no-session -- \
  aws s3api delete-objects \
  --bucket YOUR_STATE_BUCKET_NAME \
  --delete "$(aws-vault exec YOUR_PROFILE --no-session -- \
    aws s3api list-object-versions \
    --bucket YOUR_STATE_BUCKET_NAME \
    --query '{Objects: Versions[].{Key:Key,VersionId:VersionId}}' \
    --output json)"

# Destroy bootstrap
aws-vault exec YOUR_PROFILE --no-session -- \
  terraform -chdir=bootstrap destroy
Enter fullscreen mode Exit fullscreen mode

Gotchas and Cost

AOSS costs ~$350/month minimum just for the collection to exist. Whether you are querying it or not. S3, Lambda and Bedrock tokens are negligible by comparison. If you're just experimenting, destroy when you're done

On the technical side:

AOSS collection won't create: You probably hit the policies-first timing issue. Make sure encryption, network, and access policies are all in depends_on.

KB ingestion fails: Check that the KB role has s3:GetObject on the bucket and aoss:APIAccessAll on the collection. Also verify the AOSS collection is in ACTIVE state before running step 2 of the deploy.

Lambda returns 400 ("query is required"): The query Lambda parses event.body.query. Make sure your payload wraps the query in a body field as shown in Step 5.

Lambda returns 503: Bedrock is throttling. The function retries once automatically, but if you're hammering it, back off.

Empty citations array: Your documents might be in a format Bedrock can't chunk. Stick to plain text, PDF, Markdown, or HTML for best results.

terraform destroy gets stuck: The default data_deletion_policy = "DELETE" causes the teardown to deadlock when the collection is also being destroyed. Set it to RETAIN and apply before you destroy. Full steps in Step 6.

What's Next?

The Knowledge Base is live and queryable from the CLI. In Part 3, we're putting an API Gateway in front of the query Lambda and wiring up a React frontend. The query Lambda's response shape already works with API Gateway's proxy integration, and the only thing left is CORS headers and the API Gateway resource itself.

Disclaimer

This is strictly for educational purposes. You will be charged for the resources created when you follow along. Remember to clean up after use.

Links & Resources

Top comments (0)