Build a Serverless Search Engine with DynamoDB Vector Search and AWS CDK

Ashel Vasquez Palomino — Wed, 16 Sep 2026 02:00:35 +0000

Some time ago at my work, we had to implement a small search engine for help articles. The search was very simple: it used lexical, prefix-based matching. This was a problem because many words did not match.

That is exactly the kind of problem I wanted to solve with this project. Instead of relying solely on the article name or prefix matches, I wanted the search engine to understand the intent behind the query without sacrificing the speed and predictability of traditional search.

It was in this context that AWS announced support for vector indexes in DynamoDB, which was also the service where we were already storing the article data. The result is a small serverless search engine built with DynamoDB vector search, Amazon Bedrock, AWS Lambda, and API Gateway, with all the infrastructure provisioned using AWS CDK.

The main flow works as follows:

DynamoDB stores the product catalog, embeddings, and popularity scores.
A DynamoDB vector index performs semantic similarity searches.
Amazon Titan Text Embeddings V2 generates 512-dimensional embeddings.
Lambda runs lexical and semantic searches in parallel.
API Gateway exposes the HTTP endpoints.
AWS CDK defines and deploys all the infrastructure using TypeScript.

You do not need prior machine learning experience to follow this tutorial. Familiarity with TypeScript and basic AWS concepts should be enough.

This article reflects the project and AWS services available as of September 2026. DynamoDB vector search became generally available on August 5, 2026. Before using this design in production, review the official AWS documentation.

What Are Embeddings and Vector Search?

An embedding is essentially an array of numbers that represents the meaning of a piece of content.

For example, an embedding model can transform:

waterproof shoes for mountain paths

into something like:

[0.018, -0.041, 0.092, ... 509 more numbers]

The important property is that texts with similar meanings tend to produce vectors that are close to one another.

In a product search engine, this allows us to build a relatively simple workflow:

Generate an embedding for each product.
Store those embeddings alongside the product information.
Generate another embedding when the user performs a search.
Ask DynamoDB which stored vectors are closest to the query vector.

DynamoDB performs this final step using an approximate nearest neighbor (ANN) index. Its SearchVectors API returns the closest items according to the distance function configured for the index.

For this implementation, I chose cosine distance. With this metric, 0 represents identical vectors, and lower values indicate greater similarity.

One important detail is to maintain consistency across all embeddings. Both products and queries must use the same model, number of dimensions, and normalization settings.

In this project, I use:

Model: amazon.titan-embed-text-v2:0
Dimensions: 512
Normalization: enabled

Titan Text Embeddings V2 supports 256, 512, and 1,024 dimensions, according to the Amazon Bedrock documentation.

Architecture

At my work, we use help article titles, but for this tutorial we will use products.

The application has two separate flows: indexing and querying.

The indexing flow prepares products before they can be searched. The query flow processes searches coming from the frontend.

API Gateway receives the requests, a Lambda function runs the search logic, and DynamoDB uses on-demand capacity.

The vector index projects the product attributes needed in the search response. This means a SearchVectors result already contains enough information to respond without performing another read.

Prerequisites

Clone or open the repository and make sure you have:

An AWS account with your credentials configured.
Node.js 22 or later.
npm 11 or later.
An AWS Region where Amazon Titan Text Embeddings V2 is available.
Permissions for DynamoDB, Lambda, API Gateway, CloudFormation, IAM, and bedrock:InvokeModel.
An AWS environment previously initialized with CDK bootstrap.

In new Amazon Bedrock accounts, model access is usually enabled by default, although availability may still vary by Region. You can confirm Titan V2 availability in the Bedrock model catalog and Region documentation.

Install the dependencies and run the local checks:

npm install
npm run build
npm test
npm run lint
npm run format:check

If this is the first CDK application you are deploying in that account and Region, initialize the environment once:

npm run cdk -- bootstrap

Step 1: Create the DynamoDB Table with AWS CDK

The central resource is an on-demand DynamoDB table whose partition key is the product id.

const table = new dynamodb.Table(this, 'ProductsTable', {
  partitionKey: { name: 'id', type: dynamodb.AttributeType.STRING },
  billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
  encryption: dynamodb.TableEncryption.AWS_MANAGED,
  pointInTimeRecoverySpecification: { pointInTimeRecoveryEnabled: true },
});

The complete definition is available in infra/lib/search-engine-stack.ts. Each item contains both the application’s regular data and its embedding:

{
  "id": "7381bede-ffcf-4efb-8d78-3db7168d2408",
  "name": "Summit Trail Shoes",
  "normalizedName": "summit trail shoes",
  "nameInitial": "s",
  "description": "Grippy trail shoes with a rock plate for wet mountain paths.",
  "category": "footwear",
  "tags": ["hiking", "trail", "waterproof"],
  "score": 1,
  "embeddingModel": "amazon.titan-embed-text-v2:0",
  "embeddingDimensions": 512,
  "embedding": [0.018, -0.041, 0.092]
}

The actual embedding field contains 512 numbers.

I also store embeddingModel and embeddingDimensions. These fields are not required to run the search, but keeping that metadata alongside the vector simplifies future model or dimensionality migrations.

The table also contains two additional GSIs:

AutocompleteIndex enables prefix searches using the product’s normalized name.
PopularityIndex sorts products by their number of clicks.

Neither of these indexes is required for vector search. I added them because I did not want to rely solely on semantic similarity to rank the results.

Step 2: Define a DynamoDB Vector Index from CDK

I placed the vector index creation logic in a reusable construct called DynamoDbVectorIndex:

new DynamoDbVectorIndex(this, 'ProductEmbeddingVectorIndex', {
  table,
  indexName: 'ProductEmbeddingIndex',
  vectorAttribute: 'embedding',
  dimensions: 512,
  distanceFunction: 'COSINE',
  projectedAttributes: [
    'name',
    'normalizedName',
    'description',
    'category',
    'tags',
    'score',
  ],
});

The number of dimensions must match the vectors generated by Titan. I chose cosine distance because the embeddings are normalized and I am more interested in comparing the direction of the vectors than their magnitude.

CDK does not yet natively support adding a vector index, so I used an asynchronous CloudFormation custom resource.

The provider Lambda uses DynamoDB’s UpdateTable operation with VectorIndexUpdates:

await client.send(
  new UpdateTableCommand({
    TableName: props.TableName,
    VectorIndexUpdates: [
      {
        Create: {
          IndexName: props.IndexName,
          VectorAttribute: {
            AttributeName: props.VectorAttribute,
          },
          Dimensions: props.Dimensions,
          DistanceFunction: props.DistanceFunction,
          Projection: {
            ProjectionType: 'INCLUDE',
            NonKeyAttributes: props.ProjectedAttributes,
          },
        },
      },
    ],
  }),
);

This was one of the most interesting infrastructure details in the project.

Vector index creation is asynchronous. A successful response from UpdateTable does not mean that the index is ready to use. If CloudFormation considered the resource complete at that point, deployment could continue while the index was still being created.

To avoid a race condition, the custom resource provider calls DescribeTable every ten seconds and reports success only when the index reaches the ACTIVE state. It also handles deletion and replacement when immutable index properties change.

The complete implementation is available in:

I find this pattern useful beyond this specific case: when CDK does not yet expose a feature directly, you can encapsulate the low-level AWS API and its associated logic within a reusable construct.

CDK’s custom resource framework is very helpful here because it lets you separate the handler that starts the operation from the handler that checks when it has finished.

Attribute projection also deserves attention. Returning the product fields directly from SearchVectors avoids an additional read, but projected attributes also consume vector index storage. That is why we project only the fields required by the response. AWS explains this trade-off in its vector index storage documentation.

Step 3: Generate and Store the Product Embeddings

After deployment, we have an empty table and vector index. Now we need to transform the product catalog into searchable data.

The loading script is located at scripts/seed.ts, while the sample catalog is in data/products.json.

For each product, I combine several fields into a single textual representation:

return [
  `Name: ${product.name}`,
  `Category: ${product.category}`,
  `Description: ${product.description}`,
  `Tags: ${product.tags.join(', ')}`,
].join('\n');

I could have generated embeddings from the product name alone, but that would discard much of the available context.

Including the category, description, and tags gives the model more information about what each product actually represents.

The seed script then invokes Titan Embed V2:

new InvokeModelCommand({
  modelId: 'amazon.titan-embed-text-v2:0',
  contentType: 'application/json',
  accept: 'application/json',
  body: new TextEncoder().encode(
    JSON.stringify({
      inputText,
      dimensions: 512,
      normalize: true,
    }),
  ),
});

The script also:

validates the catalog,
waits until the vector index is active,
generates four embeddings in parallel,
retries transient Bedrock errors,
and performs an idempotent DynamoDB update that preserves existing click scores.

Before making calls to AWS, you can validate the catalog locally:

npm run seed -- --dry-run

That is usually the first command I run when I modify the sample data.

Step 4: Search Vectors from Lambda

Once the catalog has been indexed, the browser can make requests such as:

GET /search?q=shoes%20for%20wet%20trails&limit=5

The search Lambda first validates and normalizes the query.

It then starts two retrieval paths in parallel.

The first runs a DynamoDB Query against AutocompleteIndex to find products whose names begin with the text entered by the user.

The second generates an embedding for the query and uses SearchVectors to retrieve the 25 semantically closest candidates.

const response = await dynamo.send(
  new SearchVectorsCommand({
    TableName: tableName,
    IndexName: 'ProductEmbeddingIndex',
    SearchVector: embedding.map((value) => ({
      N: String(value),
    })),
    TopK: 25,
  }),
);

One detail that is easy to misinterpret is the Score returned by the vector search.

Because the index uses cosine distance, lower values are better. This is not a traditional relevance score in which a higher number means a better result.

For ranking, I convert that distance into a normalized similarity value:

semantic score = 1 - cosine distance / 2

The result is clamped to the range between zero and one.

For example:

distance = 0  -> semantic score = 1
distance = 2  -> semantic score = 0

Step 5: Combine Semantic and Lexical Results

I decided not to use vector similarity as the final ranking mechanism. Traditional lexical search is still very effective in certain cases.

If a user types cloud run and the catalog contains Cloud Runner Shoes, I would expect that product to appear near the top, even if another product happens to be semantically similar.

The Lambda therefore combines candidates by product ID and calculates the following weighted score:

ranking score =
    0.50 × lexical score
  + 0.45 × semantic score
  + 0.05 × popularity score

The complete implementation is in packages/shared/src/ranking.ts.

The weights slightly favor lexical matches. Semantic similarity retrieves results that a text search would not find, while popularity accounts for only 5%, mainly as a tiebreaker between candidates with similar scores.

These weights are not universal. They are simply a starting point for this example. In a real product search engine, I would tune them using real queries and expected results instead of defining them based solely on intuition.

Another decision was to run both retrieval paths with Promise.allSettled. This gives the endpoint useful fallback behavior. If Bedrock fails temporarily, the lexical search can still return results. The entire request fails only when both paths fail.

For a search endpoint, returning a slightly worse result is better than returning no result at all.

Step 6: Give Each Lambda Only the Permissions It Needs

The search Lambda mainly needs two groups of permissions.

The first allows it to query DynamoDB:

functions.search.addToRolePolicy(
  new iam.PolicyStatement({
    actions: ['dynamodb:Query', 'dynamodb:SearchVectors'],
    resources: [
      table.tableArn,
      `${table.tableArn}/index/AutocompleteIndex`,
      `${table.tableArn}/index/ProductEmbeddingIndex`,
    ],
  }),
);

The second allows it to invoke only the embedding model used by the application:

functions.search.addToRolePolicy(
  new iam.PolicyStatement({
    actions: ['bedrock:InvokeModel'],
    resources: [
      `arn:${cdk.Aws.PARTITION}:bedrock:${cdk.Aws.REGION}::foundation-model/amazon.titan-embed-text-v2:0`,
    ],
  }),
);

The other functions receive more specific permissions.

The popular-products Lambda needs only dynamodb:Query, while the Lambda responsible for recording clicks needs only dynamodb:UpdateItem. This keeps each execution role limited to the actions required by that function instead of granting broad access across the application.

Step 7: Deploy, Load Data, and Test the Search Engine

First, generate the CloudFormation template locally:

npm run cdk:synth

Then deploy the development stack:

npm run cdk:deploy -- --context stage=dev

The deployment saves its outputs to:

cdk-outputs.json

The seed script uses that file to obtain the generated names of the DynamoDB table and vector index.

Load the catalog by running:

npm run seed

To start the React interface, copy the environment variables file:

cp apps/web/.env.example apps/web/.env.local

Set:

VITE_API_BASE_URL

to the ApiUrl value generated in cdk-outputs.json.

Then start Vite:

npm run dev:web

Open http://localhost:5173 in your browser, and you will see the application with the search interface.

The most interesting searches are not necessarily product names. Try queries that express an intent:

shoes for wet mountain trails
quiet keyboard for programming
something that keeps drinks cold
gear for working out while traveling

In those cases, it is much easier to see the difference between lexical and semantic retrieval.

Problems I Encountered While Building It

`AccessDeniedException` from Amazon Bedrock

Verify that Titan Text Embeddings V2 is available in the selected Region and that your user has permission to invoke it.

The deployed Lambda receives the required permission through CDK, but the local credentials used by the seed script also need bedrock:InvokeModel.

The Seed Command Cannot Find the Stack Outputs

The project’s deployment command generates cdk-outputs.json.

Run:

npm run cdk:deploy -- --context stage=dev

before running the seed.

You can also provide both resources explicitly:

npm run seed -- \
  --table-name YOUR_TABLE \
  --vector-index-name ProductEmbeddingIndex \
  --region YOUR_REGION

The Vector Has the Wrong Number of Dimensions

Three values must always match:

The vector index dimensions.
The dimensions requested from Titan during seeding.
The dimensions requested from Titan for real-time queries.

The example uses 512.

One small piece of technical debt in the project is that this constant still appears separately in the infrastructure, seed, and runtime code.

If you change the number of dimensions, you will need to update all three places and rebuild both the index and the stored embeddings.

The First Deployment Appears to Be Stuck

This is normal. Vector index creation is asynchronous, so the custom resource may spend several minutes waiting for DynamoDB to report that the index is ACTIVE.

The provider allows up to 30 minutes for this process. As an additional safety measure, the seed script checks the index status again before inserting vectors.

The Semantic Results Look Strange

Embeddings understand semantic relationships in text, but they do not automatically know your application’s business rules.

If the search quality is poor, the first thing I would review is the text used to generate each embedding.

For example, using only the product name may not provide enough information. Adding descriptions, categories, and tags can make a significant difference.

After that, I would test the system with real queries and tune the ranking weights using an evaluation set.

For a larger catalog, other signals—such as category filters, inventory availability, or business-specific rules—will probably become just as important as vector similarity.

Before Going to Production

Search Evaluation

This is probably the most important point if the search engine starts serving real users.

Create a fixed set of representative queries and define which products should appear in the top positions for each one.

Then run the same evaluation every time you change:

the embedding model,
the number of dimensions,
the text used to generate embeddings,
the lexical scoring,
or the ranking weights.

Otherwise, it is very easy to make the search “feel better” for one query while making several others worse.

Embedding Lifecycle

Whenever relevant product information changes, its embedding should be regenerated.

Model migrations also require planning.

The embeddingModel and embeddingDimensions fields stored alongside the product can help identify which vectors need to be rebuilt or which ones can temporarily coexist during a migration.

AWS currently documents a maximum of 4,096 dimensions, up to five vector indexes per table by default, and TopK values of up to 100.

Because these limits may change, use the DynamoDB quotas documentation as the source of truth.

Clean Up the Resources

When you have finished experimenting, delete the development resources:

npm run cdk:destroy -- --context stage=dev

The stack uses a destroy removal policy and disables deletion protection for dev and demo environments.

In production, the default behavior retains the table and enables deletion protection.

Repository

The code for this tutorial is available in this repository.

DEV Community: Ashel Vasquez Palomino