Building a Bedrock Knowledge Base Over S3 Documents

A Bedrock knowledge base looks like one resource and is four, and exactly one of its settings cannot be changed afterwards. Getting the chunking configuration right at creation is most of the work; the rest is wiring, plus a synchronisation step that a surprising number of walkthroughs leave out entirely, which is why so many people end up querying an empty index.

The four resources involved

Before any API call, know what you are creating:

• A vector store. Bedrock does not host one for you implicitly in every configuration — you point the knowledge base at an OpenSearch Serverless collection, an S3 vector bucket, Aurora PostgreSQL with pgvector, or another supported store. This is where most of the running cost lives.
• The knowledge base itself, which ties the vector store to an embedding model.
• A data source, which points at your S3 bucket and prefixes and carries the chunking configuration.
• A service role Bedrock assumes to read S3, call the embedding model, and write to the vector store.

The separation matters because the knowledge base and the data source have different lifecycles. You can add and remove data sources on a live knowledge base; you cannot change how an existing one chunks.

Chunking is decided once

AWS states plainly, in the CreateDataSource API reference, that you cannot change the chunkingConfiguration after you create the data source connector. Changing your mind means deleting the data source and re-ingesting — which for a large corpus is real embedding cost, not just time. So make the decision on purpose.

The chunkingStrategy field selects between four documented behaviours:

Fixed size

fixedSizeChunkingConfiguration takes maxTokens and overlapPercentage. Predictable, cheap, and the right default for uniform prose. Overlap exists so a fact spanning a boundary appears whole in at least one chunk; it costs storage and embedding calls in direct proportion.

Default

AWS describes the default as splitting content into chunks of approximately 300 tokens while honouring sentence boundaries, so complete sentences are preserved. Fine for a first pass and rarely optimal for anything specific.

Hierarchical

hierarchicalChunkingConfiguration takes two levelConfigurations — a parent maxTokens and a child maxTokens — plus overlapTokens as an absolute token count rather than a percentage. Retrieval matches on the small, precise child chunk and then hands the model the larger parent. This is the strategy for structured documents where the answer is a sentence but the context is a section. Two caveats AWS states: the number of results returned may be lower than requested because children collapse into shared parents, and it is not recommended with an S3 vector bucket, where a high combined token count can run into metadata size limits.

Semantic

semanticChunkingConfiguration takes maxTokens, bufferSize and breakpointPercentileThreshold. Buffer size is how many neighbouring sentences are embedded together when deciding where a boundary falls — a buffer of 1 means the previous, current and next sentence. The threshold is a percentile of sentence dissimilarity; higher means fewer, larger chunks. AWS notes semantic chunking incurs additional cost because it uses a foundation model during ingestion.

You can also choose no chunking, treating each document as a single chunk. AWS notes the trade: with no chunking you lose page numbers in citations and cannot filter on the x-amz-bedrock-kb-document-page-number metadata attribute. If your users need to be told which page an answer came from, that settles it.

Creating the data source

The S3 configuration is three fields: the bucket ARN, optional inclusionPrefixes, and bucketOwnerAccountId when the bucket lives in another account.

aws bedrock-agent create-data-source \
  --knowledge-base-id ABCDEFGHIJ \
  --name policy-documents \
  --data-source-configuration '{
    "type": "S3",
    "s3Configuration": {
      "bucketArn": "arn:aws:s3:::acme-policy-docs",
      "inclusionPrefixes": ["published/"]
    }
  }' \
  --vector-ingestion-configuration '{
    "chunkingConfiguration": {
      "chunkingStrategy": "HIERARCHICAL",
      "hierarchicalChunkingConfiguration": {
        "levelConfigurations": [{"maxTokens": 1500}, {"maxTokens": 300}],
        "overlapTokens": 60
      }
    }
  }' \
  --data-deletion-policy RETAIN

inclusionPrefixes is worth using even when the bucket only holds documents. It is what lets you keep drafts and published material in one bucket without ingesting the drafts, and it is the knob you reach for when someone asks why an internal memo turned up in a customer-facing answer.

dataDeletionPolicy takes RETAIN or DELETE and governs what happens to the vectors when the data source or knowledge base is deleted. AWS is explicit that in neither case is the vector store itself deleted — only the data. Deleting a knowledge base does not stop an OpenSearch Serverless collection from billing you.

The ingestion job nobody mentions

Creating a data source does not read any documents. Nothing is embedded, nothing is queryable, and a retrieval call against the new knowledge base returns an empty result set with no error. You have to start an ingestion job, and you have to start one again every time the documents change.

1. Start it: aws bedrock-agent start-ingestion-job --knowledge-base-id ABCDEFGHIJ --data-source-id KLMNOPQRST.
2. Poll get-ingestion-job with the returned ingestionJobId. The statistics block reports documents scanned, indexed, failed and deleted — the failure count is the one to alert on, because a job can complete with documents it could not parse.
3. Re-run it after every change to the prefix. Subsequent jobs are incremental: unchanged documents are not re-embedded, so the cost of a routine sync is proportional to what moved, not to corpus size.
4. Automate it. An S3 event on the prefix, debounced, is the usual shape — see the S3-event embedding pipeline for the general pattern. A nightly EventBridge schedule is the cheaper version if freshness within a day is acceptable.

Deletion is the case people forget. Removing a document from S3 does not remove it from the index until the next ingestion job runs, so between the delete and the sync your knowledge base will happily cite a document that no longer exists. If that matters legally rather than aesthetically, the sync has to be triggered by the deletion, not by a schedule.

Querying it

Two runtime operations, and the difference is who does the generation. Retrieve returns chunks and their scores and leaves the prompt to you. RetrieveAndGenerate retrieves, builds a prompt, calls a model and returns text with citations.

aws bedrock-agent-runtime retrieve \
  --knowledge-base-id ABCDEFGHIJ \
  --retrieval-query '{"text": "What is the refund window for annual plans?"}' \
  --retrieval-configuration '{
    "vectorSearchConfiguration": {
      "numberOfResults": 5,
      "overrideSearchType": "HYBRID"
    }
  }'

Start with Retrieve, even if you intend to use RetrieveAndGenerate. It is the only way to tell whether a bad answer is a retrieval problem or a generation problem, and with hierarchical chunking it is also how you see that you asked for five results and got three because two children shared a parent. overrideSearchType takes HYBRID or SEMANTIC; hybrid adds keyword matching, which is what rescues queries containing product codes and identifiers that embeddings handle poorly.

Related

• Triggering an Embedding Pipeline on S3 Object Upload
• Wiring an Action Group to Lambda With Bedrock Agents