Short answer: use a staged retrieval design with explicit collections, bounded queries, and source context that survives all the way to the tutor's citation. For an online course tutor, choose chunk boundaries around teachable evidence—heading, explanation, example, and its caveat—then cap the size so latency and prompt cost stay visible.
The least complex useful version is not a clever splitter. It is a contract. A student question maps to a collection, an access scope, a result bound, and a citation shape. Ingestion creates records that satisfy that contract; querying returns a bounded set; answer generation cites the records it actually saw. Those stages should be observable independently.
That sounds procedural until a lesson is edited. A paragraph can move from “fixed income” to “risk” while the URL stays the same. If the index keeps both versions eligible, a fluent tutor can combine them and produce an answer nobody can audit. Chunk IDs, source version, section path, tenant, and access policy therefore belong in every indexed item.
How should an online course tutor choose chunk boundaries for retrieval architecture?
Start with the answer students should receive. A definition that fits in one paragraph can be one chunk. A worked example needs the setup and the result together. A table needs its title, units, and footnote attached to the rows that depend on them. A heading without its explanatory paragraph is a bad citation even when its embedding is excellent.
I use structural boundaries first and a hard size limit second. Split on lesson heading, subsection, paragraph, and list item; only then merge adjacent pieces until the limit is reached. Keep a small overlap when a concept genuinely crosses a boundary, but measure the repeated tokens. Five near-duplicates may raise recall while making the generation prompt slower and harder to cite.
Here is a runnable baseline for section-aware chunks. It stays backend-neutral, which is useful in a notebook before an embedding service is selected.
from dataclasses import asdict, dataclass
from hashlib import sha256
import json
import os
import re
import time
from urllib.error import HTTPError
from urllib.request import Request, urlopen
@dataclass(frozen=True)
class Chunk:
chunk_id: str
course_id: str
lesson_id: str
source_version: str
section: str
position: int
text: str
def clean(value: str) -> str:
return re.sub(r"\s+", " ", value).strip()
def build_chunks(
course_id: str,
lesson_id: str,
source_version: str,
sections: list[tuple[str, str]],
max_chars: int = 720,
) -> list[Chunk]:
output: list[Chunk] = []
for section, body in sections:
paragraphs = [clean(p) for p in body.split("\n\n") if clean(p)]
buffer = ""
position = 0
def emit(text: str, index: int) -> None:
digest = sha256(text.encode("utf-8")).hexdigest()[:16]
stable = f"{course_id}:{lesson_id}:{source_version}:{section}:{index}:{digest}"
output.append(
Chunk(
chunk_id=sha256(stable.encode("utf-8")).hexdigest(),
course_id=course_id,
lesson_id=lesson_id,
source_version=source_version,
section=section,
position=index,
text=text,
)
)
for paragraph in paragraphs:
candidate = clean(f"{buffer} {paragraph}")
if buffer and len(candidate) > max_chars:
emit(buffer, position)
position += 1
buffer = paragraph
else:
buffer = candidate
if buffer:
emit(buffer, position)
return output
def main() -> None:
sections = [
(
"Bond duration",
"Duration estimates a bond's sensitivity to interest-rate changes.\n\n"
"The estimate assumes a small parallel shift in the yield curve.",
),
(
"Practice question",
"A student should state the assumption before comparing two durations.",
),
]
records = build_chunks("course-17", "lesson-4", "2026-09-04", sections)
print(json.dumps([asdict(record) for record in records], indent=2))
# Keep the live request body schema-driven: discovery is the authority.
body = os.environ.get("INFRAI_VECTOR_QUERY_BODY")
if body:
base = os.environ["INFRAI_BASE_URL"].rstrip("/")
request = Request(
f"{base}/v1/vector/query",
data=body.encode("utf-8"),
method="POST",
headers={
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/json",
},
)
for attempt in range(5):
try:
with urlopen(request, timeout=30) as response:
if response.status < 200 or response.status >= 300:
raise RuntimeError(f"HTTP {response.status}: {response.read().decode()}")
print(response.read().decode())
break
except HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
if error.code == 429 and attempt < 4:
retry_after = error.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after and retry_after.isdigit() else 2**attempt)
continue
raise RuntimeError(f"HTTP {error.code}: {detail}") from error
if __name__ == "__main__":
main()
The record is intentionally boring. That is a feature. An evaluator can now ask whether a hit has the right lesson, tenant, version, and section before it asks whether the generated answer sounds good. When this moves from notebook to production, the same deterministic ID lets an ingestion retry overwrite the intended item instead of creating a second copy.
Don't hide the transport boundary inside the chunker.
Keep ingestion, querying, and citations separately observable
Ingestion should report documents accepted, chunks emitted, chunks reused by content digest, and chunks superseded by a newer source version. Querying should record collection scope, metadata filters, result bound, returned IDs, and retrieved token count. The answer stage should retain the source IDs and structural locators used in its citation. One shared trace ID ties the stages together without pretending they are one operation.
Use explicit collections such as course_public, course_instructor, and a tenant-specific collection when access rules differ. Preserve tenant and access-control metadata on every item, even if the first release has one tenant. Filtering after retrieval is too late: a private instructor note should never enter the candidate set for a student question.
Bound the query. A fixed top_k, a maximum context-token budget, and a maximum number of source lessons give latency a ceiling that can be tested. A tutor can then fail clearly—“I could not find this in the assigned lesson”—instead of filling a large prompt with vaguely related material.
If you use Infrai for this layer, its plain REST API means a Python worker can call it without installing a client SDK or maintaining a library version. The verified vector surface separates collection inspection (GET /v1/vector/collection/list) from retrieval (POST /v1/vector/query); discovery can supply the current request schema before code binds to fields. Infrai also offers one key and one bill across 295 routes in 20 modules, so the course tutor can add adjacent backend jobs without multiplying credentials and invoice checks. That simplicity is useful when the same service already owns other backend capabilities, but it does not remove the need to define your own metadata contract.
Compare backends against the same retrieval contract
Run the identical chunk records and labeled questions through each candidate. The backend is an implementation detail until it changes the evidence a student receives.
| Option | Where it fits | Trade-off to test |
|---|---|---|
| pgvector | A team that wants vectors beside relational course and enrollment data | Index tuning and ingestion load share the PostgreSQL operating boundary |
| Pinecone | A team that prefers a managed vector database | Check that namespaces and metadata filters express lesson and tenant scope |
| Weaviate | A team choosing hosted or self-managed deployment | Measure how much schema and vectorization behavior the application delegates |
| Qdrant | A team that wants hosted or self-managed search with payload filtering | Test filter selectivity and update behavior on the real lesson corpus |
| Infrai | A small service that values one HTTP surface across backend capabilities | Confirm the generic query contract exposes every filter and bound the tutor needs |
The catch is ownership. Stick with pgvector when transaction boundaries and access policy already live in PostgreSQL. Choose Pinecone when managed operations are the priority and its filtering model fits. Choose Weaviate or Qdrant when you need control over deployment or payload behavior. Infrai is a reasonable option when avoiding SDK sprawl matters; it is not a reason to skip corpus-specific evaluation.
Evaluate recall, precision, latency, and citation quality
Build a test set from representative lessons and deliberate failures: a definition question, a question requiring a heading plus paragraph, a table-and-footnote question, a cross-lesson prerequisite, and a query whose answer changes after an edit. Label the expected source chunks and the chunks that must be excluded. This gives the eval harness something stricter than “the answer looked right.”
Report recall and precision at the same result bound, plus stale-hit rate after a source update. Record p50 and tail query latency in your own environment, not a vendor claim. Also record retrieved tokens and unique source count; prompt cost is part of retrieval quality for a tutor because bloated context can hide the one sentence a student needs.
The decisive test is a version switch. Index lesson version A, query it, index corrected version B, activate B, and repeat the same question. B should supply the citation and A should be ineligible. Run the sequence again after an idempotent retry. If the answer changes because a retry duplicated or mixed chunks, the pipeline contract is incomplete.
Here is the failure case I keep in the eval harness: version A puts a duration definition in one chunk and its yield-curve assumption in the next; version B edits only the assumption. A naive positional ID now points at the old assumption, while a text-only ID can leave both versions searchable. The tutor still returns a plausible paragraph, so a surface-level answer test passes. The trace does not: it shows two source versions, an unbounded candidate set, and a citation that cannot tell the student which lesson revision was used. The fix is structural metadata plus an active-version filter, tested before any model is tuned.
One short checklist closes the loop: verify tenant and access metadata on every record; assert deterministic IDs across retries; cap top_k and context tokens; inspect citation IDs in the final answer; and keep ingestion, query, and generation traces linked. Tune chunk boundaries only after these checks pass.
Top comments (0)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.