DEV Community

PeterParker8991
PeterParker8991

Posted on

EdTech Code Review Assistants: TypeScript 5 Semantic Retrieval API Boundaries

#ai #typescript #semanticsearch

Short answer: define the chatbot API around a stable review-finding contract, then keep embedding, retrieval, and generation behind replaceable adapters. For an edtech app that reviews student code, the useful boundary is not a vendor SDK. It is the point where a query becomes a ranked set of evidence and then a typed finding.

Decision Prefer Why
Retrieval boundary Provider-neutral adapter Swapping an index or embedding service does not change the chat contract
Evidence shape Stable IDs, scores, and snippets A reviewer can inspect why a finding exists
Request behavior Async job plus bounded polling or streaming Large repositories do not hold an HTTP request forever
Failure policy Explicit partial results A rate limit or missing document is visible, not silently ignored

My rule is simple: make the model replaceable, but make the review evidence inspectable. That is the part that protects a one-person SaaS from spending a week rewriting UI and tests after a search change.

What should the chatbot API actually promise?

A student submits a pull request or an editor change. The in-app chatbot answers questions such as “Why is this function unsafe?” and returns structured findings that a teacher can scan. The API should promise the shape and meaning of that finding, not which retrieval engine produced it.

A useful response has an answer, citations into the submitted code or course material, and machine-readable findings. It also needs a request ID. That ID is the thread connecting a user-visible answer to logs, retries, and later evaluation.

export type ReviewFinding = {
  id: string;
  severity: "info" | "warning" | "error";
  title: string;
  explanation: string;
  evidence: Array<{
    documentId: string;
    startLine?: number;
    endLine?: number;
    snippet: string;
    score: number;
  }>;
};

export type ChatReviewResponse = {
  requestId: string;
  answer: string;
  findings: ReviewFinding[];
  incomplete: boolean;
};
Enter fullscreen mode Exit fullscreen mode

The incomplete bit matters. A chatbot that returns an authoritative-sounding answer after one retrieval shard timed out is harder to operate than one that admits the evidence set is partial. This is a product decision as much as a transport detail. Teachers need to know when to inspect the code themselves.

I would also keep the retrieval query out of the public response unless it helps the user. Internally, record the normalized query, filters, embedding model identifier, index version, and returned document IDs. Those fields let you reproduce a bad answer without binding the client to an implementation.

Why does semantic search need a boundary in code review?

Code review has two awkward data types. Source code is precise but local. Course guidance is broad but often reused across assignments. Semantic search is good at finding related material across those different documents, yet similarity is not proof. A high score can point to a nearby concept while missing the exact line that caused the defect.

The retrieval adapter should therefore return evidence, not conclusions. A minimal interface can work across a hosted search API, a self-managed index, or a relational database with vector support.

Keep it boring.

export type SearchFilter = {
  courseId: string;
  assignmentId?: string;
  language?: string;
};

export type SearchHit = {
  documentId: string;
  content: string;
  score: number;
  metadata: Record<string, string>;
};

export interface SemanticRetriever {
  search(input: {
    query: string;
    filter: SearchFilter;
    limit: number;
  }): Promise<SearchHit[]>;
}
Enter fullscreen mode Exit fullscreen mode

The adapter owns provider-specific details: vector dimensions, filter syntax, pagination, and retry headers. The review service owns limits and interpretation. That split keeps a search migration from leaking into every route handler. It also gives you one place to reject an unsafe request, such as a query that asks for documents outside the current course.

There is a less glamorous boundary that matters just as much: document identity. Store a stable documentId, a source revision, and line or character offsets when indexing code. If chunking changes, old findings should still be explainable. A snippet without provenance is a screenshot with no address.

How should the request flow handle real failures?

For a short file, a synchronous request may be fine. For a repository, indexing and review should be separate jobs. The chat route can create a review request, retrieve the latest completed index, and report its state. That avoids making a browser request responsible for every expensive operation.

The flow I use as a design target is:

  1. Validate tenant, course, assignment, and source revision.
  2. Create an idempotent request keyed by the revision and review policy.
  3. Retrieve a bounded evidence set with explicit filters.
  4. Ask the generation layer for JSON matching the finding schema.
  5. Validate that JSON and attach evidence IDs before returning it.

A provider-neutral runtime should treat throttling as a state transition, not a generic “AI failed” message. HTTP 429 means the server is limiting requests; clients can inspect Retry-After when supplied and apply bounded backoff. Do not retry forever, and do not retry a non-idempotent operation without a request key.

const delay = (attempt: number): number =>
  Math.min(8_000, 250 * 2 ** attempt);

async function withBackoff<T>(
  operation: () => Promise<T>,
  maxAttempts = 3,
): Promise<T> {
  for (let attempt = 0; ; attempt += 1) {
    try {
      return await operation();
    } catch (error) {
      if (attempt >= maxAttempts - 1 || !isRetryable(error)) {
        throw error;
      }
      await new Promise((resolve) => setTimeout(resolve, delay(attempt)));
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

The important part is the policy around this helper. A search read can usually be retried. A request that creates a review record needs an idempotency key. A generation call should be retried only when the output can be safely replaced and the request budget still permits it. Three attempts is a policy example, not a promise that every service should use the same number.

Use timeouts at each boundary. Log duration, attempt number, result count, index revision, and a redacted request ID. Do not log student source by default. A one-person team cannot afford an observability system that creates a second privacy problem.

How do I compare implementations without choosing a vendor?

Compare boundaries and operational behavior, not feature checklists. A hosted vector search service can reduce maintenance while imposing its own filter, export, or index-version constraints. A self-hosted search stack gives more control over deployment and data locality but adds upgrades, capacity planning, and on-call work. A Postgres-based vector extension keeps metadata and retrieval near application data, which can simplify transactions, while search behavior and scaling still need testing against the workload.

For concrete reference points, Pinecone exposes a managed vector index, Weaviate can be deployed as an open-source database or used as a managed service, and pgvector keeps vector columns and nearest-neighbor queries in Postgres. Those facts describe different ownership surfaces; they do not establish a winner. The adapter still has to normalize filters, result metadata, deletion behavior, and index revisions. Treat the 2020-12 JSON Schema dialect and the OpenAPI 3.1.1 contract as interface constraints, then test each implementation against the same fixtures.

These are different trade-offs. None is universally best.

The first failure to look for is a plausible answer with the wrong evidence.

Test the same corpus through each adapter. Use a small evaluation set made from real review questions: a missing null check, an incorrect authorization condition, and a course rule that conflicts with a generic style suggestion. Measure recall of the relevant evidence, precision of the returned chunks, latency at the selected limit, and the percentage of responses that fail schema validation. Keep those measurements separate from model quality. Otherwise a retrieval regression will look like a prompt regression.

Portability has a practical definition here. You should be able to change the embedding or index implementation while preserving document IDs, filters, source revisions, and the ReviewFinding contract. If changing providers requires changing the UI, prompt format, and database records together, the adapter is not a boundary. It is an SDK wrapper.

The runner-up approach is better when the main constraint is operational simplicity: one managed retrieval system may be the right call for a tiny team with a narrow corpus and no need to export indexes. Conversely, a self-managed or database-local approach becomes more attractive when course data must remain in a controlled environment or when you need direct access to indexing behavior. Make that call from retention, filter, export, and debugging requirements.

The decision I would ship this week

Start with one public chat endpoint and one internal SemanticRetriever interface. Version the finding schema before adding more providers. Persist the source revision and evidence IDs. Add fixtures for three retrieval failures and one HTTP 429 response. Then run the same review questions through every adapter in CI.

That is enough structure for weekly shipping. The undifferentiated provider plumbing stays behind one seam, while the product work remains visible: clearer feedback for students and less review triage for teachers. The payoff is not a promise of perfect answers. It is the ability to improve retrieval without invalidating the conversation, the evidence trail, or the tests that make the feature trustworthy.

References

Top comments (0)