What Is RAG? A Working Retrieval Example You Can Run With Just an Anthropic Key

#ai #rag #typescript #beginners

An agent can have every tool it needs and still answer wrong — because it doesn't know something. Not incapable of knowing it; the fact just wasn't in its training data, or it changed afterward, or it lives in a document only your team has.

This post continues my series on building agents. The previous one covered tool calling.

What RAG actually is

Retrieval Augmented Generation pulls relevant text from a knowledge source at the moment a question is asked and inserts it into the model's prompt before it answers, instead of relying only on what the model learned during training. Update the documents and the next answer reflects the change immediately — no retraining involved.

Three jobs, not one

Embedding — turn text into a vector, positioned so similar meanings land near each other.
Retrieval — compare the question's vector against every stored document vector (commonly with cosine similarity) and keep only the closest matches.
Augmentation — insert those matches into the prompt as context before the model answers.

flowchart LR
    Q[User question] --> E[Turn the question into a vector]
    E --> S[Compare against every stored document vector]
    S --> K[Keep only the closest matches above a threshold]
    K --> P[Add those chunks to the prompt as context]
    P --> M[Model generates the answer]

A working example

Anthropic doesn't ship its own embedding model — its docs point to Voyage AI instead. So this demo swaps a real embedding model for a hand-written one (word counts as vectors, cosine similarity for matching) to keep the whole thing runnable with just ANTHROPIC_API_KEY.

// agent/simple-rag.ts
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

const knowledgeBase: string[] = [
  'This blog is a Next.js app deployed on Vercel. Pushing to the main branch triggers a new production deployment automatically.',
  'Blog posts are stored as JSON files under content/posts and read at build time, so adding a new post requires a redeploy before it appears live.',
  'The blog index page uses Incremental Static Regeneration with a 60 second revalidate window.',
  'Environment variables such as API keys are set in the Vercel dashboard under Project Settings, Environment Variables, never committed to git.',
  'Every pull request gets its own preview deployment with a unique URL, separate from the production domain.',
];

function tokenize(text: string): string[] {
  return text.toLowerCase().match(/[a-z0-9]+/g) ?? [];
}

function vectorize(text: string, vocabulary: string[]): number[] {
  const counts = new Map<string, number>();
  for (const token of tokenize(text)) counts.set(token, (counts.get(token) ?? 0) + 1);
  return vocabulary.map((word) => counts.get(word) ?? 0);
}

function cosineSimilarity(a: number[], b: number[]): number {
  let dot = 0;
  let normA = 0;
  let normB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    normA += a[i] * a[i];
    normB += b[i] * b[i];
  }
  if (normA === 0 || normB === 0) return 0;
  return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

function retrieve(query: string, topK = 2, minScore = 0.22): string[] {
  const vocabulary = Array.from(new Set([query, ...knowledgeBase].flatMap(tokenize)));
  const queryVector = vectorize(query, vocabulary);
  return knowledgeBase
    .map((doc) => ({ doc, score: cosineSimilarity(queryVector, vectorize(doc, vocabulary)) }))
    .sort((a, b) => b.score - a.score)
    .filter((s) => s.score >= minScore)
    .slice(0, topK)
    .map((s) => s.doc);
}

async function askWithContext(question: string): Promise<string> {
  const context = retrieve(question);

  if (context.length === 0) {
    return "I don't have anything in the knowledge base about that.";
  }

  const response = await client.messages.create({
    model: 'claude-sonnet-5',
    max_tokens: 512,
    system:
      'Answer only using the context below. If the context does not contain the answer, say you do not know.\n\nContext:\n' +
      context.map((c, i) => `${i + 1}. ${c}`).join('\n'),
    messages: [{ role: 'user', content: question }],
  });

  const text = response.content.find((b): b is Anthropic.TextBlock => b.type === 'text');
  return text?.text ?? '';
}

askWithContext('How do new blog posts actually go live on the site?').then(console.log);

The mistake that took tuning to find

My first retrieve had no minScore — it just returned the top two matches, whatever their score. An unrelated cooking question still returned two chunks, because a few common words overlapped with the documents and cosine similarity can't tell those words carry no meaning. The model then answered confidently using the wrong context. I fixed it by printing the actual scores for questions I cared about and finding where "genuinely related" stopped and "just sharing a few words" began — 0.22, for this data.

Common mistakes

No score floor on top-k. Threshold on the score, not just the rank.
Chunking too large or too small. Start at paragraph or section level.
Never checking what got retrieved. Log it alongside the answer during development.
Skipping a real embedding model in production. Word-overlap vectors miss synonyms entirely.

Read the full version

I write each post in English and Bangla. The full version is on my blog: What Is RAG?

Where does your RAG pipeline draw the line between relevant and irrelevant?

DEV Community