A base agent with LangGraph: tools, memory, RAG and examples

When I start a new agent I always start from the same base. I use it every day in macro, my app for logging what I eat: I tell it what I had, or send it a photo of the plate, and it logs it with its calories and macros. I also ask it how today is going, or correct something I logged wrong.

It is JavaScript with LangGraph.js, Fastify and MongoDB. The code in this post is simplified so the idea shows, but the decisions are the ones macro has.

macro does not need RAG: everything it knows is the user's own data, and it reads it with tools. When an agent does need knowledge that is not in your database, like documentation or policies, I add RAG on top of the same base. I tell that part with what I did in aside.

What I mean by agent

A model that decides which tool to call, looks at the result and decides again, in a loop, until it has an answer. Anthropic describes it this way in Building effective agents: the basic building block is "an LLM enhanced with augmentations such as retrieval, tools, and memory", and the agent is that LLM using tools in a loop based on what the environment returns.

The same article recommends finding the simplest solution possible and only adding complexity when needed. That is why my base has two nodes and nothing else.

The graph: two nodes and a loop

One node calls the model. The other runs the tools the model asked for. If the model asks for a tool, it goes to the tools node and back to the model. If it asks for none, it ends.

import { StateGraph, START, MessagesAnnotation } from '@langchain/langgraph'
import { ToolNode, toolsCondition } from '@langchain/langgraph/prebuilt'
import { SystemMessage } from '@langchain/core/messages'

async function buildGraph({ userId, tools, systemPrompt }) {
  const model = (await getLLM()).bindTools(tools)

  async function callModel(state) {
    const response = await model.invoke([
      new SystemMessage(systemPrompt),
      ...windowForModel(state.messages),
    ])
    return { messages: [response] }
  }

  const graph = new StateGraph(MessagesAnnotation)
    .addNode('agent', callModel)
    .addNode('tools', new ToolNode(tools))
    .addEdge(START, 'agent')
    .addConditionalEdges('agent', toolsCondition)
    .addEdge('tools', 'agent')
    .compile({ checkpointer: getCheckpointer() })

  const config = { recursionLimit: 25, configurable: { thread_id: userId } }
  return { graph, config }
}

toolsCondition checks whether the model's last message has tool calls and picks the next node. recursionLimit is the brake: the maximum number of steps the graph can take in one run. LangGraph sets it to 25 by default, and I make it explicit so it is visible. If the model gets stuck in a tool loop, the run fails instead of burning tokens forever.

Tools: what the agent can do

macro has about twenty tools: log a meal, list, search, edit, delete, read the daily summary, set goals, log weight and water, look up a barcode or scan a photo.

Three decisions I repeat in any agent:

One source of tools. The same tools feed the chat and an MCP server, so other AI clients can use my data. Each tool calls an application use case, the same one the REST API uses. The chat only chooses which ones to expose.

The code sets the user, never the model. The userId is not a tool argument. It is injected from the session when the tool is created, so the model cannot ask for another user's data even if someone tries to slip that into a message.

Errors come back as data. If a tool fails, I return { error } instead of throwing. The model reads it and can correct itself: ask for the missing piece or try another id.

import { tool } from '@langchain/core/tools'

function createChatTools({ registry, userId }) {
  const ctx = { userId }
  return registry.listTools()
    .filter((t) => CHAT_TOOL_NAMES.includes(t.name))
    .map((t) => tool(
      async (args) => {
        try {
          const result = await registry.callTool(t.name, args || {}, ctx)
          return JSON.stringify(result ?? null)
        } catch (err) {
          return JSON.stringify({ error: err.message || 'tool failed' })
        }
      },
      { name: t.name, description: t.description, schema: t.inputSchema },
    ))
}

Each tool's description matters as much as its code. It is all the model knows about it. Anthropic calls it the agent-computer interface, and recommends putting as much care into it as into an interface for people, with usage examples and edge cases. Which brings me to examples.

Examples in the prompt (few-shot)

Few-shot means giving the model a few examples of what you want inside the prompt itself, without retraining it. The name comes from Language Models are Few-Shot Learners, the GPT-3 paper. Today it is one of the cheapest tools you have: Claude's prompting guide says examples are one of the most reliable ways to steer format, tone and structure, and recommends 3 to 5, close to the real use case, varied and kept apart from the instructions.

I use them in three places.

In each tool's description, with real phrases. I do not write "searches the meal history", I write the phrases people actually say. macro's users write in Spanish, so the examples are in Spanish:

search_meals: search the user's meal history by name and macro/date filters.
Use for "¿qué cenas con más proteína comí esta semana?", "busca tortilla",
"desayunos de más de 400 kcal de los últimos 14 días".
Prefer this over list_meals when the user is looking for specific meals.

With that, the model does not have to guess when to use search_meals and when list_meals.

In the rules, as sequences. Corrections are what breaks most if you do not spell them out. So the prompt says what to do with "it was yesterday", "make it 600 kcal" or "move it to dinner": first list_meals for the right day, then update_meal on the right id. It is an example of what to do, not only of what to say.

For tone, a written conversation. In aside I do not describe the tone with adjectives. I describe a support person and write sample sentences of theirs: how they greet, how they ask for a detail, how they say something cannot be done. Every model reads an adjective like "friendly" its own way. A written sentence, it does not.

If what you need is an exact output format, you can also put the examples as user and assistant message pairs before the conversation. In an agent with tools I prefer to keep them in the system prompt, because they do not mix with the real history.

The prompt: scope, rules and the date

macro's system prompt has four parts:

1. Scope. It only helps with the user's diet. If asked something else, it deflects in one sentence and offers something useful, without explaining why it will not answer.
2. The tools and when to use each one , with the examples above.
3. Rules. Read the daily summary before answering how today is going, never save a photo result without confirming, answer in one or two sentences after a tool, never moralize.
4. The date. Computed on every message.

The date looks trivial and it is not. The model does not know what day it is, and "yesterday" or "on Monday" have to be resolved. So the prompt carries today, yesterday and tomorrow already computed, plus the rule to use the current year when the user does not say one.

function buildSystemPrompt({ now = new Date() } = {}) {
  const today = now.toISOString().slice(0, 10)
  return `${SYSTEM_PROMPT}

Current date: ${today} (UTC).
- If the user does not mention a date, use ${today}.
- "ayer" / "yesterday" is ${shiftDate(today, -1)}.
- A day and month without a year use the current year.
Never invent a past or future year.`
}

I also pass a precomputed summary of the day: what was consumed, the target and what is left. With the instruction to use the tools for exact answers and the summary only as backup.

The last rule in the prompt treats any user message or image as data, never as instructions. A rule in the prompt is not a guarantee, but it helps. The real guarantee lives in the code: the injected userId and the confirmation before deleting.

Memory: the thread in Mongo and a short window

LangGraph keeps the conversation memory with a checkpointer. According to its persistence docs, the checkpointer saves the graph state per thread, and the thread is picked with a thread_id in the config. I use the MongoDB checkpointer with the app's own connection and the userId as thread_id: one conversation per user, which survives reloads and moves across devices for free.

import { MongoDBSaver } from '@langchain/langgraph-checkpoint-mongodb'

const checkpointer = new MongoDBSaver({
  client: mongoose.connection.getClient(),
  dbName: mongoose.connection.db.databaseName,
})

Saving everything does not mean sending everything to the model. Each message only carries the last 10, so the cost does not grow with the conversation. The full history stays in Mongo and the UI loads it from there.

There is a catch when cutting. Every tool result has to come after the call that asked for it. OpenAI's API requires each result to reference its call by id. If the window starts right on a tool result, its call was left out and the request fails. So the window never starts on one:

const MAX_CONTEXT_MESSAGES = 10

function windowForModel(messages, max = MAX_CONTEXT_MESSAGES) {
  if (messages.length <= max) return messages
  let start = messages.length - max
  while (start < messages.length && messages[start].getType() === 'tool') start++
  return messages.slice(start)
}

RAG: when the agent needs to know things

RAG is retrieving relevant text and giving it to the model so it answers with it. The term comes from this 2020 paper. What matters is where that text comes from.

In macro, what the agent needs to know is the user's meals. That is structured data, so retrieval is search_meals, a Mongo query with filters on name, date and macros. You do not need embeddings to find "breakfasts over 400 kcal".

In aside it was needed: the agent answers with each customer's documentation. I set it up the same way, as one more tool:

const searchDocs = tool(
  async ({ query }) => {
    const hits = await searchKbDocuments(workspaceId, query)
    return JSON.stringify({
      results: hits.map((h) => ({ title: h.title, text: h.excerpt.slice(0, 600), url: h.source })),
    })
  },
  {
    name: 'search_kb',
    description: "Search the knowledge base for articles relevant to the user's question. "
      + 'Use when the user asks something that might be documented.',
    schema: z.object({ query: z.string().describe('Concise search query') }),
  },
)

As a tool, the model decides when to search and with which words, and can search twice if the first try is not enough. I return it trimmed and with its source, so the UI can show the source under the answer.

My rule of thumb:

• Few documents: search by terms, which is what I use in aside.
• Many documents, or users who ask with different words than the docs use: embeddings.

And what comes out of the search is external content. OWASP calls it indirect injection: instructions that arrive inside a website or a document. It is treated as data, just like the user's message.

Before deleting, ask

delete_meal does not delete. It returns a pending confirmation with the id, and the UI shows a button. The real delete is done by the REST API when the user presses it.

if (t.name === 'delete_meal') {
  return JSON.stringify({ pendingConfirmation: true, action: 'delete', id: args?.id ?? null })
}

The prompt tells the model to call the tool once and say that the user has to confirm with the button. OWASP recommends the same for any risky action: a person approves it. Photos work in a similar way: the agent shows what it detected and saves nothing until the user confirms or corrects it.

For payments or emails, which cannot be undone, you also need idempotency. I covered it when writing about duplicated webhooks.

Streaming

To show the answer while it is written I use streamMode: 'messages'. The streaming docs explain that each chunk comes with metadata, including the node that produced it. I filter by the agent node so tool output does not leak into the chat:

const stream = await graph.stream({ messages: [userMessage] }, { ...config, streamMode: 'messages' })

for await (const [chunk, metadata] of stream) {
  if (metadata?.langgraph_node !== 'agent') continue
  const text = extractText(chunk.content)
  if (text) yield { type: 'token', content: text }
}

When it ends I read the thread state and check which write tools ran in this turn. That way the UI knows whether to refresh the daily counter or show the confirm button, without parsing the model's text.

Voice: listening and talking

aside used to have voice. I removed it because it caused problems, it was expensive and it was not the core. But it worked, and adding it on top of this base is easier than it looks, because the agent does not notice: voice comes in as text and goes out as text.

Listening (STT). I used ElevenLabs' Scribe v2 Realtime from the browser, with their React hook. The browser asks my server for a token, so the API key never leaves the server. The mic is turned into 16 kHz audio and sent over a WebSocket. Scribe detects when you stop talking (about 0.8 seconds of silence) and returns the whole sentence. That sentence enters the graph as one more user message.

What fought back the most was echo. The mic stays open while the agent talks, so you can interrupt it, and it hears the agent itself. So while it talks I drop sentences under four words, and also loose noise of one or two words.

Talking (TTS). ElevenLabs Flash v2.5, also over a WebSocket. I do not wait for the model to finish: I pass the text to ElevenLabs as it arrives through the stream, and it starts talking with the first sentence. On this base it would look like this:

const speaker = await tts.startTurn(audioSink)

for await (const [chunk, metadata] of stream) {
  if (metadata?.langgraph_node !== 'agent') continue
  const text = extractText(chunk.content)
  if (!text) continue
  yield { type: 'token', content: text }
  speaker.feedDelta(stripMarkdown(text))
}

await speaker.end()

Three details made it usable:

• One socket per turn. If each sentence goes on its own, each one sounds like it starts from scratch. With one socket per answer, the intonation carries from one sentence to the next.
• Strip the markdown before speaking. Otherwise it reads the asterisks and hashes out loud.
• A cache per sentence. I store each sentence's audio in Mongo. The ones that repeat, like the welcome line, come from there without paying again.

And if the user talks over it, the audio stops and a new turn starts.

Switching models without touching the agent

The agent does not know which model it uses. getLLM() builds the client from a setting stored in the database: OpenAI by default, Gemini, or MiniMax through its Anthropic-compatible API. The setting is cached for five seconds, so I switch provider from the admin panel without deploying.

That is what lets me try a new model on my real data in a minute. If the new model misuses a tool, I almost always fix it in the tool's description or in an example, not in the graph.

What I leave out at the start

No multiple agents, no planners, no long-term memory, no embeddings up front. The two-node loop, well-described tools, examples with real phrases, the thread in Mongo with a short window, and a confirmation before anything that cannot be undone. With that, macro works every day, and I add the rest when a real case asks for it.

None of this may be the best way. It came out of trial and error, and it is what works for me.