DEV Community: Harshavardhan Katkam

Day 03: Your repo does not fit in the context window. Now what?

Harshavardhan Katkam — Wed, 14 Oct 2026 13:19:13 +0000

Handing cartographer files one at a time had been working, so I did the obvious next thing and wrote the four lines that walk a repository and concatenate every source file into one string. Large context windows exist precisely so you can stop being clever about this.

Then, out of habit, I printed len() on the string before sending it.

I never sent it. Not because it would definitely have failed — some of it would have fit — but because I could not answer either of the two questions that immediately followed. How many tokens is that, actually? And what does one run of this cost? I had been building an agent for two days with no number attached to any part of it.

So today's build is not a feature. It is a tape measure and a meter. What they tell you is the constraint the entire project exists to work around, and it is better to meet it with a print() than with an invoice.

The Short Version

Code for this day: cartographer at day-03

A repository is measured in characters on disk; a model bills in tokens, the chunks of text it reads and writes. The ratio between them is not a constant you can look up.
Do not estimate with a tokenizer library from a different model family. You get a precise number for the wrong vocabulary, because each family splits text into tokens its own way, and the error is systematic rather than random.
Two samples and a subtraction give you the real ratio and cancel the fixed per-request overhead that a single sample would silently bake in.
The context window, the most one request can carry, is the wall you notice. The token count is the wall you hit first, because a transcript re-sends itself on every call.
Input and output are priced differently, so a meter that adds the two token counts together tells you nothing.
The conclusion is not "wait for a bigger window". It is that the agent has to choose what to read. That is a feature, not a workaround.

Measure the repository first

Walking a repository is easy. Walking it correctly is where the first mistake lives.

def source_files(root: pathlib.Path) -> Iterator[pathlib.Path]:
    for path in root.rglob("*"):
        if path.suffix not in SOURCE_SUFFIXES or not path.is_file():
            continue
        if SKIP_DIRECTORIES.intersection(path.relative_to(root).parts):
            continue
        yield path

def measure(root: pathlib.Path) -> tuple[int, int]:
    paths = list(source_files(root))
    return len(paths), sum(len(path.read_text(errors="ignore")) for path in paths)

Both live in a new module, src/cartographer/budget.py, which knows about files and arithmetic and nothing about HTTP. rglob("*") walks every path under the root. A file survives only if its extension is in SOURCE_SUFFIXES (Python, docs, config) and none of its folder names below the root is in SKIP_DIRECTORIES; the set intersection checks them all at once. measure returns the file count and the total characters, ignoring bytes that are not valid text.

The skip list is doing more work than the walk. Leave .venv and node_modules out of it and you are not measuring a project, you are measuring its dependencies, which is a much larger number that says nothing about the code. Measure the code someone wrote, not everything installed beside it.

Run that on any repository you have lying around before reading further. The character count is real and takes seconds to get.

The estimate everybody gets wrong

Characters are not what you are billed for. So you need a ratio, and there are two popular ways to get one, both wrong.

The first is characters // 4. It is a folk constant, it is not specific to your model, and it is not specific to source code, which tokenizes differently from prose because of indentation, punctuation and long snake_case identifiers.

The second is worse, because it feels rigorous: install a tokenizer library, encode the repository, report the count. That library ships one vendor's vocabulary. If you are calling a model from a different family, you have computed a precise number for a tokenizer nobody in your request is using. Precision is not accuracy, and the error is not noise you can average away — it is a systematic bias in a direction you cannot see from the outside, because you do not have the vocabulary the server used.

Two samples and a subtraction

There is an authoritative counter available and it costs almost nothing to query: the server tells you what it counted, in usage.prompt_tokens, on every response.

def prompt_tokens(text: str) -> int:
    body = post([{"role": "user", "content": text}], max_tokens=1)
    return int(body["usage"]["prompt_tokens"])

post is the old body of ask in src/cartographer/agent.py, now its own function so it can return the whole response and accept an optional max_tokens; ask now calls it and keeps its old signature. One request, capped at a single output token, because we are measuring input, not generating an answer.

Now the naive move is to send one sample and divide. Do not: prompt_tokens counts the whole rendered request, including the chat template the server wraps your message in. On a short sample that fixed overhead is a large fraction of the total, and you would fold it permanently into your ratio.

Send two samples of very different sizes instead, and take the slope. The constant cancels.

def chars_per_token(count: Callable[[str], int], small: str, large: str) -> float:
    return (len(large) - len(small)) / (count(large) - count(small))

Two requests, two output tokens, and you have a ratio measured against the model you are actually calling. It goes stale the moment you change models, which is the argument for it being a function rather than a constant in a config file.

The test that matters is offline: give it a counter with a known overhead of 11 tokens and a known true ratio of 4, and assert the result is exactly 4.0. If the subtraction were wrong, that assertion would fail.

Count input and output separately

The other half of today is the meter, and it exists because the two token counts are not interchangeable.

@dataclass
class Meter:
    input_per_million: float = 0.0
    output_per_million: float = 0.0
    calls: int = 0
    input_tokens: int = 0
    output_tokens: int = 0

    def record(self, usage: dict[str, int]) -> None:
        self.calls += 1
        self.input_tokens += usage.get("prompt_tokens", 0)
        self.output_tokens += usage.get("completion_tokens", 0)

record adds each response's usage block to running totals, and post calls it on every response, so no request can bypass the meter. A dollars property multiplies each total by its own price per million, and tests/test_budget.py pins all three pieces offline: the calibration, the skip list and the pricing. Every line is explained in today's dev notes.

Input and output are billed at different rates on every provider I have used, so a single total is not convertible into money. Keeping them apart also tells you which lever to pull: the transcript grows and every file gets re-sent, while the output stays a paragraph.

The rates themselves come from environment variables and default to zero. No price literal goes into the source, because published rates change without anyone editing your repo, and a stale number in version control is worse than no number. On a free model the dollars are honestly zero and the token counts are still true, which is the point: tokens are the thing that runs out, against the window and against rate limits, long before money does.

Three ways to make a repo fit

	Send the whole repo	Index it all up front	Let the agent choose what to read
You pay	the whole repo, on every call	one embedding pass (every file turned into searchable numbers), then cheap lookups	only for the files it opened
Breaks when	the repo outgrows the window, which is soon	the index goes stale on the next commit	the agent chooses badly
Right when	one question, one small repo	stable code, many repeated queries	large repo, open-ended questions

The third column is the one this project needs, and not reluctantly. A map made from everything is not a map: it is the territory again. Choosing what matters is the actual product, and the constraint is what forces the design to be honest about it.

My numbers for the pinned evaluation repositories live in docs/measurements.md, with a commit SHA beside each one (click 8.5.0 came to roughly half a million tokens), because a measurement against a moving branch is not a measurement and a blog post is the wrong place to freeze it.

What I'd do

Measure before you design. A few seconds of len() will tell you whether the architecture you are imagining is even legal.
Calibrate against the model you call. Two samples, subtract, never a folk constant and never another vendor's tokenizer.
Keep prices out of source. Environment variables, defaulting to zero.
Print the meter on every run, including free-tier runs. The habit is the point.
Treat selection as the feature. The moment you accept the agent must choose what to read, tools stop being a convenience and become the product.

If this was useful, a clap helps other people find it. I would like to know how other people size this up: do you estimate agent cost before you build, or find out from the first surprising bill?

All views in this post are my own. The scenarios are illustrative composites, and code examples are not drawn from any specific codebase.

Day 02: An LLM is a stateless function. Everything else is you

Harshavardhan Katkam — Wed, 14 Oct 2026 13:16:50 +0000

I pointed the forty-line version of cartographer at a second file and asked the obvious follow-up: how does this one relate to the last one?

It asked me which file I meant.

My first instinct was that I had broken something. A flag I had dropped, a session id I had forgotten to thread through. I went looking for it for longer than I am comfortable admitting. There is no session id. There was never going to be one. The endpoint had no idea I had called it ninety seconds earlier, and no amount of configuration was going to give it one.

Every piece of memory you have ever seen in an AI product is somebody, somewhere, deciding what goes into an array before a POST. That is the entire mechanism. Once you have seen it you cannot unsee it, and a surprising amount of vocabulary stops being mysterious.

The Short Version

Code for this day: cartographer at day-02

The model is a pure function of the array you send. It retains nothing between requests, and nothing you can configure changes that.
The forty-line version sent one message. Now it sends a list, and the caller owns that list — no library, no service, no hidden field.
Append the assistant's reply as well as the user's. Omitting it produces a subtler bug than amnesia.
Your agent now has exactly one piece of state, and it is a JSON file. Debug it with cat before you reach for anything else.
Re-sending a transcript is quadratic: ten equal turns bill you fifty-five turns' worth of input, not ten.
"Memory" is a product word. Every implementation of it is a policy for what to put in the array, and choosing that policy well is most of the rest of this series.

The follow-up that goes nowhere

Here is the version I wrote first, and it is worth staring at because it looks correct.

def turn(messages: list[dict[str, str]], content: str) -> str:  # Broken. Do not ship this.
    return ask(messages + [{"role": "user", "content": content}])

A list of messages goes in. A reply comes out. The signature reads like a conversation.

But messages + [...] builds a throwaway list, sends it, and drops it. The caller's list is never touched, so every call starts from the same place. This survives a read-through precisely because the type annotation implies history — you have to notice that nothing ever grows.

The endpoint this series uses is the OpenAI-shaped chat completions endpoint, and on that shape a request is self-contained. Whatever you did not put in the array did not happen.

There is no session. There is a list.

The fix is three lines and it is the whole idea of the day.

def turn(messages: list[dict[str, str]], content: str) -> str:
    messages.append({"role": "user", "content": content})
    reply = ask(messages)
    messages.append({"role": "assistant", "content": reply})
    return reply

It mutates the list it was handed. That is deliberate: there is one transcript, the caller holds it, and every append happens in one function you can put a breakpoint in.

The second append is the line people drop, and its failure mode is worse than no memory at all. Leave it out and the model receives a list of questions with no answers attached — a transcript in which it apparently never spoke. So it re-answers things it already answered, contradicts a conclusion it reached two turns ago, and generally behaves like a model that is bad at reasoning. You will spend an evening rewriting the system prompt, the standing instructions in the first message, before you find it.

Making both appends the responsibility of one function is not style. It removes the state where only half the exchange got recorded.

Around turn, the rest of src/cartographer/agent.py changes less than you would expect. Yesterday's template goes, replaced by SYSTEM, the system message that opens every list and tells the model it is mapping a repository one file at a time and should refer back to earlier files by name. ask keeps its body but now takes the whole list and sends it as the messages field, so it no longer decides what the model sees. main turns every path on the command line into one turn, then asks a closing question that only a model holding every earlier file can answer: which would you read first? tests/test_agent.py is replaced by a test that swaps ask for a fake and asserts both the order of roles and that the second call saw four messages. Today's dev notes walk through every line.

Your agent has exactly one piece of state

Because the model is a pure function of that array, the array is the complete state of your program. So write it down:

TRANSCRIPT.write_text(json.dumps(messages, indent=2))

That line ends main, and transcript.json goes into .gitignore, because it holds whole source files and is rebuilt on every run. This is the highest-value five minutes of the day, and almost nobody's tutorial does it. When the output is wrong, the question you actually need answered is not why did the model say that. It is what did the model see. There is exactly one artifact that answers it, and now it is on disk.

Nearly every bad answer I have chased at this stage has been visible in that file within thirty seconds: a file that was never appended, a source read as an empty string because the path was relative to the wrong directory, a system message clobbered by a later assignment. None of those are model problems, and all of them look like model problems until you open the transcript.

It also makes runs replayable. The input is a JSON array; you can re-send it byte for byte, change one message, and send it again.

The bill is a triangle, not a line

Now the part that gets expensive. Every call carries the whole array, so you pay for turn one again on turn two, and again on turn three.

If each turn adds roughly T tokens, the chunks of text a model reads and bills by, the n*th request carries about *nT input tokens, and the run as a whole costs T·n(n+1)/2. Ten turns is fifty-five turns' worth of input. Twenty turns is two hundred and ten. The curve is a triangle, and you are billed for its area.

For a chat assistant that is survivable, because a turn is a sentence. For an agent reading source files, each turn is an entire file, so T is enormous and the triangle gets frightening at single-digit n. That is a measurement problem before it is a design problem, which is what tomorrow is about.

There are three honest strategies, and they trade against each other:

	Send everything	Summarise older turns	Keep pointers, re-read on demand
Input cost over n turns	grows with the square	flat once the cap bites	flat, plus one call per lookup
What you give up	the context window, then your budget	detail you did not know mattered	round trips and latency
Right when	short runs, and while learning	chat, where old turns are chatty	the content is on disk and cheap to re-read

For a codebase the third column is clearly right — the repository is still sitting there, and re-reading a file costs a disk read. That is the direction this project goes, and it is why the agent eventually has to be able to fetch things for itself rather than being handed them.

What I'd do

Pass the transcript as an argument. The moment it becomes a hidden attribute on an object, you stop being able to see what you sent.
Append both sides in one function. Make the half-recorded exchange unrepresentable.
Dump the array to disk on every run, from the first day. It is four lines and it is your only observability until much later.
Assert on the sequence of roles in a test. Mine checks that two turns produce system, user, assistant, user, assistant — it would have caught the broken version above immediately.
Do not install a memory library until you can name which column of that table you need. All three are a page of code; the hard part is the choice, and no library makes it for you.

If this was useful, a clap helps other people find it. I am curious about the other side of this one: what is the longest-running agent conversation you have kept alive, and what did you eventually have to throw away to keep it affordable?

All views in this post are my own. The scenarios are illustrative composites, and code examples are not drawn from any specific codebase.

Day 01: Why your first agent should be 40 lines, not a framework

Harshavardhan Katkam — Wed, 14 Oct 2026 13:14:39 +0000

The first agent I ever got working, I did not understand. I had installed a framework, pasted eleven lines out of its quickstart, and watched a correct answer appear. Then I changed the prompt slightly and it printed nothing. Not an error. An empty string, exit code zero.

I spent the next forty minutes in a debugger. Nine frames down, past a callback manager and two abstract base classes, I found what I had been hunting: a dictionary getting built, and an HTTP POST. That was it. That was the whole thing the framework had been doing for me.

The bug was not what bothered me. What bothered me was that I could not have guessed where to look, because I had never once seen the request. I had learned a vocabulary — chains, runnables, callbacks — sitting on top of a fact I had never been shown.

So the first working version of cartographer, the codebase-mapping agent I am building over thirty days, has nothing in the way. One file, one dependency, forty lines.

The Short Version

Code for this day: cartographer at day-01

An agent is an HTTP POST in a loop. Everything else is retries, ergonomics and scheduling.
Frameworks abstract the part that was already easy and hide the part you need visible when it breaks.
Day one is forty lines: read a file, post it, print the reply. httpx is the only import that is not standard library.
Call raise_for_status() before you touch the body, or every auth and rate-limit failure reaches you as KeyError: 'choices'.
The reason to start raw is not purity. It is that you will end up reading a framework's source anyway, and this is the map you read it with.
The honest ending: it works, and the output is prose nobody can check. That, not the absent framework, is the real problem.

An agent is a POST. That is the entire secret.

Strip every layer off and what reaches the network is one request with two fields that matter: a model id, and a list of messages, each one a role (system, user or assistant) plus its text. What comes back is JSON, and the part you want is buried at a fixed address inside it. Every model provider worth using speaks this shape, which is why one endpoint can front hundreds of models.

That is not a simplification for teaching purposes. It is the actual protocol. The rest of this series — tools, protocols, graphs, memory, retrieval — is thirty days of arguing about which strings go into that array and in what order. If you do not have the array clearly in your head, none of the later arguments have anywhere to land.

The forty lines

I am running everything in this series through OpenRouter, one endpoint at https://openrouter.ai/api/v1 speaking the OpenAI wire protocol, on free models — so anyone can reproduce the whole thing for nothing. Nothing here is specific to that choice; change the URL and the model string and it works elsewhere.

import os
import pathlib
import sys

import httpx

OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions"
MODEL = os.environ.get("CARTOGRAPHER_MODEL", "nvidia/nemotron-3-super-120b-a12b:free")
TARGET = pathlib.Path("src/cartographer/agent.py")

PROMPT = """You are reading one file from a repository you have never seen.

Say what this file is for, what it depends on, and what would break if it
were deleted. Where the file does not say, answer that you cannot tell.

--- {path} ---
{source}
"""

def ask(prompt: str) -> str:
    response = httpx.post(
        OPENROUTER_URL,
        headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
        json={"model": MODEL, "messages": [{"role": "user", "content": prompt}]},
        timeout=120.0,
    )
    response.raise_for_status()
    body = response.json()
    return str(body["choices"][0]["message"]["content"])

def main() -> None:
    if not TARGET.exists():
        sys.exit(f"cannot find {TARGET} from {pathlib.Path.cwd()}")
    print(ask(PROMPT.format(path=TARGET, source=TARGET.read_text())))

if __name__ == "__main__":
    main()

That is all of src/cartographer/agent.py, the one file today adds, and it reads top to bottom in four parts.

The constants say where and what. OPENROUTER_URL is the chat completions endpoint, the address that takes a list of messages and returns a reply. MODEL reads the model id from an environment variable, with a default. TARGET is the one file to read.

PROMPT is a template. {path} and {source} are placeholders that .format() fills with the file's name and contents. Its instructions ask three concrete questions and, deliberately, permit "I cannot tell", because a model that is not allowed to say that fills the gap with something plausible.

ask is the model call. It posts a JSON body with exactly two fields, the model id and a one-message list, with the key in an Authorization header and a two-minute timeout, because free models are slow. It checks the status, parses the reply, and digs the text out of choices[0].message.content. The str() around that lookup is for the type checker, which will not accept an untyped value from a function promising a string.

main checks the file exists, fills the template, asks, and prints. The if __name__ guard runs main only when the file is executed as a program, so importing it from a test sends nothing.

The second new file, tests/test_agent.py, checks without a network that the filled prompt contains the file's text and the permission to be uncertain. Today's dev notes explain every line, including the commands to run it.

The target is hardcoded to the file you are looking at, so the agent's first job is to describe itself. That is a deliberately easy first test: you already know the right answer, so you can judge the output the moment it appears.

The model id comes from the environment with a default rather than being pinned in source, because free-tier availability churns constantly. When one model is busy you swap it in your shell, not in a commit.

Three failures a framework would have swallowed

No key. os.environ['OPENROUTER_API_KEY'] raises KeyError at the exact line that needed it. Subscript, not .get(). A missing credential should not be allowed to become an empty string that travels three functions before failing.

A bad key, or a busy model. Without raise_for_status(), a 401 or a 429 gives you a perfectly valid JSON body containing an error object and no choices — so the failure surfaces as KeyError: 'choices' from the last line of ask, pointing at the one place where nothing is wrong. One method call converts that into an exception naming the status code.

An empty answer. choices can come back as an empty list. Under raw HTTP you see that immediately, because you are the one doing the indexing. Under a framework you get None from a helper and start wondering about your prompt.

None of these are exotic. They are the first three things that happen to everybody, and all three are cheap to diagnose only because there is no layer between the traceback and the wire.

So when is a framework right?

Not never. It is right at the point where you are paying for something you have already built badly.

	Raw HTTP	Provider SDK	Agent framework
You own	the request body	the retry and typing layer	almost nothing
You see when it breaks	status code and body	a typed exception	a stack you have not read
Costs you	boilerplate per feature	portability across providers	the shape of your program
Right when	learning, or the loop is the product	one provider, production traffic	you need durability, branching and resumption, and know why

The third column becomes correct later in this series, when the agent has to survive a crash halfway through a scan and resume where it stopped. Writing that yourself is a bad use of a month. But arriving there having never seen the request means you cannot debug it, and you will need to.

It works, and I cannot tell if it is right

Running it prints a fluent paragraph about agent.py. It reads well. It is also completely unverifiable: there is no field to assert on, no count to compare, nothing a test can hold. Prose is unfalsifiable by construction, and a mapping tool whose output cannot be checked is a tool nobody should trust with a codebase they do not know.

That gap is the honest state of the project tonight, and it is a more interesting problem than any missing framework. A framework would have produced the same unverifiable paragraph, faster, from behind a wall.

What I'd do

Write the POST by hand once, before installing anything. Half an hour, and it never stops paying.
Put raise_for_status() in from the start. It is the difference between a diagnosis and a hunt.
Read the key with a subscript, not .get(). Fail at the boundary.
Point the first version at a file you already understand, so bad output is obvious rather than plausible.
Adopt the framework the day you can name the feature you need from it — resumption, branching, durability — and not one day earlier.

If this was useful, a clap helps other people find it. I want to hear the other side too: what was the first thing that made you reach for an agent framework, and was it something you had already tried to build yourself?

All views in this post are my own. The scenarios are illustrative composites, and code examples are not drawn from any specific codebase.

Day 00a: Six ideas that explain every AI agent, before any code

Harshavardhan Katkam — Wed, 14 Oct 2026 13:12:03 +0000

Somewhere in the first hour of reading about AI agents, you meet a diagram. A box labelled Thought, an arrow to Action, an arrow to Observation, and an arrow curling back to the start. The captions say the agent reasons, plans and decides. It makes building one sound like work for a research lab.

Then someone asks a plain question: when the agent "thinks", what does the model actually receive? The honest answer is a list of messages, written as JSON, sent in an ordinary web request. The reply is more JSON. Nothing in between remembers anything.

That gap, between the vocabulary and the mechanism, is what this primer closes: six ideas, explained from zero, each with something you can see for yourself.

The Short Version

A model call is text in, text out, over HTTP. It predicts likely text, which is why it can be fluently wrong.
Models read and are priced in tokens, chunks often smaller than a word.
The context window is the most one call can carry. Anything outside it does not exist for the model.
The model remembers nothing between calls. Your code resends the conversation.
Tool calling does not let the model act. It lets the model ask your code to act.
An agent is a loop around a model call: ask, run any requested tool, send the result back, repeat until it answers. Your code owns the limits.

Before you start

What you will learn. The six ideas, tried for real where possible, and the vocabulary for the next thirty days.

Prerequisites. None to read it. You have used a chat assistant, and you can read a little Python. The optional "Try it yourself" boxes need a macOS or Linux terminal (or WSL on Windows) with curl and python3, and a free OpenRouter account: one web address that forwards requests to hundreds of models, some of them free.

Time. About twenty-five minutes to read, fifteen more for the experiments.

How to read it. Each idea has the same parts: plain words, an everyday picture, what goes over the wire, the common mistake, why it matters. At "Predict, then check", decide before opening the reveal.

Get a free OpenRouter key

The experiments need an API key, a secret string that tells OpenRouter which account is asking. Getting one takes two minutes and no payment.

Go to openrouter.ai and sign up, with an email address or one of the sign-in buttons it offers.
Open openrouter.ai/keys and click Create Key.
Name it something you will recognise, such as cartographer. Leave the credit limit empty; :free models cost nothing.
Copy the key at once. It is shown only one time.
Free models need no payment, but they are rate-limited: a busy one may ask you to wait.
Never paste the key into a file, a chat or a screenshot. Load it straight into your terminal instead:

read -rs "?OpenRouter key: " k && export OPENROUTER_API_KEY="$k" && unset k

Paste the key when asked; nothing shows as you paste. That is zsh, the macOS default; in bash, start with read -rsp "OpenRouter key: " k. The key now sits in an environment variable, a named value your terminal hands to the commands it runs, never printed and never saved. It lasts until you close that terminal, and Day 01 loads it the same way.

If the key ever leaks, delete it on the Keys page and create a new one. The old one stops working the moment you delete it.

The picture the word suggests

The word "agent" suggests an assistant that remembers what you told it, looks at your files, and goes off and does things. Each part of that picture is wrong in a way that decides how you build one. It does not remember: your code resends everything. It cannot see your files, only the text you send. It cannot act: it can only ask, and your code acts. The six ideas below replace the picture one piece at a time.

Idea 1: A model call is text in, text out

In plain words. Your code sends an HTTP request to a provider, the service that runs the model, holding a model name and a list of messages. The useful part of the response is one field: the generated text. The model does not browse, run code or open files. It returns the text it predicts should come next, one small piece at a time.

An everyday picture. The word suggestions above your phone keyboard: the same job, guessing what comes next, at an enormously larger scale.

On the wire. A complete request. Each message has a role saying who wrote it: user for you, assistant for the model, system for standing instructions at the top.

{
  "model": "nvidia/nemotron-3-super-120b-a12b:free",
  "messages": [
    {"role": "user", "content": "Name one Python web framework, in one word."}
  ]
}

The response, trimmed to the fields that matter. The shape is real; your wording and numbers will differ.

{
  "choices": [
    {
      "message": {"role": "assistant", "content": "Flask"},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": <number>,
    "completion_tokens": <number>,
    "total_tokens": <number>
  }
}

Three fields carry everything. choices[0].message.content is the answer: the first entry of choices, its message, its content. finish_reason says why it stopped: stop means finished, length means cut off at a size limit. usage counts tokens, Idea 2.

Try it yourself. One real model call. Paste it into the terminal where you loaded your key.

curl -s https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "nvidia/nemotron-3-super-120b-a12b:free", "messages": [{"role": "user", "content": "Name one Python web framework, in one word."}]}' \
  | python3 -m json.tool

Part	What it does
`curl -s <url>`	Sends an HTTP request to the endpoint, quietly
`-H "Authorization: Bearer ..."`	A header carrying your key, read from the environment variable
`-H "Content-Type: application/json"`	Says the body is JSON
`-d '{...}'`	The body: the model name and the messages
`python3 -m json.tool`	Prints the reply neatly; it changes nothing

Predict, then check. How much of what comes back will be the answer itself?

Check your prediction
One word, inside a page of bookkeeping: an id, the model name, the finish reason, the token counts.

What you should see. After a few seconds, JSON with a framework name in content, finish_reason set to stop, and three counts under usage. A reasoning field, if present, is not the answer. If you see an error instead:

You see	It means
`401`	The key is missing or wrong in this terminal; run the `read` line again
`429`	The free model is busy; wait a little and retry
The model id is unknown or unavailable	Free models come and go; swap in any id ending in `:free` from openrouter.ai/models

Common mistake. Treating a fluent answer as a checked one. The model produces what is likely, not what is verified, so when it does not know, it still produces something likely. A confident, plausible, wrong answer is hallucination. No prompt removes it entirely. Grounding, making the model answer from text you gave it, reduces it.

Why it matters for agents. Everything an agent does is built from this call, and grounding is why cartographer, the agent this series builds, only cites code it has read off disk.

Idea 2: Tokens are the unit everything is measured in

In plain words. Models do not read characters or words. Text is split into tokens: often a whole short word, sometimes a fragment of a long one, sometimes one punctuation mark. The model reads and writes tokens, and paid providers charge per token both ways. The splitter is the model's tokenizer, and different models split the same text differently.

An everyday picture. The data on a phone plan: what you pay for, and what runs out. Tokens are both the price and the limit.

On the wire. You send text; the provider reports the cost in usage: prompt_tokens for what you sent, completion_tokens for what the model wrote, total_tokens for the sum.

A worked example by hand. A tokenizer might keep "the" whole and cut "unbelievably" into pieces like "un", "believ" and "ably"; where it cuts depends on the model. So eight words need not be eight tokens, and code, with its brackets and indentation, usually costs more than its word count suggests.

Predict, then check. Your question in the first experiment had eight words. Will prompt_tokens say 8?

Check your prediction
Very likely not. Words and tokens do not line up one to one, and the provider also counts the formatting that wraps your message. Run the experiment again and read the real number.

To watch a tokenizer cut text live, try Tiktokenizer; its counts are for particular models, so they illustrate, not predict.

Common mistake. Estimating cost or size in words. Providers bill and limit in tokens, and only usage tells you the true count.

Why it matters for agents. An agent makes many calls, each billed and limited in tokens. Day 03 measures a real repository instead of guessing.

Idea 3: The context window is one page

In plain words. Every model has a context window: the most tokens one request can hold, your messages and the reply together. It is not memory and not storage. It is the size of the one page the model sees, and anything off that page does not exist for that call. Sizes differ by model, so check yours.

An everyday picture. An exam where you bring one sheet of notes and write your answer on the same sheet.

A worked example by hand. Toy numbers, chosen for easy arithmetic, not taken from any real model. Pretend the window is 100 tokens.

What is on the page	Tokens	Running total
Instructions (the system message)	15	15
One source file	60	75
Your question	10	85
Room left for the reply	15	100

Add a second 60-token file and the input alone is 145: it does not fit, and the request fails with an error that no rewording fixes. Keep one file but ask for a long explanation, and the reply runs out of room after 15 tokens, stopping mid-sentence with finish_reason set to length.

Common mistake. Thinking a better prompt fixes a full window. The problem is arithmetic, so the fix is sending less: a slice instead of the whole file, a summary, only the files that matter. Chat apps may quietly drop your oldest messages to make room, which is why a very long chat seems to forget its start.

Why it matters for agents. A real codebase is far bigger than any page, so an agent that reads code must choose what goes on it. That choice is most of what this series builds.

Idea 4: The model remembers nothing

In plain words. Send two requests in a row and the second knows nothing about the first; the endpoint keeps no conversation for you. A chat feels continuous because your code keeps the conversation as a list and resends all of it, plus the new message, every time. The model is not remembering. It is rereading. This is what stateless means.

An everyday picture. A help line where a different person answers every call and nobody keeps notes. To continue, you read the whole conversation out again, every time.

Predict, then check. Three separate requests: the first tells the model a name, the second asks for it back, the third sends the whole conversation in one list. What will the second answer?

Try it yourself: prove it forgets. Three requests, each printing only the answer.

curl -s https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "nvidia/nemotron-3-super-120b-a12b:free", "messages": [{"role": "user", "content": "My name is Asha. Reply only with OK."}]}' \
  | python3 -c "import sys, json; print(json.load(sys.stdin)['choices'][0]['message']['content'])"

curl -s https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "nvidia/nemotron-3-super-120b-a12b:free", "messages": [{"role": "user", "content": "What is my name?"}]}' \
  | python3 -c "import sys, json; print(json.load(sys.stdin)['choices'][0]['message']['content'])"

curl -s https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "nvidia/nemotron-3-super-120b-a12b:free", "messages": [{"role": "user", "content": "My name is Asha. Reply only with OK."}, {"role": "assistant", "content": "OK"}, {"role": "user", "content": "What is my name?"}]}' \
  | python3 -c "import sys, json; print(json.load(sys.stdin)['choices'][0]['message']['content'])"

What you should see
The first replies OK. The second says some version of "I don't know your name". The third says Asha. Wording varies; the pattern does not. Nothing changed between the second and third except what you sent. If the second names Asha, it guessed: rerun it. If a command prints a KeyError, the request failed: swap the last line for python3 -m json.tool to see the error.

A worked example by hand. Toy numbers again: every question is 10 tokens, every answer 20.

Turn	What your code sends	Input tokens
1	Q1	10
2	Q1, A1, Q2	40
3	Q1, A1, Q2, A2, Q3	70

You typed 30 tokens of questions but sent 120 tokens of input, and the gap widens every turn.

Common mistake. "But my chat app remembers me." The app does, not the model: it saves notes and puts them back into the request. Some providers offer APIs that store a conversation for you, but the model still receives all of it on every call.

Why it matters for agents. An agent's working memory is a list your code owns: what goes back in, what gets cut, what it costs. Day 02 builds it.

Idea 5: Tool calling lets the model ask, not act

In plain words. If the model only produces text, how does an agent read a file? In the request, you describe functions it may ask for, each a tool with a name, a description and its arguments (its tool schema). The model can then reply with a tool call, a structured request to run one, instead of an answer. Your code runs it and sends the result back as a new message. The model asked; your program acted.

An everyday picture. A doctor on the phone says "take your temperature and tell me the number". You hold the thermometer and report back. The doctor only asked.

On the wire. The request gains a tools list. One entry, in the format Day 06 uses:

{
  "type": "function",
  "function": {
    "name": "read_file",
    "description": "Read one file from the repository and return its text.",
    "parameters": {
      "type": "object",
      "properties": {
        "path": {
          "type": "string",
          "description": "Path relative to the repository root."
        }
      },
      "required": ["path"]
    }
  }
}

When the model wants it, content is null, finish_reason is tool_calls, and the message carries the request:

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {
      "id": "call_1",
      "type": "function",
      "function": {
        "name": "read_file",
        "arguments": "{\"path\": \"src/app.py\"}"
      }
    }
  ]
}

Your code appends that message, runs read_file, and appends the result with the role tool, tied to the request by its id:

{"role": "tool", "tool_call_id": "call_1", "content": "<the text of src/app.py>"}

Then it sends everything again. The id here is made up.

Predict, then check. Look at arguments. Is it a JSON object?

Check your prediction
No. It is a string containing JSON, written by the model like any other text. Your code must parse it and expect it to be malformed, name a missing file, or ask for a tool you never described.

Common mistake. Saying the model "ran" a tool. It can only request what you described; nothing runs unless your code runs it. One reply can request several tools, and each gets its own result message.

Why it matters for agents. This is where the control sits: the tools you describe are all the agent can do, and checking each request is your job. Day 06 makes this round trip by hand.

Idea 6: An agent is a loop

In plain words. Put the last two ideas together and you have an agent:

Send the conversation and the list of available tools.
If the reply is an answer, stop.
If the reply is a tool request, run it, append the result to the conversation, and go back to step 1.

That is the whole of it. The model chooses the next step. Your code decides the rest: which tools exist, how many times the loop may run, what happens at that limit. Each pass is a turn.

An everyday picture. The phone doctor asks for your temperature, then about your throat, then diagnoses. You decide how many tests you will do.

On the wire: one run, traced by hand. An agent with list_files and read_file tools is asked for a project's entry point.

Turn	Your code sends	The model replies	Your code then
1	system, your task, the tools	tool call: `list_files`	runs it, appends the file list
2	all of the above, plus that result	tool call: `read_file` on one file	runs it, appends the file's text
3	all of the above, plus that result	an answer, `finish_reason` `stop`	stops and prints it

Every turn sends everything before it: Idea 4, inside the loop.

Predict, then check. What happens if the model never replies with an answer, and keeps asking for one more file?

Check your prediction
Nothing in the model stops it. The loop runs, spending tokens, until your code stops it, which is why every real loop has a ceiling, such as a maximum number of turns.

Common mistake. Reading "the agent decided" as something beyond this loop. The model returned a tool request and the loop ran it. Thought, Action, Observation is this loop drawn with bigger words.

Why it matters for agents. A loop the model can keep alive needs a limit you control. Day 06 builds the loop with two such limits.

What an agent is not

Not a chatbot. A chat window is an interface. Some chat apps run a tool loop behind it, and the loop is what makes the agent.
Not a workflow. A workflow is a fixed sequence your code decides in advance, and it is often the better choice.
Not a smarter model. It is the same model, with a loop and tools around it.

The question that sorts them is who decides the next step:

	Who decides the next step	Good for	Breaks when
Chat model	You, every turn	A single question	The task needs many steps
Workflow	Your code, fixed in advance	Predictable, repeatable jobs	The path depends on what it finds
Agent	The model, inside a loop you bound	The next step depends on the last result	You cannot afford it to explore

Reading an unfamiliar codebase is the third kind: you cannot know the second file to open until you have read the first.

Words you will meet in the next thirty days

Every later post defines its terms again, in this wording.

Term	What it means	Built on
Model	The program that predicts text; you reach it over the network	Day 01
Provider	The service that runs models and answers your requests	Day 01
Endpoint	The web address a request is sent to	Day 01
API key	A secret string that identifies your account; never write it in a file	Day 01
Message	One entry in the conversation: a role plus its text	Day 01
Role	Who wrote a message: `system`, `user`, `assistant` or `tool`	Day 01
Prompt	The text you send for the model to respond to	Day 01
System prompt	Standing instructions in the `system` message at the top	Day 01
Hallucination	A fluent answer that is not backed by anything the model was given	Day 01
Grounding	Making the model answer from text you supplied	Day 01
Transcript	The conversation list your code keeps and resends	Day 02
Stateless	Keeps nothing between calls	Day 02
Token	The chunk of text a model reads, writes and is billed in	Day 03
Context window	The most tokens one request and its reply can hold	Day 03
Structured output	Making the model reply in a fixed JSON shape you can check	Day 04
JSON Schema	A standard way to describe the shape a piece of JSON must have	Day 04
Temperature	A setting that controls how much randomness goes into picking each token	Day 05
Tool	A function you describe to the model by name, description and arguments	Day 06
Tool call	The model's structured request to run one of your tools	Day 06
Turn	One pass of the agent loop: send, reply, act	Day 06
MCP	Model Context Protocol, a standard way to offer tools to many agents	Day 09
Graph	An agent written as named steps and the edges between them	Day 15
State	Everything the agent has accumulated so far, in one place	Day 16
Checkpoint	A saved copy of state, so a run can resume after a crash	Day 19
Human in the loop	Pausing the agent for a person to approve a step	Day 20
Memory	What is kept beyond one conversation, and how it is put back in	Day 22
Retrieval	Fetching only the relevant text, so it fits on the page	Day 23
Eval	A repeatable test of the agent's answers	Day 25
Trace	A record of every call and tool run in one agent run	Day 26
Guardrail	A limit on what the agent is allowed to do	Day 29

And a translation for what you will read elsewhere:

When someone says	It means
"The agent thought about it"	The model generated text in reply to the conversation
"The agent decided to read a file"	The model returned a tool call, and the loop ran it
"The agent remembers"	Your code resent earlier messages, or put saved notes back into the request
"The agent forgot"	That text was not in the request it was sent
"The agent used a tool"	Your code ran a function the model asked for
"The agent observed the result"	Your code appended the tool's output as a message
"The context is full"	The request plus the reply would exceed the window
"It hallucinated"	It produced likely text that nothing it was given supports
"Thought, Action, Observation"	Reply, tool call, tool result: the same loop

Where each idea comes back

Idea	Where you build it
1. Text in, text out	Day 01, the raw call; Day 04, making the text checkable; Day 05, how each next piece is picked
2. Tokens	Day 03, measuring a real repository and a cost meter; Day 27, cost control
3. The context window	Day 03, the hard wall; Day 23, retrieval as a tool
4. The model remembers nothing	Day 02, the transcript; Day 22, memory
5. Tool calling	Day 06, by hand; Day 08, tool descriptions; Days 09 to 13, MCP
6. The agent loop	Day 06, the loop and its limits; Days 15 to 21, the loop as a graph

Check yourself

Answer each before opening it. If one will not come, reread that idea.

1. A model confidently describes a function that does not exist. Why, and what reduces it?

Answer
It produces likely text, not checked text (Idea 1). Give it the real source to answer from.

2. Your question is twelve words. Will prompt_tokens be 12?

Answer
Almost certainly not. Tokens are not words, and formatting counts too (Idea 2).

3. A toy window holds 100 tokens. Instructions 15, a file 70, a question 10. How much room is left for the reply, and what if it needs more?

Answer
5 tokens. The reply is cut off with finish_reason length (Idea 3). Send less.

4. Turn two of a chat clearly "remembers" turn one. Where does that memory live?

Answer
In your program's list, resent with every request (Idea 4).

5. Questions are 10 tokens, answers 20. How many input tokens does turn three send?

Answer
70: two questions, two answers, the new question (Idea 4).

6. An agent "read a file". Who opened it?

Answer
Your code, after the model returned a tool call (Idea 5).

7. What is in a tool call's arguments field, and why check it?

Answer
A string of JSON the model wrote; it can be malformed or name things that do not exist (Idea 5).

8. What stops an agent loop running forever, and who owns it?

Answer
A limit in your code, such as a maximum number of turns (Idea 6).

9. What one question separates a chatbot, a workflow and an agent?

Answer
Who decides the next step.

Recap

"The agent decided" means the model returned a tool request and the loop ran it.
When something "forgets", look at the list it was sent.
When something is too big, it is a context-window problem; no prompt fixes it.
When an answer is confidently wrong, ask what text it was given.
Trust the usage block, not an estimate.
Your code owns the ceiling on every loop.

What's next

Day 01 writes the first agent: the request from Idea 1, sent from about forty lines of Python, reading the same content field. Today's dev notes list what Day 01 needs on your machine.

Day 00: Learn AI agents by building one, in 30 days

Harshavardhan Katkam — Wed, 14 Oct 2026 13:03:46 +0000

Someone hands you an empty file and says: build an agent. Perhaps you have read the explainers and can recite that an agent is "an LLM in a loop with tools". Perhaps you have read none of them, and the word only means software that does things on its own. Either way the cursor blinks, and you do not know what the first line should be.

Most people in that spot open a framework's quickstart. Three lines later something impressive happens, and the interesting part — the loop, the decision about what to send, the thing that goes wrong at scale — happened inside a function you never opened. That is not your fault. Almost everything written about agents demonstrates a library, so what it teaches is vocabulary rather than mechanism.

This is a thirty-day path to closing that gap. One concept a day, each one built by hand before any framework touches it, all of it assembling into a single real tool by the end. You will write the agent loop before you use a framework that hides it, and by the time you do adopt one you will be able to say what it is doing for you.

The Short Version

Thirty days, one concept each, from a bare model call to a production-shaped agent with evaluation and cost control.
Everything is built by hand first. Week one uses no SDK and no framework — forty lines of Python over raw HTTP.
It all assembles into one real tool: cartographer, an agent that reads a codebase you have never seen and tells you where to start.
Every day ships build instructions you can follow, and the code is public: github.com/Harshavardhan17/cartographer.
Every model call runs on a free tier. Following along costs nothing.
Day 25 scores the finished agent against repositories with known-good answers. If it turns out to be bad at this, that is the post.

Why build one thing instead of thirty demos

Most tutorials teach agent concepts with a fresh toy per concept: a weather tool for tool calling, a PDF chatbot for retrieval. Each one works, and none of them teach you much, because a toy cannot push back. You never hit the wall that makes the concept necessary — so you learn the API and not the reason.

So this series builds a single tool across all thirty days, chosen because it fights you.

cartographer points at a repository you have never seen and returns an onboarding map: where execution actually begins, which small fraction of the files carry the system, one data path traced end to end, and where your first change would go. Everyone who has joined a team knows the week this replaces.

It is a good teacher because of one property: a repository does not fit. A mid-sized open-source project runs to several million tokens, the chunks of text a model reads and is billed by. There is no context window, the most one request can hold, large enough, no prompt clever enough, and no bigger model next year that makes it go away. The agent must decide what to read.

That single constraint turns every concept in the curriculum from a demonstration into a necessity. Tool calling becomes the only way in. Retrieval stops being a stage you were told to add. Cost control becomes arithmetic. You meet each idea at the moment you need it, which is the only way any of it sticks.

It has a second useful property: there is a right answer. Point it at a codebase you know well and you can check the map, which is what makes a real evaluation possible on Day 25 instead of a vibe-based "seems to work".

What you will build, week by week

Week one: the model, with nothing in front of it. The agent loop in about forty lines of Python over raw HTTP, with no SDK and no agent library. You will handle the field that separates a finished answer from a truncated one, build conversation memory by hand and see why "memory" is a misleading word, do the token arithmetic that governs every design decision later, make the model return typed records instead of prose, and hand-roll tool calling so you have seen exactly what crosses the wire. By Friday you have a working agent and a clear list of everything wrong with it.

Week two: MCP. The protocol that lets any editor or client talk to your agent instead of it living inside one script. This is a good moment to learn it, because MCP removed its initialization handshake and protocol-level sessions in the 2026-07-28 revision — which means a large share of what is currently written about how MCP works describes a protocol that no longer exists. You will learn the version that is true today, build a server worth keeping, and handle the security question that arrives the moment an agent reads code it did not write.

Week three: LangGraph, and why a loop is not enough. Reading a codebase is not a pipeline. You read a file, fail to understand it, and go back for more context — that is a cycle, and a cycle is the thing a chain cannot express. You will also add the two features that turn a script into software: surviving a crash four thousand files into a scan, and stopping to ask a human before doing something consequential.

Week four: the half that decides whether it is real. Memory as three separate problems rather than one word. Retrieval as a tool the agent chooses to call. An evaluation set with known-good answers, so "did it get worse?" has an answer. Tracing, cost routing, and the seven ways this breaks in production: the part most tutorials skip.

How to follow along

Each day has two pieces: a post explaining the concept and why it matters, and dev notes that say which file to create in which folder, what to type into it, and what every line does.

Today's build has no agent code. It creates the repository, and every file in it has a job. pyproject.toml names the project, its Python version and its one runtime dependency. src/cartographer/__init__.py makes the folder an importable package and holds the version. .gitignore keeps the environment, caches and any secrets out of history. LICENSE makes the code reusable. README.md says what the tool will do and admits nothing works yet. .github/workflows/ci.yml lints, type-checks and tests every push, and tests/test_package.py is one real test. It also clones click 8.5.0, the repository the agent studies all series, pinned so every day measures the same code.

You need Python 3.12, a terminal, and about an hour a day. Every model call runs against free models through one OpenAI-compatible endpoint, so there is no bill.

You do not need prior agent experience. If agents are new to you, read the primer first, "Six ideas that explain every AI agent, before any code": about fifteen minutes, no code, covering what a model call is, tokens, the context window, why the model remembers nothing, tool calling, and the loop that makes it an agent. If they are familiar, start at Day 01. You do need to be comfortable reading Python.

Two rules for the writing

Nothing gets described that has not been built. The code ships before the post does. When something does not work, that is the post, and there will be several, because thirty days of building something hard is not thirty days of wins.

No invented numbers. Any claim about accuracy comes from the Day 25 evaluation, against repositories with known-good answers, with the methodology published alongside it. Where a figure is illustrative, the sentence will say so.

Tomorrow: the forty-line agent, and why your first one should not be a framework.

If you have lost a week to an unfamiliar codebase — and most of us have — which part actually cost you the time: finding the entry point, or working out which files mattered? The answer changes what gets built first, so I would genuinely like to know.

All views in this post are my own. The scenarios are illustrative composites, and code examples are not drawn from any specific codebase.

The Senior Developer Mindset: How to Turn Abstract Ideas Into Production Code

Harshavardhan Katkam — Thu, 04 Jun 2026 08:04:16 +0000

The single skill in my career that has compounded the most isn't a language, a framework, or a system design pattern. It's the ability to take a vague, half-formed idea — a feature request from a PM, a "we should make this resilient" from an architect, a new project ask from an account, a half-formed integration plan that crosses three teams — and turn it into something shipped, working, and useful.

This is the skill nobody teaches in a course. It's also one of the most rewarding to develop, because it's what allows you to grow into the engineer your team can hand ambiguous problems to with confidence. Here's the mindset, the framework, and the lessons that helped me grow toward it.

The Skill That Doesn't Have a Name

Every engineer learns to write code. Most learn to debug, design, and ship. Few practice the skill that comes before all of that — the skill of taking ambiguity and giving it shape.

You've felt this if you've ever been on the receiving end of:

"We need the dashboard to feel faster."
"We should split this service — it's getting unwieldy."
"Can we deliver a POC for the client conversation by Friday?"
"Let's partner with team X to ship this end-to-end."
"Can we set up a new module for this account?"
"The system should be more robust."
"There has to be a better way to do this."

Some of these are features. Some are architectural decisions. Some are new projects within an account. Some are cross-team integrations. None of them is a specification. The gap between any of them and a working outcome is where most real engineering effort goes — and the gap is the same shape whether the work is one PR or six months across three teams.

What follows is the framework I've built from doing this work. I'm sharing it in case it saves you a few of the wrong turns I took.

Where Abstract Ideas Come From

Not all abstract ideas are equal. Each source has its own failure mode and its own opportunity.

Source 1 — Product / Business. PMs, BAs, and clients describe outcomes ("users should be able to..."). They rarely describe systems.

Trap: building exactly what they asked for, only to discover what they meant was different. Opportunity: catching the misalignment before any work begins.

Source 2 — Technical Leadership. Architects and tech leads describe direction ("we should make this more resilient," "we need to migrate off X"). Vision, but rarely a roadmap.

Trap: treating the suggestion as either a command (over-investing) or a passing thought (ignoring it). Opportunity: turning the suggestion into a concrete proposal that leadership can green-light or refine.

Source 3 — Your Own Observations. The bug keeps coming back. The deployment keeps breaking. The on-call rotation keeps paging at 2 AM.

Trap: complaining without doing, or doing something massive without communicating. Opportunity: building the smallest fix or proof-of-concept that demonstrates value, then asking forgiveness rather than permission.

Source 4 — Improvisations That Become POCs. Some of the best work I've shipped started as a quick script, an internal tool, an idea jotted during a meeting.

Trap: treating it as throwaway and never investing in making it solid. Opportunity: recognizing when an improvisation is actually a POC for something bigger.

The framework that follows works for any of these — feature, architectural, project-level, or cross-team.

Before the Framework: Understand the Terrain

Before you can translate an abstract idea into anything real, you need to understand the system the work will live inside. Whether the "system" is a codebase, an architecture, an account, or a set of stakeholders — the principle is the same.

It's tempting to jump from "great idea" to "let's start" without pausing to ask: what's already here, why is it set up this way, and what will my work actually touch? I've fallen into this many times, and the result is fairly predictable. A clean new piece of code that conflicts with an existing pattern. An architectural change that breaks a downstream system nobody mentioned. A new project that quietly steps on another team's roadmap.

Slow down at the start. Before any meaningful translation work, ask:

What does the existing system look like? Read the code. Trace the data flow. Read the contracts between services. Talk to the teams adjacent to your work.
Why was it built this way? Git history, old PRs, design docs, informal Slack threads. The current state reflects past decisions you don't see in the present.
What's the messy part nobody wants to touch? Every system has zones of accumulated complexity. Knowing where they are tells you where the risks live.
Who knows this best? Find that person. Ten minutes with them saves a week of trial and error.

This stage is unglamorous. It can feel like you're not making progress. But the time you invest in understanding the terrain compounds — every later decision becomes faster and less risky.

Sometimes You Have to Take Ownership of the Mess

Sometimes the existing structure isn't just unfamiliar — it's tangled, undocumented, and getting in the way. Code paths nobody fully understands. Configuration buried across services. A "we'll clean it up later" that's been there for two years.

It's tempting to route around it. The more rewarding move is often to engage with it. That can look like:

Documenting what you've understood, even if no one asked you to
Refactoring the parts you've had to touch, leaving them better than you found them
Naming the problem out loud — in a writeup, a Slack thread, a one-on-one — so it's visible instead of invisible
Proposing a small, low-risk cleanup as a side-quest to the main work

Not every mess needs cleaning, and not every cleanup is yours to drive. But occasionally picking up the mop, when the cost is fair and the impact is real, tends to be one of the most rewarding kinds of work — and quietly earns trust over time.

The Framework: Five Stages

Translation work has five stages. They're not always linear — sometimes you loop back — but they're always present. The discipline is not skipping any of them.

Stage 1: Capture

Write down the idea verbatim. Not your interpretation. The exact words.

If a PM said it, copy from Slack. If an architect said it, summarize back and confirm. If it's your own observation, note the symptom that triggered it. Your brain immediately starts interpreting — capturing the original wording gives you something to come back to when you realize three days in that you've drifted.

Every translation effort starts with one written line that nobody can later argue about. That line is your anchor.

Stage 2: Clarify

Find the real problem, not the stated one.

The stated request is rarely the real problem. "The dashboard should feel faster" might mean: data takes too long, layout shifts on render, no loading indicator, comparison to a competitor, slow connection. Five interpretations, five different solutions ranging from one day to one quarter.

The habit worth building: ask "what does success look like?" and don't start work until you can describe success in measurable terms. Five questions I always ask:

Who experiences this? (Specific persona, not "the user")
What are they trying to accomplish?
How will we know it's better? (Measurable, even approximately)
What's the cost of not doing this? (If "nothing concrete," the priority is probably wrong)
What would the absolute simplest version look like?

If I can't answer these five, I don't write code, draw architecture, or kick off a project. I go back to whoever raised the idea and have a conversation. This is rarely glamorous and often slightly awkward. It's also the single most valuable habit I've built.

Break It Down to Layman Terms — The Feynman Move

A habit I've made non-negotiable: before I start anything, I make sure I can explain the problem in the simplest possible language, with an analogy a non-engineer could follow.

This is the Feynman technique applied to engineering. If you can't explain something simply, you don't really understand it.

In practice, my first move on any meaningful task is to ask either a peer or an AI tool: "Explain this to me like I'm new to the system. Use an analogy if it helps."

If the ask is "improve the consistency of our distributed cache," a layman framing might be: "Imagine three people each have a copy of a shared notebook. When one writes something, how do the others find out, and how fast?" That sentence — minus the engineering vocabulary — anchors my thinking for everything that follows.

I sometimes write the layman version at the top of a doc before any technical detail. It serves as a north star — every time the work feels tangled, I come back to that one sentence and check whether I'm still solving the problem I started with.

The senior engineers I admire most can describe extremely complex problems in remarkably simple terms. That's not a coincidence. It's a habit, built deliberately.

Stage 3: Constrain

Decide what not to build.

This is the stage that's easiest to skip — and the one I've come to value most. Once you're convinced something is worth doing, the next question is: given my time, my team's bandwidth, and the actual cost of getting this wrong — what's the smallest meaningful version?

Constraints to think about:

Time: A week, a quarter, a spare hour?
Scope: What's in, what's out, what's a "later"?
Risk: What could go wrong, and what's the worst case?
Reversibility: One-way door (hard to undo) or two-way door (easy to undo)?
Audience: Who uses it, who maintains it, who approves it?

A useful tool: the 3-day version. Force yourself to design what you'd ship if you had three days, no more. Even if you eventually have three weeks, the 3-day version teaches you what's truly load-bearing and what's optional. Often, the 3-day version is what should ship first anyway.

This stage prevents most "scope creep" — not by saying "no" to features after they're requested, but by deciding upfront, in writing, what's not going to be built.

A Lesson I Wish I'd Learned Earlier: Timebox, or Start Fresh

For a long time, I had a habit I didn't recognize. When faced with an ambiguous problem, I'd dive into analysis. Open documentation, sketch designs, weigh trade-offs, talk to people, draw more diagrams. Hours would pass. The work felt productive — I was thinking deeply, after all.

A tech lead I worked with eventually pointed it out: I was in analysis paralysis. I was so focused on getting the design right before starting that I was burning hours on decisions that would have taken five minutes to validate with a quick prototype.

The advice he gave me changed how I work: timebox the analysis, and if you're not converging, start fresh.

Practically:

Give yourself a fixed amount of time to clarify and constrain — usually no more than half a day for typical work
If you're still spinning at the end of that window, stop
Either commit to the best option you have and start building, or reset entirely with a blank page and 30 minutes — often the second pass is faster and clearer than the first

Starting fresh feels like wasted work, but it's almost always faster than continuing to refine a tangled first pass. Your second attempt benefits from everything you learned in the first one, but isn't burdened by the dead-ends you got stuck on.

Stage 4: Construct

Build the thing.

By this point, you usually know what you're building, why, and what you're not building. The actual construction is faster because the ambiguity has been resolved upfront. This is often the stage we think is the job — but it's actually the stage where the upstream work pays off.

A few principles I've internalized:

Decompose into demoable units. If you can't show progress in 2–3 day increments, the unit is too big.
Sketch before building. A diagram, a flowchart, a written API contract — anything that lets you reason about the design before code, architecture, or commitments lock you in. Five minutes of sketching saves five hours of rework.
Use AI as a force multiplier, not a replacement. AI tools accelerate boilerplate, surface edge cases, draft tests, review for blind spots. They do not make the design decisions or skip the clarify stage. The judgment is yours.
Lean on your peers. Quick gut-check messages — "does this approach make sense?" — to a senior peer or architect cost five minutes of their time and can save you days. The habit worth building is asking early and specifically.

Stage 5: Confirm

Close the loop with whoever raised the idea.

Once it's working — even partially — show it back. Not a deck. Not a status update. The actual thing.

This stage catches the misalignments that survived all earlier stages. You'll find that what you built is 95% right and 5% wrong, and the 5% only became visible when they saw it. Better to find it now than after the next sprint.

Sometimes this stage reveals that the original idea was wrong, and what they actually needed was different. That's the whole point of staging it iteratively. The faster you close the loop, the less wasted work.

What Makes the Framework Actually Work

Three force multipliers worth investing in:

1. Dedicated, focused time. Translation work cannot happen in 15-minute slots between meetings. The clarify and constrain stages especially require uninterrupted thinking. I block calendar time explicitly — not for "writing code," but for "translating idea X into a plan." When I started doing this deliberately, my output quality changed more than from any tool or technology I adopted.

2. AI as a thinking partner. AI tools, used well, accelerate the framework. I lean on them for: generating "what could go wrong" lists (Stage 2), asking "what's the simplest version?" (Stage 3), boilerplate and test scaffolding (Stage 4), drafting demos and writeups (Stage 5). I do not lean on them for the judgment calls — whether to build, whether the simpler version is enough, whether the design will hold up. The AI removes friction around your thinking. It doesn't replace it.

3. Mentorship — both ways. Earlier in my career, having a senior to validate or redirect my translation work saved me from countless wrong turns. Now, increasingly, I'm the one being asked. Both directions of this exchange compound the skill. Senior engineers don't translate ambiguity better because they're smarter — they've just seen the same shapes of ambiguity hundreds of times. Mentorship is compressed exposure.

Improvisations Compound

Some of the highest-leverage work I've shipped started as something improvised — a small script, an internal tool, an idea jotted in a meeting. The pattern looks like this:

You notice a recurring annoyance — manual process, flaky test, something that keeps breaking
You spend an hour or two building a quick fix
The fix works for your case
A teammate asks if it could work for theirs
You generalize it — another half-day
By the end of the month, three people are using it
By the end of the quarter, it's the recommended approach

None of this was on a roadmap. None of it was in a sprint plan. But it shipped because you noticed something, took it seriously, and applied the same translation framework you'd apply to any ask.

This is, in my experience, where engineers create the most outsized value — not in the work that's handed to them, but in the work they spot and pursue. The framework still applies, just with you as both originator and implementer. The discipline keeps you from two equally common pitfalls: ignoring your ideas entirely, or over-investing in them without validation.

Three Failure Modes I Watch in Myself

1. Skipping clarify because the request "seems clear." It rarely is. The clearer it seems, the more likely you've assumed something the asker didn't say.

2. Building first, asking later. A common reflex, and one I've fallen into many times. Once you've built, you're emotionally invested in the solution, and it gets harder to walk away from it even when it's the wrong shape.

3. Using AI to skip thinking. AI can generate output that looks right but isn't the right thing. The framework — clarify, constrain, confirm — is exactly what AI tools cannot do for you. Don't outsource it.

What This Skill Grows You Into

If I had to point at the single thing this framework has unlocked, it's the chance to grow into what some teams call a feature anchor.

A feature anchor is the engineer the team can trust to take an ambiguous problem — feature, architectural change, new project, cross-team integration — and bring it to a working outcome. Not because they're the best coder. Not because they know the most. Because they're willing to ask the right questions, decide what to build and what to skip, work through the messy middle, and demo something useful at the end.

That role is rarely assigned. It tends to grow on you quietly, as you keep running through this kind of process on enough problems that people start handing you the ambiguous work because they know it'll get done. It's one of the more rewarding milestones in an engineering career, because it's the moment the judgment part of the job — not just the code — starts to compound.

If becoming the engineer your team turns to when something fuzzy lands on the table sounds appealing — this skill, more than any specific technology, is the path.

Closing Thoughts

The framework is simple. The discipline of running it every time is what's hard.

So much of growing as an engineer is not about the code. It's about everything that happens before and after the code. The clarify conversation that prevents weeks of rework. The patience to understand the existing system before changing it. The discipline of restating the problem in layman terms. The constraint exercise that ships something useful in days instead of months. The improvisation that becomes a team-wide tool. The willingness to demo half-built work and absorb feedback. The wisdom to timebox analysis and restart fresh. The willingness to take ownership of the messy parts when the team needs someone to.

None of this is glamorous. None of it makes for a flashy pull request. But over time, it's the work that compounds the most.

If you're earlier in your career and this resonates — try one stage of the framework deliberately this week. Pick one ambiguous ask, run it through clarify and constrain explicitly, and notice what changes.

If you're further along — I'd love to hear what's missing or what you'd phrase differently. Every engineer I've learned from has a slightly different version of this, and I'm always curious to hear theirs.

If this helped, a clap goes a long way. And if you've shipped something that started as an improvisation, I'd love to hear that story.

All views in this post are my own. The framework is drawn from my own learning and not from any specific employer, project, or codebase. Examples are illustrative.

RestTemplate vs RestClient vs WebClient

Harshavardhan Katkam — Fri, 24 Apr 2026 08:41:33 +0000

The HTTP Client Showdown Every Spring Developer Needs to Understand

If you've been writing Spring applications for a while, you've probably had this conversation: "Should I use RestTemplate? Someone said it's deprecated. What about WebClient? And wait, what on earth is RestClient?"

Let's settle this once and for all.

The Short Version (For the Impatient)

Client	Introduced	Style	Status in 2026
RestTemplate	Spring 3.0 (2009)	Blocking, imperative	In maintenance mode — not deprecated, but not evolving
WebClient	Spring 5.0 (2017)	Non-blocking, reactive	Actively maintained, still the async champion
RestClient	Spring 6.1 (2023)	Blocking, fluent	The new recommended default for synchronous code

If you're starting a new project today and your code is synchronous — use RestClient. If you're doing reactive or truly async I/O — use WebClient. If you have an existing RestTemplate codebase — you don't have to migrate tomorrow, but plan for it.

Now let's dig into why.

A Quick History Lesson (Because Context Matters)

To understand these three clients, you need to understand the problems each was born to solve.

2009 — RestTemplate arrives. Back then, calling REST APIs in Java was painful. You were stitching together HttpURLConnection, Apache HttpClient, and custom serializers. RestTemplate wrapped all that ceremony behind a clean, synchronous API. It was revolutionary — and for a decade, it was the way.

2017 — WebClient arrives. The reactive revolution (RxJava, Project Reactor) changed how we think about I/O. Microservices meant one request often fanned out into ten. Blocking threads became expensive. Spring introduced WebFlux and, with it, WebClient — a non-blocking HTTP client built on Reactor's Mono and Flux.

The problem? WebClient became the only modern choice. If you wanted modern features (builder pattern, better error handling, interceptors that don't fight you), you had to adopt reactive types — even if your app had zero reactive code. Teams were writing .block() everywhere and pretending they weren't.

2023 — RestClient arrives. The Spring team finally admitted the obvious: most applications are synchronous, and they deserve a modern, fluent, blocking API that doesn't force reactive types down their throat. RestClient is what RestTemplate would look like if we designed it today.

RestTemplate: The Veteran

@Service
public class UserService {
    private final RestTemplate restTemplate;

    public UserService(RestTemplateBuilder builder) {
        this.restTemplate = builder
            .rootUri("https://api.example.com")
            .build();
    }

    public User getUser(Long id) {
        return restTemplate.getForObject("/users/{id}", User.class, id);
    }

    public User createUser(User user) {
        return restTemplate.postForObject("/users", user, User.class);
    }
}

What's good:

Everyone knows it. Zero learning curve.
Massive ecosystem, battle-tested, boring in the best way.
Works synchronously — matches how most business logic thinks.

What hurts:

The API is overloaded beyond recognition. Look at the Javadoc — exchange, execute, getForObject, getForEntity, postForObject, postForEntity, postForLocation... twenty ways to do one thing.
No fluent builder. Headers, query params, and body-building are awkward.
Error handling defaults are surprising. A 4xx throws; a 2xx with no body returns null.
Not evolving. New features go to RestClient and WebClient.

Official status: Not deprecated. This is important. The Spring team has explicitly said RestTemplate will not be removed. But it's in maintenance mode — no new features, just bug fixes.

WebClient: The Reactive Powerhouse

@Service
public class UserService {
    private final WebClient webClient;

    public UserService(WebClient.Builder builder) {
        this.webClient = builder
            .baseUrl("https://api.example.com")
            .build();
    }

    public Mono<User> getUser(Long id) {
        return webClient.get()
            .uri("/users/{id}", id)
            .retrieve()
            .bodyToMono(User.class);
    }

    public Flux<User> getAllUsers() {
        return webClient.get()
            .uri("/users")
            .retrieve()
            .bodyToFlux(User.class);
    }
}

What's good:

Truly non-blocking. Perfect for high-concurrency fan-out (a single Netty event-loop thread can handle thousands of in-flight requests).
Fluent, modern, discoverable API.
First-class streaming support — Flux<User> lets you process a 10GB response without loading it into memory.
Backpressure. If you're consuming a fast producer from a slow consumer, WebClient handles it natively.

What hurts:

You're now in Reactor-land. Mono, Flux, flatMap, zip, onErrorResume — there's a real learning curve.
Stack traces are ugly. Debugging reactive code is genuinely harder.
If you .block() everything, you get WebClient's complexity with RestTemplate's performance characteristics — the worst of both worlds.
Needs spring-webflux on the classpath even in a servlet app.

When it shines: WebFlux apps, streaming responses, SSE/WebSockets, fan-out aggregation where latency matters, any place you're genuinely doing async I/O.

RestClient: The Goldilocks Client

@Service
public class UserService {
    private final RestClient restClient;

    public UserService(RestClient.Builder builder) {
        this.restClient = builder
            .baseUrl("https://api.example.com")
            .build();
    }

    public User getUser(Long id) {
        return restClient.get()
            .uri("/users/{id}", id)
            .retrieve()
            .body(User.class);
    }

    public User createUser(User user) {
        return restClient.post()
            .uri("/users")
            .contentType(MediaType.APPLICATION_JSON)
            .body(user)
            .retrieve()
            .body(User.class);
    }

    public User getUserWithCustomErrorHandling(Long id) {
        return restClient.get()
            .uri("/users/{id}", id)
            .retrieve()
            .onStatus(HttpStatusCode::is4xxClientError, (req, res) -> {
                throw new UserNotFoundException(id);
            })
            .body(User.class);
    }
}

Look at that code. Notice something? It reads exactly like WebClient — but there's not a Mono or Flux in sight. It's synchronous. It blocks. It returns the value directly.

What's good:

Same fluent API as WebClient, zero reactive types.
Modern error handling with .onStatus().
Can use any underlying HTTP client — JDK's HttpClient, Apache, Jetty, Reactor Netty.
Integrates cleanly with Spring's @HttpExchange declarative clients (think Feign, but built in).
Familiar mental model for anyone coming from RestTemplate or WebClient.

What hurts:

Newer. Smaller community base of examples and Stack Overflow answers (though that's changing fast).
Not reactive. If you need non-blocking, it's the wrong tool.

Pro tip: You can build a RestClient from an existing RestTemplate — a huge win for gradual migration:

RestClient restClient = RestClient.create(existingRestTemplate);

All your existing interceptors, message converters, and error handlers come along for the ride.

The Deep Comparison

1. Syntax and Ergonomics

RestClient and WebClient share near-identical fluent APIs. RestTemplate is the outlier with its overloaded method zoo.

2. Threading Model

RestTemplate — one thread per request, blocked until response arrives.
RestClient — same model (blocking), but the underlying transport is pluggable.
WebClient — event-loop based, one thread handles many in-flight requests.

For a typical CRUD service handling a few hundred req/sec on a well-sized thread pool, the difference is invisible. For a gateway aggregating 50 downstream calls per request at 10K RPS, WebClient wins by orders of magnitude.

3. Error Handling

RestTemplate throws HttpClientErrorException / HttpServerErrorException by default. You override a ResponseErrorHandler globally to change it. Not fine-grained.

RestClient and WebClient offer per-request .onStatus() handling — you decide what a 404 means in the context of this specific call. Huge ergonomic win.

4. Testing

RestTemplate — MockRestServiceServer is mature and easy.
WebClient — needs either ExchangeFunction mocking or an MockWebServer setup.
RestClient — works with MockRestServiceServer (shares the infrastructure). Big win for teams migrating.

5. Declarative Clients (the underrated feature)

Both RestClient and WebClient support Spring's @HttpExchange:

@HttpExchange(url = "/users")
public interface UserApi {
    @GetExchange("/{id}")
    User getUser(@PathVariable Long id);

    @PostExchange
    User createUser(@RequestBody User user);
}

Wire it up once and you have a type-safe client — no implementation code to write. This is effectively what Feign gave us, now built into Spring. RestTemplate doesn't support this.

The Decision Framework

Ask yourself three questions, in order:

1. Is your codebase reactive (WebFlux, R2DBC, Kafka reactive)?
→ Use WebClient. Don't fight the paradigm.

2. Do you need genuine non-blocking I/O for performance (high fan-out, streaming, SSE)?
→ Use WebClient. The complexity pays for itself.

3. Everything else?
→ Use RestClient. It's the modern synchronous default.

And for legacy code on RestTemplate — there's no emergency. Plan a gradual migration as you touch code. RestClient.create(restTemplate) makes incremental migration almost free.

A Word on Performance

I'll save you the benchmark drama: for synchronous workloads, RestTemplate and RestClient are effectively identical. They share transport layers. The real performance story is WebClient vs. everything else in high-concurrency async scenarios — and even there, it only matters if you're actually constrained on threads.

Don't pick WebClient for "performance" if your app does 50 RPS. You'll pay the complexity tax for benefits you'll never measure.

Common Mistakes I See on Code Reviews

Using WebClient with .block() everywhere. You're burning all of WebClient's benefits and eating its complexity. Switch to RestClient.
Creating a new RestTemplate/RestClient per request. These are designed to be singletons. Inject the builder, build once, reuse.
Ignoring timeouts. Default timeouts are "forever." Always set connect and read timeouts — production will thank you.
Catching RestClientException broadly. Use .onStatus() for semantic error handling. Catch specific exceptions at the boundary.
Not using @HttpExchange. If you're writing repetitive CRUD wrappers, you're writing code that Spring will generate for you.

Closing Thoughts

Spring's HTTP client story went from "just use RestTemplate" to "use WebClient and suffer" to — finally — a clean, ergonomic split based on what your code actually needs. RestClient in particular is one of the best Spring improvements of the last five years. It quietly fixes the biggest complaint developers had without breaking anyone's existing code.

If I had to sum it up in one sentence: use RestClient by default, reach for WebClient when you're genuinely reactive, and migrate off RestTemplate at your own pace.

That's it. No dogma, no hype. Pick the tool that matches your workload.

If this helped, a clap goes a long way. And if you disagree — especially on the "don't migrate tomorrow" take for RestTemplate — I'd love to hear your war stories in the comments.

DEV Community: Harshavardhan Katkam

Day 03: Your repo does not fit in the context window. Now what?

The Short Version

Measure the repository first

The estimate everybody gets wrong

Two samples and a subtraction

Count input and output separately

Three ways to make a repo fit

What I'd do

Day 02: An LLM is a stateless function. Everything else is you

The Short Version

The follow-up that goes nowhere

There is no session. There is a list.

Your agent has exactly one piece of state

The bill is a triangle, not a line

What I'd do

Day 01: Why your first agent should be 40 lines, not a framework

The Short Version

An agent is a POST. That is the entire secret.

The forty lines

Three failures a framework would have swallowed

So when is a framework right?

It works, and I cannot tell if it is right

What I'd do

Day 00a: Six ideas that explain every AI agent, before any code

The Short Version

Before you start

Get a free OpenRouter key

The picture the word suggests

Idea 1: A model call is text in, text out

Idea 2: Tokens are the unit everything is measured in

Idea 3: The context window is one page

Idea 4: The model remembers nothing

Idea 5: Tool calling lets the model ask, not act

Idea 6: An agent is a loop

What an agent is not

Words you will meet in the next thirty days

Where each idea comes back

Check yourself

Recap

What's next

Further reading

Day 00: Learn AI agents by building one, in 30 days

The Short Version

Why build one thing instead of thirty demos

What you will build, week by week

How to follow along

Two rules for the writing

The Senior Developer Mindset: How to Turn Abstract Ideas Into Production Code

The Skill That Doesn't Have a Name

Where Abstract Ideas Come From

Before the Framework: Understand the Terrain

Sometimes You Have to Take Ownership of the Mess

The Framework: Five Stages

Stage 1: Capture

Stage 2: Clarify

Break It Down to Layman Terms — The Feynman Move

Stage 3: Constrain

A Lesson I Wish I'd Learned Earlier: Timebox, or Start Fresh

Stage 4: Construct

Stage 5: Confirm

What Makes the Framework Actually Work

Improvisations Compound

Three Failure Modes I Watch in Myself

What This Skill Grows You Into

Closing Thoughts

RestTemplate vs RestClient vs WebClient

The Short Version (For the Impatient)

A Quick History Lesson (Because Context Matters)

RestTemplate: The Veteran

WebClient: The Reactive Powerhouse

RestClient: The Goldilocks Client

The Deep Comparison

1. Syntax and Ergonomics

2. Threading Model

3. Error Handling

4. Testing

5. Declarative Clients (the underrated feature)

The Decision Framework

A Word on Performance