DEV Community: Aditya Negandhi

Going Event-Driven? Read the Bill First.

Aditya Negandhi — Sat, 10 Oct 2026 06:02:01 +0000

Picture this. You ship a gorgeous event-driven architecture. Everything is async, decoupled, cloud-native, the works. You're proud of it. Three weeks later someone drops a screenshot of the AWS bill in Slack, and suddenly nobody's talking about how elegant it is.

Pretty architecture and cheap architecture are two different things.

I spent the past year working closely with solution architects, and I kept seeing the same movie: a team goes event-driven for all the right reasons, then gets blindsided by costs nobody modelled. So here's my claim. An event-driven system is billed per event, per consumer, and per kilobyte, which means three whiteboard decisions set your bill: fan-out, payload size, and retry behaviour. You can do that math before writing a line of code. I did it below so you can steal it.

The short version

Events are great when services don't need to wait on each other. Service A publishes "this happened" and gets on with its life. The catch is that every one of those events is metered. On public AWS prices, one event fanned out to 12 small Lambda consumers costs about $5.90 per million events. That's pocket change at a million events a month. It's about $5,900 at a billion, and that's before a chunky payload or a retry loop gets involved.

What's wrong with good old HTTP calls?

When services call each other over REST, the whole chain is as fast as its slowest link. Checkout Service calls Payment Service, which calls Fraud Service, which calls Analytics Service. One of them hangs, the user's request hangs. One flaky service downstream and your p99 latency is on the moon.

That's the distributed monolith. It looks like microservices and behaves like a monolith. You split the code and kept all the coupling. Worst of both worlds.

Event-driven architecture fixes it by deleting the waiting. Service A publishes an event (say, "Order Placed") to a broker like Kafka or Amazon EventBridge. Whoever cares about that event picks it up and reacts in their own time. Service A is already serving the next request. Nobody waits on anybody.

So when should something be an event?

Not every call deserves to be one. Here are the two rules I use.

If Service A can finish its job without knowing what Service B did with the info, make it an event. When someone checks out, writing the order to the database has to succeed right now, so that stays synchronous. The confirmation email, the analytics update, the loyalty points? Nobody at the checkout button is waiting on those. Make them event consumers and your checkout gets faster for free.

If your traffic is spiky, events are your shock absorber. Synchronous APIs fall over under sudden load. A broker soaks up the spike and lets your workers chew through it at a pace they can survive.

OK, the bill

This is the part I wish someone had sent me before we shipped. To be clear, these numbers are a model I built from the public price list, not a bill from my employer. Prices are AWS list prices in US East, checked on 9 October 2026, free tier ignored.

The inputs:

EventBridge charges $1.00 per million custom events published. Delivery to a target in the same account is free. Each 64 KB chunk of payload counts as one event.
Lambda charges $0.20 per million requests, plus $0.0000166667 per GB-second of compute.
My assumption for a consumer: 128 MB of memory and 100 ms per invocation. That's 0.0125 GB-seconds, or about $0.21 per million invocations in compute.

So one "Order Placed" event with 12 consumers costs this much per million events:

Line item	Calculation	Cost per 1M events
Publish to EventBridge	1M x $1.00/M	$1.00
Lambda requests	12M x $0.20/M	$2.40
Lambda compute	12M x 0.0125 GB-s x $0.0000166667	$2.50
Total		$5.90

And here it is at three volumes:

Events per month	1 consumer	12 consumers	12 consumers, 256 KB payload
1 million	$1.41	$5.90	$8.90
100 million	$141	$590	$890
1 billion	$1,408	$5,900	$8,900

Three things jump out.

Fan-out is the multiplier. The broker is the cheap part. More than eighty percent of the 12-consumer cost is the consumers. Every new subscriber on a busy event adds its own request line and compute line, forever. So be picky about who gets to listen to what.

Keep events skinny. A 256 KB payload is billed as four events, which turns the $1.00 publish line into $4.00. Send an ID and what changed, like {"order_id": "abc123", "status": "placed"}. If a consumer needs the whole entity, it can go fetch it from the database or a cache. That's the claim-check pattern, and the 256 KB column is what it saves you.

Infinite loops will ruin your week. A service that hears an event, processes it, and accidentally emits the same event again will keep going until something stops it. At $5.90 per million that sounds harmless. But a loop doesn't run at your traffic rate. It runs as fast as the platform will scale. Set up dead-letter queues, retry limits, and loop detection before you ship. Lambda has recursive loop detection for some services now. It only covers some of them, so go read which.

Where I'd stop trusting this post

It's a small model and I picked the inputs. Twelve consumers at 128 MB and 100 ms is a believable fan-out, not a measured one. Double the duration or the memory and the compute line doubles too. I also left out logs, data transfer, the database reads claim-check adds, and whatever your consumers call downstream. Any of those can cost more than my whole table.

At small volume, ignore all of this. Under a few million events a month the entire event bill is lunch money, and your time is the expensive thing. Optimise for clarity and close this tab.

The best objection is that the bill is the wrong thing to worry about. Martin Fowler's survey of event-driven patterns argues the real price is losing sight of the flow: there's no single place in the code that shows what happens after "Order Placed". I think he's right, and I didn't put a number on that cost.

Kafka changes the math. A cluster you run or rent is priced by brokers and storage, not per event. Fan-out gets close to free and idle capacity becomes the thing you pay for. Everything above is about serverless brokers.

The takeaway

Most teams go event-driven too early and for the wrong reasons. Events aren't the default way for services to talk. They're what you reach for when you need things to happen async and at scale. If a service needs another service's answer to finish its own job, a REST call is still the right tool, and there's zero shame in that.

When you do make the jump, do the multiplication first: events per month, times consumers per event, times 64 KB chunks per payload. If that number scares you on a whiteboard, wait until it shows up on an invoice.

Microservices? Do You Really Need Them?

Aditya Negandhi — Sat, 10 Oct 2026 06:00:49 +0000

Feels like every team runs microservices now. Split the monolith, they said. It'll scale better, they said. And honestly, they're not wrong. But chopping your app into a pile of independent services is the easy part. Keeping those services in sync when something blows up halfway through? That's the actual job.

I've spent the past year as an associate engineer shipping and scaling production microservices, working closely with solution architects and principal engineers. And early on, one thing really bugged me. Our UI called an orchestrator, and the orchestrator called the real service. Why the extra hop? Just call the service directly. One less network call, less latency, simpler everything... right?

Boy, was I wrong.

So, do you really need microservices? Only if you're ready to own distributed transactions, because that's what you sign up for the second each service gets its own database. And once you're there, my claim is that an orchestrated saga is how you run them. One component that owns the workflow beats a bunch of services reacting to each other's events.

The short version

When your data lives in several independent services, a boring multi-step operation (like placing an order) becomes a distributed transaction. If step three fails, something has to cleanly undo steps one and two. The saga pattern is how you do that. It comes in two flavours: choreography (decentralised, event-driven) and orchestration (centralised, command-driven). Orchestration takes more setup and is almost always the right call as you grow. Here's why.

What even is a distributed transaction?

In a monolith, if something breaks mid-operation, the database rolls everything back for you. ACID has your back and life is good.

With microservices, that safety net is gone. Order Service, Payment Service, and Inventory Service each own their own database (the database-per-service rule). No single transaction covers all three. So if the payment goes through and the inventory reservation fails, congrats, you have a half-finished order in production and real money stuck in the middle.

That's the distributed transaction problem. It's the first wall every team runs into when they go all in on microservices.

Enter the saga

A saga is a chain of local transactions. Each service does its bit. If a step fails, you run compensating transactions to undo the steps that already worked. It's an undo plan baked into your architecture.

There are two ways to wire one up.

Choreography vs orchestration

Think of a restaurant kitchen.

Choreography is a kitchen with no head chef. Every station listens for the previous one to yell that they're done, then does its thing. Grill hears "prep done!" and fires the steak. Fry hears "steak done!" and drops the fries. It works, right up until somebody misses a shout, or two stations react to the same shout, and now nobody knows what state the order is in.

Orchestration is a kitchen with a head chef holding the ticket. They tell grill to fire, wait for the nod, then tell fry to drop. If something goes wrong, they know exactly what happened and who has to undo what.

Choreography

Services publish events. Other services listen and react. No central controller, everything decoupled.

The good part: services truly don't know about each other. On paper it's clean, pure microservices.

The bad part: once a workflow grows past 3 or 4 services (my rule of thumb, not a measured threshold), debugging turns into a murder mystery. There's no single place to check where a transaction is. You're stitching together logs from six services and praying the timestamps line up. And cyclic dependencies creep in, because services start listening to each other's failure events just to compensate for them.

Orchestration

One central orchestrator runs the whole workflow. It sends each service an explicit command, waits for a reply, and decides what happens next.

Command-driven: the orchestrator tells Payment Service to charge and gets back "success". It tells Inventory Service to reserve and gets back "failed". It tells Payment Service to refund. Done.
You can see everything: at any moment you can ask the orchestrator which step a transaction is on and why it stopped.
Compensation lives in one place: when something fails, the orchestrator knows what already succeeded and fires the rollbacks in order. No guessing.

Written out, that failed order is a short script you can read top to bottom, which is the whole point:

UI            -> Orchestrator : placeOrder(order-42)
Orchestrator  -> Payment      : charge(order-42)        <- ok
Orchestrator  -> Inventory    : reserve(order-42)       <- FAILED: out of stock
Orchestrator  -> Payment      : refund(order-42)        <- ok   (compensation)
Orchestrator  -> UI           : order-42 rejected: out of stock

With choreography the same five steps still exist. They're just scattered across the event handlers of three services, and no file has them in order.

Back to my "why the extra hop?" question

Here's why calling the service directly would have been worse.

When our UI calls the orchestrator, the orchestrator isn't a middleman forwarding traffic. It's running a multi-step transaction that might touch 3 or 4 services. If the UI called each service itself, the UI would have to run the saga: decide what to call next, handle partial failures, send the compensation calls. You'd be building an orchestrator in your frontend. Please don't.

Route through the orchestrator and the UI stays dumb and focused. The orchestrator stays smart and focused. Each service stays dumb and focused. Everybody has one job. That "extra network call" isn't overhead. It's the architecture doing its job.

Why orchestration wins as you grow

Visibility. One place to check the state of any business transaction. Priceless when you're on call at 2am.

Simpler services. A service runs commands and reports back. It doesn't need to know the bigger business flow. That's the orchestrator's problem.

No cyclic dependencies. Choreography breeds coupling in disguise, because services end up reacting to each other's events to handle failures. With orchestration the arrows only point one way.

Easier testing. Mock the service replies and you can test the whole workflow. No need to spin up the entire event mesh.

Where I'd stop trusting this post

The best objection: the orchestrator is a single point of failure and a bottleneck. AWS says so in its own guidance on saga orchestration. Fair hit. Every workflow now leans on one component, so it has to persist its state, survive a restart mid-saga, and scale with all your traffic. That's why teams reach for a durable workflow engine like AWS Step Functions or Temporal instead of rolling their own, and that's real operational weight choreography doesn't carry.

Simple flows don't need it. Two or three services and no compensation logic? Choreography is less to build and less to run.

Orchestrators attract business logic. Keep yours to sequencing and compensation. Once pricing rules and validation move in, you've rebuilt the monolith with extra network calls.

You might not need the split at all. Martin Fowler's MonolithFirst points out that most successful microservice systems started as a monolith that got too big. A single database transaction is still the cheapest saga there is.

And this isn't a benchmark. The comparison comes from working inside an orchestrated system and from reading, not from running both styles side by side on the same workflow.

The takeaway

Choreography is a fine place to start. Orchestration is where a microservices setup that ships and scales tends to end up.

Lots of teams start with choreography because it feels elegant. And for simple stuff, it is. But complexity always shows up eventually, and then you're spending more time debugging event flows than building features. An orchestrator forces your business logic to be explicit, visible, and testable. That's not overhead. That's just being kind to future you.

My rule: if a business operation needs a compensating step in more than one service, give it an orchestrator. Future you, and whoever's on call, will say thanks.

LLMs Don't Read Your Prompt. They Count It.

Aditya Negandhi — Sat, 10 Oct 2026 05:59:24 +0000

Ever typed a prompt into ChatGPT or Claude and wondered how this thing actually understands you? Spoiler: it doesn't read text the way we do. I got curious, went digging into how Large Language Models (LLMs) handle input, and it changed how I write prompts.

The whole post in one sentence: a model never sees your text, it sees a list of integers called tokens, and that list is what you get billed for and what your context limit counts.

The short version

LLMs don't work with raw text. They turn it into tokens, which are just numbers, using algorithms like byte-pair encoding. Tokens are the currency of AI. You pay per token, not per character. Below I show text turning into tokens, tokens turning back into text, and why any of this matters for cost and quality. There's code, so you can try it yourself.

What's a token, and why should you care?

If you've ever looked at your OpenAI bill (and winced), you'll have noticed you're charged by tokens, not words. So what is one? Picture a sentence chopped into bite-sized pieces. Sometimes a piece is a whole word, sometimes half a word, sometimes a comma. Each piece is a token. That's how models like GPT-4o or Llama take in text, and everything they do runs on it.

Why care? More tokens in or out means more money. And once you get how tokens work, you write tighter prompts and get better answers.

Step 1: text goes in, numbers come out

Every bit of text you send gets chopped into tokens by a tokenizer. I played with js-tiktoken, a JavaScript port of OpenAI's tiktoken and the tokenizer GPT-4o uses, to watch it happen:

   import { Tiktoken } from 'js-tiktoken/lite';
   import o200k_base from 'js-tiktoken/ranks/o200k_base';
   import { readFileSync } from 'node:fs';
   import path from 'node:path';

   const tokenizer = new Tiktoken(
     // NOTE: o200k_base is the tokenizer for GPT-4o
     o200k_base,
   );

   const textToTokens = (text: string) => {
     return tokenizer.encode(text);
   };

   const input = readFileSync(
     path.join(import.meta.dirname, 'input.md'),
     'utf-8',
   );

   const output = textToTokens(input);

   console.log('Content length in characters:', input.length);
   console.log(`Number of tokens:`, output.length);
   console.dir(output, { depth: null, maxArrayLength: 20 });

I ran that on a markdown file with 2,294 characters. Guess how many tokens came out? Just 484.

  Content length in characters: 2294
  Number of tokens: 484

That's about 4.7 characters per token for this file. Tokens aren't letters. They're chunks. One token might be a whole word like "hello", or a slice of a longer word. That's why the token count is way lower than the character count. It's still the number on your bill.

Step 2: what the model actually chews on

Here's the kicker. LLMs are trained on tokens, not text. All that data they learned from got tokenized first. When a model "writes" a reply, it's predicting the next token, a number, based on patterns it picked up in training. It isn't writing words. It's producing numbers that get turned into words afterwards.

So the loop is: your prompt becomes a list of numbers, the model crunches them to predict more numbers, and those numbers get decoded into text. Kind of wild, right?

Step 3: numbers back to text

Now the reverse. After the model outputs tokens, the tokenizer decodes them into something readable. I decoded a random token to see what I'd get:

  const tokensToText = (tokens: number[]) => {
     return tokenizer.decode(tokens);
   };

  const tokens = [13984];
  const decoded = tokensToText(tokens);
  console.log(decoded);

Token 13984 decodes to "VC". Random, and kind of funny. This decoding step is how a pile of numbers ends up as the reply you read.

Why tokens matter beyond the bill

Tokens aren't only about cost. They're the unit the model works in. More text means more tokens, which means more compute and more money. Same for the response. Want to save tokens? Keep prompts tight and skip the fluff.

It also explains context limits. A model can only handle a fixed number of tokens at once (like 8k or 128k), so a long conversation eventually gets cut off.

What changed for me

When I started tuning prompts for my AI projects, thinking in tokens made a real difference. Shorter prompts didn't only save money. Responses came back faster and were often more accurate, because the model wasn't wading through stuff it didn't need. If you build with LLMs, go play with a tokenizer. Watching a paragraph shrink into a list of numbers is weirdly satisfying.

Where I'd stop trusting this post

These numbers are one tokenizer and one file. o200k_base is OpenAI's encoding. Claude, Gemini, and Llama each have their own, so the same text gets a different count on each. For Claude you ask the API through its token counting endpoint. My 4.7 characters per token came from English Markdown. Code, JSON, and non-English text usually split into more tokens per character.

"Shorter prompts are more accurate" is what I saw, not a law. Cutting junk context helped my projects. Cutting context the model needed would have hurt. The habit worth keeping is measuring, not trimming.

Tokens aren't the whole bill. Output tokens cost more than input tokens on most price lists, and cached input is priced differently again. A count alone won't tell you the cost.

The takeaway

LLMs don't speak text. They speak tokens. It's all numbers back there, and knowing that makes you better at working with these things. Before you ship a prompt, run it through the tokenizer for the model you're actually calling. It's ten lines of code, and it swaps a guess for a number.

Semantic Search: What It Is and How I Built It

Aditya Negandhi — Sat, 10 Oct 2026 05:58:06 +0000

A while back I got a fun one at work: add semantic search to our API pipeline so our LLM could answer customer questions more accurately, in real time. I had a rough idea of how semantic search worked (shoutout to my ML professor, that class finally paid off), but I had no clue where to start building it. So here's how it went.

What I learned, in one sentence: most of the work in semantic search isn't the embedding model. It's picking a vector size and an index your database can actually serve fast.

The short version

Semantic search matches what you mean, not which words you typed. I built it with OpenAI's embeddings model, trimmed the vector dimensions so queries stayed quick, added an index, and switched our query flow from parameter matching to similarity. The result was search that got smarter and faster, which made the LLM's answers better too.

What is semantic search, and how is it different?

Old-school search does lexical matching. It looks for the exact word, or part of it. Search for "laptop" and you get results with the word "laptop" in them. It has no idea that "chromebook" or "MacBook" might be what you're after.

Semantic search tries to match meaning instead of strings. So even if you don't type the exact keyword that's in the data, it can still find the stuff that's conceptually close. That's the whole trick, and it's why it beats plain keyword search for questions written like a human wrote them.

How I built it

1. Pick a model and generate embeddings

First you turn your data into embeddings. An embedding is a long list of numbers (a vector) that captures what a piece of text means.
I used OpenAI's text-embedding-3-large. You send text, you get a vector back. Mine came out at 3,072 dimensions. Great for accuracy, rough for performance, so I cut it to 2,000. More on why in a sec.
You don't need multimodal embeddings unless you're dealing with images or video. Text-only did the job for me.

2. Pick the right number of dimensions

More dimensions means more nuance and better results. It also means more compute and slower queries, unless you happen to own a supercomputer.
Bigger vectors slow queries down and lean on your database, especially at scale. So you experiment until you find the balance. For me, going from 3,072 to 2,000 was a good trade between speed and relevance.
That number is less random than it looks. The text-embedding-3 models are designed to be shortened through the API's dimensions parameter, and pgvector can only index its vector type up to 2,000 dimensions.
Your right number depends on your data, your use case, and how fast you need it to be.

3. Index your embeddings

Once the vectors are stored, you need an index. Without one, every query scans the whole dataset, and that gets slow fast.
This is where approximate nearest neighbor (ANN) search comes in. You give up a little recall and get a huge speedup, which is a trade most production setups happily take.
The three index types I looked at:
1. IVFFlat
  - Uses k-means clustering to split your data into lists. A search only scans some of the lists, and you choose how many to probe, so you can dial speed against accuracy.
  - It needs training, and it's good when you want knobs to tune.
2. HNSW (original paper)
  - Builds a multi-layered graph of vectors. The search starts at the top layer and narrows in as it gets closer to the best matches.
  - Usually a better speed-to-recall trade than IVFFlat, with no training. You can even create the index on an empty table.
3. DiskANN
  - Built for scale. High recall and low latency even with billions of records. It takes more time and memory to build, but it's a solid pick when you need high throughput.

4. Change the query flow

In a normal search setup you parse the user's query, pull out parameters, and build a SQL or NoSQL query. Semantic search flips that.
You embed the user's plain-language question, then compare that vector against the ones stored in your database.
There are a few ways to measure how close two vectors are:
I went with cosine similarity because it worked well for text. Compare, grab the top n matches, return them. Now people can ask in plain language and still get relevant results.

Was it worth it? Here are the benchmarks I ran:

Where I'd stop trusting this post

Those benchmarks are one dataset on one system. They show what happened with our data and our queries. Trust the direction, not the exact numbers, and rerun them on your own data before you believe anything.

Semantic search is bad at exact matches. Embeddings blur a product code, an error ID, or a rare name into its neighbours. Keyword search nails those instantly. Production systems usually run both and merge the results (hybrid search), and Postgres already ships full-text search for the keyword half.

ANN indexes trade recall for speed, and they do it quietly. An approximate index can miss the true best match and nothing will tell you. Measure recall against an exact scan on a sample before you trust it.

Fewer dimensions means losing information. Shortening the vector kept quality good enough for us. On a different corpus the cut-off that works could be higher or lower.

The takeaway

Semantic search and LLMs are a great pair. Instead of shoving all your data at the model, you use semantic search to narrow it down to the relevant bits first.

That gets you faster responses, fewer tokens, and better answers. It also cuts down on hallucinations, because the model gets focused, relevant context and not a giant haystack to reason over.

Short version: semantic search filters, the LLM interprets, and your users get something that feels a lot smarter.

Do You Really Need AI Agents?

Aditya Negandhi — Sat, 10 Oct 2026 05:56:42 +0000

Whether you work with LLMs or not, you've heard "agent" and "agentic AI" about four thousand times by now. After months of building projects (hobby ones and production ones), eating a ton of content, and working daily with agentic frameworks like LangGraph and Google Vertex AI, I have some thoughts.

The short answer to the title: usually not. Most of what gets built as an agent would be cheaper, faster, and easier to trust as a fixed workflow. An agent should have to earn its spot with a measured result.

The short version

Agents are great when a task needs adaptability: messy inputs, lots of steps you can't predict, automation at scale. They're also wildly overused. For simple, predictable jobs, a workflow or even a single LLM call is more efficient and more reliable. So start with the boring solution, measure it, and bring in an agent only when the extra complexity clearly pays for itself.

Why is everyone so hyped about agents?

LLMs opened up automation that old-school algorithms couldn't touch. They can reason through a problem on the fly, so naturally every dev and every company wants that in their product, automating the repetitive stuff like updating docs or reviewing PRs. Even coding is semi-automated now, which backs up what OpenAI, Anthropic, and Google have been saying about LLMs handling the small stuff.

Agentic AI wants to take that all the way and automate the whole process. As a software engineer, I get the urge. But the hype also brought a lot of FOMO, and now agents get thrown at problems a much simpler solution would handle. It's like using a bazooka to kill a fly.

Workflows vs agents: what's the difference?

I use the split Anthropic draws in Building effective agents: in a workflow, your code decides the path. In an agent, the model does.

Workflows

A workflow is a predefined sequence. The LLM follows the code path you wrote, so it's predictable and structured. Every step has a clear goal.

Say a user sends a message. A workflow could classify it as "query", "complaint", "feedback", or "information", then send back the matching predefined response. Perfect when consistency matters more than flexibility.

Agents

An agent is more free-range. It doesn't follow a set path. It decides, step by step, how to find and use information. That makes it non-deterministic: it adapts to the context and to whatever tools it has.

So where the workflow just classifies the message, an agent might go poke around several data sources, combine what it finds, and write a tailored answer. Powerful for complex tasks, and also a lot harder to predict and control.

When should you use an agent?

When small time savings add up. Take expense reports: an agent can scan receipts, categorise expenses, and build the report. A person spends several minutes per receipt on that. Multiply by hundreds of reports and the manual work mostly disappears.

Agents also shine when you need flexibility and lots of decisions that can't be scripted ahead of time. Just know the deal you're making: you pay in cost and latency to get better results on the task. Only take that deal when it's worth it.

When should you NOT use an agent?

When the inputs are so ambiguous that you can't verify the output. Think automated legal contract review, where the nuance of language and intent needs a human expert. Or generating personalised investment strategies, where subjective judgment is the whole game. In cases like that, a small misread turns into an expensive mistake.

If the task needs strict reliability, a workflow is the better pick, because predictability and consistency are what get you there. And for a lot of apps, one well-tuned LLM call with retrieval and a few in-context examples gets the job done without any of the extra machinery.

Where I'd stop trusting this post

I wrote this in March 2025, and models have moved since. Things that needed a carefully scripted workflow back then can go to a single capable model call now, and agents fail less often than they used to. The line between "workflow" and "agent" shifts with every model generation. Treat my examples as dated and the method as the part that lasts.

The best case for going agent-first is that a workflow encodes how you understand the task today, and you pay to rewrite it every time the task shifts. For open-ended work like coding or research, where you can't list the steps up front, starting with an agent is the honest design. Anthropic's guide carves out the same exception.

I didn't put numbers on any of this. The cost and latency trade-off is stated, not measured. A fair version of this post would run one task both ways and report tokens, seconds, and error rate.

The takeaway

My prediction when I wrote this: 2025 would be the year we go from "agentic" to "multi-agent" systems, with several specialised agents working together on complex workflows. Exciting stuff. It's also a great way to pile on complexity and cost if you're not measuring at every step.

My rule: write the task as a single prompt first. Then as a fixed workflow. Reach for an agent only when you can point at the exact step where a fixed path fails, and you have an eval showing the agent does better.

What Exactly Is MCP?

Aditya Negandhi — Sat, 10 Oct 2026 05:55:23 +0000

If you hang out in tech corners of LinkedIn, YouTube, or X, your feed has probably been buried in MCP posts. Mine was. So a few days into the noise I decided to actually dig in, and this post is everything I managed to find and understand so far.

My one-line take: MCP is to AI tools what a common plug is to hardware. It doesn't make a model smarter. It makes a tool you write once usable from every app that speaks the protocol.

The short version

MCP (Model Context Protocol) is not just another API. It's a standard way to hand structured context to AI models. Think USB-C for AI: one plug for data sources, tools, and workflows.

Why does that matter? Because models are often context-blind. They struggle to use real-world information well. MCP helps by bridging data silos, tightening up security, and making integrations reusable across models. For devs, that means less glue code, faster builds, and setups that don't break every time you swap a model.

What's the big deal?

Lots of people assume MCP is one more API. If you're picturing REST endpoints and JSON payloads, think bigger. MCP is a protocol that gives a model proper context about the tools it has, then lets the model's own reasoning decide which tool it needs and when.

Why does context matter so much?

Today's models are seriously capable, but they trip up when they don't have the right context. In frameworks like LangChain, we handled that by giving models tools and teaching them when and how to use each one. MCP goes a step further and structures that context from the start, so the model isn't leaning on a vague system prompt and some tool descriptions.

More context ~= better, more accurate output

Here's an analogy. Think about asking a friend for homework help. You wouldn't just say "How do I solve this?" You'd say:

"Hey, I'm working on algebra homework." (Domain)
"We're learning about quadratic equations." (Specific topic)
"I need to solve x² + 5x + 6 = 0." (Task)
"Can you walk me through the steps?" (Request)

The traditional way, a model gets a prompt, reads whatever the system prompt says, and tries to work out which tools to use and how.

MCP gives it structured context up front, so it can make a much better call about which tool to pick.

What's wrong with how AI uses tools today?

Image credit: Matt Pocock

Right now, models don't magically know how to use external tools. You have to teach them. Usually that means building custom data access tools, writing prompts (like docstrings) explaining when and how to use each one, and attaching it all to the LLM through a system prompt. Then you hope the model picks the right tool for the user's question.

That approach has some real downsides:

Glue code overload. You write a pile of extra code just to make the tools behave.

Prompt engineering headaches. The model depends on carefully worded system prompts to understand its tools, and getting that wording right is fiddly.

Tight coupling. Change a tool and you have to update its docs, the system prompt, and the implementation. That's a maintenance headache that never goes away.

How does MCP fix this?

Image credit: Matt Pocock

MCP takes out the guesswork by standardising how tools and data are accessed, so you're not starting from scratch for every new source. You build a tool once and reuse it across agents and apps.

The model gets structured context up front, which cuts the ambiguity and the dependence on system prompts. You're no longer asking it to infer everything from fuzzy instructions. The key details are just there.

Bridging data silos. A big problem in enterprise AI is that the important data is everywhere. Some in databases, some in cloud storage, some buried in Slack. MCP gives the model one unified way to reach all of it, so it's working with the full picture.

Works with any model. MCP isn't tied to one model. Claude, GPT, your own custom LLM, it handles context the same way across all of them. And it's modular, so it plays nicely with different data sources and systems.

Security and access control. Dealing with sensitive data? MCP builds in controlled access to data and tools, so a model only touches what it's supposed to.

Pre-built integrations. MCP ships with ready-made connectors for things like Google Drive, Slack, GitHub, and databases, so you can hook a model up to common data sources quickly.

How do you get started?

MCP is still new, but it's an open standard, so you can start playing with it today.

Read the official docs. The architecture overview and the specification are the primary sources. The reference servers are the best code to read.

Try pre-built MCP servers. Since it's open source, npm-style registries are popping up with servers for tools we use every day. The most useful ones I've found so far: smithery, mcp-get, glama, mcp.so.

Build your own. The quick start guide walks you through creating a custom MCP server.

Where I'd stop trusting this post

You don't need MCP for one app with a few tools. Plain function calling does the same job with less machinery. The protocol pays off when the same tool has to work across several clients, or when you want to use servers other people wrote.

The best objection is security. An MCP server is somebody else's code and instructions, running with your credentials and feeding text straight into your model. The spec's own security section says descriptions of tool behaviour should be treated as untrusted unless they come from a trusted server, and that users must explicitly consent to data access and tool use. Installing a server from a registry deserves the same care as installing a package.

More tools isn't more capability. Every connected server adds its tool definitions to the context window. Connect ten and your model burns tokens, and attention, reading menus.

This is a summary of my reading. Nothing here is a measurement or a production report. Treat it as an explainer.

The takeaway

MCP is early. It promises a smarter way to connect AI to real-world data, and it's too soon to say how big it'll get. On the upside, standardising context could mean better accuracy, easier integrations, and smoother automation. The pre-built connectors and modular design make it tempting if you want simpler workflows.

There are open questions too. Will companies actually adopt it? Can it scale across different models and enterprise systems? Right now it's an interesting idea with a lot of potential. If you're in the AI space, it's worth watching and worth a weekend of tinkering. Will it truly change the agentic AI game? Guess we twiddle our thumbs and find out.

A Folder That Contains Nothing Is a Lie

Aditya Negandhi — Sat, 10 Oct 2026 05:44:02 +0000

Have you ever noticed that Gmail doesn't have folders?

It has labels. An email can carry three labels, or none, and "removing" one doesn't delete anything. I never thought much about why, until I ran into the exact problem labels solve.

Here's what I learned, and I learned it by getting it wrong: a folder makes you decide where something belongs at the moment you save it. That's a bet on where you'll need it later, and for some kinds of data you lose that bet constantly.

The short version

A folder is a decision made at write time about where you'll need something later.
I filed every note by where it was written. Ask from anywhere else and the note was invisible, which broke the one thing the project was for.
The fix: search everything, and treat "where it was written" as a ranking hint, not a wall.
If a name in your interface promises containment and the system doesn't contain, rename it.

The design that looked totally fine

I've been building a memory layer for AI tools called Hivemind. The idea is that you tell one tool something and your other tools already know it.

My first data model was the obvious one. Every note gets a folder when it's written. Every search looks inside exactly one folder. Projects stay separate, results stay relevant, and everybody understands folders. Lovely.

Then I made saving automatic. Nobody wants a "where should I file this?" popup every time they say something worth keeping. So the folder had to be guessed from wherever you were:

In a coding tool, connected over MCP: the repo you're in.
In a browser chat: the site you're on.

Still sounds reasonable, right?

The bug you can't fix with code

Now walk through the one thing the project exists to do:

Step	What happens	Folder
1	In Claude Code, inside `acme/payments`, you say you're migrating off Stripe Checkout	`acme/payments`
2	Later, in ChatGPT, you ask about the migration	`chatgpt.com`
3	The search looks in one folder	`chatgpt.com`
4	Result	Nothing. The note never comes back.

The note is sitting right there in the database. The search will never see it, because it's in a different folder.

That's not an edge case. That's the entire point of the thing, blocked by the storage model. And no ranking tweak can save you, because the note gets excluded before ranking even runs.

Two smaller cracks pointed the same way. A company-wide convention you happen to mention inside one repo isn't a fact about that repo. And "the site you're on" says almost nothing about which project is in your head.

Decide at read time

The fix was to stop filing things.

Notes don't go anywhere now. Every search looks at everything. What used to be a folder became a saved search, the same move as Gmail's labels or a database view: a named way of looking at the data, not a box the data lives in.

The "where was this written" signal didn't get thrown away. It got demoted from a wall to a weight. Each note still remembers where it came from, and if you ask from that same place, notes from there rank a bit higher. They're just never the only thing you can get.

const work = hm.withOrigin("acme");
await work.remember({ content: "Staging runs Postgres 16" });

Ask from acme and that note gets a boost. Ask from anywhere else and it can still show up, if it's the best answer.

Why I had to rename it too

I could've kept calling them folders. Everyone knows what a folder is.

And that's the problem. If you take something out of a folder, you expect it to be gone from somewhere. With a saved search it isn't, because it was never "in" there. The name promises containment and the system can't deliver it. Keep the name, keep the broken promise.

So the word is "view" now, everywhere.

I also resisted reviving an older name I'd already retired once. Recycling dead vocabulary is how I ended up with one word, "session", meaning two unrelated things in the same schema: a stretch of work, and a browser login. That's exactly as confusing as it sounds.

Where I'd stop trusting this post

A weight is not a wall. If you need hard separation, say client A's context must never show up while you're working for client B, ranking won't give you that. A preference isn't a boundary. For real isolation you need a real partition, like separate accounts.

Searching everything costs you. With no folder to shrink the haystack, your relevance cutoff has to be good, or unrelated projects leak into each other's answers. I traded a structural guarantee for a quality problem. I think it's the right trade here. It's still a trade.

Folders are great for plenty of things. Files, photos, anything you'll go looking for yourself. Hierarchies are how people navigate. This is about data that has to find you, somewhere you didn't predict.

"Decide at read time" can go too far. Schema-on-read is the same idea taken to its limit, and plenty of data lakes turned into swamps that way. Some structure at write time is what keeps data usable.

The takeaway

Any time you pick a container when something is saved, you're predicting the future. For things a person will go fetch, that's fine. For things that have to turn up on their own, the prediction is wrong often enough to break what you're building.

My rule now: decide at read time when you can, and keep the write-time signal as a hint, never a gate. And if a name in your interface promises something the system doesn't do, change the name before a user finds out for you.

5% a Day Sounds Gentle. It Isn't.

Aditya Negandhi — Sat, 10 Oct 2026 05:42:51 +0000

I found a bug recently that I can't stop thinking about, mostly because it was one innocent-looking line.

I've been building a memory layer for AI tools (Hivemind, if you're curious). Part of its job is ranking: you ask a question, it scores every saved note, and the best ones get sent to the model. Somewhere in that scoring was a small age penalty. Older notes lose 5% of their score per day. Recent stuff matters more. Seems fair.

It was quietly destroying everything older than a month.

The lesson I took, and the reason I'm writing this down: compounding turns "a little" into "everything" much faster than your gut says, and a test can only catch a bug its data is able to express.

The short version

A 5% daily penalty compounds. It halves a score every 13.5 days and leaves one ten-thousandth after six months.
In my ranking code that meant any note older than a couple of months could never win, however well it matched.
The benchmark missed it because every test note was the same age: zero.
The fix is a half-life in months with a floor underneath, and a test input where age actually varies.

How bad can 5% be?

Bad. The trick is that it's 5% of what's left, every single day. That's exponential decay, the same maths as radioactive stuff.

Go on, guess what a note is worth after six months. "Maybe a third" feels about right, doesn't it? Here's the real answer:

Age of the note	Score multiplier (0.95 ^ days)
1 day	0.95
7 days	0.70
30 days	0.21
90 days	0.0099
180 days	0.0001
365 days	0.00000001

After a month, a note has lost four fifths of its score. After six months it's worth one ten-thousandth of an identical note written today. It could match the question word for word and still lose to basically anything from this week.

A handy way to feel this: 5% a day is a half-life of about 13.5 days. Every two weeks, half of what was left is gone. Thirteen halvings in six months. Nothing survives thirteen halvings.

The funny part

A different part of the same system was doing the exact opposite.

It tracked how often each note got used, and after five uses it promoted the note to a "long-term" tier. The comment in the code literally called it earned permanence.

So one piece of code was carefully deciding which notes deserved to live forever. And another piece, a single multiplication nobody had looked at in ages, was killing them anyway. Two subsystems, opposite policies, and the quiet one won.

And there was a second one hiding next to it

While I was in there I found its sibling.

The system is supposed to fit the most useful notes into a fixed token budget. I'd been calling that a knapsack: pick the most valuable set that fits. What the code actually did was sort by score and fill from the top until the budget ran out.

That's not packing. That's top-k with a cutoff. Cost never enters into it, so one 2,000-token note scoring 0.9 would shove out twenty 100-token notes scoring 0.85. Same budget. One answer where there could've been twenty.

Why didn't the tests catch any of this?

This is the bit that gets me, because I had a benchmark and it looked great.

It looked great because every note in a benchmark run gets loaded at the same moment. Every age is zero. Every multiplier is 1. I could have set the decay to 99% a day and the score wouldn't have moved.

Same story for the packing bug. If your test notes are all about the same length, "sort by score" and "pack by value" pick nearly the same set.

The benchmark wasn't wrong. It just had no way to see the thing that was broken.

What I changed

Age is a nudge now. A half-life measured in months, with a floor that age can never push a note below:

// Simplified from the real thing.
function recencyMultiplier(tier: Tier, days: number): number {
  // Long-term notes don't decay. That's what the tier means.
  if (tier === "longterm") return 1;

  const halfLife = 180; // days
  const floor = 0.35; // age can never push a note below this
  return floor + (1 - floor) * 0.5 ** (days / halfLife);
}

Same ages, both curves:

Age of the note	Before	After
7 days	0.70	0.98
30 days	0.21	0.93
90 days	0.0099	0.81
180 days	0.0001	0.68
365 days	0.00000001	0.51

A year-old note keeps about half its weight. If it's the best match, it still wins.

Packing actually packs. Candidates get ranked by value per token (score divided by size) and added greedily, then a swap pass puts back any high-value note the small ones were blocking. It takes microseconds on fifty candidates.

Where I'd stop trusting this post

Every constant in there is a guess. 180 days and 0.35 aren't measured truths. They live in one config so I can sweep them later, and I fully expect them to move.

Greedy isn't optimal. Value-per-token plus a swap gets within a constant factor of the best possible set. Exact dynamic programming is affordable at this size. I haven't seen it change a result, so I haven't switched.

More notes isn't always better. Twenty small notes mean twenty things for the model to read, and models are worse at the middle of a long context. Packing is only half the job. Ordering matters too.

Sometimes you want things to fade fast. If it's a scratchpad for this week's task, steep decay is a feature. My bug wasn't that decay existed. It ran on everything, including stuff the system had already decided to keep.

The takeaway

Two habits I'm keeping.

First, whenever I see "a small percentage, repeatedly", I stop and work out the half-life. It takes one line and it has embarrassed me every time.

Second, for every signal that feeds a ranking, I go find the test input where that signal isn't a constant. If I can't find one, the signal is untested, however green the suite looks.

Why Does Every AI Tool Forget You?

Aditya Negandhi — Sat, 10 Oct 2026 05:41:43 +0000

Here's something that kept bugging me. I tell Claude Code how my project is set up. The next day I open Cursor and explain it again. Then I paste the same paragraph into ChatGPT, because of course it has no idea either.

These tools are scary smart. So why do they all have the memory of a goldfish?

I went digging, and the answer is simpler than I expected: a language model doesn't remember anything, ever. Every "memory" feature is an app pasting text into your prompt before the model sees it. Once that clicks, a lot of weird behaviour starts to make sense.

The short version

A model keeps no state between calls. A chat only feels continuous because the app resends the conversation every time.
"Memory" is a search over saved notes, plus a paste into your prompt.
Those notes live inside each app, which is why your other tools know nothing.
I compared four ways of choosing what to paste. Fusing keyword and meaning search found the right note 79.5% of the time, at one fortieth the tokens of pasting everything.

Wait, the model really remembers nothing?

Nothing. A model is a function. Text goes in, text comes out, and it keeps no state between calls.

The reason a chat feels continuous is that the app resends the whole conversation every time you hit enter. All of it has to fit in the context window, which is the fixed amount of text a model can look at in one go. When a conversation gets too long, something has to be dropped or summarised, and that's when it "forgets" what you said an hour ago.

Start a new chat and the window is empty again. Hello, goldfish.

So what is "memory", then?

A database and a paste.

When a tool says it remembers you, here's what's actually happening:

At some point it saved a few sentences about you somewhere. "Prefers TypeScript." "Deploys on Fridays, bravely."
When you ask something, it searches those sentences for ones that look relevant.
It pastes the winners into the prompt, above your question, where you can't see them.

That's it. It's retrieval-augmented generation pointed at your own notes. The model isn't remembering. It's reading a cheat sheet somebody slipped it.

Then why doesn't my other tool know?

Because the cheat sheet lives inside the app that wrote it.

ChatGPT's notes sit on OpenAI's servers. Your editor keeps its own rules file. Your CLI agent has a different file in a different format. None of them can read each other's, so you become the sync mechanism, copy-pasting your own context around like it's 2005.

This is the gap MCP is nudging at. If tools can call out to a shared server, the cheat sheet doesn't have to live inside any one of them.

The part I got curious about

Saving sentences is easy. The hard part is step two: picking what to paste.

Paste too little and the model doesn't know the thing. Paste too much and you've burned your context window on trivia, and models are measurably worse at using stuff buried in the middle of a long prompt anyway.

So I wanted a number. How often does a search actually bring back the note that contains the answer?

I've been building a memory layer on the side (it's called Hivemind, and it's the shared cheat sheet idea from the last section). I pointed it at LoCoMo, a public benchmark of long conversations with questions about them, and compared three ways of picking ten notes per question. Same 1,536 questions for each.

How the notes get picked	Found the note with the answer	Tokens pasted per question
Paste the whole conversation	99.7%	18,853
Keyword search (BM25)	63.9%	349
Meaning search (embeddings)	70.4%	389
Both searches fused, then filtered and packed	79.5%	465

Three things surprised me.

Pasting everything "wins", and it's a trap. You find the answer 99.7% of the time because the answer is in there somewhere, under 18,000 tokens of everything else. That's forty times the cost per question, every question.

Keyword search is better than its reputation. 63.9% from an algorithm from the nineties. It's fast, it's cheap, and it nails exact names and IDs that embeddings blur.

Neither search wins alone. Fusing the two beat the better one by nine points. They fail on different questions, so together they cover for each other.

Where I'd stop trusting this post

That's retrieval, not answers. "The note came back" isn't "the model answered right". When I measured the full loop with a tiny model answering, accuracy dropped a lot. Finding the note is necessary. It isn't sufficient.

One benchmark, in English, run offline. LoCoMo is chatty conversation. Your codebase notes aren't. Against the live service, on a smaller sample, the same pipeline scored 78.2%, and I wrote up the caveats for that one too.

I built one of the rows. I tried to keep the comparison fair: same questions, same store, same number of notes. You should still read it knowing who ran it.

Bigger context windows change the maths. As windows grow and tokens get cheaper, "paste everything" gets less silly. I don't think it wins, because attention is the scarce thing, not space. But I could be wrong about where the line ends up.

The takeaway

Your AI tools don't have bad memories. They have no memories, and a thin layer of search-and-paste on top that happens to be locked inside each app.

If you want to see it for yourself, try this. Tell one tool a fact. Open a different tool and ask a question that depends on it. Wherever that breaks is exactly where the cheat sheet stops, and now you know why.

What Are LLMs, and How Do You Build Stuff With Them?

Aditya Negandhi — Fri, 09 Oct 2026 22:52:19 +0000

Ever wondered how ChatGPT, Claude, Gemini, Deepseek, Llama and friends understand you and write back like a person? Here's the idea this whole post hangs on: an LLM only predicts the next token. So every useful app is the scaffolding you build around it: what context it sees, which tools it can call, and what it remembers.

So what is an LLM?

A Large Language Model is an AI system built to process and generate human-like text. Under the hood it's deep learning, specifically the Transformer architecture from the paper Attention Is All You Need. These models are trained on enormous piles of text like fineweb, which is how they pick up language patterns, context, and meaning. That's what makes them good at generating text, translating, summarising, and a lot more.

Old rule-based systems followed instructions someone wrote. LLMs are probabilistic. They predict the most likely next word based on what they saw in training. That's why they can handle such a wide range of questions and still give answers that fit the context.

The five words you'll keep hearing

Tokenization. Breaking text into smaller units called tokens. A token can be a word, part of a word, or a single character, depending on the tokenizer. Models work on tokens, not raw text.
Embeddings. Lists of numbers (vectors) that represent words or phrases and capture how they relate to each other. They're how a model knows two different words mean similar things.
Context window. The most text a model can handle at once. Bigger window, more it can keep in mind, more coherent the answers. Go past it and the early parts of the conversation fall off.
Temperature. The randomness dial. Turn it up for creative, varied output. Turn it down for predictable, repeatable output.
Fine-tuning. Training a model further on a specific dataset so it gets better at a specialised job, like medical, legal, or financial work.

What LLMs are good and bad at

LLMs, and AI in general, are NOT a magic bullet. Here's where they shine and where they fall on their face.

Where they're awesome

Writing human-like, coherent text that fits the context.
Pulling out and summarising big chunks of information fast.
Working across lots of languages.
Producing structured output.

Where they struggle

They hallucinate, and they're confident about it.
Short memory and a limited context window.
They can be biased, depending on the training data.
Logical reasoning isn't their strong suit.
They're expensive and energy-hungry to run.

OK, that's the model. Now, how do you actually build something with it?

How to build with LLMs

On its own, an LLM can't do much. If you want to solve a real problem and not just ship one more chatbot, you have to give it resources: internet access, tools for the tasks you need (calling an API, hitting a database), and so on. For that, you want LangChain.

LangChain

If you're serious about building apps with LLMs, LangChain is worth having in your toolbox. It gives you a structured way to talk to models and makes it much easier to hook them up to outside data sources, databases, and APIs. It comes with pre-built pieces for memory, for chaining several LLM calls together, and for using external tools like search APIs. Chatbot, research assistant, AI-powered knowledge base, whatever you're making, it takes a lot of the plumbing off your plate.

Why LangChain rocks

Integrations. Plug into different models, databases, and external APIs without much fuss.
Tools. The model can use search APIs, calculators, and other tools to give better answers.
Chains. String several LLM calls together like a conversation flow.
Memory. Keep conversation context across multiple exchanges.

Keeping your AI chat on track

LLMs have a limited attention span (that context window again), so long conversations get messy. Three ways to keep them manageable:

Trimming. Cut the parts of the conversation you don't need.
Filtering. Keep only the most relevant messages.
Summarising. Turn a long-winded chat into a short recap.

What can you build with it?

Plenty: chatbots, smart search, writing assistants, automated research tools. It's great for anything that needs to talk to databases, external APIs, and knowledge retrieval systems.

It has limits though. LangChain structures your AI interactions, but it's still stuck with the model's own constraints, like the context window and no long-term memory. And on its own it doesn't give you advanced decision-making or complex multi-step orchestration. That's where LangGraph comes in.

LangGraph

LangGraph is a flexible framework for building dynamic, multi-step AI apps. Where a simple app is prompt in, answer out, LangGraph lets you build workflows with multiple decision points, several agents talking to each other, and structured automation. It's especially handy when your AI needs to go back and forth with users, juggle different workflows, or remember things across sessions.

With it you can route queries, break a big task into smaller steps, and run things in parallel. Automating customer service flows, walking users through a process step by step, generating content with AI in the loop, it gives you the pieces to orchestrate all of that.

Storing conversation data

Want your AI to "remember" things? LangGraph gives you options:

Memory Saver. Stores data locally for quick access.
Postgres Saver. Uses PostgreSQL for external storage.
MongoDB. Stores conversations in a NoSQL database.
Redis Saver. Fast, efficient storage for AI memory.

Getting it production-ready

Use a different thread ID for each conversation.
Save conversation history with persistent checkpoints.
Avoid huge user prompts that overwhelm the model.
Stream responses so it feels smooth.
Handle errors (rate limits, content moderation, and so on).

Prompt engineering

To get accurate, relevant answers out of an LLM, the prompt matters a lot. Prompt engineering is part art, part science, and how you structure the question changes the quality of the answer.

Who's who in a conversation

SystemMessage. Sets the ground rules, behaviour, and personality of the AI. It's where you define a persona, a tone, or overall guidelines.
HumanMessage. What the user sends, usually text typed by a person.
AIMessage. What the model sends back: text, or a request to call a tool.
ToolMessage. Carries the result of a tool call back to the model, usually when outside data got fetched or processed.

Where I'd stop trusting this post

You might not need a framework at all. The best counter-argument comes from Anthropic's Building effective agents, which says to start with direct API calls, because frameworks add layers that hide the prompts and make debugging harder. If your app is one or two model calls, the plain SDK is less to learn and less to break.

This is a map from February 2025. Framework APIs in this space change every few months, and my list of things LLMs "struggle" with has shrunk since. Check the current docs before you copy a pattern from here.

It's an overview, not a build log. There's no code in this post and no measurements. For a worked example, Finchat is the app these notes came out of.

The takeaway

Building with LLMs is easier than ever with frameworks like LangChain and LangGraph. Chatbots, smart assistants, knowledge-search tools, they all come down to prompt engineering, context management, and conversation workflows.

If you remember one thing: decide what the model needs to see, what it's allowed to do, and what it should remember. In that order. The right framework is whichever one makes those three decisions easiest to read.

✌️ Stay curious, Keep coding, Peace nerds!

Updated 10 October 2026: rewrote this in plainer language and dropped the GIF. The central idea, primary sources, and limits section were added on 9 October 2026.

I Wrote a JSON Parser From Scratch

Aditya Negandhi — Fri, 09 Oct 2026 18:25:51 +0000

Ever wondered how an app actually understands the data an API sends it? Most of the time that data is JSON, a human-readable format that's all over the internet. But computers don't read "human-readable". Something has to turn that text into real data. That something is a JSON parser.

In this devlog I build one with Deno and TypeScript. It can parse local JSON files and JSON responses from an API. The thing I want you to walk away with: a parser is two small programs. A tokenizer turns characters into tokens, and one recursive function per grammar rule turns tokens into a tree. JSON's grammar is small enough that you can see the whole idea in one sitting.

What is JSON, and why is it everywhere?

JSON, or JavaScript Object Notation, is a lightweight, text-based format for exchanging data. It's the default for sending data around the internet, especially through APIs. Devs love it because it's simple and easy to read.

Language-independent. It isn't tied to any one programming language.
Real data structures. You can nest objects and arrays as deep as you need.
Easy to parse. Which makes it fast to process.

//  This is what a valid JSON looks like:
{
  "name": "Alice",
  "age": 25,
  "isDeveloper": true
}

You'll find it in API responses, config files, saved app settings, you name it. So how does software make sense of it? Parsers.

What's a parser?

A parser is a program that turns text into structured data, and rejects anything that doesn't follow the grammar. For JSON, its job is to turn a raw string into objects you can use. That usually happens in two steps:

Tokenizing (lexical analysis). Chop the input into meaningful chunks called tokens, like {, "name", :, "Alice".
Parsing (building an AST). Arrange those tokens into a tree, called an Abstract Syntax Tree (AST), that represents the data.

Example

// Here's a JSON snippet:
{ "id": 1, "isActive": true }

After tokenizing, it looks like this:

[
  { type: "BraceOpen", value: "{" },
  { type: "String", value: "id" },
  { type: "Colon", value: ":" },
  { type: "Number", value: "1" },
  { type: "Comma", value: "," },
  { type: "String", value: "isActive" },
  { type: "Colon", value: ":" },
  { type: "True", value: "true" },
  { type: "BraceClose", value: "}" },
];

And after parsing, the Abstract Syntax Tree (AST) looks like this:

{
  type: "Object",
  value: {
    id: { type: "Number", value: 1 },
    isActive: { type: "Boolean", value: true }
  }
}

Here's the flow for a valid JSON:

And for an invalid one:

Building it with Deno and TypeScript

I went with Deno because it supports TypeScript out of the box and ships with handy tools like a test runner and a coverage checker.

# The project structure is as follows:

/json-parser
      ├── main.ts          # Entry point for the application
      ├── parser.ts        # Contains the parser logic
      ├── tokenizer.ts      # Contains the tokenizer logic
      ├── types.ts         # Type definitions for tokens and AST nodes
      └── utils.ts         # Utility functions for type checking
      │
      ├── main_test.ts     # Contains tests for the parser and tokenizer
      └── test-data.json   # Sample JSON data for testing
      │
      ├── deno.json                # Deno configuration file
      ├── .vscode/settings.json     # VSCode settings for Deno
      └── deno.lock                # Deno lock file for dependencies

The tokenizer (lexical analysis)

Tokenizing is always step one when you write an interpreter or a compiler. Even your favourite code formatter tokenizes the whole file before it prettifies anything. You're breaking the input into small parts the parser can reason about, so it knows where each thing starts and stops.

It also gives you the structure of the input: keywords, symbols, literals. And it's a big part of error handling. Once you can identify and categorise tokens, catching syntax errors gets a lot easier.

One rule here: only use the grammar from ECMA-404 when tokenizing JSON. (The diagrams on json.org are the same grammar, drawn out.) That's how you make sure the tokenizer recognises JSON tokens correctly.

Here's what tokenizer.ts looks like:

export const tokenizer = (input: string): Token[] => {
  let current = 0;
  const tokens: Token[] = [];

  while (current < input.length) {
    let char = input[current];

    if (char === "{") {
      tokens.push({ type: "BraceOpen", value: char });
      current++;
      continue;
    }
    // handlers for other token types (e.g., strings, numbers, etc.)
  }
};

The parser (building the AST)

The parser is where the tokens start to mean something. Now we build the Abstract Syntax Tree. The AST captures the structure of the data as a tree, with the relationships between all the pieces: which values belong to which keys, what's nested inside what.

Here's what parser.ts looks like:

function parseValue(): ASTNode {
  const token = tokens[current];
  switch (token.type) {
    case "String":
      return { type: "String", value: token.value };
    case "Number":
      return { type: "Number", value: Number(token.value) };
    case "True":
      return { type: "Boolean", value: true };
    case "False":
      return { type: "Boolean", value: false };
    case "Null":
      return { type: "Null" };
    case "BraceOpen":
      return parseObject(); // recursive fn
    case "BracketOpen":
      return parseArray(); // recursive fn
    default:
      throw new Error(`Unexpected token type: ${token.type}`);
  }
}

Testing it

To check the parser works and actually follows the ECMA-404 standard, we need tests. And we need the annoying edge cases too: \n, \t, \r, \f, and \b in strings, - for negative numbers, and so on.

Here's what main_test.ts looks like:

Deno.test("parser should correctly parse an array with mixed types", () => {
  const jsonString = `[-1, "string\n\t\r\f\b", true, null, {}]`;
  const tokens = tokenizer(jsonString);
  const ast: ASTNode = parser(tokens);

  assertEquals(ast, {
    type: "Array",
    value: [
      { type: "Number", value: -1 },
      { type: "String", value: "string\n\t\r\f\b" },
      { type: "Boolean", value: true },
      { type: "Null" },
      { type: "Object", value: {} },
    ],
  });
});

Deno.test("parser should throw an error for invalid JSON", () => {
  const jsonString = `{
    "key": "value",
    "invalid": 
  }`;

  const tokens = tokenizer(jsonString);

  try {
    parser(tokens);
    throw new Error("Expected an error to be thrown for invalid JSON");
  } catch (e) {
    assertEquals((e as Error).message, "Unexpected token type: BraceClose");
  }
});

// More tests...

What I learned

Tokenizing is the foundation. Break the input into small, understandable parts and the parser's job becomes manageable.

The tokenizer takes a JSON string and hands back an array of tokens that describe the structure of the data. The parser takes those tokens and builds an AST that neatly organises the key-value pairs and nested structures.

Put the two together and, yep, that's a JSON parser. I built one. Feels good.

Where I'd stop trusting this post

Passing my tests isn't the same as being correct. My tests cover the cases I thought of. JSONTestSuite has hundreds of files that real parsers disagree on, and it's the honest benchmark for any parser that claims to follow the standard.

The standards leave things open. ECMA-404 defines the syntax only. RFC 8259 adds interoperability advice and still leaves choices to whoever's implementing it: what to do with duplicate keys, how big a number can be, how deep nesting can go. My parser makes those choices without telling you.

Recursive descent has a depth limit. Every nested array or object is another function call, so a deep enough document will blow the call stack. Production parsers either cap the depth or use an explicit stack.

Please don't use this in production. JSON.parse is native code and heavily optimised. The reason to write your own is to understand the one you already have.

Want the full code? Here's the GitHub repo.

✌️ Stay curious, Keep coding, Peace nerds!

Updated 10 October 2026: rewrote this in plainer language and gave it a new title. A broken sentence, the ECMA-404 link, and the limits section were fixed or added on 9 October 2026.

I Built a Spell Checker to Watch an Algorithm Think

Aditya Negandhi — Fri, 09 Oct 2026 12:07:37 +0000

Hey fellow devs! 👋 Let me tell you about my little adventure with the Levenshtein distance algorithm. You know those moments when you find something that makes you go "wait, that's actually pretty cool"? This was one of those.

The idea in one line: a spell checker is an edit-distance function plus a dictionary. Count how many single-character edits separate the typed word from each known word, and suggest the closest ones.

The discovery

There I was, grinding through DSA problems (like we all do), when I ran into Levenshtein distance. At first it looked like one more dynamic programming problem to stuff into my interview prep. Then I dug a little deeper and found out this algorithm is the backbone of spell checkers. You know, the thing that saves us from embarrassing typos in MS Word.

The lightbulb moment

My first thought was, "What if I could actually see this algorithm work?" Not just input and output, but the whole process. So I built a spell checker visualizer. Nothing fancy, just something to help me (and maybe you) understand how the algorithm thinks. There's a live demo, and the source is on GitHub.

The algorithm: Levenshtein distance

How it works (time complexity: O(m*n))

Say you're comparing two words and you want to know how many changes it takes to turn one into the other. Let's use "wrld" and "world".

Make a grid (a matrix) where:
- The first row is the letters of the first word, plus an empty slot.
- The first column is the letters of the second word, plus an empty slot.
Fill the first row and first column with 0, 1, 2, 3 and so on. That's how many changes it takes to get from an empty string to each prefix.
Now the fun part. For each cell, look at the two letters it lines up with and ask: "Are these the same?"
- If they're the SAME:
  - Copy the number from the diagonal upper-left cell.
- If they're DIFFERENT:
  - Look at three neighbours (up, left, and diagonal upper-left).
  - Take the smallest number.
  - Add 1.
The number in the bottom-right cell is your answer: the minimum number of changes.

A basic implementation

function levenshteinDistance(str1: string, str2: string): number {
  const matrix: number[][] = [];

  // Matrix initialization
  for (let i = 0; i <= str1.length; i++) {
    matrix[i] = [i];
  }
  for (let j = 0; j <= str2.length; j++) {
    matrix[0][j] = j;
  }

  // The magic happens here
  for (let i = 1; i <= str1.length; i++) {
    for (let j = 1; j <= str2.length; j++) {
      if (str1[i - 1] === str2[j - 1]) {
        matrix[i][j] = matrix[i - 1][j - 1];
      } else {
        matrix[i][j] =
          Math.min(
            matrix[i - 1][j - 1], // substitution
            matrix[i][j - 1],     // insertion
            matrix[i - 1][j]      // deletion
          ) + 1;
      }
    }
  }

  return matrix[str1.length][str2.length];
}

The dictionary

Here's a simple example of how the dictionary is set up:

interface Dictionary {
  words: Set<string>;
}

const dictionary: Dictionary = {
  words: new Set([
    'world',
    'hello',
    'programming',
    // ... more words
  ])
};

Building the visualizer

I grabbed my favourite tools for the job:

Vite + React (because who doesn't love a speedy dev experience?)
TypeScript (to keep my errors honest)
Tailwind CSS + shadcn/ui (for a clean look, and yes, there's dark mode)

What it does is pretty simple:

You type some text.
The app spots the misspelled words.
And the cool part: it shows the Levenshtein matrix for each suggestion.

What it looks like

Other ways to do it

Levenshtein is great, but it's not the only game in town:

Damerau-Levenshtein distance: also handles two adjacent letters being swapped.
Soundex: matches words that sound alike.
N-gram similarity: useful for longer text and for finding similar phrases.

Where I'd stop trusting this post

Comparing against every word doesn't scale. The matrix costs O(m*n) per pair, and a naive checker runs it against the entire dictionary for every typo. Fine for a hardcoded word list, way too slow for a real one. The usual fix is to index the dictionary so most words never get compared, for example with a BK-tree.

Plain Levenshtein gets common typos wrong. Swapping two adjacent letters ("teh") counts as two edits, so it can rank a worse suggestion first. It also has no idea which words are common. Peter Norvig's spelling corrector shows how much word frequency matters, in about twenty lines of Python.

It checks words, not sentences. "Their" when you meant "there" is spelled correctly and will never get flagged. Context needs a language model, and that's a different tool.

The takeaway

Is it basic? Absolutely. The dictionary is hardcoded and it won't be replacing Grammarly any time soon. But that was never the goal. I wanted to watch the algorithm turn "wrld" into "world" one edit at a time. And you know what? It works.

Sometimes the simplest projects are the most satisfying. This wasn't about building the next big thing. It was about taking an algorithm out of interview prep and watching it solve a real problem.

Not every project has to change the world. Sometimes just understanding how a thing works is reward enough. So here's to the small wins, the "aha" moments, and the joy of seeing an algorithm come to life.

✌️ Stay curious, Keep coding, Peace nerds!

Updated 10 October 2026: rewrote this in plainer language and gave it a new title. The core idea, sources, and limits section were added on 9 October 2026.