<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: ADITYA DUTT PANDEY</title>
    <description>The latest articles on DEV Community by ADITYA DUTT PANDEY (@aditya_duttpandey_b33d2c).</description>
    <link>https://dev.to/aditya_duttpandey_b33d2c</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F2062508%2F615d3947-ff52-4092-a6d4-8c945494586d.jpg</url>
      <title>DEV Community: ADITYA DUTT PANDEY</title>
      <link>https://dev.to/aditya_duttpandey_b33d2c</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/aditya_duttpandey_b33d2c"/>
    <language>en</language>
    <item>
      <title>Harness engineering is not a joke. I have the failed runs to prove it.</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Wed, 14 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/harness-engineering-is-not-a-joke-i-have-the-failed-runs-to-prove-it-5gjf</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/harness-engineering-is-not-a-joke-i-have-the-failed-runs-to-prove-it-5gjf</guid>
      <description>&lt;p&gt;I built a coding harness in six hours, and on small tasks it was better than the tools I'd been using.&lt;/p&gt;

&lt;p&gt;Then I started trying to make it good, and that's where it got humbling. I've lost more rounds than I've won, and this post is the honest account. I tried, I lost, and I went back in.&lt;/p&gt;

&lt;h2&gt;
  
  
  Small tasks flatter a harness
&lt;/h2&gt;

&lt;p&gt;A task that runs for ten minutes hides almost every design mistake. The conversation is short, a few files are in play, and nothing has had time to rot.&lt;/p&gt;

&lt;p&gt;A thirty-minute task is a different animal. The model has read dozens of files, formed three theories and dropped two of them. It's carrying its own history, and paying for that history again on every call. Most of what I'd built was tuned for the ten-minute case, and I couldn't see that until a long one broke it.&lt;/p&gt;

&lt;h2&gt;
  
  
  The thing I fought first: tokens
&lt;/h2&gt;

&lt;p&gt;An agent run isn't one request. Every tool call is another API call, and every API call sends the whole conversation again. So a tool result isn't paid for once. It's paid for on every call that comes after it.&lt;/p&gt;

&lt;p&gt;In one real mew run, a single file read cost 1,922 tokens and was re-sent 15 times, which is 28,830. Overall, 52% of that run's input was earlier tool results travelling along again. The price of reading a file is its size times the number of calls that follow.&lt;/p&gt;

&lt;p&gt;That pushes you two ways at once. You want fewer calls, and you want less in each one.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Fewer calls.&lt;/strong&gt; Search, outlines and symbol reads let the model find a function without reading a whole file. A read-only scout on a cheap model investigates first, so the main model starts from findings.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Less per call.&lt;/strong&gt; mew remembers which lines of each file the model has already seen. Ask again and you get only what's new, or a diff if the file changed.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Those held up. Then I got ambitious.&lt;/p&gt;

&lt;h2&gt;
  
  
  Round one: the variable idea
&lt;/h2&gt;

&lt;p&gt;If files are what fill the context, why keep them in the context at all?&lt;/p&gt;

&lt;p&gt;My idea was a variable. Read a file, keep the contents somewhere else, and leave the model a handle. The data lives outside the conversation, so it can't be re-sent on every call.&lt;/p&gt;

&lt;p&gt;On paper it was perfect. In practice, the same file was being read four or five times, the tokens were coming back as uncached, and the bill went up. That was the opposite of the point.&lt;/p&gt;

&lt;p&gt;I still can't fully explain why they were uncached. I had a theory that prompt caching depends on the start of the conversation staying identical, so anything that rewrites an earlier message breaks it. I never confirmed that, and given what happened next, it may not be the explanation at all.&lt;/p&gt;

&lt;p&gt;What did happen was dull. I removed one of the pieces I'd stacked on top of each other, and it behaved a bit better than before. Then I found one very small glitch in my Python and fixed it, and the variable approach now works. All that theory, and the thing standing in the way was a small bug in my own code.&lt;/p&gt;

&lt;p&gt;I won't turn that into a lesson about clever design. The lesson is to check the bill, because the bill was the only thing that told me the idea was failing. What gave me something to check it against was the cost report. &lt;code&gt;/cost&lt;/code&gt; now says why a call was billed in full: an idle gap, a model change, or a changed prefix. In the run I screenshotted, it reported 25,381 tokens re-billed because the prefix changed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Round two: the thirty-minute bug
&lt;/h2&gt;

&lt;p&gt;This is the one I'm least proud of.&lt;/p&gt;

&lt;p&gt;I gave mew a bug I'd have found myself in an hour or two of digging in my own codebase, and let it plan and investigate. It failed badly. It wasn't designed for work like that, and I tried a few things on the spot that didn't rescue the run.&lt;/p&gt;

&lt;p&gt;A long investigation isn't a bigger version of a short task. It needs things a short one never asks for. So I went back and added what I thought was missing:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Memory that survives compaction.&lt;/strong&gt; When the context fills up, something has to go, and a paraphrase of the whole history loses the facts the investigation stands on. mew now keeps the newest ~20k tokens word for word, folds only older turns into a running summary, and adds a plain list of the files read and changed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A way to stop reading in circles.&lt;/strong&gt; The prompt now teaches hypothesis-first investigation: say what you think is wrong, then look for where the real behaviour differs from the expected behaviour. Loop detection catches repeated actions and escalates to a stronger model instead of burning the budget.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A budget that ends cleanly.&lt;/strong&gt; At 70% mew warns, and when the budget is gone it makes one last call with no tools, so the work so far becomes an answer.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I'm not going to tell you that fixed it. I built those because that run showed me what was missing, and I haven't shown that mew now beats me at a two-hour bug.&lt;/p&gt;

&lt;h2&gt;
  
  
  The scoreboard
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Attempt&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Search first, outlines, symbol reads, a scout&lt;/td&gt;
&lt;td&gt;Kept&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Line map and diffs instead of re-reading files&lt;/td&gt;
&lt;td&gt;Kept&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Variable handles instead of files in the context&lt;/td&gt;
&lt;td&gt;Lost, then fixed a small glitch and it works now&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A long, hard bug investigation&lt;/td&gt;
&lt;td&gt;Lost, tried again, still not proven&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  What I'd tell anyone starting
&lt;/h2&gt;

&lt;p&gt;Measure before you optimise. A harness has too many moving parts to improve by feel, and the cost report told me things I'd never have guessed, like one file being re-sent 15 times.&lt;/p&gt;

&lt;p&gt;Test long tasks early. Ten-minute runs tell you the loop works and nothing else. The design problems show up at thirty minutes, and by then the architecture is set.&lt;/p&gt;

&lt;p&gt;Expect the boring answer. The variable idea sounded like a design problem, and a small Python glitch was in the way.&lt;/p&gt;

&lt;p&gt;I started this in six hours and thought I understood it. The ten-minute runs agreed with me. The thirty-minute one didn't, and it was right.&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>llm</category>
      <category>softwareengineering</category>
    </item>
    <item>
      <title>I built a coding agent in six hours. It argues with me the way I argue with myself.</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/i-built-a-coding-agent-in-six-hours-it-argues-with-me-the-way-i-argue-with-myself-206l</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/i-built-a-coding-agent-in-six-hours-it-argues-with-me-the-way-i-argue-with-myself-206l</guid>
      <description>&lt;p&gt;When I write code well, it's because I argued with myself first. One half of me writes it. Another half attacks it. A third decides whether it ships.&lt;/p&gt;

&lt;p&gt;Every AI coding tool I tried gave me the first half. So I built mew, a harness that runs all three, and it took six hours.&lt;/p&gt;

&lt;p&gt;It's now how I write code. Against Claude Code and Pi, it has been better for me. That's my experience from using it every day, on my own work, and I'll be careful about what it does and doesn't prove at the end.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: split the writer from the critic and the judge
&lt;/h2&gt;

&lt;p&gt;mew doesn't wait to be asked for feedback. While I work, other agents run alongside me and push back on my decisions. They aren't a list of if-this-then-that rules. They read what I'm doing and challenge it, which is what I'd be doing to myself in my head.&lt;/p&gt;

&lt;p&gt;The critic and the judge are separate on purpose. When a run changes files, your checks run first, and a failure goes back to the model as a repair request. Then a second model, a different one, reads the real diff. My reasoning is that a model reviewing its own work tends to agree with itself, and a different one has different blind spots. I haven't measured that.&lt;/p&gt;

&lt;p&gt;Here's a real run on tunyt's web app. The checks passed and the review passed. The final answer still opened with "Partly correct. I wouldn't ship it as is", and said it had read the code and never run it.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://lab.adpandey.dev/posts/mew-session.webp" rel="noopener noreferrer"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fk7g9kmhbctbddcqi43l7.webp" alt="A mew terminal session on the tunyt repo. The result is verified, with a note that typecheck and lint don't exercise the changed behaviour, followed by a cost report showing 21 calls and 85% of input cached." width="800" height="504"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That's the part I care about. Typecheck tells you the types still line up and lint tells you the code follows the rules. Neither runs the change. Confirming this fix means liking an event, coming back hours later, and looking at the count and whether the like is still selected. mew said exactly that, in the output, without me asking.&lt;/p&gt;

&lt;p&gt;Every run ends in one of five statuses: &lt;code&gt;verified&lt;/code&gt;, &lt;code&gt;unverified&lt;/code&gt;, &lt;code&gt;verification_failed&lt;/code&gt;, &lt;code&gt;failed&lt;/code&gt; or &lt;code&gt;done_no_changes&lt;/code&gt;. None of them is a bare "done".&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Flab.adpandey.dev%2Fposts%2Fmew-flow.svg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Flab.adpandey.dev%2Fposts%2Fmew-flow.svg" alt="mew request flow: classify, scout, edit, verify, review, citation check. Failed checks and invented references loop back to the edit step." width="720" height="160"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: I set the budget before the work starts
&lt;/h2&gt;

&lt;p&gt;I like to decide how many tokens a kind of task deserves, and I want the tool to respect that. So mew sorts each request by difficulty without spending a model call on it. Easy requests go to a fast chain of models, hard ones to a strong chain.&lt;/p&gt;

&lt;p&gt;There's also a hard ceiling. At 70% of the budget mew warns me, and when the budget is gone it makes one last call with no tools, so the work done so far becomes an answer and not a bare failure.&lt;/p&gt;

&lt;p&gt;I also use it to divide a task and rethink how I'd approach it before any code gets written. Having to say how much a task is worth turns out to change how I plan it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 3: run on the cheapest models and make the context do the work
&lt;/h2&gt;

&lt;p&gt;Most of this runs on small, cheap models, DeepSeek V4 Flash and GPT-5.6 Luna among them. The aim is the best output the cheapest models can give. My bet is that a cheap model given a tight, clean context does well, and that a lot of what looks like a model's weakness is really a context problem. It's a bet. I haven't isolated it.&lt;/p&gt;

&lt;p&gt;So most of mew is about the context. The surprising part is where the tokens go. A long run sends the whole conversation again on every call, so a tool result isn't paid for once, it's paid for on every call after it. In the run above, one file read cost 1,922 tokens and was re-sent 15 times, which is 28,830. Half of that run's input, 52%, was earlier tool results travelling along again. A file's real cost is its size times the number of calls that follow it.&lt;/p&gt;

&lt;p&gt;So mew keeps a map of which lines of each file are already in context. Ask again and you get only what's new, or a diff if the file changed. It finds code with search and outlines before reading anything, and a read-only scout on a cheap model goes first, so the main model starts from findings. A cold index of a large repo takes 13.7 seconds and a warm one 1.2.&lt;/p&gt;

&lt;p&gt;Caching works on the start of the conversation, so one early change re-bills everything after it. That run re-billed 25,381 tokens because the prefix changed. The Codex backend only caches when you send &lt;code&gt;session_id&lt;/code&gt; and &lt;code&gt;prompt_cache_key&lt;/code&gt;, so mew sends both and warns if a model isn't caching. I also cut the system prompt from about 3k tokens to about 0.7k. That run: 21 calls, 418.8k tokens in, 85% cached.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 4: any provider, and say so when it switches
&lt;/h2&gt;

&lt;p&gt;mew runs Claude, GPT and Codex, DeepSeek, and anything behind an OpenAI-style API. I pick which model fills which role: fast, strong, review, summarise.&lt;/p&gt;

&lt;p&gt;The run in the screenshot was routed to the strong model, which hit a 429 rate limit. mew said so and finished the work on GPT-5.6 Luna. Nothing fails over silently.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 5: the rules are code
&lt;/h2&gt;

&lt;p&gt;"Please don't touch my SSH keys" is a request, and a model can forget a request. Editing a file the agent hasn't read this session is refused, and so is editing one that changed on disk after it was read. File tools can't leave the folder you started in. Shell commands are scanned for paths before they run, and on macOS an OS sandbox sits under all of it.&lt;/p&gt;

&lt;p&gt;Every &lt;code&gt;path:line&lt;/code&gt; in the final answer is checked against your real tree too. A file that doesn't exist goes back for one correction, and anything still unverified shows up as a warning. That's the sentence in an answer people believe fastest, so it's the one worth checking.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I haven't proven
&lt;/h2&gt;

&lt;p&gt;I built this for one person, and I'm that person. Six hours of work that suits how I think doesn't mean it suits everyone, and "better than Claude Code" is my daily experience, not a result. Nobody has run a head-to-head benchmark against other harnesses. There's a bench you can aim at your own tasks and models, and 224 offline tests.&lt;/p&gt;

&lt;p&gt;It also has gaps. There's no plan mode, no image input and no structured JSON output for CI, and the kernel-level sandbox is macOS only.&lt;/p&gt;

&lt;p&gt;Six hours was enough because I already knew what I wanted it to argue about.&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>coding</category>
      <category>programming</category>
    </item>
    <item>
      <title>The bug where nothing was wrong</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/the-bug-where-nothing-was-wrong-gih</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/the-bug-where-nothing-was-wrong-gih</guid>
      <description>&lt;p&gt;Some bugs throw errors. The worst ones don't.&lt;/p&gt;

&lt;p&gt;There's no exception, no alert, no red graph and no failing test. Every service is healthy. An organiser changes their ticket types, and people keep seeing the old ones for up to ten minutes.&lt;/p&gt;

&lt;p&gt;Nothing is wrong. Everything is wrong. Welcome to cache invalidation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: cache the hottest read
&lt;/h2&gt;

&lt;p&gt;Every event page on tunyt needs that event's ticket types. That makes it one of our hottest reads, read far more often than it changes, which is the textbook case for a cache. So we put one in front of it. The pattern is called cache-aside, and every backend has some version of it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;getTicketTypes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;eventId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;`cache:ticket:type:event:&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;eventId&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;hit&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;cacheGet&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;hit&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;hit&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;rows&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ticketType&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;findMany&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;where&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;eventId&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="na"&gt;take&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
  &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;cacheSet&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;ttlSeconds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;600&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Read from the cache. On a miss, read from the database and write the result back.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: ten minutes
&lt;/h2&gt;

&lt;p&gt;The TTL is a trade. Longer means less load on the database and more risk of serving something stale, and shorter means fresher data and more load. Ticket types don't change often, so ten minutes saves a lot of database work.&lt;/p&gt;

&lt;p&gt;The TTL is also our upper bound on how long any staleness bug can live, which matters more than it sounds. Hold that thought. Those ten lines are fine, and they're never where the bug is.&lt;/p&gt;

&lt;h2&gt;
  
  
  The bug is the writer you add later
&lt;/h2&gt;

&lt;p&gt;The first way this breaks is a person, and it shows up months later.&lt;/p&gt;

&lt;p&gt;Somebody adds a new way to change ticket types. A bulk edit, an admin tool, a one-off script. It updates the database correctly and passes its tests. It never touches the cache, because the person writing it had no reason to know the cache existed.&lt;/p&gt;

&lt;p&gt;So the database says one thing and the cache says another, and the cache wins, for up to ten minutes. If what changed was a price or a sold-out flag, that's a support ticket at best. This class of bug is on our own performance audit list, which tells you how normal it is.&lt;/p&gt;

&lt;h2&gt;
  
  
  The race nobody notices
&lt;/h2&gt;

&lt;p&gt;The second way it breaks is subtler, and it lives inside the ten lines. Two requests, almost the same moment:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;A reader misses the cache and reads the ticket types from the database.&lt;/li&gt;
&lt;li&gt;A writer updates the ticket types and deletes the cache key.&lt;/li&gt;
&lt;li&gt;The reader, holding data from before the update, writes it back into the cache.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The cache now holds stale data, and nothing will remove it until the TTL runs out. Every step was correct. The order was the bug.&lt;/p&gt;

&lt;p&gt;You can engineer around this with versioned keys or a short delayed second delete. But it's the honest reason the TTL exists. The TTL isn't there to make data fresh. It's there to put a ceiling on every bug you haven't found yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 3: don't shorten the TTL
&lt;/h2&gt;

&lt;p&gt;The tempting fix for both bugs is a shorter TTL. Make it a minute and the damage is smaller.&lt;/p&gt;

&lt;p&gt;We didn't. The cache exists to take load off the database, and every time you shorten the TTL you hand some of that load back and still serve stale data, just for less time. You'd be paying permanently to shrink a bug instead of removing it. The TTL is the safety net, and it shouldn't be the plan.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 4: every writer invalidates, through one helper
&lt;/h2&gt;

&lt;p&gt;There's one helper that deletes the ticket-type cache for an event, and every code path that changes ticket types calls it. It's a written rule, and it's on the checklist anyone, person or coding agent, runs through before shipping a cache: invalidation wired at every mutation site.&lt;/p&gt;

&lt;p&gt;A helper beats deleting the key inline because a helper is searchable. You can find every caller in seconds, and a code path that changes ticket types without calling it stands out in review.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 5: one shape for every key
&lt;/h2&gt;

&lt;p&gt;Every key looks like &lt;code&gt;cache:{type}:{identifier}&lt;/code&gt;. That sounds cosmetic, and it isn't. Search the codebase for a type and you find every place that reads or writes it. If you can't find all the readers, you can't be sure you've invalidated them.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I review caches now
&lt;/h2&gt;

&lt;p&gt;A cache is a second copy of the truth. Every second copy needs someone keeping it honest, and in code that someone is every function that writes the first copy.&lt;/p&gt;

&lt;p&gt;So when a new cache shows up in a review, I don't start with the read. The read is ten lines and it's always fine. I ask to see every write.&lt;/p&gt;

</description>
      <category>architecture</category>
      <category>backend</category>
      <category>debugging</category>
      <category>performance</category>
    </item>
    <item>
      <title>I trained a language model that writes by filling in blanks</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/i-trained-a-language-model-that-writes-by-filling-in-blanks-2fk8</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/i-trained-a-language-model-that-writes-by-filling-in-blanks-2fk8</guid>
      <description>&lt;p&gt;Every language model you've used writes the same way. Left to right, one token at a time, each word chosen after the one before it.&lt;/p&gt;

&lt;p&gt;I wanted to know what happens if you don't do that. So I trained a small model that starts with a sentence that's entirely blanks, and fills them in.&lt;/p&gt;

&lt;h2&gt;
  
  
  How it writes
&lt;/h2&gt;

&lt;p&gt;The model is a BERT-style transformer, which means it looks at the whole sequence in both directions at once, not just at what came before.&lt;/p&gt;

&lt;p&gt;Generation works like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Start with a sequence where every position is &lt;code&gt;[MASK]&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;The model predicts a token for every masked position at the same time.&lt;/li&gt;
&lt;li&gt;Keep the predictions it's most confident about. Put the mask back on the rest.&lt;/li&gt;
&lt;li&gt;Repeat, until nothing is masked.
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[MASK] [MASK] [MASK] [MASK] [MASK] [MASK]
[MASK] little [MASK] [MASK] [MASK] park .
The little [MASK] went to the park .
The little dog went to the park .

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It's diffusion, the same idea behind image generators, applied to words. Image models start from noise and remove it step by step. This one starts from blanks and fills them in step by step.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: diffusion instead of left to right
&lt;/h2&gt;

&lt;p&gt;The appeal is that nothing forces the model to commit to the first word before it knows the last. It can settle the shape of a sentence early, the subject here and the full stop there, and fill in the middle once the edges make sense. And it fills many positions per step instead of one, which is where the speed promise of diffusion language models comes from.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why it's awkward
&lt;/h2&gt;

&lt;p&gt;This is where it got interesting, and honest.&lt;/p&gt;

&lt;p&gt;Image diffusion works with gentle noise. A pixel can be slightly wrong and get nudged towards right. Words don't work like that. A token is either "dog" or it isn't, and there's no such thing as slightly-dog. So "noise" for text becomes a blunt on or off mask, and a lot of the smoothness that makes diffusion work for images just isn't there.&lt;/p&gt;

&lt;p&gt;Other awkward parts showed up quickly. You have to decide the length of the output before you start, because you're filling a fixed row of blanks. And deciding which predictions to keep at each step matters enormously. Commit too early and the model locks in a mistake it can never undo. Commit too late and it barely makes progress.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: train it on a laptop
&lt;/h2&gt;

&lt;p&gt;I trained it on TinyStories, a dataset of simple short stories, locally on Apple Silicon. No cloud cluster.&lt;/p&gt;

&lt;p&gt;That was deliberate. A small model on a small dataset is something you can actually finish, and finishing is where you learn. You get to watch it train, break it, change one thing and see what happens, all in an afternoon instead of a week of waiting on rented GPUs. It's the same instinct as every other small version of a big thing I've built: shrink it until you can hold all of it, then look closely.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I took from it
&lt;/h2&gt;

&lt;p&gt;Left-to-right generation isn't a law of nature. It's one choice, and seeing the alternative up close makes it much clearer what that choice buys you and what it costs.&lt;/p&gt;

&lt;p&gt;The model writes worse stories than the ones you're used to. It taught me more than any of them.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>machinelearning</category>
      <category>nlp</category>
    </item>
    <item>
      <title>Why I ran a ticketing backend on two Raspberry Pis</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/why-i-ran-a-ticketing-backend-on-two-raspberry-pis-10id</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/why-i-ran-a-ticketing-backend-on-two-raspberry-pis-10id</guid>
      <description>&lt;p&gt;It sounds like a joke: Tixin's backend ran on two Raspberry Pis.&lt;/p&gt;

&lt;p&gt;It also held about 16,000 requests a second in a load test. This is how, and more importantly why I made each choice.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: hardware that's too small on purpose
&lt;/h2&gt;

&lt;p&gt;Tixin was the ticketing pilot that later grew into tunyt. I built its Kubernetes cluster by hand, on two Raspberry Pi 5s.&lt;/p&gt;

&lt;p&gt;On a big cloud instance, a slow query costs you nothing you can see. You have headroom, so you shrug, and the slow query becomes a permanent resident. On two small boards there's no headroom to hide anything in. Every slow query, every chatty service and every lazy default shows up immediately, and the only way past it is to fix it. You can't buy your way out.&lt;/p&gt;

&lt;p&gt;I wanted that pressure, and I'd do it again.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem ticketing actually has
&lt;/h2&gt;

&lt;p&gt;Ticketing traffic isn't a curve. It's a cliff.&lt;/p&gt;

&lt;p&gt;For most of the day almost nothing happens. Then sales open and a lot of people try to do things at the same moment. Most of them are reading: looking at the event, checking prices, refreshing to see what's left. A smaller number are writing, which means actually booking. If reads and writes share one path and one database, they slow each other down at exactly the moment it matters.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: split reads from writes
&lt;/h2&gt;

&lt;p&gt;So Tixin split them. The pattern is called CQRS, command query responsibility segregation, which is a long name for a simple idea: the code that changes things and the code that reads things don't have to share a path.&lt;/p&gt;

&lt;p&gt;Commands, like "book this ticket", go onto Kafka. Queries, like "show me this event", never touch that path and are served from a Redis cache. A person refreshing an event page and a person booking a seat are now doing completely different work on completely different infrastructure.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 3: Kafka on the write path
&lt;/h2&gt;

&lt;p&gt;Kafka is an append-only log, and appending is about the cheapest thing a computer can do. A burst of bookings becomes a burst of appends. The log absorbs the spike, and the work behind it catches up at its own pace instead of falling over at the peak. At peak, the pipeline sustained more than 50K events a second.&lt;/p&gt;

&lt;h2&gt;
  
  
  What CQRS costs you
&lt;/h2&gt;

&lt;p&gt;CQRS gets sold as free. It isn't.&lt;/p&gt;

&lt;p&gt;When reads come from a cache and writes go somewhere else, there's a moment where the two disagree. A booking has happened and the read side hasn't caught up yet. Usually the window is tiny, but in ticketing it forces real questions. What does someone see in that window? Where do you make sure a seat can't be sold twice? On which side does the truth live?&lt;/p&gt;

&lt;p&gt;The pattern doesn't answer those questions. It makes you answer them on purpose instead of by accident, which is most of what good architecture does.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 4: tune the boring parts
&lt;/h2&gt;

&lt;p&gt;Splitting the paths was most of the win. The rest was unglamorous. After tuning PostgreSQL and MongoDB on the booking read path, query latency dropped by 70%. Rebuilding the CI/CD automation cut deploy time by 60%.&lt;/p&gt;

&lt;p&gt;None of that is clever, and all of it was forced by the hardware. On a big instance I'd have shrugged. On a Pi it was the difference between holding and falling over.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 5: refuse to quote the number
&lt;/h2&gt;

&lt;p&gt;This is the decision people find strangest. I don't quote 16K as Tixin's capacity.&lt;/p&gt;

&lt;p&gt;A load test isn't production. I chose the workload, so the result tells you where the limits are under conditions I picked. It says nothing about real people on flaky phones, retries, double-taps on the pay button and everything else real users do.&lt;/p&gt;

&lt;p&gt;So 16K is evidence of one specific thing: the architecture wasn't the bottleneck. That's useful to know. It isn't a promise, and the fastest way to lose a senior engineer's trust is to dress one up as the other.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I took into tunyt
&lt;/h2&gt;

&lt;p&gt;Tixin is retired now, but two things from it run through everything I build.&lt;/p&gt;

&lt;p&gt;One is the habit of measuring first and fixing what the measurements show, and not what you suspect. The other is the pattern itself. I later pulled the read and write split out of Tixin into its own repository, &lt;a href="https://github.com/Adityaadpandey/cqrs" rel="noopener noreferrer"&gt;cqrs&lt;/a&gt;, so it could be read without a product wrapped around it.&lt;/p&gt;

&lt;p&gt;The Pis didn't make Tixin fast. They made it impossible to pretend it was.&lt;/p&gt;

</description>
      <category>architecture</category>
      <category>hardware</category>
      <category>kubernetes</category>
      <category>performance</category>
    </item>
    <item>
      <title>Our traffic grew seven times and we got faster. I don't fully trust that.</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/our-traffic-grew-seven-times-and-we-got-faster-i-dont-fully-trust-that-7eo</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/our-traffic-grew-seven-times-and-we-got-faster-i-dont-fully-trust-that-7eo</guid>
      <description>&lt;p&gt;Here are tunyt's numbers from one stretch of this year:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Weekly requests&lt;/td&gt;
&lt;td&gt;69K&lt;/td&gt;
&lt;td&gt;485K&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Latency&lt;/td&gt;
&lt;td&gt;110 ms&lt;/td&gt;
&lt;td&gt;59 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Apdex&lt;/td&gt;
&lt;td&gt;0.55&lt;/td&gt;
&lt;td&gt;0.89&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Error rate&lt;/td&gt;
&lt;td&gt;2.84%&lt;/td&gt;
&lt;td&gt;0.32%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Deploy time&lt;/td&gt;
&lt;td&gt;20+ min&lt;/td&gt;
&lt;td&gt;under 9 min&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The error rate moved between June and August 2026, while traffic was still climbing.&lt;/p&gt;

&lt;p&gt;Traffic went up about seven times and every other number got better. That's the kind of table that makes a great LinkedIn post, and it's exactly the kind of table I've learned to be suspicious of, my own included. So this post is about how we got there, and what I'm not willing to claim.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: measure before touching anything
&lt;/h2&gt;

&lt;p&gt;Before we changed a line, we made the server tell us how it was doing. A metrics endpoint reports, for every route, request duration at the 50th, 95th and 99th percentile. It also reports how many jobs are waiting in each queue and the state of every circuit breaker.&lt;/p&gt;

&lt;p&gt;You usually already suspect what's slow, and suspicion is how you spend a week optimising the wrong thing. The measurements decide what we fix. What I think is slow doesn't get a vote.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: per route, and in percentiles
&lt;/h2&gt;

&lt;p&gt;An average across the whole API is a comfortable lie. Ninety fast endpoints will hide the one that's slow for every single person who touches it, and that one is usually the endpoint people came for. So every route gets its own numbers.&lt;/p&gt;

&lt;p&gt;Within a route, we watch percentiles. The 50th is your typical user. The 95th is your unlucky one, the person on a slow phone and bad data who decides your app is broken and doesn't come back. The 99th is where the strangest problems hide. If you only watch the average, you're optimising for someone who doesn't exist.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 3: one number for "does it feel fast"
&lt;/h2&gt;

&lt;p&gt;Percentiles are precise but hard to watch day to day. So alongside them we track Apdex, a single score from 0 to 1.&lt;/p&gt;

&lt;p&gt;You pick a target time. Requests faster than it count as satisfied. Requests up to four times slower count as tolerating. Anything slower is frustrated.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Apdex = (satisfied + tolerating / 2) / total

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Going from 0.55 to 0.89 means the bulk of requests moved into satisfied. It's a blunt number, but it's the closest single number I know to "does this feel fast?"&lt;/p&gt;

&lt;h2&gt;
  
  
  What changed in that window
&lt;/h2&gt;

&lt;p&gt;I'll list what we did, and I won't pretend I can tie each number to one change.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Every list query got a ceiling.&lt;/strong&gt; Pagination on endpoints, an explicit &lt;code&gt;take&lt;/code&gt; everywhere else.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Loops of lookups became one query.&lt;/strong&gt; N round trips turned into one &lt;code&gt;IN&lt;/code&gt; query and a &lt;code&gt;Map&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hot reads got a cache with real invalidation.&lt;/strong&gt; Every write path deletes what it changes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Expensive reads got single-flight.&lt;/strong&gt; One request rebuilds a missing value while the rest serve the last good copy, so a cache expiry can't stampede the database.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Analytics moved to rollups.&lt;/strong&gt; Finished days come from a daily table, and only today is computed live.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The database got a budget.&lt;/strong&gt; One connection limit for the whole deployment, split between the API and the workers, with capped fan-out.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hot pages start on the server.&lt;/strong&gt; Event pages fetch their data in parallel on the server and hand it straight to the client's cache, so the browser doesn't wait for a second round of requests.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Decision 4: treat deploy time as reliability
&lt;/h2&gt;

&lt;p&gt;Deploys went from more than twenty minutes to under nine. Docker layer caching meant unchanged dependencies stopped being rebuilt. Lint and tests became parallel jobs instead of one long line. Model weights got baked into the base image instead of downloaded on every build.&lt;/p&gt;

&lt;p&gt;Deploy time quietly controls how you ship. When a fix takes twenty minutes to reach production, people batch changes together, and big batches are harder to review, harder to roll back and more likely to break something. Nine minutes makes small changes cheap again. Deploy time is a reliability metric pretending to be a convenience.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 5: refuse to take full credit
&lt;/h2&gt;

&lt;p&gt;Some of the improvement came from the changes above. Some almost certainly came from things I didn't control: traffic patterns shifting, caches staying warmer at higher volume, effects I never measured.&lt;/p&gt;

&lt;p&gt;If I told you the rollup table saved exactly 30 ms, I'd be making it up. I'd rather lose a little credit than gain a little dishonesty, because the moment someone catches you overclaiming one number, they stop believing the rest.&lt;/p&gt;

&lt;p&gt;What I can say honestly is that the system got faster and more reliable while doing about seven times the work. Every change on that list is now a written rule the team and our agents check before shipping, so the numbers don't quietly slide back.&lt;/p&gt;

&lt;p&gt;The table is real. The story that every number came from a fix I made is the part I won't tell you.&lt;/p&gt;

</description>
      <category>backend</category>
      <category>devops</category>
      <category>performance</category>
      <category>webdev</category>
    </item>
    <item>
      <title>I keep building worse versions of software that already exists</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/i-keep-building-worse-versions-of-software-that-already-exists-1m0i</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/i-keep-building-worse-versions-of-software-that-already-exists-1m0i</guid>
      <description>&lt;p&gt;I've spent weeks building worse versions of software that already exists. On purpose.&lt;/p&gt;

&lt;p&gt;A Redis-compatible server in Go. A load balancer in Bun. A four-service gRPC mesh. An SFU on mediasoup. A small operating system in C.&lt;/p&gt;

&lt;p&gt;None of them will ever replace the original. If you're running production on my Redis, please stop. But each one changed how I make decisions as a CTO, and that's worth explaining.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: rebuild what holds my system up
&lt;/h2&gt;

&lt;p&gt;I don't rebuild things at random. The rule is simple: if a tool is load-bearing for what I run, I want to understand it from the inside.&lt;/p&gt;

&lt;p&gt;Every convenience is rent. What I can't rebuild owns me.&lt;/p&gt;

&lt;p&gt;That's not a rule against using tools. I run Kafka, Redis and Kubernetes in production and wrote none of them. It's a rule about knowing them well enough to answer three questions: what does it cost, where does it break, and what will I do when it does?&lt;/p&gt;

&lt;h2&gt;
  
  
  Redis taught me about TCP
&lt;/h2&gt;

&lt;p&gt;Redis speaks a protocol called RESP, and it's beautifully simple. A line starting with &lt;code&gt;+&lt;/code&gt; is a string, &lt;code&gt;-&lt;/code&gt; is an error, &lt;code&gt;:&lt;/code&gt; is an integer, &lt;code&gt;$&lt;/code&gt; is a blob of bytes with its length up front, and &lt;code&gt;*&lt;/code&gt; is an array.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;*3\r\n$3\r\nSET\r\n$3\r\nkey\r\n$5\r\nvalue\r\n

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's &lt;code&gt;SET key value&lt;/code&gt;. My version speaks it well enough that &lt;code&gt;redis-cli&lt;/code&gt; connects and doesn't notice the difference. I wrote the parser, the storage and the command handling by hand.&lt;/p&gt;

&lt;p&gt;The first real lesson wasn't about Redis at all. TCP gives you a stream of bytes, not messages. One command can arrive in two pieces, or three commands can arrive glued together in one read. Your parser has to handle both, or it works perfectly on your laptop and breaks under load. A whole class of "works locally, fails in production" bugs made sense to me after that.&lt;/p&gt;

&lt;p&gt;The second lesson: the real Redis runs commands one at a time, on a single thread. Write the command loop yourself and that stops looking like a limitation. No locks around the data, no races between commands, very predictable latency.&lt;/p&gt;

&lt;h2&gt;
  
  
  The load balancer taught me what "healthy" means
&lt;/h2&gt;

&lt;p&gt;Configuring a load balancer, you tick "health check" and move on. Building one, you have to decide what healthy means. Is a server that answers &lt;code&gt;/health&lt;/code&gt; instantly but takes four seconds on real requests healthy? How many failures before you stop sending it traffic? How do you bring it back without stampeding it?&lt;/p&gt;

&lt;p&gt;Mine does auto-scaling for Bun backends, active health checks, routing and live monitoring. I learned more about connection handling from it than from a year of configuring someone else's.&lt;/p&gt;

&lt;h2&gt;
  
  
  The SFU taught me why video calls need a middleman
&lt;/h2&gt;

&lt;p&gt;In a naive group video call, everyone sends their video to everyone else. That's a mesh. With five people, each one uploads four copies of their own video. Home internet is usually much worse at uploading than downloading, so the mesh falls over long before you'd expect.&lt;/p&gt;

&lt;p&gt;An SFU, a selective forwarding unit, fixes it. Everyone uploads once, to the server, and the server forwards streams to everyone else. You only really get why every serious video product works this way after you've built the mesh and watched it struggle.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: stop at small
&lt;/h2&gt;

&lt;p&gt;I never try to finish these. A small version is enough to teach the shape of the thing, and the last 10% of a database is years of other people's work I have no business redoing.&lt;/p&gt;

&lt;p&gt;That's a real limitation. A small version hides the hard parts: replication, persistence, failure handling at scale. When I say I understand Redis, I mean I understand why it's shaped this way. I don't mean I could run it better than the people who maintain it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 3: let it change real choices
&lt;/h2&gt;

&lt;p&gt;This is where it pays off.&lt;/p&gt;

&lt;p&gt;When we choose infrastructure for tunyt, I'm not choosing from a feature table. I'm choosing from a picture of how the thing works inside. And when someone proposes adding a new moving part, my first question is what it costs per instance and where it breaks, because I've built enough small versions to know every moving part has an answer to both.&lt;/p&gt;

&lt;p&gt;The other projects in the set:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;go-grpc-micro.&lt;/strong&gt; Account, catalog and order services over gRPC behind a GraphQL gateway, on PostgreSQL and Elasticsearch. Small enough to hold in your head, real enough to break.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;cqrs.&lt;/strong&gt; The read and write split Tixin ran on, extracted into its own repo. I could only pull it out cleanly because I'd built the pieces underneath.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;OsU.&lt;/strong&gt; A small operating system written from scratch in C.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I don't build these to own better software. I build them so the software I depend on stops being a black box.&lt;/p&gt;

</description>
      <category>architecture</category>
      <category>backend</category>
      <category>programming</category>
      <category>softwaredevelopment</category>
    </item>
    <item>
      <title>People said my photo was AI. So I built a camera that can prove it wasn't.</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/people-said-my-photo-was-ai-so-i-built-a-camera-that-can-prove-it-wasnt-2ekm</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/people-said-my-photo-was-ai-so-i-built-a-camera-that-can-prove-it-wasnt-2ekm</guid>
      <description>&lt;p&gt;I took a photo. It was real. A lot of people looked at it and decided it was AI.&lt;/p&gt;

&lt;p&gt;There was nothing I could do. I could say "it's real", and they could say that's what someone faking it would say. A photo used to be its own evidence. That stopped being true, quietly, sometime in the last few years, and I found out the way most people will, by being on the wrong end of it.&lt;/p&gt;

&lt;p&gt;Veris is what I built instead of arguing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: prove it at the source, don't detect it afterwards
&lt;/h2&gt;

&lt;p&gt;The usual answer to fake images is detection. Train a model to spot generated pixels, run it on a suspicious photo and get a score.&lt;/p&gt;

&lt;p&gt;I went the other way, and the reason is in what each approach can actually tell you. A detector gives you a probability, and it's in a permanent race with the generators it's trying to catch. Every time generation gets better, detection gets worse. Even when it works, "87% likely to be real" doesn't end an argument.&lt;/p&gt;

&lt;p&gt;A signature made at the moment of capture is a different kind of answer. It doesn't guess whether a photo looks real. It records that this exact image came out of this specific camera at that moment. So Veris doesn't try to catch fakes. It makes real photos provable.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: sign inside the hardware
&lt;/h2&gt;

&lt;p&gt;The obvious way to sign a photo is in an app, and it's also the weak way. If the signing key lives in software, software can copy it, and anyone who extracts the key can sign anything they like, including a fake.&lt;/p&gt;

&lt;p&gt;So the key lives in a TPM, a small security chip built to create and use keys without ever handing them out. Every photo is signed at capture with a key rooted in that chip. Trust starts in the hardware, and no app that could be changed or copied sits in the middle of it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 3: let anyone check it without trusting me
&lt;/h2&gt;

&lt;p&gt;If proving a photo is real means asking my server, you haven't removed trust. You've moved it to me, and Veris is then only as trustworthy as the company running it.&lt;/p&gt;

&lt;p&gt;So verification lives somewhere nobody controls, me included. A Groth16 zero-knowledge proof anchors the photo's signature on Ethereum as an ERC-1155. Groth16 proofs are small and cheap to verify, which is what you want when the verifying happens on a blockchain. Anyone can check a Veris photo without asking me anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 4: keep the evidence off-chain, but tamper-proof
&lt;/h2&gt;

&lt;p&gt;Putting a photo itself on a blockchain would be absurdly expensive. So the evidence goes to Filecoin and IPFS instead.&lt;/p&gt;

&lt;p&gt;IPFS has a property that suits this well: a file's address is a fingerprint of its contents. Change one pixel and you get a different address. The chain holds the small proof, IPFS holds the evidence, and the link between them can't be quietly swapped.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 5: under a second, or nobody will use it
&lt;/h2&gt;

&lt;p&gt;All of that would be worthless if checking a photo took a minute, because proof that's slow becomes proof nobody bothers with. From capture to an on-chain check, Veris takes under a second.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 6: meet people where their camera already is
&lt;/h2&gt;

&lt;p&gt;Veris works today in three forms: a clip that attaches to a phone, a hot-shoe module that sits on a camera, and a standalone camera. Most photos are taken on phones, and serious photographers won't give up the camera they already trust, so the same signing has to fit both. The standalone camera is the purest version, and it's the one we took to the Google Solution Challenge 2026, where it made the top 14 of more than 85,000 teams.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem nobody has fully solved
&lt;/h2&gt;

&lt;p&gt;Anyone technical will ask, so I'll say it first.&lt;/p&gt;

&lt;p&gt;A camera that signs what it captures will faithfully sign a photo of a screen. Point Veris at a monitor showing a generated image and you get a genuine signature on a picture of a fake. This is the hardest open question for every capture-signing camera, not just mine, and it's where a lot of the interesting work on Veris still is.&lt;/p&gt;

&lt;p&gt;I'd rather say that here than have you find it in a pitch meeting.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where it's going
&lt;/h2&gt;

&lt;p&gt;The code is public, and I'm looking for pre-seed funding to take Veris from a working prototype to something people carry every day. If that's interesting to you, I'd like to talk.&lt;/p&gt;

&lt;p&gt;A detector tells you a photo is probably fake. Veris tells you this exact image came from this camera at that moment. Only one of those holds up in an argument, and I'd know. I lost one.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>hardware</category>
      <category>security</category>
    </item>
    <item>
      <title>I spent a whole night looking for money that was never missing</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/i-spent-a-whole-night-looking-for-money-that-was-never-missing-40m7</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/i-spent-a-whole-night-looking-for-money-that-was-never-missing-40m7</guid>
      <description>&lt;p&gt;I once spent an entire night looking for money that was never missing.&lt;/p&gt;

&lt;p&gt;It was tunyt's second event. By morning I had accounted for every rupee, and the problem I'd chased all night turned out to be hiding inside a fix I'd already made.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: let the webhook drive everything
&lt;/h2&gt;

&lt;p&gt;When someone pays, Razorpay sends our server a webhook, an HTTP request saying the payment succeeded. We made that the trigger for everything after a payment:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;create the ticket,&lt;/li&gt;
&lt;li&gt;send the QR code,&lt;/li&gt;
&lt;li&gt;increase the number of tickets sold,&lt;/li&gt;
&lt;li&gt;add the amount to the organiser's dashboard.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;I'd defend most of that today. A browser can't be trusted to say it paid, because anyone can fake a success screen. The webhook comes from Razorpay's servers to ours, which makes it the one confirmation you can believe.&lt;/p&gt;

&lt;p&gt;The weakness was that it was the only path. One event in, four effects out, and nothing downstream ever checked.&lt;/p&gt;

&lt;h2&gt;
  
  
  The night it broke
&lt;/h2&gt;

&lt;p&gt;Three people bought tickets and never got them. The money left their accounts. On our side there was no ticket, no QR code and no email. I traced it back and found that Razorpay had disabled our webhook. Payments were arriving and our system never heard about them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: people first, then the books
&lt;/h2&gt;

&lt;p&gt;I had two jobs, and they weren't equal. One was to understand what had happened. The other was that three people had paid for an event and had nothing to show at the door. That one had a deadline. The investigation didn't.&lt;/p&gt;

&lt;p&gt;So I turned the webhook back on and emailed all three their QR tickets myself, by hand. They were getting in, whatever else was true.&lt;/p&gt;

&lt;p&gt;I'd make that call again. It's also the call that set up the rest of my night.&lt;/p&gt;

&lt;h2&gt;
  
  
  Then the money didn't add up
&lt;/h2&gt;

&lt;p&gt;While making sure I'd covered everyone, I went through the payments, and Razorpay showed more money than our dashboard did. It looked like three payments had arrived with no tickets behind them at all.&lt;/p&gt;

&lt;p&gt;Three people with no tickets I understood. Three payments with no people was worse. Where had that money come from, and where was it on our side?&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 3: go through it line by line
&lt;/h2&gt;

&lt;p&gt;I knew very little about reconciliation then. I didn't know how much data I could pull straight from Razorpay, or that I could store it and compare it with our own records using a short script. I wasn't good with Excel either.&lt;/p&gt;

&lt;p&gt;So I did the only thing I knew how to do. I exported everything to CSV and went through it by hand. This is the payment ID. This is the order string. Does it exist on our side? Next line. It's slow in the way that makes you start doubting your own spreadsheet.&lt;/p&gt;

&lt;h2&gt;
  
  
  The twist
&lt;/h2&gt;

&lt;p&gt;By morning I had the answer, and it was almost funny. There were no extra payments. They were the same three people.&lt;/p&gt;

&lt;p&gt;When the webhook stopped firing, none of the four steps ran. I'd fixed step 2 by hand, because step 2 is the one people can see. Steps 3 and 4 had never happened. The ticket count didn't go up and the amount never reached the dashboard. The money was all there. Our records just hadn't caught up, and my manual fix had made the customers whole while leaving the books exactly as wrong as before.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a webhook actually promises
&lt;/h2&gt;

&lt;p&gt;A webhook is a notification, not a record. Providers try to deliver them and retry when they fail, but the guarantees are looser than they look. One can arrive late, or twice, or not at all, and it can be switched off on the provider's side without anything in your app noticing.&lt;/p&gt;

&lt;p&gt;A design where one notification triggers four effects, and nothing ever double-checks, doesn't fail loudly. It drifts. Your records slowly stop matching reality, and nobody finds out until the numbers stop adding up.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 4: trust the provider, check ourselves
&lt;/h2&gt;

&lt;p&gt;So we stopped relying on the webhook alone. We pull payment data from Razorpay, store it in our own database, and compare the two with a script. If money arrived and our side doesn't fully reflect it, the script finds it, and neither an organiser squinting at a dashboard nor an attendee at the door has to. The check now runs automatically, with AI in the loop to flag anything that doesn't match.&lt;/p&gt;

&lt;p&gt;If I were designing it today I'd go one step further and make every post-payment step safe to run twice, keyed by the payment ID. A missed webhook would then mean "run it again", and I wouldn't be fixing it by hand at night.&lt;/p&gt;

&lt;p&gt;What I keep from that night is that a manual fix only repairs what you can see. I had completed one step of a four-step transaction, and the other three don't complain when they're skipped.&lt;/p&gt;

&lt;p&gt;The money was there the whole time. What was missing was our record of it, and I'd only repaired the part that hurt.&lt;/p&gt;

</description>
      <category>architecture</category>
      <category>backend</category>
      <category>debugging</category>
    </item>
    <item>
      <title>The obvious fix would have worked. That's why I didn't take it.</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/the-obvious-fix-would-have-worked-thats-why-i-didnt-take-it-1fph</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/the-obvious-fix-would-have-worked-thats-why-i-didnt-take-it-1fph</guid>
      <description>&lt;p&gt;In July 2026 tunyt's database hit its ceiling, and there was an obvious fix sitting right there.&lt;/p&gt;

&lt;p&gt;Find the slowest query. Add an index. Watch the graph drop. Everyone relaxes.&lt;/p&gt;

&lt;p&gt;It probably would have worked, for a while, and that's why I didn't take it. The fix wasn't wrong. It was going to work just well enough to stop us asking the real question.&lt;/p&gt;

&lt;h2&gt;
  
  
  What saturation looks like
&lt;/h2&gt;

&lt;p&gt;Every connection in the pool was busy, and every request that needed one waited in line behind them. Pages that had nothing to do with the cause got slow too, because they were queueing for the same connections.&lt;/p&gt;

&lt;p&gt;That detail matters most. When unrelated pages slow down together, you're not looking at a slow query. You're looking at a shared resource that ran out.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: don't chase the query
&lt;/h2&gt;

&lt;p&gt;Speed up the slowest query and a different one becomes the slowest. Traffic grows, the pool saturates again, and you're back adding another index. It's whack-a-mole, and every round feels like progress.&lt;/p&gt;

&lt;p&gt;So instead of asking which query was slow, I asked why one part of the product could take the whole database down with it.&lt;/p&gt;

&lt;h2&gt;
  
  
  A little maths
&lt;/h2&gt;

&lt;p&gt;Every connection pool obeys one law:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;connections in use ≈ requests per second × seconds each request holds a connection

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's Little's law, and it's the most useful equation I know for backend work. A page that loads 20 times a second and holds a connection for a quarter of a second each time keeps about 5 connections busy, permanently.&lt;/p&gt;

&lt;p&gt;Now make that page run fourteen queries at once.&lt;/p&gt;

&lt;h2&gt;
  
  
  What was actually happening
&lt;/h2&gt;

&lt;p&gt;Our organiser dashboard needed fourteen pieces of data, and the first version fetched all fourteen in parallel. One page load took fourteen connections.&lt;/p&gt;

&lt;p&gt;The results were cached, so most loads should never have reached Postgres. But caches expire. When ours did, every request in that window missed it, and every one of them ran all fourteen queries itself. We saw two identical waves of the same queries hit the database together. That's a cache stampede: the cache protects you right up until it expires, and then everyone rushes the database at once.&lt;/p&gt;

&lt;p&gt;Meanwhile the analytics worker was running its own jobs against the same database, five at a time.&lt;/p&gt;

&lt;p&gt;No single query was the problem. Nothing in the system said how much of the database any one part of it was allowed to use.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: a budget, not a bigger database
&lt;/h2&gt;

&lt;p&gt;The other easy answer was a bigger database, with more connections and more headroom.&lt;/p&gt;

&lt;p&gt;Look at the equation again. Demand grows with traffic. A bigger pool moves the ceiling up, but nothing stops the next fan-out or the next stampede from filling it too. And Postgres connections aren't free, because each one is a whole server process, so you can't keep adding them.&lt;/p&gt;

&lt;p&gt;So we set one number for how many connections the whole deployment may hold, and split it. The API gets one slice and the workers get a smaller one. A busy worker can't starve the API any more, and the API can't starve the workers. "The database is down" becomes, at worst, "the analytics are a bit slow", which is a much better day.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 3: put a ceiling on fan-out
&lt;/h2&gt;

&lt;p&gt;No more &lt;code&gt;Promise.all&lt;/code&gt; over an open-ended list of queries. How many run at once is capped, and no longer depends on how much data a page happens to want.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Run fn over items with at most `limit` in flight at once.&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;mapLimited&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;T&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;R&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;items&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;T&lt;/span&gt;&lt;span class="p"&gt;[],&lt;/span&gt; &lt;span class="nx"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;number&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;fn&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;item&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;T&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;Promise&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;R&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;out&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nb"&gt;Array&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;R&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;items&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;next&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;lanes&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Array&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;length&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;items&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="k"&gt;async &lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;while &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;next&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="nx"&gt;items&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;next&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="nx"&gt;out&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;fn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;items&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;});&lt;/span&gt;
  &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nb"&gt;Promise&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;all&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;lanes&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;out&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Decision 4: one request rebuilds, everyone else waits politely
&lt;/h2&gt;

&lt;p&gt;When the cache misses, the first request takes a short Redis lock and rebuilds the value. Everyone else keeps getting the last good copy meanwhile. There's also a guard inside each process, so a server doesn't race itself for the lock.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Simplified. The real one also stamps an expiry so stale values can be detected.&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;getDashboard&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;eventId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kr"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;`cache:dashboard:&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;eventId&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;cached&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;redis&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;cached&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nf"&gt;isStale&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;cached&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;parse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;cached&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="c1"&gt;// Only one request rebuilds. The rest serve the last good copy.&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;won&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;redis&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;:lock`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;1&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;PX&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="nx"&gt;_000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;NX&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;won&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nx"&gt;cached&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;parse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;cached&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;fresh&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;buildDashboard&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;eventId&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;redis&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;stringify&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;fresh&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;fresh&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The trade-off is that some people see data a few seconds old while it rebuilds. For an analytics dashboard that's an easy trade. Nobody's decision changes because a chart is five seconds behind, and everyone's day changes when the database is down.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decisions 5 to 7: the small ones
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Stop recomputing the past.&lt;/strong&gt; A finished day of analytics never changes, so completed days now come from a daily rollup table. Only today is computed live.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The background never wins.&lt;/strong&gt; The analytics worker went from five concurrent jobs to three. That's slower analytics, on purpose. Background work shouldn't be able to win a fight with the people actually using the product.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Indexes that don't block writes.&lt;/strong&gt; When we do add an index to a busy table, it's built with &lt;code&gt;CREATE INDEX CONCURRENTLY&lt;/code&gt;. It's slower to build, but it doesn't lock writes while it runs. A fix for slow reads shouldn't take down the writes.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why I'd make the slow call again
&lt;/h2&gt;

&lt;p&gt;The index would probably have made the graph go green, and it would have taught the system nothing. The next stampede, or the next dashboard with fourteen queries, would have found the same hole.&lt;/p&gt;

&lt;p&gt;Budgets feel like paperwork until the day they're the reason the rest of the app keeps working while one page misbehaves. Every one of these decisions is now a written rule that the team checks before shipping a list endpoint, a cache or a worker, and our coding agents read the same rules. The budgets live in one table, and changing a number needs a reason in the pull request, because some of those numbers were paid for with this incident.&lt;/p&gt;

&lt;p&gt;The database had no budget, so one page spent all of it.&lt;/p&gt;

</description>
      <category>backend</category>
      <category>database</category>
      <category>performance</category>
      <category>systemdesign</category>
    </item>
    <item>
      <title>We gave users a choice. That was the bug.</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/we-gave-users-a-choice-that-was-the-bug-4d8n</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/we-gave-users-a-choice-that-was-the-bug-4d8n</guid>
      <description>&lt;p&gt;The hardest problem we had on a recent tunyt event had no stack trace.&lt;/p&gt;

&lt;p&gt;Nothing crashed. No alert fired. Every line of code did exactly what we wrote it to do, and people still kept writing in, stuck and confused, asking us questions we couldn't answer for them.&lt;/p&gt;

&lt;p&gt;It was a team event. People registered as teams, and the team had to pay. That sounds like a checkbox. It turned out to be four separate decisions, and the interesting part is how a series of reasonable choices added up to one that wasn't.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: the leader pays
&lt;/h2&gt;

&lt;p&gt;Our first plan was the simplest thing that could work: the team leader pays for everyone. One team, one payment, one receipt.&lt;/p&gt;

&lt;p&gt;In a payment system every extra state is a place for money to get lost, and this design had almost none. A team was either paid or it wasn't. I still think that was right.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: let teams split it too
&lt;/h2&gt;

&lt;p&gt;After a lot of back and forth inside our team, we added a second option. A team could split the payment, so every member pays for their own ticket. It felt friendlier. Nobody has to put up money for four people and chase the others afterwards.&lt;/p&gt;

&lt;p&gt;So we shipped both, and let the team leader pick.&lt;/p&gt;

&lt;p&gt;That last sentence is the bug. I didn't see it yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  What people actually did
&lt;/h2&gt;

&lt;p&gt;Two kinds of chaos showed up, and on the surface they had nothing to do with each other.&lt;/p&gt;

&lt;p&gt;The first was a team that changed shape. A team of three formed and the leader paid for all three. Then they removed one member and added two new ones, and wrote to us asking why they were suddenly being asked to pay for two more people. From their side they'd paid for "the team", and the team had changed. From our side, three tickets had been paid for, one belonged to someone no longer on the team, and two new people had none. The person they'd removed still held a valid ticket of their own, and nobody on the team knew.&lt;/p&gt;

&lt;p&gt;The second was a payment button that never came. Members started writing in to say they had no button, no link and no email. Support asked the same question every time: has your team leader chosen to split the payment, or to pay for the whole team? Almost every time, the answer was that nobody knew.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 3: say no
&lt;/h2&gt;

&lt;p&gt;The first problem pulled us toward building more. If teams change after paying, the proper answer is refunds, ticket transfers, moving a ticket from one person to another, and handling every team that changes its plan halfway through.&lt;/p&gt;

&lt;p&gt;I sat with what that would mean. Each of those is its own small product. Each moves money or ownership, and each has its own edge cases, its own support load and its own new ways to get a payment wrong. We'd be spending weeks to support behaviour that a few teams had stumbled into.&lt;/p&gt;

&lt;p&gt;So we built none of it. We kept the model we had and stopped trying to make every possible team change free. It's the decision I'm surest about, and the hardest to make with someone upset in your inbox. "We won't support that" sounds like failure. Most of the time it's just scope, and a small team that says yes to every edge case ends up with a product made of edge cases.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 4: move the choice
&lt;/h2&gt;

&lt;p&gt;Saying no didn't fix the second problem, and the second problem was the real one.&lt;/p&gt;

&lt;p&gt;When I drew the flow out as states, it became obvious. With the leader choosing at runtime, a team could be in any combination of: no choice made yet, leader pays, or split; paid, partly paid or unpaid; the same members as when it paid, or different ones. We understood every one of those states. The people inside them couldn't see any of them. Every support message was someone stuck in a state the product never showed them.&lt;/p&gt;

&lt;p&gt;So we moved the choice. Now the organiser decides, once per event, how teams pay: either the leader pays for everyone, or it's split and each person pays as they join. Attendees never make the choice, so they can't be confused about what was chosen, and "no choice made yet" stops existing as a state at all.&lt;/p&gt;

&lt;p&gt;We had almost all the logic already. In the end it was mostly a flag on the event:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Simplified. Set by the organiser when they create the event.&lt;/span&gt;
&lt;span class="kd"&gt;type&lt;/span&gt; &lt;span class="nx"&gt;TeamPayment&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;LEADER_PAYS&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;SPLIT&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One more rule closes the gap that started all this. If a team has already paid and someone new joins, that person buys their own ticket. They join as a new member with a ticket of their own and never reopen a payment that's already settled. Money that's settled stays settled.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I took from it
&lt;/h2&gt;

&lt;p&gt;A choice you give users is a state you have to show them. We thought we were adding flexibility, and we were adding a hidden state that only one person on the team could see. How a team pays is the organiser's call, and the moment we let whoever clicked first decide, everyone else inherited a choice they never made.&lt;/p&gt;

&lt;p&gt;Support asking the same question over and over was the product asking it in the wrong place. "Did your leader split or pay?" belonged on the event page, and it never needed to be a support chat.&lt;/p&gt;

&lt;p&gt;Decision 1 was right. Decision 2 was reasonable. Putting decision 2 in the wrong hands was the bug.&lt;/p&gt;

</description>
    </item>
    <item>
      <title>The most dangerous change I ever shipped was a cleanup</title>
      <dc:creator>ADITYA DUTT PANDEY</dc:creator>
      <pubDate>Sat, 10 Oct 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/aditya_duttpandey_b33d2c/the-most-dangerous-change-i-ever-shipped-was-a-cleanup-4e6o</link>
      <guid>https://dev.to/aditya_duttpandey_b33d2c/the-most-dangerous-change-i-ever-shipped-was-a-cleanup-4e6o</guid>
      <description>&lt;p&gt;The most dangerous change I've ever shipped was a cleanup.&lt;/p&gt;

&lt;p&gt;No new feature and no risky migration. One line that made the code tidier. On 1 May 2026 it took tunyt's production down thirty seconds after CI went green.&lt;/p&gt;

&lt;p&gt;The line was mine, and so is this write-up.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 1: put everything that starts in one place
&lt;/h2&gt;

&lt;p&gt;tunyt does a lot of work in the background: notifications, analytics, AI jobs. That work lives in queues, and next to the queues live sweepers. A sweeper is a small loop that keeps a queue healthy. It finds jobs that got stuck, cleans up old ones and nudges things along.&lt;/p&gt;

&lt;p&gt;Our workers all started in one function, &lt;code&gt;startWorker()&lt;/code&gt;. The sweepers started somewhere else, and that bothered me. Two places where background work begins means two places to forget something and two places to read when you're debugging at night.&lt;/p&gt;

&lt;p&gt;So I moved the sweepers into &lt;code&gt;startWorker()&lt;/code&gt;. One line. Lint passed, tests passed, the build passed.&lt;/p&gt;

&lt;p&gt;I'd still defend the instinct, because having one place where things start is good engineering. What I didn't do was count.&lt;/p&gt;

&lt;h2&gt;
  
  
  What that line actually did
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;startWorker()&lt;/code&gt; doesn't just call a function. It gives the work its own worker thread. In Node, a worker thread isn't a lightweight coroutine. It's a separate JavaScript runtime with its own heap and its own event loop, and in our setup its own connections.&lt;/p&gt;

&lt;p&gt;So every sweeper that moved in got the full package. On a single node, all at once, that meant:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;21 V8 runtimes,&lt;/li&gt;
&lt;li&gt;21 thread pools,&lt;/li&gt;
&lt;li&gt;21 Redis connections,&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;booting together on one machine, fighting for the same CPU and memory while the API tried to serve real people. The node didn't slow down gracefully. It just went.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why CI couldn't have caught it
&lt;/h2&gt;

&lt;p&gt;I think about this part the most.&lt;/p&gt;

&lt;p&gt;CI did its job perfectly. It checked that the code was correct. The sweepers still swept, the workers still worked, the types lined up. All true.&lt;/p&gt;

&lt;p&gt;What CI never checks is what happens when you start 21 of something on one box at the same moment. That isn't a correctness question. It's a capacity question, and our pipeline had no opinion about capacity. A green pipeline tells you the code does what you wrote. It can't tell you whether you should have written it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decision 2: threads only for work that needs them
&lt;/h2&gt;

&lt;p&gt;The fix came from asking what sweepers actually do. They're almost pure waiting. Ask Redis something, wait. Ask the database something, wait.&lt;/p&gt;

&lt;p&gt;Node's event loop is built for exactly that. Thousands of things can wait at once on a single thread, each costing almost nothing while it waits. A thread helps when you have CPU work that would otherwise block everything else. For work that mostly waits, a thread buys you nothing and costs you a whole runtime.&lt;/p&gt;

&lt;p&gt;So the sweepers went back in-process, on the main event loop, where they belonged. The rule we kept is that only CPU-heavy work gets its own thread.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Mostly waiting on I/O: stays on the event loop, where waiting is cheap.&lt;/span&gt;
&lt;span class="nf"&gt;startSweepers&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;queues&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="c1"&gt;// CPU-heavy: worth a thread, because it would otherwise block everything else.&lt;/span&gt;
&lt;span class="nf"&gt;startWorker&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;some-cpu-heavy-job&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Decision 3: count before you multiply
&lt;/h2&gt;

&lt;p&gt;Isolation has a price, and you pay it per instance.&lt;/p&gt;

&lt;p&gt;A thread, a runtime, a connection pool, a container: each one costs memory and start-up time on its own, and the cost multiplies by however many you create. One extra worker is nothing. Twenty-one at once, on one node, during a deploy, is an outage.&lt;/p&gt;

&lt;p&gt;So before I split anything into its own worker, process or service, I write down two numbers: what one instance costs, and how many there will be. If I can't answer both, I'm not ready to make the change.&lt;/p&gt;

&lt;h2&gt;
  
  
  What changed after
&lt;/h2&gt;

&lt;p&gt;"Does this multiply something?" is now my question for anything that touches how processes, workers or connections start. It's really a version of the question I ask before shipping anything: if 100K users hit this tonight, what breaks first? On 1 May the 100K was my own code.&lt;/p&gt;

&lt;p&gt;The cleanup was a good idea. Shipping it without counting was the bug.&lt;/p&gt;

</description>
      <category>backend</category>
      <category>production</category>
      <category>refactoring</category>
      <category>softwareengineering</category>
    </item>
  </channel>
</rss>
