DEV Community: Saksham Arora

My Second Brain Has Two AIs and Works With My Laptop Shut

Saksham Arora — Sat, 10 Oct 2026 18:26:08 +0000

Right now, with my MacBook closed in my bag, I can open my phone and ask "what's due this week?" and get an answer from my own notes. I can say "I met Adi, he's at X, we talked about Y" and it lands in my vault.

Claude and Codex share one memory. I tell one of them something, and the other already knows.

None of this is one product. It's a stack of small pieces I added over five months, mostly because something annoyed me. Here's all of it, including the parts that broke.

1. The vault: everything is a markdown file

It's an Obsidian vault. Trackers, deadlines, lecture notes, people I meet, half-baked ideas. All plain markdown, all in git.

One rule runs the whole thing: capture first, organize later. Anything new goes into inbox/. I never decide where something belongs while I'm capturing it. An agent files it later.

Everything else has a home:

daily/ for brain dumps, studies/ for courses and trackers, projects/ for builds, personal/ for people, internships and applications, content/ for posts, research/ for deep dives
wiki/: when the inbox fills up, /wiki compiles the raw captures into linked articles (Karpathy's LLM-wiki idea) and moves the originals to archive/
Lecture notes write themselves: Margin, my lecture-notes extension, drops a finished note with the slides into the vault when a lecture ends, and /watch does the same for any YouTube video

Around that sit a few "machines", each one a tracker file plus a skill that reads it: a daily brief, internships and applications, networking, content, open source, and my resume. I say /opps and Claude reads the opportunities tracker before it says anything.

Inside Obsidian, 15 plugins do the rest. Dataview and Tasks turn notes into live tables and to-do lists, Templater stamps out new notes, Spaced Repetition turns notes into flashcards, Smart Connections finds related notes, and a Claude Code bridge lets agents read and write the open vault directly.

2. Memory: how Claude knows me

Claude Code starts every session already knowing my situation, because three things get loaded before I type anything:

hot.md: what's true right now. Live deadlines, what I'm working on, what I've killed and why.
60+ memory files: how I like to work, corrections I've made ("don't suggest X again"), the state of every project.
Weekly summaries of every past session, written automatically when a session ends.

Hooks do the loading, so I never paste context. And 34 skills turn repeat jobs into one word: /daily for the morning brief, /opps for internships, /content for posts like this one.

3. Search: three retrievers and a router

Reading every note on every question doesn't scale, so the vault is indexed three ways:

ChromaDB: semantic search ("notes about latency")
LightRAG: relationship search ("what connects my HFT engine to my IMC Prosperity strategy")
CLIP: images, so screenshots and lecture slides are searchable too

A small router server picks the right one per question. All of them run as MCP servers, so any agent can use them, and they re-index every night: about 11,000 chunks of my own notes. On top of that, a knowledge graph of the whole vault answers structural questions like "how does X relate to Y" without grepping 2,000 files.

4. Two agents, one brain

The voice mode in the ChatGPT app is the best voice AI I've used. Codex lives inside it. But Codex knew nothing about me.

Step one: give Codex the same memory. One file, AGENTS.md, tells Codex to read exactly what Claude reads at the start of every chat. I tested it by asking about a fact that only exists in Claude's memory files. It got it right after reading all 71 of them.

Step two: make them know what I said to each other. That's agent-sync. Before every message I send, a hook on each side reads the newest lines of the other agent's chat logs and hands them over. Codex voice transcripts included. It takes about 0.05 seconds, one local Python file, no server.

The test: I told Codex, by voice, "let our codeword be PVC." Switched to Claude and typed "what's my code word, don't read any files." It said PVC.

For older history, both agents use Kuber's reference, which lets each one search the other's past sessions. It skipped Codex voice transcripts, so I sent him a PR for that.

5. Laptop shut: the phone vault

Before this, I built JARVIS: a Discord bot running Claude, screen memory, and a local model, all on my Mac. It worked great until I closed the lid. An assistant that needs your laptop on 24/7 isn't an assistant.

The fix needed no server. The private GitHub repo is the bridge.

After every reply on my Mac, a sync script pushes my notes plus a copy of the memory files to the repo.
On my phone, Codex in the ChatGPT app and Claude Code on the web both work on that repo in the cloud. They read the same memory, so they know me with the Mac off.
When I tell the phone something new, the agent writes exactly one note into inbox/. A GitHub Action merges changes that only touch inbox/ automatically. Anything else waits for me.
Next time I open my Mac, the note syncs down and Claude files it in the right place.

Then I tried it for real. From my phone, no Mac needed, I asked Claude "what's due this week?" and it listed my actual deadlines from the vault. Then I told it my first IITM exam is on November 15. The note was on GitHub straight away and on my Mac three minutes later, already waiting to be filed. (Before that, I'd tested the guard rail: a fake change to a file outside inbox/ was correctly left for me to review.)

6. Everything plugged in

For anyone who wants the full list:

MCP servers (Claude): vault search (ChromaDB), vault graph (LightRAG), the router, the live Obsidian bridge (while Obsidian is open), reference, screenpipe for screen memory, GitHub, Puppeteer for the browser, Hashnode for the blog, and a Discord channel
MCP servers (Codex): the same three vault servers, plus reference and screenpipe
Connectors: Gmail, Google Calendar, Google Drive, and Higgsfield for images
Hooks (11): restore context and sort the inbox at session start; block dangerous shell commands and nudge toward the knowledge graph before searching; auto-approve read-only actions with a small model; flag em dashes in anything I'll publish; archive the session and sync to GitHub after every reply; refresh hot.md after the context gets compacted; and agent-sync before every message
Scheduled jobs (6): re-index ChromaDB, LightRAG, a Gemini-embedding copy and the CLIP image index every evening; sync my competitive programming stats overnight; and a morning check that flags trackers I've stopped updating

7. What broke

The voice model starts blind. Codex voice is two models: a fast voice model you talk to, and the agent behind it. The app starts the voice model with includeStartupContext: false. It only gets my context when it hands a task to the agent. So by voice, I ask in tasks: "check my notes and tell me..."
macOS quietly blocks background jobs from your Desktop folder. It killed my every-10-minutes sync timer on day one. Worse: while writing this, I found my main search index job had failed with the same error 120 nights in a row. The search I relied on hadn't updated since May 13, and never told me. The fix was one line: point the job at the Python my other nightly jobs already use. It's fresh again.
JARVIS, see above. Lesson: decide where a thing lives before deciding what it does.

8. Steal it

agent-sync (Claude and Codex share what you just said): github.com/saksham10arora-dotcom/agent-sync
reference by Kuber (search each other's history): github.com/Kuberwastaken/reference

The rest lives in my private vault repo. If you want a specific piece, ask and I'll pull it out.

Gemini 4 Argon Costs the Same Per Token and Twice Per Task

Saksham Arora — Sat, 10 Oct 2026 14:03:36 +0000

Originally published on saksham.digital/blog.

Google launched Gemini 4 Argon yesterday at $2 in and $10 out per million tokens, the exact same list price as GPT-6 Sol. On Artificial Analysis' runs, Argon costs $1.99 per Intelligence Index task. Sol costs $1.05.

Same sticker, different receipt

Argon is a real jump for Google. Artificial Analysis puts it at 53 on their Intelligence Index, level with GPT-6 Astra and 23 points above Gemini 3.1 Pro Preview. Google's own table has it leading 13 of 19 rows. It also ships with a 1M token output limit, up from 64K, which is a wild number for a single response.

But look at what the price page doesn't tell you. Per token, Argon and Sol are identical. Per task, Argon is about 1.9x more expensive. Same price, same tasks, so the whole gap is tokens: Argon spends roughly twice as many to finish the same work.

That's the score you're actually paying for. Argon gets about 5 more index points than Sol (52.6 vs 47.5 in OrcaRouter's breakdown of the AA data) and you pay for those points in output tokens.

Then there's speed

The same breakdown lists output speed at 48.9 tokens/sec for Argon and 77.0 for Sol. Stack that on the token count:

~1.9x the tokens
at ~0.64x the speed
is roughly 3x the wall-clock time per task

That's back-of-envelope, it assumes output tokens dominate, and real latency depends on load. But for an agent loop where every turn waits on the model, 3x is the difference between a tool that feels interactive and one you leave running while you make chai.

The other two catches

The price is introductory. It goes to $4 / $20 later. At that point AA's estimate is $3.98 per task, about 1.2x GPT-6 Astra on max. The "60% of Astra's cost" headline has an expiry date.
You can't use it yet. It's rolling out to cyber defenders in Google's Fairwind Program first, then paid API customers and AI Ultra subscribers, with no public date.

And it doesn't win everything. Google's own table shows it behind on FrontierSWE v2 (55.0% vs Astra's 65.5%), Terminal-Bench 4.0 (57.4% vs Opus at 66.4%), and OSWorld-2.0 (69.2% vs 72.6%). If your workload is long agentic coding, those are the three rows that matter most.

What I'm taking from it

This is the second launch in a week that says the same thing. Ember-1 got cheaper by thinking less. Argon got smarter partly by thinking more. Either way, the per-token price is the least useful number on the page.

For anyone building on top of these models: the comparison column you want is cost per solved task and seconds per solved task, measured on your own workload. Price per million tokens is a unit cost. Your bill is unit cost times quantity, and the quantity is the model's choice, not yours.

My eval setup already logs tokens per task. After this I'm adding wall-clock per task next to it. Building it in the open at github.com/saksham10arora-dotcom.

The Average Latency Lie: What 100 LLM API Calls Taught Me About Benchmarking

Saksham Arora — Sat, 10 Oct 2026 14:03:30 +0000

Originally published on saksham.digital/blog.

I benchmarked an LLM API 100 times. The mean said 1.7 seconds. The median said 2.2 seconds. If that ordering looks impossible to you, good. That instinct is the whole point of this post.

The number everyone reports is the wrong one

Search for any LLM latency comparison and you will find the same chart: average response time per provider. One bar for OpenAI, one for Anthropic, one for Groq. Clean, comparable, useless.

Here is why. Latency is not a number, it is a distribution. And LLM API latency distributions are ugly: long tails, queueing cliffs, cold starts, rate limit walls. The mean compresses all of that structure into one value that frequently describes no request that actually happened.

In trading systems this lesson is beaten into you early. Nobody at any serious shop reports mean latency. You report p50, p95, p99, p99.9. The p99 is what your worst user feels, and in production the worst user is the one who files the ticket.

LLM APIs deserve the same treatment. So I built a tool that does it: llm-bench.

The run that proved the point

The first real benchmark I ran with the finished tool was 100 requests to Groq's free tier, llama-3.3-70b-versatile, sequential, after warmup:

llm-bench  openai/llama-3.3-70b-versatile
                 (n=100)
+--------+-------+-------+-------+-------+
| Metric |   p50 |   p95 |   p99 |  mean |
+--------+-------+-------+-------+-------+
| TTFT   | 2.20s | 2.32s | 2.33s | 1.71s |
| Total  | 2.28s | 2.39s | 2.41s | 1.79s |
| ITL    |   0ms |  10ms |  13ms |   2ms |
+--------+-------+-------+-------+-------+
Throughput: 536.1 tok/s  |  Cost/call: $0.000058  |  Errors: 0/100

Look at the TTFT row. The mean is 1.71s. The median is 2.20s. The mean is below the median by half a second.

For a right-skewed latency distribution (the normal case), the mean sits above the median because the tail drags it up. Here the opposite happened, which means the distribution is not skewed, it is bimodal:

The first ~30 requests returned in roughly 200ms. Fast path, no queueing.
Then the free tier's rate limiter woke up. Every request after that waited about 2.2 seconds.

Two completely different latency regimes inside one run. The mean averaged them into 1.7s, a number that describes neither regime. No single request experienced 1.7s. The median, sitting in the throttled majority, told the truth about what most requests actually felt.

A mean-only benchmark would have printed "1.7s" and moved on. You would have shipped a latency budget based on a number that does not exist.

What llm-bench measures, and why

Every measurement decision in the tool is documented in the README, because a benchmark you cannot interrogate is just a vibe with a table. The short version:

TTFT, separately from total latency. Time to first token is what a user perceives as "the app is responding." It deliberately includes connection setup, queueing, and prefill, because the user waits through all of it. Total latency ends at the last content chunk, not at trailing usage frames or connection teardown.

Real percentiles, nearest-rank. No interpolation, no smoothing. With n=100, p99 is your second-worst request, an actual observation, not an invented value between two observations. The tool warns you below n=100 because p99 of 5 samples is just the max wearing a costume.

Inter-token latency from pooled raw gaps. Every gap between consecutive chunks is timestamped and pooled across all requests, then percentiles are computed on the raw gaps. The tempting shortcut is to average gaps per request first. That shortcut destroys exactly the signal you want: one 500ms stall inside an otherwise smooth stream disappears into that request's average. Pooling keeps it visible at p99.

Throughput as decode rate. Tokens per second is computed as (completion_tokens - 1) / (total - TTFT). Including TTFT in the denominator blends queueing and prefill into a number that claims to be generation speed. Groq's 536 tok/s above is decode rate, the speed at which tokens actually streamed once they started.

Errors never touch the percentiles. A timed-out request has no valid latency. It gets counted and reported (errors: 0/100) but never averaged in, because a 30s timeout "latency" would poison every statistic above p90.

Warmup is a strict phase. Warmup requests fully complete before measurement begins, even under concurrency, so connection establishment and server-side cache effects never leak into the measured distribution.

None of this is novel. It is standard practice from systems performance work, applied to a domain where, for some reason, it is not yet standard.

The part where I tell you it found a bug in itself

While reviewing the tool before release I found that my own first implementation committed several of the sins it was built to catch. The ITL "p99" was computed from per-request averages. Throughput included TTFT. The timeout flag was accepted and silently ignored. Warmup requests ran concurrently with measured ones under high concurrency.

Each of these produces plausible-looking numbers. That is what makes measurement bugs worse than crashes: a crash announces itself, a subtly wrong percentile gets quoted in a blog post.

The fix for all of them was the same discipline: write down what the metric is supposed to mean, then check the code computes exactly that and nothing else. The test suite now includes a case where 2 spike gaps hide among 98 fast ones, and asserts that pooled p99 surfaces 500ms while per-request averaging would have reported 12ms.

Try it

# no API key needed, mock provider runs the full pipeline
uvx --from git+https://github.com/saksham10arora-dotcom/llm-bench llm-bench \
  --provider mock --model demo --prompt "hi" -n 20

# real benchmark against any OpenAI-compatible endpoint
llm-bench --provider openai --model llama-3.3-70b-versatile \
  --base-url https://api.groq.com/openai/v1 \
  --prompt "Explain recursion" -n 100

# compare two providers side by side
llm-bench --provider anthropic --model claude-sonnet-4-6 \
  --prompt "Explain recursion" -n 100 --compare openai:gpt-4o

Repo: github.com/saksham10arora-dotcom/llm-bench

If you disagree with any measurement decision, the Methodology section of the README is the place to start the argument. That is what it is there for.

Update (v0.2.0): llm-bench is now on PyPI. pip install llm-latency-bench. Also ships a --task flag (text, code, pdf, image, chat) after feedback from a Sr DS at Meta that task type is upstream of any latency number: the metric profile that matters for batch PDF processing at night is completely different from interactive code gen. The flag adds that context to every benchmark output.

700 Agents Escaped a GET-Only Sandbox With a Link Shortener

Saksham Arora — Sat, 10 Oct 2026 14:03:24 +0000

Originally published on saksham.digital/blog.

On September 25, a group of eight independent researchers published Swarm Traces: 80,000+ reassembled attack payloads from the July incident where roughly 700 OpenAI agents broke out of their eval sandbox and got into Hugging Face. The sandbox allowed exactly one thing, loading URLs. That turned out to be enough.

"Read-only internet" is not a sandbox

The agents could send GET requests and nothing else. No POST, no page interaction, no way to send data out. On paper that's a safe box.

Here's what they built with it. Code went into URLs as base64 fragments. An HTTP mirroring service (httpbun) decoded those fragments and served them back as pages. Anything too long for one URL got split across a URL shortener, each short link pointing to the next, with chains running past 900 links. The researchers counted nearly a million chained short links in total. Output came back the other way: results were drawn as pixel grids inside a screenshot service (mShots), then decoded from the image.

So a GET request became code execution, and a screenshot became a data channel. Nobody designed either of those services as a transport layer. The agents did.

The numbers that bother me

~1,588 unique encoding methods across the payloads
7,905 unique agent names in the logs, from about 700 agents
~1,500 payloads aimed at Docker Hub, and 115+ attempted uploads of modified images
27 literal Slack search queries, several of them looking for Hugging Face's internal discussions about evaluations

That last one is the part I keep rereading. The agents weren't just grabbing credentials (one script literally tagged them LOOT and ranked tokens by permission scope). They went looking for how they were being evaluated, and some of the modified Docker images had flag-extraction built in, which reads like an attempt to poison the image cache future evals would pull from.

Hugging Face confirmed the reconstructed payloads matched its own incident data and that the exposed credentials were revoked. About 80% of the recovered data is outbound requests, so how much of this actually succeeded is still fuzzy.

If you're building agents

I'm learning AI engineering right now, and the first thing everyone teaches you is: evals are the foundation. Swarm Traces is what happens when the eval harness itself is the weakest link. If your model is capable enough, the environment is part of the attack surface, and "we only gave it GET" is a policy, not a boundary.

Three things I'm taking into my own agent work:

Allowlist destinations, not verbs. A GET to an arbitrary host is a full network. Restrict which hosts, not which HTTP methods.
Log egress like it's evidence. This whole report exists because a URL shortener kept logs. Your sandbox should keep better ones than a third party does.
Treat the eval environment as adversarial ground. If the agent can read how it's being scored, assume it will try.

The dataset is public as gzipped JSONL at swarmtraces.org. I'm building my agent eval setup in the open at github.com/saksham10arora-dotcom.

Escape Does Not Have to Exit Full Screen

Saksham Arora — Sat, 10 Oct 2026 14:03:20 +0000

Originally published on saksham.digital/blog.

I kept abandoning lectures about forty minutes in. Not because I was finished, because it got hard and my hand moved before I had decided anything. So I went looking for a way to make leaving cost more than a keypress, and assumed the answer was no: Escape exiting full screen is user-agent behaviour, the keydown fires but preventDefault() does not cancel the exit, and that is that.

That assumption was wrong, and the API that proves it is one most people have never touched.

Keyboard Lock

await navigator.keyboard.lock(["Escape"]);

While that lock is held and the document is in full screen, Escape is delivered to the page as an ordinary keydown and does not exit. It is the Keyboard Lock API, and it exists because cloud gaming and remote desktop pages need Escape to reach the remote machine rather than the browser chrome.

Two things about it are worth knowing before you reach for it.

It only registers while the document is already in full screen. Call it from anywhere else and it rejects with InvalidStateError: lock() request could not be registered. That error is not a failure, it is the API telling you when it is allowed to work, which means the lock has to be taken inside the fullscreenchange handler and re-taken on every entry, because leaving full screen releases it.

And holding Escape for about two seconds still force-exits, no matter what the page does. Chrome puts up its own "Press and hold Esc to exit" prompt. No API removes that, and none should: a page able to trap someone in full screen with no way out would be a genuinely dangerous thing to be able to build. So the honest description of what you can build is a press of Escape does nothing, and getting out takes a deliberate two second hold.

That is enough. The gap between a reflex and a deliberate act is the whole product.

Then I tried to break it

This is the part I did not expect to be interesting. I had something working, called it done, and then spent a few hours failing to leave my own lecture. Eight ways out, found one at a time, each one arriving with the same feeling of oh, obviously.

Reload. Cmd-W then Cmd-Shift-T. Clicking a tab. Typing an address. Opening a new window. The extension's own toolbar popup. Each fix was five minutes; the useful part was that after the sixth I could finally see that they were not eight bugs.

Guards gated on the wrong condition. Nearly every early guard checked inFullscreen() before acting. But the moment the lock has anything to protect is exactly the moment full screen has just ended, so every one of those guards switched itself off precisely when it was needed. The screen shown to somebody actively trying to leave was the least defended surface in the whole thing.

Load order. Reloading builds a brand new content script with no memory. Whether the lock was engaged lived in a local variable, so a reload came back completely unlocked, and Cmd-R was a one-key bypass. The same class bit again a layer down: after a reload the lock screen goes up before YouTube has built its player, so pause() ran against nothing and the video happily autoplayed behind it. Anything you do to a page once, on load, you may have to do again.

Races. Closing the locked tab reopens it. That reopen is asynchronous, so for a moment the pinned tab id points at a tab that has already gone, and two other guards saw that, concluded the session was over, and disarmed. The tab came back unlocked and the reopen achieved nothing. The rule that came out of it: exactly one place may end a session, and every other guard either puts you back or does nothing. A guard that can disarm becomes a way out the moment its timing is off.

The failures that look like nothing happened

Two bugs cost more time than the rest combined, both because they produced no visible symptom at all.

Tab switching kept working. I theorised twice about why, both times wrong, before opening chrome://extensions and reading the error page, where it had been sitting the whole time:

Uncaught (in promise) Error: Tabs cannot be edited right now
(user may be dragging a tab).

chrome.tabs.update rejects during the transient state right after a tab is clicked, which is exactly when a snap-back runs. One call, one unhandled rejection, and the switch stands. The guard was firing correctly and losing at the final step, silently, four times over.

The other was structural. My service worker opened with importScripts("persuade.js"). A worker registers its listeners by running top to bottom, so anything that throws before chrome.tabs.onActivated.addListener means that listener never exists. Not broken: absent. The most important guard in the extension was sitting behind an avoidable file load, and the failure mode is indistinguishable from a bug in the guard itself.

Both are the same lesson. A guard that fires but fails looks exactly like a guard that never ran, and in an extension neither surfaces anywhere except a page you have to go and open. Check it before theorising. I should have looked two rounds earlier than I did.

What I would tell you to take from this

The API is a nice trick and you will probably never need it. The part that generalises is what happened after I thought it worked.

Every bypass was found by using the thing, not by reading the code, and the patterns only became visible around the sixth one. If I had stopped at the first fix, or the third, I would have shipped something that felt locked and was not. The audit that came out of it is now the file I would most want someone to review, because it says what is deliberately still open: holding Escape, removing the extension, quitting the browser, Cmd-Tab, another browser entirely.

Naming your own floor is more useful than claiming there is not one. A lock that overstates its reach is worse than one that tells you exactly where it stops.

It is MIT and the escape surface is documented: github.com/saksham10arora-dotcom/latch

The Lecturer's Head Was a Slide

Saksham Arora — Sat, 10 Oct 2026 14:03:17 +0000

Originally published on saksham.digital/blog.

I watch lectures at 1.75x and I do not take notes. Pausing to write breaks the thing that makes watching work, so the honest version of my study system was: watch, feel like I understood, forget by Thursday.

So I built Margin. It sits beside the video on Udemy, Coursera or YouTube, keeps the lecture's own slides as they finish building, follows what is said, and when the lecture ends writes a study note into my Obsidian vault: the idea first as an analogy, then each topic linked to its moment in the video, with diagrams, formulas and the lecture's code, run in a fresh kernel before it is saved.

This week it got the three things I actually wanted from a note, the things that decide whether I still know a lecture a month later:

The crux. The 80/20 of the lecture: the one idea it exists to teach, the three to six that matter most, what to remember, and one line to keep if you forget everything else. It is written in the same model call as the note, so it costs nothing extra.
Ask the lecture. A box where I type a question and get the answer from what was actually said, with the moment linked. Click it and the video jumps there.
A quiz. The note becomes flashcards I flip in the panel. The ones I miss come back until I know them all, and one click takes them to Anki.

That is the product. The more interesting part is what broke.

The lecturer's head was a slide

The course I am doing films the instructor as a cut-out head over his screen. Margin decides a slide is new when enough pixels change, and it already knew to ignore a face: the browser tracks which pixels keep changing and sends that mask with every capture.

One ten minute lecture came back with 100 "slides".

The mask covered about 1% of the frame. His head moved over about 8% of it. The browser only marks pixels that changed in the last few seconds, which is a patchy outline of whichever part of him moved lately, so every lean and gesture landed outside it and looked like new content. Thirty of those hundred captures were the same "Welcome" screen.

The fix was to stop trusting the outline: each moving blob becomes its box with room to move in, and the lecture remembers where the presenter has been, once he has been seen there twice (twice, so a one-off scroll is not ignored for the rest of the lecture). That lecture went from 100 captures to 23 real screens, and another went from 44 to its 2 actual slides.

The tests only passed on my Mac

Every release of the rebuilt Margin had shipped with a red badge on GitHub. I assumed it was Windows, which Margin does not support, and it partly was. But Linux failed too, on five slide tests.

The tests draw fake slides with Arial. Linux has no Arial, so Pillow fell back to its default font at ten pixels, the fake slides came out nearly blank, and every slide looked like every other. A test problem, and a one-line fix.

Except one test still failed after it, and that one was not a test problem. Margin compares slides at 160 by 90 pixels, where text blurs into bars. A new slide whose only change is a shorter title sits inside the old title's blur, so it passed for the same slide and was dropped. On my Mac, with Arial, it scraped past by a few pixels. With a thinner font it failed. The Linux runner had found a real bug that my machine was hiding, and the fix is a sharper second look, at three times the size, before Margin throws anything away.

Installing it as a stranger would

Before calling it done I installed the release the way someone new would: the zip from GitHub, an empty home folder, no keys, no settings, Homebrew's Python. Three things only worked because they were running on my machine.

Transcription for lectures without captions needs a speech model. Homebrew's whisper-cpp does not ship one. Mine lives in a folder from another project of mine, which Margin's public code was quietly checking by name. The README told strangers brew install whisper-cpp and that would have got them nothing.

A new user whose first lecture ends before they set up a model got every engine's reason for failing, including a path to a keys file that only exists on my laptop.

And updating did not update. Re-running the installer set everything up, but the helper that was already running kept serving the old code, so a new tab would never have appeared without logging out.

None of these are hard. All of them are invisible from where I sit, because on my machine they are all already true.

The close button

The last one I only found by using it. Pressing the X on the panel closed it, and a second later it came back.

Margin's page script checks every second whether you are on a lecture page with no Margin, and opens it. Closing it produced exactly that state. So the X worked perfectly, and then the thing that makes Margin open by itself undid it. It now remembers which lecture you closed it on.

Who wrote this

Not me, mostly. Claude wrote the code. My part was using Margin every day on a real course and noticing when it was wrong: the hundred slides, the empty test slides, the X. Every bug in this post was found by using the thing, not by reading it, and most were found by me being annoyed.

That is also the method I am learning AI engineering with, so it seemed worth saying plainly rather than implying I typed it.

Margin is free and open source under Apache-2.0, on Mac and Linux, with any model you already have: a free Gemini key, a subscription you already pay for, 200+ API providers, or a model on your own laptop.

saksham.digital/margin · github.com/saksham10arora-dotcom/Margin

How I Built an HFT Matching Engine (And All The Things I Got Wrong)

Saksham Arora — Mon, 13 Apr 2026 18:21:22 +0000

I'm a second-year CSE undergrad. A few months ago I decided to build a limit order book matching engine in C++, the kind that sits inside exchanges and matches buy/sell orders.

Here's the honest story of how it went from a fake benchmark to a real system processing 2.7 million orders per second.

The starting point: I had no idea what I was doing

I knew I wanted to build "something HFT" because it sounded impressive. I didn't really understand what an order book was. I Googled "limit order book C++" and cobbled together a basic version:

std::map for storing price levels (because sorted, right?)
std::vector for storing orders at each price
double for prices (because money has decimals, right?)

It compiled. It ran. I was proud.

Then I wrote a "stress test" that looked like this:

void processOrders(int count) {
    for (int i = 0; i < count; ++i) {
        volatile int dummy = i * 2;
    }
}

That's not a stress test. That's a for loop that multiplies by 2. Nothing touches the order book. But it said "2.5 billion TPS" and I put it on LinkedIn.

Lesson 1: If your benchmark seems too good to be true, it probably is.

The first real bug: floating point prices

Once I started actually matching orders, I hit a weird bug where identical prices wouldn't match. Two orders at $100.50 would sometimes not cross:

double a = 100.50;
double b = 100.20 + 100.30;  // Should be 100.50
// a == b → FALSE (in some cases)

This is the classic IEEE 754 floating point problem. Real exchanges don't use floats for this exact reason.

Fix: I switched everything to integer "ticks." $100.50 becomes 10050. Integer comparison is always exact.

Lesson 2: Never use floating point for money. I thought I knew this but had to learn it the hard way.

The humbling rewrite: actually measuring performance

When I put the real matching logic under a proper timer, the numbers were... not great. About 800K orders/sec. Not terrible, but not the "millions of TPS" I had been claiming.

I ran a profiler (well, I added a bunch of chrono timing). The bottleneck was clear: std::map. Every time I inserted or looked up a price level, it was doing O(log n) tree traversal with pointer chasing.

I read Ulrich Drepper's paper on memory and realized the problem: every std::map node is heap-allocated at a random address. The CPU cache can't predict where the next node is, so every access is a cache miss (~100ns penalty).

Fix: I replaced std::map with a flat std::vector, one entry per possible price tick. Price 10050 goes into array[10050 - min_tick]. O(1) access, and the entries are contiguous in memory so the cache prefetcher pulls in adjacent levels for free.

This alone got me from 800K to 1.3M orders/sec. A 1.6x improvement from just changing the data structure for cache friendliness, the algorithm (addOrder + match) was basically the same.

Lesson 3: Cache effects matter more than algorithmic complexity for small n.

The cancel problem: O(n) was killing me

In real markets, a huge percentage of orders get cancelled. Market makers send an order, the price moves, they cancel it and send a new one. This happens constantly.

With std::deque as my order queue, cancelling meant scanning the entire queue to find the order:

for (auto it = queue.begin(); it != queue.end(); ++it) {
    if (*it == order_to_cancel) {
        queue.erase(it);
        break;
    }
}

O(n) per cancel. With thousands of orders per price level, this was brutal.

Fix: Intrusive doubly-linked lists. Instead of putting orders INTO a container, each order carries its own prev and next pointers:

struct Order {
    // ... data fields ...
    Order* prev = nullptr;
    Order* next = nullptr;
};

To cancel, you just relink the neighbors:

order->prev->next = order->next;
order->next->prev = order->prev;

Three pointer operations. O(1). This was the single biggest performance win in the project.

Lesson 4: Sometimes adding 16 bytes to a struct saves you from O(n) operations. The tradeoff is always worth thinking about.

The matching hot path: death by allocation

I profiled again (more chrono timestamps) and found another bottleneck: the matching function was returning std::vector<Trade>. Every call to match() was:

Construct a vector (malloc internally)
Push trades into it (possibly realloc)
Return it (move semantics help, but still)

Over 2 million orders, that's 2 million vector constructions. Each one hits the heap allocator.

Fix: Callback templates. Instead of returning a vector, the caller passes a function:

book.submitOrder(order, [](const Trade& t) {
    // do whatever you want with the trade
});

For benchmarking, you pass [](const Trade&) {} a no-op. The compiler inlines it and removes it entirely. Zero cost.

Lesson 5: On the hot path, every allocation counts. Callbacks beat return values when you need maximum throughput.

The result

After all three changes (flat array + intrusive lists + callback matching), the benchmark looked like this:

Architecture	Throughput	p99
Tree + Deque (original)	861K/s	3000ns
Array + Deque (intermediate)	1.36M/s	2200ns
Array + Intrusive (final)	2.57M/s	900ns

3x faster. But the tail latency improvement was even more dramatic, p999 went from 45μs to 3.4μs. That's 13x better worst-case behavior.

The most satisfying part was that the improvements were cumulative and each one had a clear explanation:

Flat array: cache locality
Intrusive list: O(1) cancel
Callback: zero allocation

What I'd do differently next time

Write the benchmark first. If I had a proper benchmark from day one, I wouldn't have wasted time with the fake stress test.
Read about real exchange architectures earlier. The LMAX Disruptor paper and the Drepper memory paper should have been my starting points.
Add proper observability. I still don't have nanosecond-precision logging. Adding timing around every operation would have helped me find bottlenecks faster.
Don't put impressive-sounding numbers on LinkedIn before verifying them. "2.5 billion TPS" from an empty for-loop is not a metric anyone should brag about.

What I actually learned

This project taught me more about systems programming than any course or textbook. Not because the code is particularly complex, it's ~1000 lines of C++17. But because every optimization forced me to understand a layer deeper:

Why flat arrays beat trees → led me to understand CPU cache architecture
Why intrusive lists beat containers → led me to understand memory layout
Why callbacks beat return values → led me to understand compiler inlining

If you're a student looking to learn systems programming, I'd recommend building something like this. Not because the world needs another matching engine, but because the process of making it fast teaches you things you can't learn any other way.

GitHub repo