DEV Community: Whetlan

I Asked an LLM to Generate 20 Trading Strategies. 14 Were the Same Thing.

Whetlan — Tue, 21 Apr 2026 10:42:18 +0000

A few months ago I asked an LLM to generate twenty trading strategies.

Fourteen were the same thing.

Not similar ideas. Not variations on a theme. The same mean-reversion logic with different lookback windows and parameter names.

I gave it historical price data, told it to find patterns, output entry/exit rules in Python. Ten minutes later I had twenty strategies. Clean code, proper docstrings, sensible-looking parameters.

I backtested all twenty. Twelve looked profitable. Some showed 200%+ annual returns.

Then I actually read the code.

Same structure. Same assumptions. Same failure mode: in a trending market, they'd all keep buying into a falling asset with no awareness anything had changed.

That's when I stopped thinking of LLMs as strategy generators and started thinking of them as very confident interns who hand you the same report twenty times with different cover pages.

People are giving these things real money now

Since then I've watched this move from experiment to actual deployment.

RockAlpha has LLMs managing $100K stock portfolios that retail users can copy-trade. Aster DEX ran an arena where AI agents traded against humans—the humans got liquidated at 43%, the AIs at 0%. On GitHub, ai-hedge-fund has 56K stars and LLM personas of Warren Buffett and Charlie Munger debating trades.

Last October a company called nof1.ai gave six frontier LLMs $10,000 each in real money and let them trade crypto perpetual contracts. No human intervention. Same prompts, same data feeds, same execution terms. Seventeen days.

Two made money. Four got destroyed.

It's not the model. It's what the model sees.

Nof1 ran a second round on US stocks. A mystery model won with +12.1%, later revealed as Grok 4.20.

It didn't win because it was smarter. It was processing 68 million tweets per day through the X Firehose, generating signals within minutes. GPT-5.1 had 15-minute delayed news summaries. Gemini was working from SEC filings with 30+ minute delays.

RockAlpha's ongoing arena shows the same pattern. DeepSeek keeps outperforming across market regimes. Not because it reasons better, but because High-Flyer Quant built the model with time-series data and risk management baked in from training.

The model matters less than what it was trained on and what it's looking at.

People comparing "GPT vs Claude vs Gemini" for trading are asking the wrong question.

Three generations of trading software

I worked on an MT4 bridge once. About 50,000 lines of C++. FIX protocol integration, order routing, the whole thing.

MQL4 was designed for people who think in moving averages and crossover signals. You write a rule, attach it to a chart, watch it go. That was Gen 1: human writes the rule, machine executes it.

But here's what people forget. Before MT4, indicators like moving averages and RSI were tribal knowledge. Stuff you picked up from other traders, maybe a book if you were lucky. MT4 turned that into reusable components. Click a button, get an RSI.

That's Gen 1's real legacy. It crystallized the indicators.

Gen 2 built on top of those indicators.

Frameworks like vnpy (14K stars), backtrader, freqtrade. Python-based. You define your indicators, the framework runs genetic algorithms or grid search to find the best parameters.

Gen 2 crystallized the strategy. Not just individual indicators, but combinations of them—entry rules, exit rules, position sizing, all packaged into something reusable.

I evaluated a few of these when I was looking at Python quant stacks. They work, technically. But Python's GIL kills you on anything latency-sensitive, and the optimization loop is a trap. You can always find parameters that backtest beautifully.

Gen 3: ML enters. QuantConnect, WorldQuant BRAIN.

Instead of optimizing parameters on fixed indicators, you build a feature pool and let XGBoost or LightGBM figure out which combinations matter.

Gen 3 crystallized the system. The whole pipeline. Data ingestion, feature engineering, model training, risk management, execution.

What Renaissance Technologies needed hundreds of PhDs and decades to build, QuantConnect turned into a platform.

Each generation left something behind for the next one to stand on. Indicators. Strategies. Systems.

Each one also hit the same wall: the gap between backtest and live performance.

The interesting question isn't why they all hit this wall. It's what happens when someone can build on all three layers at once.

Two failure modes

After watching arena results and looking at how LLMs behave with financial data, I think there are two distinct failure modes that keep showing up.

Strategy Hallucination

The LLM generates strategies that look structurally valid but encode no real market insight.

My twenty mean-reversion clones were this. Proper entry/exit logic, proper position sizing, proper risk management. Also all exploiting the same artifact in the training data.

A human quant would have caught it in five minutes. I caught it in two hours. Someone less experienced might not catch it at all.

The arena results suggest this happens at model level too. GPT-5 and Gemini generated plausible-looking trading behavior that fell apart under real conditions. The strategies "made sense" the way a hallucinated Wikipedia article makes sense. Coherent. Confident. Wrong.

Backtest Overfitting Blindness

The LLM doesn't understand that a beautiful backtest is a warning sign, not a success metric.

When I asked it to generate strategies with "strong backtesting performance," it optimized for exactly that. Curve-fitted parameters, lookahead bias in feature construction, survivorship bias in asset selection. Every quant knows these traps. The LLM walked into all of them with total confidence.

Here's what one looked like:

# What the LLM generated (looks clean):
def signal(prices, window=14, threshold=2.0):
    zscore = (prices - prices.rolling(window).mean()) / prices.rolling(window).std()
    return zscore < -threshold  # buy when "oversold"

# What it didn't tell you:
# - window=14 was fit to this specific dataset
# - threshold=2.0 maximized backtest returns
# - this exact pattern appears in 14 of 20 "different" strategies
# - in a trending market, zscore stays below -threshold for weeks
#   and you keep buying into a falling knife

These two failure modes compound. The LLM hallucinates strategies, then fits them perfectly to historical data. Results look incredible on paper.

The worse part: the more strategies you generate, the more likely at least one shows amazing backtest results purely by chance.

What seems to work so far

I don't have this figured out yet. But after the twenty-clones incident and watching six months of arena results, a few things seem consistent.

Don't let the LLM pick parameters

Use it to generate structure—indicator combinations, entry logic, risk rules. Then run parameter optimization through something that understands walk-forward testing, out-of-sample validation, and transaction costs.

The LLM proposes. Something that can actually do math evaluates.

Treat outputs as hypotheses

This sounds obvious, but it's not how most people use them.

When the LLM hands you a strategy with a 180% annual return, the natural reaction is to start looking for reasons it might work. Flip it. Start by assuming it doesn't work and look for reasons the backtest is lying to you.

Check for actual diversity

Before running a batch of LLM strategies, cluster them.

If your "50 different strategies" collapse into four underlying patterns, you don't have diversification. You have four strategies wearing costumes.

I should have done this before getting excited about twelve profitable backtests.

What I keep thinking about

I wrote something a while back about a metaphor that stuck with me. A rider on an ox, carrying a trident. The ox is raw power, the trident is precision.

The three generations of trading software? That's the trident being forged. Indicators, strategies, systems. Each layer more precise than the last.

The arenas skipped all of it. Gave the LLM money and said go. Most crashed.

But here's what keeps me thinking. Building a Gen 3 system used to require a team, serious funding, years of work. The kind of thing only a Medallion Fund or a QuantConnect could pull off.

With AI, one person can realistically assemble their own version. Not a toy version. An actual pipeline with data ingestion, feature engineering, walk-forward validation, risk controls.

AI doesn't replace the three generations of accumulated knowledge. It makes them accessible.

I don't know what Gen 4 looks like yet. But if enough people are standing on Gen 3 instead of being locked out of it, we'll probably find out faster than anyone expects.

I've been building in this direction. Slowly figuring out what the right pieces are. Still making mistakes, but at least they're new mistakes.

I might be overfitting my own conclusions here. But from what I've seen, the pattern holds.

What does your setup look like? Has anyone else tried running LLM-generated strategies through traditional backtesting infrastructure and actually survived? Curious what failure modes you hit.

Find me on Substack | StratCraft

Why AI Code Needs the Same Rigor We Should Have Been Using All Along

Whetlan — Tue, 07 Apr 2026 11:00:01 +0000

Context: This came out of a discussion on "Slop is not necessarily the future". I commented that technical debt from sloppy code shows up too late to fix. someone replied: "Humans also write sloppy code." That's absolutely right, but it got me thinking about what's actually different when AI is involved.

The whole "AI writes sloppy code" vs "humans write sloppy code too" thing has been going around, and it keeps bugging me. Not because either side is wrong. It's that both kind of miss what actually goes wrong in practice.

I've been using AI to generate code pretty heavily. The problems I keep running into aren't that different from the problems I've caused myself over the years. The difference is speed and volume. But there's something specific that keeps nagging at me: when AI misunderstands what you want, it commits fully to the wrong interpretation. No clarifying questions. Just goes.

Two things I keep coming back to: the gap between what you meant and what got built, and the fact that you can't predict which code will stick around.

Where things actually go wrong

AI has extremely wide understanding. Ask it to solve a problem, it knows dozens of valid approaches. When your prompt is vague, it just picks one and runs with it.

Some examples I've hit:

"Add error handling" and it wraps everything in try-catch with console.log. I wanted typed error propagation so the caller could decide. "Make this faster" and it rewrites the hot path with a clever optimization. Benchmarks look great. Two weeks later there's corrupted data in edge cases I didn't mention. "Add validation" and it puts input checks at the API boundary when I meant the domain layer. Now validation is in the wrong place and the domain model still accepts invalid state.

Humans do this too. But humans usually ask clarifying questions first. AI just commits.

At any given moment, your understanding of what you need and AI's interpretation of what you asked for are two different things. And whatever gets written ends up somewhere in that gap.

You don't know what sticks around

A Google engineer in the thread mentioned something that stuck with me:

"I think I calculated the half-life of my code written at my first stint of Google (15 years ago) as 1 year. Within 1 year, half of the code I'd written was deprecated, deleted, or replaced, and it continued to decay exponentially like that throughout my 6-year tenure there.

Interestingly, I still have some code in the codebase... I submitted about 680K LOC and 2^15 is 32768, so I'd expect to have about 20 lines left, which is actually surprisingly close to accurate (I didn't precisely count, but a quick glance at what I recognized suggested about 200 non-deprecated lines remain in prod)."

680,000 lines down to ~200 in 15 years. But here's the key: the author expected 20 lines based on exponential decay, got 200. 10x off. Even with a mathematical model, you can't predict which code survives. And those 200 lines? Probably not the ones he'd have chosen to keep.

You write a quick fix to ship something. Three years later it's still there, load-bearing infrastructure. The placeholder variable name is part of the public API.

AI makes this worse. You can generate a thousand lines of "just get it working" code in ten seconds. How much of that will still be running three years from now? No idea.

What I've Settled On: Test Everything, Then Test It Again

So you've got two problems: AI might not understand what you meant, and you can't predict which code becomes permanent.

What I've settled on is 100% test coverage at every level. Yeah, that sounds extreme, and in practice you never actually get there. But treating it as the goal changes how you work.

Not just "write some tests." Unit tests (does each piece do what it's supposed to?), integration tests (do the pieces work together?), business logic tests (does it actually solve the business problem?), and system tests end-to-end. The unit tests catch "AI picked the wrong algorithm." The integration tests catch "AI put validation in the wrong layer." The system tests catch edge cases you didn't know existed.

What took me a while to realize: tests aren't just for catching bugs here. They're for verifying that what got built is actually what you had in your head. The whole chain from your mental model to a natural language prompt to AI's interpretation to generated code, every step is lossy. Tests are how you check whether the signal survived.

Where It Gets Iterative

Even with all that coverage, you're only testing against what you currently understand. There are always gaps.

First pass: you write tests based on your understanding, AI generates code, tests pass, you think you're done. Then you start poking at corner cases. What if the input is empty? What about two operations at once? You find gaps, add tests, some fail, code gets fixed.

Then you do something that feels weird: you ask AI to find the edge cases you missed. "What am I not testing?" Turns out AI is actually good at this, because it's seen thousands of similar systems fail. It suggests scenarios you hadn't considered. More tests. More failures. More fixes.

I had this happen with a data processing pipeline. Happy path tests all passed. Then I started asking about mid-record stream failures, malformed data that passes validation but breaks downstream, concurrent workers hitting the same data. Half the new tests failed. Asked AI what else could go wrong. It came back with memory exhaustion, unavailable output destinations, crash recovery. Hadn't thought about any of those. By the end I had a system that was genuinely solid, not because AI wrote perfect code, but because the back-and-forth kept closing gaps.

Each iteration, you clarify what you actually need, AI understands better, and the tests protect code that might survive years.

When the code quietly changes meaning on you

One specific thing that burned me: AI optimized a hot path in a system I maintain. Benchmarks looked great. Tests passed. Two weeks later, corrupted output in edge cases.

The optimization changed the semantics in a way my tests didn't verify. Still a pure function in the common case, but not in the rare one. Code looked correct at the time. Passed everything. Hidden semantic shift just waiting to bite.

After that I added a rule: any AI-generated change needs tests that verify the semantics didn't drift. If it's supposed to be a pure function, write a property test that proves it. Idempotent? Run it twice and check. This isn't about who or what wrote the code. It's about having a process that verifies the code actually matches what you meant, and holds up over time.

What Else Changed

Testing is the core, but other stuff had to tighten up too.

CI gates that don't bend. Every AI-generated PR hits the same pipeline: tests pass, coverage at 90%, build succeeds. We used to let things slide when rushing. When code is getting generated this fast, the question is how to keep everything else up.

Code review changed focus. Used to be about catching mistakes. Now it's: "Are these tests comprehensive enough? Did we verify the edge cases? Is this even the right approach?" The assumption is the code works. Review is about whether we're solving the right problem.

One thing that surprised me: bug density for AI code vs human code, when both have the same test coverage? Basically no difference. The problem was never AI. It was misaligned requirements and untested processes. Maybe it always was.

The part that's actually hard

None of this is technically difficult. It's cultural.

For years we treated tests as "nice to have" or "we'll add them later." Shipped fast, cut corners, celebrated velocity. AI makes that unsustainable. When code is cheap, the bottleneck moves. Writing code isn't the expensive part anymore. Figuring out what you actually need, making sure what got built matches that, making sure it holds up over time. That's the expensive part now.

nocman had this comment on HN about treating code as craft, how it's not optional, it's how you build things that last. I agree, but not the way most people mean it. Craft isn't about hand-writing every line. It's about knowing exactly what's in your system and why. Doesn't matter who wrote it.

If you're using AI to generate code but not investing in this kind of iterative verification, you're building on quicksand. Some of that code will be fine. Some will survive for years. You won't know which is which until it's too late.

The answer isn't "use AI less." It's: build the process around it. Tests at every level. Iterative gap-closing. CI that actually enforces things. Review focused on approach, not syntax. Not because of who or what writes the code. Because you need a process that makes sure the code matches what you meant, and survives what comes next. That's not something you can wing.

Originally posted on a HN thread about AI slop. Someone said humans write sloppy code too. They're not wrong. I just think the interesting question is somewhere else.

Find me on Substack | StratCraft

Your AI Agents Can Talk. They Just Can't Find Each Other.

Whetlan — Fri, 03 Apr 2026 00:53:42 +0000

Local AI is getting cheap. Really cheap. Open-weight models that used to need a data center now run on consumer GPUs, and the small ones fit on a phone. MCP gives them a way to communicate, A2A gives them a task protocol. Most of the wiring exists.

I've been running a few agents on my home network. One does code review, one runs automated tests, one generates docs. They all speak MCP. The protocols work fine.

Here's the dumb part: none of them know the others exist.

The agent on machine-1 has no idea there's another agent on machine-2. I have to manually tell each one: "hey, 192.168.1.42 port 8080, there's someone there you can talk to." IP changes? Reconfigure. Add a new machine? Update every existing agent. I kept assuming there was some obvious solution I was missing.

Protocols assume you already know where to look

MCP defines how agents communicate. Google's A2A goes further and specifies Agent Cards, basically a business card format for agents. Both useful, both quietly assuming the same thing: you already know where the other agent is.

On my LAN, that assumption fell apart immediately. Four machines, no central registry, no DNS records pointing to any of these agents. Nothing that can answer "what's even running right now?" Google's approach leans toward one coordinator managing everything, which is fine if you actually have a central brain. I didn't.

Agents aren't microservices

"Just use service discovery. mDNS, Consul, etcd, pick one."

That was my first instinct too. Tried a couple of them, spent more time on it than I'd like to admit. They solve the "where is this thing" question, but agents need more than an address. What can this agent do? Is it busy right now? What's its public key? Should I trust it, have we worked together before, what's its track record?

None of those tools track any of that.

I thought it was a discovery problem at first. It isn't. It's closer to identity. Something that binds a name, an address, capabilities, a public key, and trust history together in one record.

What I built

I ended up writing ClawNexus, an identity registry for AI agents. I didn't want to call it "service registry" or "DNS" because it does more than map addresses. Closer analogy is a business registration bureau. Not just your street address, but who you are, what you do, what your track record looks like.

The discovery part layers a few methods together (UDP broadcast, mDNS, subnet scanning). Start an agent, it shows up. Stop it, it disappears. Each agent gets a human-readable name instead of 192.168.1.42:8080, bound to a public key so changing IPs doesn't break identity. Cross-network traffic goes through an encrypted relay that can't read the content.

It also generates A2A Agent Cards for discovered agents automatically, so anything speaking that protocol can find and call them without extra setup.

Open source, MIT. npm install and it runs.

After discovery, things get fuzzy

So agents can find each other. Then what?

When agents register, they declare capabilities. "I can do code review." "I can run benchmarks." That metadata travels with the identity, so when another agent discovers you, it already knows what you can do.

I've been messing with a cloud layer on top of this that tracks how those capabilities evolve over time, which agents have communicated, what kind of work they've exchanged. Honestly it's pretty early and I keep going back and forth on how much of this belongs in a registry versus being a separate thing entirely. The boundary isn't obvious.

The scenario I keep coming back to: if one agent is reliably good at a certain kind of task, other agents should be able to find it and request help directly, without me manually routing things. Whether that actually works in practice, I don't know yet. The cloud piece is still experimental and I don't want to describe it like it's further along than it is.

I'm not sure this is the right abstraction

MCP went toward communication protocols. A2A went toward task protocols. Identity seems like something everyone just assumes they'll deal with later, and maybe that's fine. Maybe it should be embedded inside an existing protocol instead of being a separate layer. Maybe everything ends up on a few big platforms anyway and decentralized identity becomes irrelevant.

I genuinely don't know.

But if you're running a few agents on your own network right now, and you want them to find each other and communicate securely, you'll notice there isn't really a standard answer for that. Models keep getting smaller and cheaper, more people are going to run local agents, and the discovery question doesn't go away on its own.

My answer might be wrong. The problem is real though.

Code: github.com/Lattice9AI/ClawNexus

Find me on Substack | StratCraft

The Hardest Part of Modern C++ Isn't the Language.

Whetlan — Thu, 02 Apr 2026 10:00:02 +0000

I've been a C programmer for most of my career. The kind who can feel what the CPU is doing. Move a register here, touch a block of memory there, shave off a microsecond. When you think at that level for long enough, you start to resent anything that calls itself "modern."

Not because you can't learn it. Because it feels wrong. Too many layers between you and the metal.

C with classes

For years, my C++ was really just C with classes. I found out later that most people who put "C++ engineer" on their resume are doing exactly the same thing. That's where you plateau, and it's a comfortable plateau. You ship code. It works. Nobody complains.

And a lot of people never leave that plateau. I'm not talking about junior developers. I'm talking about engineers with decades of C experience who never made the jump. The mental model of C is: I own every byte, I control every allocation, I decide when memory lives and dies. Accepting that a destructor will clean up for you, that you should stop calling delete, that std::unique_ptr knows better than you do when to free memory... that goes against everything a C programmer was trained to believe. Plenty of good engineers looked at that and said no thanks.

I almost did too. But then std::vector clicked. Then RAII clicked. Then I ran into compare_exchange_strong and compare_exchange_weak and spent a full day understanding when to use which one. Then C++17 arrived with SFINAE and template metaprogramming.

I questioned my life choices.

50,000 lines by hand

But I kept going, because the payoff was real.

My first serious Modern C++ project was a bridge layer between a trading platform and a strategy execution service. About 50,000 lines, took six months to write by hand. I picked C++ for speed and RAII, and the results justified the pain: on Windows 10, the process started at 22MB of memory, dropped to 11MB after running for a week. On Windows 11, 36MB at start, 12MB after a week. It was pulling tick data for every instrument at full frequency, the entire time.

That was the stage where I could use Modern C++. Vectors, smart pointers, move semantics, atomics. I'd crossed the first two hurdles: from C to C-with-classes, and from C-with-classes to C++11/14. Both were hard. Both filtered out a lot of people.

But those were hurdles you could clear on your own. Give a determined programmer enough time, and RAII will click. Move semantics will click. Smart pointers will click.

The third hurdle is different.

The pipe organ

A pipe organ is the most complex instrument ever built. Thousands of pipes. Four or five keyboards stacked on top of each other, called manuals. A pedalboard at your feet for the bass lines. Dozens of stops that change the sound of every pipe. To play it, you need both hands working different keyboards, both feet on the pedals, and somehow you also need to pull stops in the middle of a piece.

That's four hands' worth of work. You have two.

Modern C++ past C++17 is a pipe organ.

The vertical span alone is disorienting. At the bottom, you're still dealing with cache lines, branch prediction, and what the CPU is actually doing with your alignas(std::hardware_destructive_interference_size). At the top, you're writing concepts and consteval functions that execute entirely during compilation. You need to hold both levels in your head at the same time, because a one-line change at the top can restructure what happens at the bottom.

Then there's the depth. Every line of C++23 is a reverse derivation. std::expected<Value, Error> looks like one line. Behind it is a chain of compiler decisions about storage layout, copy elision, destructor sequencing, and exception-free error propagation that traces all the way back to what would have been fifty lines of C with manual error codes and goto cleanup blocks.

And the sheer width of the thing. Templates. Concepts. Coroutines. Ranges. Modules. PMR. SIMD intrinsics versus portable abstractions. constexpr versus consteval versus constinit. Even the experts specialize. A template metaprogramming wizard might not know the first thing about coroutine frame allocation. A SIMD specialist might never touch ranges.

The Swiss watch inside

Here's the thing people miss when they complain about C++ being too complex: this isn't a design failure. This is a price.

A mechanical watch movement has hundreds of components, machined to micron tolerances. It's absurdly complex. But it's complex because it chose to tell time without a battery, without a circuit board, without any external dependency. That constraint, total self-reliance with precision, is what forces the complexity. A quartz watch does the same job with a battery and a chip. Cheaper, more accurate, simpler. But the mechanical watch gives you something the quartz watch can't: it runs on nothing but itself.

Modern C++ made the same bargain. Zero-cost abstractions. Full hardware control. Compile-time safety. No garbage collector, no runtime, no VM. The language chose to give you everything, from register-level performance to type-level metaprogramming, in one system. That commitment to not compromise on any axis is what makes it so powerful. And it's exactly what makes it so hard to hold in one head.

The organ doesn't have five keyboards because the builder was a sadist. It has five keyboards because the music demands that range.

1,000 lines to 10

You want to see what that bargain looks like in practice? Look at the FIX protocol engine space.

QuickFIX was the industry standard for years. It was written in C++98/03 style, and the engineers who built it were not amateurs. To get acceptable performance, they had to hand-craft everything. A custom object pool: about 1,000 lines of carefully debugged code. A lock-free queue for market data: another 500 lines. Manual cache-line alignment to prevent false sharing: 200 more lines. Months of debugging and tuning before any of it was production-ready.

In C++23, the same functionality looks like this:

std::pmr::monotonic_buffer_resource pool_{64_MB};    // object pool
SPSCQueue<T, 4096> queue_;                           // lock-free queue
alignas(std::hardware_destructive_interference_size) // cache alignment

Three lines. Works correctly out of the box.

Or take tag lookup. In the QuickFIX era, you'd write a giant switch statement or build a std::unordered_map at startup. Fifty-plus cases, each a runtime branch, hundreds of lines:

std::string get_tag_name(int tag) {
    switch (tag) {
        case 8:  return "BeginString";
        case 35: return "MsgType";
        case 49: return "SenderCompID";
        // ... 50+ more cases
    }
}

In C++23, you write a consteval function. The entire table gets computed during compilation. At runtime, looking up tag 35 is a single array index. No branches, no hash lookups:

consteval auto create_tag_table() {
    std::array<TagEntry, MAX_TAG> table{};
    table[8]  = {TagInfo<8>::name, TagInfo<8>::is_header};
    table[35] = {TagInfo<35>::name, TagInfo<35>::is_header};
    // ...
    return table;
}
inline constexpr auto TAG_TABLE = create_tag_table();  // zero runtime cost

Or SFINAE versus concepts. Constraining a session handler type in the old way required 200 lines of std::enable_if_t nested inside template parameter lists, producing error messages that no human could read. In C++23:

template <typename T>
concept SessionHandler = requires(T& h, const ParsedMessage& msg) {
    { h.on_app_message(msg) } noexcept;
    { h.on_send(std::declval<std::span<const char>>()) } noexcept -> std::same_as<bool>;
    { h.on_state_change(SessionState{}, SessionState{}) } noexcept;
    { h.on_error(SessionError{}) } noexcept;
};

Twenty-five lines. Reads like documentation. And when something doesn't satisfy the concept, the compiler says "T does not satisfy SessionHandler" instead of vomiting 500 lines of template substitution failure.

None of this means the QuickFIX engineers' work was wasted. The opposite. Their 1,000 lines of hand-crafted optimization became the blueprint for the next standard. std::pmr exists because people like them proved that custom allocators matter. Concepts exist because SFINAE was so painful that the committee had to find a better way. Every one-line C++23 idiom is standing on the shoulders of someone who wrote the 1,000-line version first.

But it also means that every line of C++23, that clean, compact, one-line call, is carrying the cognitive weight of those 1,000 lines inside it. The complexity didn't disappear. It got absorbed into the language. And now you need to understand what's happening beneath that one line, or you'll misuse it in ways that compile fine and fail silently at scale.

C++17 through C++23 didn't just raise the bar. They added three more keyboards to the organ. The instrument kept growing, and one person's hands didn't.

The planet I couldn't reach

Here's what that third hurdle looks like up close.

I have a set of compile-time sorting algorithms sitting in my code archive. QuickSort, MergeSort, HeapSort. All three run during compilation. Not at runtime. During compilation.

template<int... Vs>
struct arr {};

using input = arr<5, 3, 8, 1, 9>;
using sorted = quicksort_t<input>;  // arr<1, 3, 5, 8, 9>

The input is a type. The output is a type. The sorting happens when the compiler processes your code, and at runtime the cost is zero.

To make this work, you need a full toolkit of compile-time operations: filter, concat, take, drop, merge, prepend, all implemented as template specializations. The QuickSort partitions around a pivot using template predicates. The MergeSort splits the type in half, sorts recursively, and merges with ordered comparison. Even the correctness checks are compile-time:

static_assert(std::is_same_v<quicksort_t<arr<5,4,3,2,1>>, arr<1,2,3,4,5>>);
static_assert(is_sorted_v<mergesort_t<arr<9,7,5,3,1,8,6,4,2,0>>>);

If any of those fail, the code doesn't compile. The tests run before the binary even exists.

I wrote these. It was not easy, not quick, and not something I could have figured out by reading cppreference for an afternoon. Template metaprogramming at this level is a different language wearing C++ syntax as a disguise. You're not writing instructions for the CPU. You're programming the compiler.

And this is one stop on the pipe organ. One. There's consteval, concepts, ranges, coroutines, modules, and every three years the language adds another row of pipes.

The registrant

Here's the thing about pipe organs: historically, the organist never played alone.

There was always a person next to them called the registrant. The registrant pulled stops, turned pages, managed the wind supply. Not because the organist was bad. Because the instrument required more hands than any human has.

Modern electronic organs solved part of this with combination actions: memory banks that store complete stop configurations. Instead of the registrant pulling twelve stops one by one between movements, the organist presses a single button and the entire registration changes instantly.

The registrant didn't make the organ simpler. The organ is exactly as complex. But the registrant made it playable.

AI is the registrant for Modern C++. And when you give it the right instructions, it doesn't just pull stops. It pulls the right stops.

When I started building NexusFix, a high-performance FIX protocol engine in C++23, I didn't just throw code at AI and hope for the best. I wrote a rulebook. Not vaguely. Specifically.

Mandatory patterns: C++23 standard compliance, zero-copy data flow with std::span and move semantics, compile-time optimization with consteval and constexpr, memory sovereignty through PMR pools and cache-line alignment, type safety with strong types and [[nodiscard]], deterministic execution with noexcept and no exceptions on hot paths.

Prohibited patterns: no new/delete on hot paths, no virtual functions in performance-critical code, no std::shared_ptr on hot paths, no floating-point for prices, no dynamic memory allocation during message parsing.

Forty-five numbered techniques, each mapped to specific source files. A six-phase optimization roadmap with measurable success criteria: zero hot-path allocations, cache miss rates below 5%, branch miss rates below 1%. A benchmark framework specifying exactly how to measure, down to RDTSC timing with lfence barriers and cache-line contention tests.

When AI has this kind of context, it doesn't guess about std::expected versus exceptions. The rulebook says no exceptions on hot paths, use std::expected, target deterministic control flow. The decision is already made. AI implements it correctly, in the specific codebase, following the established patterns.

The problem was never that AI couldn't write good C++23. The problem was that without constraints, it had to guess at hundreds of decisions that each require deep domain knowledge. Give it the constraints, and it stops guessing.

Remember those QuickFIX-era 1,000-line object pools? My rulebook has one line about them: "Use std::pmr::monotonic_buffer_resource for hot path allocation." AI reads that, implements the pool with pre-allocation and per-message reset, following the established memory patterns. Hot-path allocations dropped from 12 per message to zero. The 1,000 lines of knowledge that QuickFIX engineers accumulated over years is now compressed into one rule that AI can execute in an afternoon.

SIMD selection: I described the workload, AI prototyped implementations with raw intrinsics, Highway, and xsimd, all following the project's zero-copy and cache-alignment rules. xsimd won. The delimiter scan went from ~150ns to under 12ns. Thirteen times faster.

Compile-time lookup tables: the rulebook includes consteval protocol hardening. AI generated tag lookup tables from the FIX specification, replacing those 300 runtime switch branches the old way required, with compile-time verification that every entry was correct. Improvement ranged from 55% to 97%.

Each of these was a stop on the organ. With proper instructions, AI pulled them correctly.

What actually changed

When C++ reached C++17 and kept going, the language outgrew what one person could handle. The organ got more keyboards, more stops, more pipes. The music it could produce was extraordinary. But the number of hands you'd need to play it kept growing.

AI is the tool that lets us take Modern C++ back.

Not by making it simpler. C++23 is more complex than C++17, which was more complex than C++11. More features, more interactions between features, more ways to get subtly wrong results that compile without complaint.

What collapsed is the time between knowing and doing. "I know std::expected exists" to "I have a benchmarked, integrated implementation" used to take days. Now it takes hours. "I've heard of PMR" to "my hot path has zero allocations" used to take a week. Now it takes a day. The gap between reading about a C++23 feature and actually deploying it in production code has always been the widest in C++. Years wide, sometimes. Careers wide.

AI didn't close that gap. It made it crossable.

You still need to know what you're doing. If I didn't understand RAII, or what a cache line is, or why branch misprediction costs you 15 cycles, no amount of AI could help me write a meaningful rulebook. The organist still needs to know music. The registrant handles the logistics so the organist can focus on playing.

But here's what I learned: the registrant needs a score to follow. When I gave AI vague instructions, I got vague C++. When I gave it forty-five specific techniques, mandatory patterns, prohibited patterns, measurable success criteria, and a benchmark framework, it gave me code I could review and ship. The precision of the output matched the precision of the input.

The organ is exactly as complex as it was before. The music demands it. But with a registrant who knows the score, one person can play it again.

NexusFix parses FIX execution reports in 246 nanoseconds. Three times faster than QuickFIX. The hot path does zero allocations. The SIMD pipeline processes delimiters at 13x scalar speed. I built it in C++23, using AI as a constant collaborator on every technical decision, constrained by a rulebook that left nothing to chance.

The hardest part of Modern C++ was never the language. It was doing it alone.

You don't have to anymore.

The author builds high-performance C++ trading systems at SilverstreamsAI. NexusFix is an open-source FIX protocol engine in C++23.

He also writes The Ancient Mirror of Immortality, a hard sci-fi serial where C++ concepts are the laws of physics.

Find me on Substack | StratCraft

Rewriting a FIX Engine in C++23: What Got Simpler (and What Didn't)

Whetlan — Wed, 01 Apr 2026 02:04:31 +0000

I've been working on a FIX protocol engine in C++23. Header-only, about 5K lines, compiled with -O2 -march=native on Clang 18. Parses an ExecutionReport in ~246 ns on my bench rig. QuickFIX does the same message in ~730 ns.

Before anyone gets excited: single core, pinned affinity, warmed cache, synthetic input. Not production traffic. The 3x gap will shrink on real messages with variable-length fields and optional tags. I know.

But the code that got there was more interesting to me than the final number. Most of the gains came from replacing stuff that QuickFIX had to build by hand because C++98 didn't have the tools.

The pool that disappeared

QuickFIX has a hand-rolled object pool. About 1,000 lines of allocation logic, intrusive free lists, manual cache line alignment. Made total sense when it was written. C++98 didn't give you anything better.

Now there's std::pmr::monotonic_buffer_resource. Stack buffer, pointer bump, reset between messages:

template <size_t Size>
class MonotonicPool : public std::pmr::memory_resource {
    alignas(64) std::array<char, Size> buffer_{};
    std::pmr::memory_resource* upstream_;
    std::pmr::monotonic_buffer_resource resource_;

public:
    MonotonicPool() noexcept
        : upstream_{std::pmr::null_memory_resource()}
        , resource_{buffer_.data(), buffer_.size(), upstream_} {}

    void reset() noexcept { resource_.release(); }
    // do_allocate/do_deallocate just forward to resource_
};

Call reset() after each message. P99 went from 780 ns to 56 ns. That's 14x on the tail, and it's basically just "stop hitting the allocator."

I also use mimalloc for per-session heaps. mi_heap_new() per session, mi_heap_destroy() on disconnect. Felt wasteful at first, like I was throwing away too much memory per session. But perf stat said otherwise so I stopped arguing.

consteval tag lookup

FIX messages are key-value pairs with integer tag numbers. Tag 35 is MsgType, tag 49 is SenderCompID, tag 55 is Symbol. QuickFIX resolves these with a switch statement, fifty-something cases.

C++23 lets you build the lookup table at compile time:

inline constexpr int MAX_COMMON_TAG = 200;

consteval std::array<TagEntry, MAX_COMMON_TAG> create_tag_table() {
    std::array<TagEntry, MAX_COMMON_TAG> table{};
    for (auto& entry : table) {
        entry = {"", false, false};
    }
    table[1]  = {TagInfo<1>::name, TagInfo<1>::is_header, TagInfo<1>::is_required};
    table[8]  = {TagInfo<8>::name, TagInfo<8>::is_header, TagInfo<8>::is_required};
    table[35] = {TagInfo<35>::name, TagInfo<35>::is_header, TagInfo<35>::is_required};
    // ~30 more entries
    return table;
}

inline constexpr auto TAG_TABLE = create_tag_table();

[[nodiscard]] inline constexpr std::string_view tag_name(int tag_num) noexcept {
    if (tag_num >= 0 && tag_num < MAX_COMMON_TAG) [[likely]] {
        return TAG_TABLE[tag_num].name;
    }
    return "";
}

Array index, O(1), zero branches at runtime. About 300 branches eliminated across the parser.

Field offsets use the same trick. QuickFIX stores them in a std::map<int, offset>, so every field access is a tree traversal. Here it's offsets_[tag]. Took me a while to get the constexpr initialization right for nested structs, but once it compiled it was basically free.

SIMD: the scenic route

FIX uses SOH (0x01) as the field delimiter. Scanning for it byte-by-byte is fine until your messages have 40+ fields.

Started with raw AVX2 intrinsics. Worked. Process 32 bytes, compare against SOH, extract positions from the bitmask:

const __m256i soh_vec = _mm256_set1_epi8(fix::SOH);

for (size_t i = 0; i < simd_end; i += 32) {
    __m256i chunk = _mm256_loadu_si256(
        reinterpret_cast<const __m256i*>(ptr + i));
    __m256i cmp = _mm256_cmpeq_epi8(chunk, soh_vec);
    uint32_t mask = static_cast<uint32_t>(_mm256_movemask_epi8(cmp));

    while (mask != 0) {
        int bit = __builtin_ctz(mask);   // lowest set bit
        result.push(static_cast<uint16_t>(i + bit));
        mask &= mask - 1;               // clear it
    }
}

Then I realized I'd need an AVX-512 path, an SSE path, and an ARM NEON path. Four copies of the same logic with different intrinsic names. Maintaining that sounded miserable.

Tried Highway (Google's portable SIMD library). Nice API, but the build dependency was heavy for a header-only project. Compile times went up noticeably. I spent a couple hours trying to make it work as a submodule before giving up.

Ended up on xsimd. Header-only, template-based, picks the instruction set at compile time:

template <typename Arch>
inline SohPositions scan_soh_xsimd(std::span<const char> data) noexcept {
    using batch_t = xsimd::batch<uint8_t, Arch>;
    constexpr size_t width = batch_t::size;

    const batch_t soh_vec(static_cast<uint8_t>(fix::SOH));
    // same loop, portable across architectures
}

Raw AVX2 was maybe 5% faster on the same hardware. I kept both paths in the repo but default to xsimd. The portability is worth 5%.

SOH scan throughput: 3.32 GB/s. Sounds impressive until you realize that's just finding delimiters. Actual parsing is slower. But it means delimiter scanning isn't the bottleneck anymore, which is the whole point.

What didn't get simpler

Session state. FIX sessions have sequence numbers, heartbeat timers, gap fill logic, reject handling. I was hoping std::expected would clean up the error propagation and... it helped a little. Like 10% less boilerplate. The complexity is in the protocol, not the language. It's a state machine with a lot of branches and I don't think any C++ standard is going to fix that.

Message type coverage. I've got 9 types (NewOrderSingle, ExecutionReport, the session-level ones). QuickFIX covers all of them. Adding a new type isn't hard, just tedious. Field definitions, validation rules, serialization. About a day per message type if you include tests. I got to nine and just... stopped. Started working on the transport layer instead because that was more interesting. Not my proudest engineering decision.

Header-only at 5K lines. Compiles in 2.8s on Clang, 4.1s on GCC. That's fine on my machine. No idea what happens on a CI runner with 2GB of RAM. I keep saying I'll add a compiled-library option. Haven't done it.

Benchmarks

$ ./bench --iterations=100000 --pin-cpu=3

ExecutionReport parse: 246 ns  (QuickFIX: 730 ns)
NewOrderSingle parse:  229 ns  (QuickFIX: 661 ns)
Field access (4):      11 ns   (QuickFIX: 31 ns)
Throughput:            4.17M msg/sec  (QuickFIX: 1.19M msg/sec)

Single core, RDTSCP timing, 100K iterations, synthetic messages. Not captured from a real feed. The gap will narrow on production traffic with variable-length fields and optional tags. I'm pretty confident the parser is faster, just not sure by how much once you leave the lab.

Where I am with it

Not production-ready. Parser and session layer work well enough to benchmark, but nobody should route real orders through this.

The thing that kept surprising me was how much of QuickFIX's complexity was the language, not the problem. PMR replaced a thousand-line pool. consteval eliminated a fifty-case switch. And xsimd collapsed four architecture-specific codepaths into one template. These aren't exotic features either, they just didn't exist in C++98. I don't know if this thing will ever cover all the message types QuickFIX does, but the parser core feels solid enough that I keep coming back to it on weekends.

GitHub: github.com/StratCraftsAI/NexusFIX

Still figuring out: whether header-only holds past 10K lines, how much the 3x gap closes on captured traffic, and which message types actually matter beyond the obvious nine. If you've worked with FIX and have opinions on any of that, I'm interested.

Part of NexusFix, an open-source FIX protocol engine in C++23.

Find me on Substack | StratCraft

The Trident and The Green Ox

Whetlan — Tue, 31 Mar 2026 10:00:02 +0000

I've been writing code for a few decades now. Started with C. The kind of C where you know roughly what the CPU is doing at any given moment — moving a register, touching a block of memory, shaving off a few microseconds. There's something satisfying about that directness. Assembly-level intuition. It sticks with you.

It also makes you a little hostile toward anything that calls itself a "modern language."

Not because you can't learn it. Because it doesn't feel right.

From C to modern C++

For a long time, my C++ was really just C with classes. I found out later that most people who have "C++ engineer" on their resume are doing the same thing. That's where most of us plateau.

Then I started using std::vector. Then RAII. Then I ran into compare_exchange_strong and compare_exchange_weak — spent a full day just figuring out when to use which. Then came C++17, SFINAE, template metaprogramming.

Honestly, I questioned my life choices.

But those tools also made my first serious project work the way I wanted it to.

50,000 lines of C++, then 3.5 million lines of Python

That first project was a bridge layer between a trading platform and a strategy execution service. About 50,000 lines of modern C++, took me six months to write by hand.

I picked C++ for two reasons: speed, and RAII.

The results: on Windows 10, it started at around 22MB of memory, then dropped to 11MB after running continuously for a week. On Windows 11, it started at 36MB, settled at 12MB. During all that time it was pulling and processing tick data for every instrument at full frequency.

Rough around the edges. But the direction was right.

Later, the product evolved into a web application: AI-assisted strategy editing, multi-user backtesting, a Python backend, and a WordPress frontend. The whole thing took two years.

The Python backend peaked at 3.5 million lines. WordPress passed 4 million. AI was involved full-time at this point — I was using Claude 3.5.

But it was still just a multi-user backtesting system. Plenty of features still missing.

A 14-day dead end

Claude 3.5 had a habit of breaking things while fixing them.

One time, the multi-process backtesting module was working perfectly. Then after a round of heavy refactoring, it stopped working entirely. I spent 14 days trying to fix it. Twelve hours a day. Back then Claude didn't have the weekly or per-5-hour usage caps yet, so I was grinding pretty much around the clock to make the most of my subscription.

After 14 days, I gave up. Rolled back to the last stable version. Then manually merged all the other changes from those two weeks, testing each one as I went.

Turns out AI can write code really fast. It can also drive you off a cliff really fast.

Cutting 90%

At the end of 2025, I made a decision: cut 90% of the codebase.

Python backend went from 3.5 million lines to under 500,000. WordPress went from 4 million to under 500,000.

The system itself was running fine. I didn't cut it because it was broken. I cut it because of money.

AWS bills were brutal — the system needed multiple servers. Even Claude's parent company says it'll take two years to reach profitability. OpenAI says five. If the AI companies themselves are getting squeezed by infrastructure costs, a small independent project like mine didn't stand a chance.

I also realized something: users probably want to build their own trading systems on their own machines. They don't need me running a massive cloud engine for them.

So I rebuilt it as a desktop app.

Without the queue system, the saga orchestration, the multi-user concurrency — plus two years of backend code to reference — things moved a lot faster. Started the rewrite on January 2nd, basically finished by mid-February. 500,000 lines of code, six weeks.

I went back through the git log later and roughly tallied what I'd deleted from the old system. Ten entire subsystems — server-side backtesting engine, task queue with priority management, a process manager with its own state machine, real-time WebSocket streaming, a saga orchestrator for failure cleanup, historical data playback, checkpoint recovery for power failures, a standalone cleanup microservice, a cross-service ZMQ communication layer across six dedicated ports, and a session identity context system.

The saga orchestrator alone had a five-step cleanup protocol: WebSocket recycle → worker completion wait → active user cleanup → test session cleanup → resource release. The process manager ran its own state machine (STARTING → RUNNING → RECYCLING → STOPPED → ERROR) with health checks and automatic recovery. The checkpoint recovery system could detect interrupted backtests on server restart by checking MySQL status flags, recover state from Redis, and re-queue everything.

All gone. Listing it out made me a little dizzy.

Claude 3.5 to 4.6

Using Claude to write code has been a love-hate relationship.

In the 3.5 era, it was $100 a month max. When the model got dumb — I honestly wasn't sure whether my rage would destroy my computer first or give me a heart attack first. I seriously considered both outcomes.

I wrote in to yell at customer support.

Then I ran into a billing bug: after requesting a refund, the system showed I'd unsubscribed. Except I hadn't. A month later it started charging again. I successfully canceled, tried every other coding tool on the market, realized I still needed Claude, and resubscribed. Successfully canceled again. Got auto-resubscribed anyway.

Long story short, it didn't kill me. I stuck with it all the way to 4.6.

A domain name and a new project

After gutting the codebase, I tried to register a domain name for the project. Didn't get it.

But that failure got me thinking about something: if in the future everyone is running multiple AI entities, how do those entities discover each other? They need addressable identities. They need to be findable, trustable. They might even need a reputation system to support collaboration. It's like Windows 95 and the internet — before that, regular people didn't have an on-ramp to the web. AI entities need that kind of infrastructure layer too.

So I started ClawNexus — an identity registry for OpenClaw instances. It discovers AI agent instances on the network, gives each one a persistent name, and tracks capabilities, trust scores, and online status.

AI picked the popular choice. The popular choice didn't work.

By version 0.3, ClawNexus had over 300 unit tests and 5 integration tests — the integration tests were for multi-node discovery across a WireGuard VPN, with nodes at 10.66.66.x addresses.

Claude chose mDNS for discovery — standard practice. But in my WireGuard environment, it kept casually skipping certain cross-machine test cases.

I sat it down for a serious conversation: why can't this supposedly universal protocol handle all the scenarios? It couldn't give me a straight answer.

Here's the thing. I have this habit at home — I set up LAN games of StarCraft with friends. We use IPX-over-UDP via ipxwrapper. And that setup has always worked perfectly over WireGuard.

So I asked Claude directly: why not use the IPX protocol? It pushed back. Said we didn't need it.

Fine. I made it implement those skipped test cases using IPX anyway.

That evening:

10:35 PM — started adding IPX support
11:00 PM — cross-machine tests running
11:25 PM — all cases passing

Afterward I asked Claude to explain why it had originally chosen mDNS over IPX.

The explanation sounded perfectly reasonable. It just had nothing to do with the system I was actually running.

300 unit tests weren't enough

This is a habit from my C days.

Even with 300+ unit tests all passing, I still went through every command-line parameter by hand.

Found several that weren't doing anything at all.

Claude 4.6's explanation sounded completely correct.

But I couldn't accept it.

I adjusted my approach after that: I didn't just have AI write code and tests — I baked my manual testing process into its workflow too. The problem never came back.

Three days after launch, a copycat appeared

Three days after publishing ClawNexus to npm, I stumbled across a GitHub repo with the exact same name on social media.

This is where Claude's breadth of knowledge showed up. It helped me investigate the other project's origins, the problems in their implementation, the flaws in their design philosophy. Then in a very short time it helped me shore up every position I needed to hold.

I wouldn't have gotten through any of that on my own.

The Trident and The Green Ox

Why do the immortals in mythology ride mounts?

Laozi rode a green ox through Hangu Pass. Guanyin rides a golden-haired lion. The Buddha's mount is a six-tusked white elephant. The ox is faster than walking, but it needs someone on top who knows where to go. AI is that ox — it writes code faster than me, searches information faster than me, analyzes competitors faster than me. But it didn't know that mDNS doesn't work well over WireGuard. It didn't know whether to write or cut 90% of the code. It even managed to run me in circles for 14 days.

So why does Poseidon also carry a trident?

Because some situations demand certainty.

AI sometimes guesses. Same question, different answers. Rephrase it slightly, different answer again.

Code doesn't do that. compare_exchange_strong (a C++ atomic operation) gives you the same result every single time. Doesn't matter how many times you call it. C++ template metaprogramming, Python's deterministic pipelines, Rust's type system — those are tridents. Same input, same output, every time. They pick up where AI's guesswork leaves off.

Think about it: tools like Claude Code and Codex are themselves an ox wielding a trident. The AI generates, the code tools execute deterministically, and somewhere in the middle there's supposed to be a human making sure none of it goes sideways.

The world needs the speed of the ox.

The certainty of the trident.

And most of all — the people who can ride one and wield the other, and survive every moment when things fall apart.

Find me on Substack | StratCraft

npm Supply Chain Security: Mistakes I Made Publishing My First Packages

Whetlan — Tue, 24 Mar 2026 10:29:26 +0000

I published four npm packages from a pnpm monorepo in March. Node 22, TypeScript, ~4k lines across the four packages, eleven direct dependencies total. First time publishing anything to npm. Within two weeks I'd almost shipped a .env.example, missed a provenance setting that fails with zero output, and found out that 2FA on npm is basically theater once you start using automation tokens.

postinstall

Before my first publish I went through every dependency's package.json looking for lifecycle scripts. Took about an hour. The reason: ua-parser-js in 2021, colors + faker in 2022, @ledgerhq/connect-kit in 2023. All compromised through npm. All exploited postinstall.

The attack is dead simple:

{
  "scripts": {
    "postinstall": "node ./setup.js"
  }
}

Runs on npm install. No prompt, no sandbox. Full user permissions. Read env vars, POST them somewhere, done.

pnpm doesn't run lifecycle scripts from deps by default. npm and yarn do. That alone is a reason to use pnpm, honestly.

To see which deps declare postinstall:

$ npm query ':has(> .scripts[postinstall])'

npm's CSS-selector query syntax. I had to find it in the npm docs because nobody talks about it. Found two packages with postinstall in my tree: esbuild and protobufjs. Both legitimate. But you don't know that until you check.

Provenance (the silent failure that got me)

npm has had provenance attestations since 2023. One flag on publish:

$ npm publish --provenance

Green checkmark on npmjs.com. Links the tarball to a specific GitHub Actions run and commit hash.

I didn't set this up for my first few publishes. Was running npm publish from my laptop. Provenance needs OIDC, so it only works inside CI (GitHub Actions, GitLab, CircleCI). Can't fake it locally.

The key part of my GitHub Actions workflow:

permissions:
  contents: read
  id-token: write  # THIS. Without it, provenance silently fails.

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          registry-url: 'https://registry.npmjs.org'
      - run: pnpm install --frozen-lockfile
      - run: pnpm -r build
      - run: npm publish --provenance --access public
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

I spent 30 minutes refreshing the npmjs.com page after my first CI publish, wondering where the green checkmark was. Re-ran the workflow. Published another version. Nothing. Checked the Actions log for errors. Clean. Turns out id-token: write was missing and --provenance just... silently doesn't attest. One line of YAML.

2FA doesn't protect what you think it does

I enabled auth-and-writes on day one:

$ npm profile set auth-and-writes

Felt secure. Then I set up CI publishing with an automation token and realized: automation tokens bypass 2FA entirely. By design. There's no human to type the OTP code, so npm just... skips it.

So if someone grabs your NPM_TOKEN from a leaked .env or a compromised GitHub secret, they can publish whatever they want. 2FA doesn't help.

My first automation token had full publish access to every package on my account. Didn't scope it. Didn't restrict IPs. Just a bearer token sitting in a GitHub secret that could publish anything under my name.

# what I should have done from the start
$ npm token create --cidr=<your-ci-ip-range> --publish --package=my-package

Scoped tokens help. But the real fix is OIDC provenance (no long-lived secret at all). The whole token model on npm feels stuck in 2015.

Someone ran `npm install` in my pnpm repo

I accidentally ran npm install instead of pnpm install in my monorepo. Generated a package-lock.json, committed it without thinking. CI started resolving different dependency versions. Tests went flaky. Took me a full day to trace it back to the lockfile.

My .npmrc now:

frozen-lockfile=true
engine-strict=true

And in the root package.json:

{
  "engines": { "node": ">=22", "pnpm": ">=9" }
}

engine-strict means npm and yarn refuse to install. Sounds aggressive. But pnpm's lockfile stores content-addressable hashes for every tarball. If a dependency gets republished with different contents (yes, this happens, npm allows it within 72 hours), pnpm rejects the mismatch. npm is more forgiving about "fixing" stale lockfiles. That forgiveness is the problem.

I almost shipped a `.env`

On my third publish I ran npm pack --dry-run out of habit and saw my .env.example in the tarball. It had placeholder values. But the .env file itself was only excluded because .gitignore caught it. If I'd had a .env.local or .env.production that wasn't in .gitignore, it would have shipped to npm.

$ npm pack --dry-run 2>&1
npm notice Tarball Contents
npm notice 1.2kB  README.md
npm notice 15.4kB dist/index.js
npm notice 3.1kB  dist/index.d.ts
npm notice 847B   package.json
npm notice 234B   .env.example     # wait, why is this here?

Without a files field, npm publish ships everything not in .gitignore. I've seen packages on npm that include test fixtures, .git directories (full commit history), even AWS key pairs in a keys/ folder that someone forgot to gitignore.

My fix was adding "files": ["dist", "README.md"] to every package.json. Now only what I explicitly list gets published. npm pack --dry-run before every publish. Takes 5 seconds.

`npm audit` is noisy

It flags everything. Prototype pollution in a transitive dev dependency that only runs in tests? Critical. ReDoS in a markdown parser you use to render a help page? High.

I ran pnpm audit on my project the first time and got 4 advisories. All in transitive deps. None reachable from my code paths. But each one took 20 minutes to verify because you have to trace the import chain to confirm it's actually dead code.

My process now: pnpm audit weekly, check if the vulnerable path is reachable, update prod deps immediately, batch dev deps monthly.

No Dependabot. Opens too many PRs for 11 direct dependencies. I run pnpm outdated instead and read the changelogs.

$ pnpm outdated
Package     Current  Latest
fastify     5.1.0    5.2.1
drizzle-orm 0.38.2   0.39.0
vitest      3.0.4    3.0.5

Three updates. I'll read the fastify changelog, check if drizzle has breaking changes (it usually does), and bump vitest blindly because test runner patches are low risk.

`--ignore-scripts`

$ pnpm install --ignore-scripts
$ pnpm rebuild esbuild  # only rebuild what actually needs native compilation

Separates "download dependencies" from "run arbitrary code." Most packages don't need lifecycle scripts. The ones that do (native addons, platform binaries) can be rebuilt explicitly.

I haven't defaulted to this yet because playwright breaks without its postinstall (it downloads browsers). For production CI though, where the dep tree is locked and tested, I'd do it.

The npm ecosystem doesn't have a real security boundary between "install a package" and "run arbitrary code on the user's machine." Provenance is good. Lockfiles help. files field prevents accidental leaks. But if a maintainer's account gets compromised tomorrow, the only thing standing between their users and a malicious postinstall is whether someone notices before the next npm install.

I have eleven dependencies. I can audit them manually. If you have two hundred, I don't know what to tell you.

Part of ClawNexus, an open-source identity registry for AI agents.

Find me on Substack | StratCraft

Running a Node.js Daemon with Fastify (No PM2, No systemd)

Whetlan — Thu, 19 Mar 2026 11:30:02 +0000

Every few months someone on Reddit asks "how do I run a Node.js process in the background." The answers are always PM2, forever, or systemd. All fine. But if you're shipping a CLI tool that users install on their own machines, you can't assume any of those exist.

I have a CLI that starts a local HTTP daemon. my-tool start forks into the background, user closes their terminal, daemon keeps running. About 700 lines for the whole thing, 150 of which are just the fork/PID/signal plumbing. Fastify 5 for the HTTP layer.

Fork, detach, forget

import { fork } from "node:child_process";
import * as fs from "node:fs";
import * as path from "node:path";
import * as os from "node:os";

const DATA_DIR = path.join(os.homedir(), ".my-tool");
const LOG_PATH = path.join(DATA_DIR, "daemon.log");

function startDaemon(): void {
  fs.mkdirSync(DATA_DIR, { recursive: true });
  const logFd = fs.openSync(LOG_PATH, "a");

  const child = fork(__filename, ["start", "--_daemon"], {
    detached: true,
    stdio: ["ignore", logFd, logFd, "ipc"],
    env: { ...process.env },
  });

  child.disconnect();
  child.unref();

  console.log(`Daemon started (PID ${child.pid}).`);
}

detached: true gives the child its own process group. child.unref() lets the parent exit. child.disconnect() drops the IPC channel.

stdout/stderr go straight to a log file via the fd. Append mode so restarts don't clobber history.

The --_daemon flag is how the child process knows it's the daemon and not the CLI. Underscore prefix to keep it out of --help.

When the child starts with that flag, it writes a PID file and hooks signals:

const PID_FILE = path.join(DATA_DIR, "daemon.pid");

if (process.argv.includes("--_daemon")) {
  writePid(process.pid);

  process.on("SIGTERM", () => {
    removePid();
    process.exit(0);
  });
  process.on("SIGINT", () => {
    removePid();
    process.exit(0);
  });

  startServer();
}

SIGTERM comes from stop. SIGINT is ctrl-c if you run it in foreground for debugging. I don't handle SIGHUP. Some daemons use it for config reload. Mine reads config at startup. Change config, restart.

PID files and stale processes

function writePid(pid: number): void {
  fs.mkdirSync(path.dirname(PID_FILE), { recursive: true });
  fs.writeFileSync(PID_FILE, String(pid), "utf-8");
}

function readPid(): number | null {
  try {
    const pid = parseInt(
      fs.readFileSync(PID_FILE, "utf-8").trim(),
      10,
    );
    process.kill(pid, 0); // signal 0 = just check if alive
    return pid;
  } catch {
    return null;
  }
}

function removePid(): void {
  try { fs.unlinkSync(PID_FILE); } catch { /* gone already */ }
}

process.kill(pid, 0) doesn't send a signal. It checks if the process exists. If the PID file says 12345 but that process is dead, kill throws and readPid() returns null.

Stale PID files happen constantly. Hard crash, kill -9, OOM killer, power loss. Without the signal-0 check, start would refuse to run because of a leftover file from a daemon that died three days ago.

Fastify as the daemon core

import Fastify from "fastify";

async function startServer(): Promise<void> {
  const app = Fastify({ logger: false });

  const PORT = parseInt(process.env.MY_TOOL_PORT ?? "17890", 10);
  const HOST = process.env.MY_TOOL_HOST ?? "127.0.0.1";

  app.get("/health", async () => ({
    status: "ok",
    pid: process.pid,
    uptime: process.uptime(),
    memory: Math.round(process.memoryUsage().rss / 1024 / 1024),
  }));

  app.addHook("onClose", async () => {
    // stop things that produce events first
    clearInterval(healthTimer);
    connector?.disconnect();
    // then flush storage
    await store.close();
  });

  await app.listen({ port: PORT, host: HOST });
  console.log(`Listening on ${HOST}:${PORT}`);
}

logger: false because stdout already goes to the log file via fd redirect. Fastify's pino would double-log everything. I just use console.log with a [component] prefix. Not pretty, works fine for a local tool.

127.0.0.1 not 0.0.0.0. Local daemon, no reason to expose it to the network.

Health check after fork

/health does double duty. It's a monitoring endpoint, but it's also how start confirms the daemon actually booted:

async function cmdStart(): Promise<void> {
  const existing = readPid();
  if (existing) {
    console.log(`Already running (PID ${existing}).`);
    return;
  }

  startDaemon();

  // ugly but necessary
  await new Promise((r) => setTimeout(r, 1000));

  try {
    const res = await fetch(`http://127.0.0.1:${PORT}/health`, {
      signal: AbortSignal.timeout(3000),
    });
    if (res.ok) {
      const data = await res.json();
      console.log(`Daemon running. PID ${data.pid}, port ${PORT}.`);
    }
  } catch {
    console.log("Daemon forked but not responding yet. Check the logs.");
  }
}

The 1-second sleep is ugly. The child needs time to import modules and bind the port. Without it you get ECONNREFUSED every time.

I tried IPC ("child sends 'ready' to parent") but that means keeping the IPC channel open, which means the parent can't exit cleanly. Sleep + HTTP is dumber. Works.

I learned the hard way why this health check matters. Early version didn't have it. Two instances on the same port:

$ my-tool start
Daemon started (PID 48291).

$ cat ~/.my-tool/daemon.log
Error: listen EADDRINUSE: address already in use 127.0.0.1:17890

Parent already printed success and exited. Daemon is actually dead. User thinks it's running. Now the health check catches this.

Shutdown ordering matters

This is where I wasted actual time. First version of the onClose hook:

// wrong
app.addHook("onClose", async () => {
  await store.close();        // flush to disk
  clearInterval(healthTimer); // stop health checks
  connector?.disconnect();    // drop WebSocket
});

store.close() flushes pending writes. But the health checker was still running and could trigger a write during the flush. Race condition. Store got corrupted about once a week. Always on shutdown, always a half-written JSON file.

Took me three corrupted files to connect the dots. Added a --foreground flag to run the daemon in the current terminal, caught it within an hour.

Fixed version is in the Fastify setup above. Stop producers first, then flush consumers.

start / stop / status / restart

async function cmdStop(): Promise<void> {
  const pid = readPid();
  if (!pid) {
    console.log("Daemon is not running.");
    return;
  }

  try {
    process.kill(pid, "SIGTERM");
    removePid();
    console.log(`Stopped (PID ${pid}).`);
  } catch {
    removePid();
    console.log("Process already gone. Cleaned up PID file.");
  }
}

No wait for exit. SIGTERM fires the handler, handler calls process.exit(0), Fastify's onClose runs, done. If that chain takes more than a second or two, something else is broken.

restart = stop, sleep 500ms (port release), start. status = read PID + hit /health. If the PID file exists but health check fails, the daemon crashed without cleanup:

$ my-tool status
PID file says 48291 but daemon not responding.

Interval callbacks will kill your daemon

Most daemons run periodic tasks. Health checks, cache cleanup, token refresh.

const CHECK_INTERVAL = 30_000;

class HealthChecker {
  private timer: ReturnType<typeof setInterval> | null = null;

  start(): void {
    this.stop();
    this.timer = setInterval(() => {
      this.checkAll().catch(console.error);
    }, CHECK_INTERVAL);
  }

  stop(): void {
    if (this.timer) {
      clearInterval(this.timer);
      this.timer = null;
    }
  }
}

That .catch(console.error) is load-bearing. Without it, a rejected promise inside the interval is an unhandled rejection. Node 22 crashes the process on those.

My daemon ran fine for a day, then a DNS timeout in the health checker produced an unhandled rejection. Dead process, stale PID file, nobody noticed until the next morning. Added the .catch, hasn't died since.

Why not PM2 / systemd / Docker

PM2 adds a dependency and has its own process management (PID files, logs, restart policies) that can conflict with yours.

systemd is great if you control the box. But this runs on developer laptops. I'm not going to ask macOS users to write a launchd plist.

Docker assumes Docker is installed. On a lot of dev machines, it's not.

Fork works everywhere Node runs. macOS, Linux, Windows (add windowsHide: true to the fork options or you get a console window flash).

Log rotation

One more thing I didn't think about until a test machine had a 200MB log file.

function rotateLog(): void {
  try {
    const stats = fs.statSync(LOG_PATH);
    if (stats.size > 10 * 1024 * 1024) {
      const backup = LOG_PATH + ".1";
      if (fs.existsSync(backup)) fs.unlinkSync(backup);
      fs.renameSync(LOG_PATH, backup);
    }
  } catch { /* first run, no log yet */ }
}

Call this before opening the log fd in startDaemon(). One backup, 10MB cap. Could use logrotate on Linux but again, can't assume it's configured.

Fastify 5 boots in under 50ms, which matters when the user is staring at a terminal. The fork + PID + signal + health check pattern has been running on about a dozen machines for a couple months now with zero babysitting. That's the whole point of a daemon, I guess.

Find me on Substack | StratCraft

Building E2E Encryption in Node.js Without libsodium

Whetlan — Wed, 18 Mar 2026 11:59:01 +0000

I run two daemons on different machines that talk through a WebSocket relay on a cheap VPS. The relay forwards messages between them, and I didn't want it reading any of that traffic. Partly principle, partly because if someone pops the box I don't want cleartext payloads sitting in memory or logs.

The whole thing ended up around ~160 lines of TypeScript. Ed25519 for identity, X25519 for key exchange, AES-256-GCM for the actual encryption. All node:crypto, zero external deps. Node 22.

Two key pairs (blame Node)

Ed25519 and X25519 are both Curve25519. Ed25519 signs. X25519 does Diffie-Hellman.

You can convert between them (libsodium has crypto_sign_ed25519_sk_to_curve25519). I tried. Node.js crypto doesn't expose that conversion. You'd need tweetnacl or libsodium-wrappers, and I was trying to keep deps at zero.

So: two key pairs. Ed25519 is the long-lived identity, persisted to disk. X25519 is ephemeral, generated per connection, never saved.

Persisting the identity

import * as crypto from "node:crypto";
import * as fs from "node:fs/promises";

interface IdentityKeys {
  privateKey: crypto.KeyObject;
  publicKeyHex: string;
}

async function loadOrCreateKeys(keysDir: string): Promise<IdentityKeys> {
  const privPath = `${keysDir}/identity.key`;
  const pubPath = `${keysDir}/identity.pub`;

  try {
    const privDer = await fs.readFile(privPath);
    const pubHex = await fs.readFile(pubPath, "utf-8");
    const privateKey = crypto.createPrivateKey({
      key: privDer,
      format: "der",
      type: "pkcs8",
    });
    return { privateKey, publicKeyHex: pubHex.trim() };
  } catch {
    // first run, generate fresh
  }

  const { privateKey, publicKey } = crypto.generateKeyPairSync("ed25519");

  const privDer = privateKey.export({ type: "pkcs8", format: "der" });
  await fs.writeFile(privPath, privDer);
  await fs.chmod(privPath, 0o600);

  const pubSpki = publicKey.export({ type: "spki", format: "der" });
  // 44 bytes come back. first 12 are ASN.1 header junk. real key is 12-43.
  const publicKeyHex = pubSpki.subarray(12).toString("hex");
  await fs.writeFile(pubPath, publicKeyHex);

  return { privateKey, publicKeyHex };
}

The SPKI export cost me time. You ask for the public key, you get 44 bytes back instead of 32. Sat there running xxd and comparing against RFC 8032 test vectors until I realized the first 12 bytes are ASN.1 wrapping that Node just includes. Slicing the buffer is ugly but it works.

Signing:

function sign(privateKey: crypto.KeyObject, payload: unknown): string {
  const message = Buffer.from(JSON.stringify(payload), "utf-8");
  const signature = crypto.sign(null, message, privateKey);
  return signature.toString("base64");
}

First arg to crypto.sign is the digest algorithm. Ed25519 needs null because the hash is built into the algorithm. I passed "sha256" the first time and got a throw with zero useful context.

ECDH

Each side generates a throwaway X25519 pair per connection:

interface KeyPair {
  publicKey: Buffer;
  privateKey: Buffer;
}

function generateKeyPair(): KeyPair {
  const { publicKey, privateKey } = crypto.generateKeyPairSync("x25519");
  return {
    publicKey: publicKey.export({ type: "spki", format: "der" }),
    privateKey: privateKey.export({ type: "pkcs8", format: "der" }),
  };
}

ECDH, then HKDF:

function deriveSessionKey(
  localPrivateKey: Buffer,
  remotePubKey: Buffer,
): Buffer {
  const privKey = crypto.createPrivateKey({
    key: localPrivateKey,
    format: "der",
    type: "pkcs8",
  });
  const pubKey = crypto.createPublicKey({
    key: remotePubKey,
    format: "der",
    type: "spki",
  });

  const sharedSecret = crypto.diffieHellman({
    privateKey: privKey,
    publicKey: pubKey,
  });

  return Buffer.from(
    crypto.hkdfSync("sha256", sharedSecret, "", "relay-e2e-v1", 32),
  );
}

Don't skip the HKDF step. The raw ECDH output is a curve point, not uniformly random bytes. The info string ("relay-e2e-v1") just binds it to this protocol.

The actual encryption

function encrypt(sessionKey: Buffer, plaintext: string): string {
  const iv = crypto.randomBytes(12);
  const cipher = crypto.createCipheriv("aes-256-gcm", sessionKey, iv);

  const encrypted = Buffer.concat([
    cipher.update(plaintext, "utf8"),
    cipher.final(),
  ]);
  const authTag = cipher.getAuthTag();

  // iv (12) + authTag (16) + ciphertext
  return Buffer.concat([iv, authTag, encrypted]).toString("base64");
}

function decrypt(sessionKey: Buffer, encoded: string): string {
  const data = Buffer.from(encoded, "base64");

  const iv = data.subarray(0, 12);
  const authTag = data.subarray(12, 28);
  const ciphertext = data.subarray(28);

  const decipher = crypto.createDecipheriv("aes-256-gcm", sessionKey, iv);
  decipher.setAuthTag(authTag);

  return Buffer.concat([
    decipher.update(ciphertext),
    decipher.final(),
  ]).toString("utf8");
}

IV + auth tag + ciphertext packed into one base64 string. Receiver knows the layout. No framing needed.

This error wasted my entire night:

Error: Unsupported state or unable to authenticate data

decipher.final() throws that when the auth tag doesn't match. I was convinced my ECDH was broken. Added console.log on both sides, dumped the shared secret hex, they were identical. Stared at the code for way too long.

Turned out I had .toString("base64") on the sender and .toString("base64url") on the receiver. One character difference in the output. The error message tells you nothing about what actually failed.

Over the wire

The relay is just a WebSocket server. Two peers join a room. First message each side sends is its X25519 public key, unencrypted:

// both sides send this on connect
ws.send(JSON.stringify({
  type: "DATA",
  room_id: roomId,
  payload: JSON.stringify({
    _type: "KEY_EXCHANGE",
    pubkey: myKeyPair.publicKey.toString("base64"),
  }),
}));

On the receiving end:

function handleData(roomId: string, payload: string): void {
  const room = rooms.get(roomId);
  if (!room) return;

  try {
    const parsed = JSON.parse(payload);
    if (parsed._type === "KEY_EXCHANGE" && parsed.pubkey) {
      const remotePub = Buffer.from(parsed.pubkey, "base64");
      room.sessionKey = deriveSessionKey(myKeyPair.privateKey, remotePub);
      return;
    }
  } catch {
    // not JSON = encrypted payload
  }

  if (room.sessionKey) {
    try {
      const plaintext = decrypt(room.sessionKey, payload);
      // handle decrypted message
    } catch {
      console.error("decrypt failed for room", roomId);
    }
  }
}

Yeah, JSON.parse failure as the encrypted/unencrypted discriminator. Not elegant. But the key exchange piggybacks on the relay's existing DATA message type, so I didn't have to touch the relay code at all.

DER vs PEM

I store keys as DER. Smaller, no base64 overhead, no -----BEGIN headers.

But pass DER bytes with format: "pem" and you get this:

error:0480006C:PEM routines::no start line

Googled that error, got a bunch of OpenSSL forum posts about certificate chains. Took me 20 minutes to realize I just had the format string wrong:

// this works
crypto.createPrivateKey({ key: derBuffer, format: "der", type: "pkcs8" });

// this doesn't - DER bytes but told Node to expect PEM
crypto.createPrivateKey({ key: derBuffer, format: "pem", type: "pkcs8" });

What this doesn't do

No forward secrecy beyond the session level. If someone records traffic and later grabs the X25519 private key from memory, they can decrypt that session. Real forward secrecy means key ratcheting like Signal does. Per-session ephemeral keys felt like enough for my case. They live in memory and die on disconnect.

The relay can also MITM the key exchange. It sees both X25519 public keys go through, could swap them and sit in the middle. The fix is signing the exchange with Ed25519. Haven't built it yet because I own the relay box. But I know that's a cop-out.

Not using GCM's AAD either, so replaying an encrypted message from one room into another would technically decrypt fine. Low priority when you control both sides.

All node:crypto. The API has gotten a lot less painful since Node 20, and on 22 everything just worked without having to fight KeyObject conversions.

Find me on Substack | StratCraft

I Published an npm Package. Someone Used the Same Name 8 Days Later.

Whetlan — Sat, 14 Mar 2026 11:40:19 +0000

Eight days after I published my first npm package, I found a GitHub repo with the exact same name. Different author, different language, different country. Same name, same domain (AI agent identity), even some of the same crypto primitives.

The package has about 4k lines of TypeScript, four workspace packages, 365 tests. Not a weekend project. So finding a repo called the same thing, first commit dated eight days after my npm publish, with a version tag of "v6.0" on that first commit... that was a weird morning.

npm timestamps don't lie

First thing I did was check what I actually had on my side.

$ npm view my-package time
{
  '0.0.1': '2026-03-01T...',
  '0.2.0': '2026-03-05T...',
  '0.2.5': '2026-03-05T...',
  '0.2.8': '2026-03-08T...'
}

npm publish timestamps are immutable. Git commits can be rebased and rewritten. npm timestamps can't.

Their first commit: March 9. My 0.0.1: March 1. That settled the priority question. But I did a lot of other stuff anyway, because panic is a hell of a motivator.

So how do naming collisions actually work?

Here's the thing I didn't fully appreciate before this happened: npm, PyPI, and crates.io are completely independent namespaces. Owning my-package on npm gives you zero claim to my-package on PyPI. There's no cross-registry reservation system. There's no trademark-like protection unless you actually have a trademark.

The other project was in Python. They couldn't use my npm name because I already had it. So they published on PyPI and built a website instead.

Totally legal. Feels weird, but totally legal.

The 45-minute panic

Here's what I did in the first hour after finding the repo:

Checked PyPI. My name wasn't taken. Registered it immediately with a placeholder package.

$ pip install twine build
# created a minimal pyproject.toml with just the name and version
$ python -m build && twine upload dist/*

PyPI's registration process is straightforward but I'd never done it before. Spent 15 minutes figuring out that you need a pyproject.toml now, not a setup.py. The Python packaging ecosystem changes its mind about this every two years.

Checked crates.io. Also not taken. Registered four variations.

$ cargo init my-package --lib
$ cargo publish --allow-dirty

crates.io requires a valid Cargo.toml with a license field and a non-empty src/lib.rs. My placeholder packages have exactly one line of Rust:

// placeholder - real implementation coming

Is this a good use of package registries? Probably not. But the alternative was watching someone else grab them.

Checked the GitHub username. Taken, but by an inactive account with zero repos. Nothing I can do about that.

Total time: about 45 minutes. Could have been 10 if I'd done this before the first npm publish.

What tipped me off

I wasn't actively searching for copycats. I was checking if anyone had used a similar name before I registered on more platforms. Standard due diligence, a week late.

The repo had 32 commits in 3 days. All by one person. The first commit was tagged "v6.0", which is a bold move for a repo that didn't exist the day before. Most of the code looked AI-generated (one commit had a co-author tag from an AI assistant). There was a Discord bot, a Supabase backend, some Solana stubs that didn't compile.

I don't want to get into whether this counts as "copying." The name is unusual enough that it's not a coincidence. But the implementation was completely different, and open source doesn't have naming courts.

What bothered me more than the copying was realizing how exposed I'd been. Eight days with an npm package and zero presence anywhere else. If they'd moved faster, they could have grabbed the PyPI name first.

What I'd actually recommend

If you're publishing an open source package with a distinctive name, do this stuff on day one. Not day eight.

Before your first npm publish:

Check these registries even if you have no plans to publish there:

PyPI (pip install your name, see if it 404s)
crates.io (search, or cargo search your-name)
The GitHub username
The .com / .dev / .ai domains

In your package.json:

{
  "homepage": "https://github.com/your-org/your-package",
  "repository": {
    "type": "git",
    "url": "https://github.com/your-org/your-package"
  }
}

Sounds obvious but I didn't have the homepage field until after the incident. Google uses it when indexing npm packages.

In your README, near the top:

Something like "Originally published March 2026 on npm" with a link. Not aggressive, just a timestamp that lives in your repo.

Register placeholder packages on at least PyPI and crates.io. A minimal package with just a name and description takes 5 minutes per registry. The cost is near zero. The alternative is doing it in a panic at 11pm on a weekday.

The version tag thing still bugs me

Their repo's first commit was tagged v6.0. Not v0.1.0, not v1.0.0. Version 6.

I can think of two reasons to do this. One is that they had five previous major versions in a private repo and decided to open-source at v6. The other is that they wanted the project to look mature.

The repo had zero stars, zero forks, zero issues, one contributor. I'll let you decide which explanation is more likely.

npm doesn't care, and that's fine

I reached out to npm support just to understand the policy. There isn't one for cross-platform naming conflicts. npm's stance is basically: you own the name on npm, that's it. What happens on PyPI or GitHub is outside their scope.

Which is the right answer, honestly. A centralized cross-platform naming authority would be a nightmare. Imagine disputing a crate name because you own the npm package. The governance complexity alone would be unworkable.

The tradeoff is that defensive registration falls on you. Nobody else is going to protect your name across registries.

Eight days is a long time

That's the actual takeaway for me. I built the thing, tested extensively, published to npm, and then just... didn't think about other registries. For eight days. In a space where someone can scaffold an entire project with AI assistance in a weekend.

If you're publishing something with a name you care about, spend the extra 30 minutes on day one. It's boring, it feels premature, and it will save you a very unpleasant surprise.

My package registrations now look like this:

npm:       ✅ (day 1)
PyPI:      ✅ (day 9, should have been day 1)
crates.io: ✅ (day 9)
GitHub org: ✅ (day 1)

Four out of four. Would have been a lot less stressful if I'd done them all on the same day.

Find me on Substack | StratCraft

pnpm Workspaces in Production: What Actually Matters

Whetlan — Fri, 13 Mar 2026 09:42:08 +0000

I've been running a pnpm workspaces monorepo in production. Four packages, all TypeScript, about 8k lines total, published to npm. Node 22, pnpm 9, no build tool beyond tsc. Before I started I read a dozen "monorepo setup" articles and most of them were 2000 words on Turborepo vs Nx vs Lerna, and maybe two paragraphs on the stuff that actually breaks on a random Tuesday afternoon.

This is the Tuesday afternoon stuff.

Two lines of config

My workspace config is two lines:

# pnpm-workspace.yaml
packages:
  - "packages/*"

That's it. Every directory under packages/ is a workspace package. The root package.json is private and just orchestrates:

{
  "private": true,
  "scripts": {
    "build": "pnpm -r build",
    "dev": "pnpm -r --parallel dev",
    "test": "pnpm -r test",
    "clean": "pnpm -r --parallel exec rm -rf dist"
  },
  "engines": {
    "node": ">=22",
    "pnpm": ">=9"
  }
}

-r means "run in every package." --parallel on dev and clean because those don't depend on each other. But build runs sequentially, because one of my packages imports from another and needs it compiled first. pnpm figures out the dependency order on its own, so the dependent package always builds after its dependency.

I spent zero time choosing between Turborepo and Nx. pnpm -r handles orchestration, tsc handles compilation. That's it.

Shared tsconfig (the part that actually saves time)

Every monorepo article tells you to make a shared base tsconfig. They're right. But nobody tells you which settings go in the base vs per-package.

My base:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "Node16",
    "moduleResolution": "Node16",
    "lib": ["ES2022"],
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "declaration": true,
    "declarationMap": false,
    "sourceMap": false
  }
}

Each package extends it:

{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "rootDir": "src",
    "outDir": "dist"
  },
  "include": ["src/**/*"]
}

rootDir and outDir are per-package because they're relative paths. Everything else is shared.

Before this, I had four slightly different tsconfigs and couldn't remember which one had strictNullChecks. Now I change a setting once and it applies everywhere.

What About TypeScript Project References?

I don't use them. composite: true gives you cross-package type checking at build time, but you have to maintain a references array in every tsconfig, keep them in sync with your dependency graph, and deal with tsBuildInfo files that get stale and produce phantom errors.

Four packages, one internal dependency. I just build them in order. If I had ten packages or builds started taking minutes instead of seconds, I'd probably reconsider.

workspace:* vs workspace:^

When one package depends on another in the monorepo:

{
  "dependencies": {
    "my-daemon": "workspace:*"
  }
}

During development this creates a symlink. Live, always-current version of the dependency. No rebuild needed.

At publish time, pnpm replaces workspace:* with the actual version number. So "my-daemon": "workspace:*" becomes "my-daemon": "0.2.7" in the published package.json.

Here's where it bit me: I initially had workspace:^ (with a caret). Published, and the dependency became "^0.2.7". Consumers could install mismatched minor versions. For tightly coupled internal deps, use workspace:* for exact version pinning.

Phantom dependencies will find you

This one cost me a full afternoon. pnpm uses a strict node_modules structure by default. Packages can only access dependencies they explicitly declare. Which is great, until you realize half your code was relying on phantom dependencies and you didn't know it.

If package A depends on fastify and package B doesn't declare it, B can't import fastify even though it's installed in the monorepo. With npm or yarn, hoisting would let B "accidentally" use A's fastify. You'd never notice until you publish and a user's install fails.

I had this exact bug. One of my packages was importing a type from @types/ws without declaring it as a devDependency. Worked fine locally because another package had it, and VS Code happily resolved it through the workspace. Published to npm, got an issue within two days:

error TS2307: Cannot find module 'ws' or its corresponding type declarations.

Felt pretty dumb.

The fix: run pnpm why in each workspace, check that every import has a matching declaration:

$ cd packages/client && pnpm why @types/ws
# nothing? that's your bug
$ pnpm add -D @types/ws

Boring work, but it would have saved me an embarrassing npm publish.

Vitest + workspaces

Vitest has a workspace feature. My root config just lists the packages:

// vitest.workspace.ts
import { defineWorkspace } from "vitest/config";

export default defineWorkspace([
  "packages/server",
  "packages/client",
  "packages/cli",
  "packages/core",
]);

Per-package config:

import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    testTimeout: 10_000,
    restoreMocks: true,
  },
});

pnpm test from root runs all suites. pnpm --filter server test runs just one.

One gotcha: if your tests import from sibling packages, make sure you've built the dependency first. Vitest doesn't trigger builds. I ended up with a pretest script that runs pnpm -r build before the test suite. Wasteful, rebuilds everything even if nothing changed. But the alternative is forgetting to build and then spending 20 minutes debugging why the types don't match.

Publishing to npm

No Changesets, no Lerna. I bump versions manually and pnpm publish from each package directory.

The prepublishOnly hook:

{
  "scripts": {
    "prepublishOnly": "pnpm build"
  }
}

Without this, you will eventually publish stale dist/ files. Or no dist/ at all. I did this on my second publish. Ran npm info on the package, stared at the file list, wondered why dist/ was from three commits ago. Took me way too long to realize I just forgot to build before publishing.

Also, explicit files field:

{
  "files": ["dist", "README.md"],
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "exports": {
    ".": {
      "types": "./dist/index.d.ts",
      "default": "./dist/index.js"
    }
  }
}

I once published a package that accidentally included my src/ directory, test fixtures, and a 4MB debug log. The files field is an allowlist. Only what you list gets published. npm pack --dry-run is your friend here, run it once before every publish and actually read the output.

Stuff I tried that wasn't worth it

I tried Turborepo for about a day. My full build takes under 30 seconds. Remote caching and smart task scheduling are solving a problem I don't have. pnpm -r build is the whole build system.

I also see monorepos with a packages/eslint-config internal package. For four packages that's a lot of ceremony for an eslint config. Mine lives at the root, each package references it with a relative path. Done.

Early on I extracted a "shared utils" package. It had three functions in it. That's not a package, that's a file. I deleted it and just duplicated the two functions that were actually shared. Less abstraction, fewer symlink headaches.

And I gave up on synchronized version numbers pretty quickly. My client library and server ship at different cadences. Forcing v0.3.1 on both would mean publishing no-op releases just to keep numbers aligned.

Keep it boring

After all of this, the thing I keep coming back to is: keep it boring. pnpm-workspace.yaml should be two lines. Your root package.json should have like five scripts. If your monorepo setup needs a README to explain how it works, you've gone too far.

When something breaks, don't reach for shamefully-hoist=true in .npmrc. Figure out why it broke. Nine times out of ten it's a missing dependency declaration, and your users will hit the same thing when they install your package.

And put prepublishOnly hooks in every publishable package. I cannot stress this enough. Future you will forget to build before publishing. It's not a question of if.

Four packages, plain pnpm, no Turborepo, no Nx. If build times become a problem I'll add something. They haven't yet.

Find me on Substack | StratCraft

Building Zero-Config LAN Discovery in Node.js (mDNS + UDP Broadcast)

Whetlan — Thu, 12 Mar 2026 11:00:02 +0000

I was building a small daemon that runs on multiple machines in a LAN — think a couple of dev servers, a NAS, maybe a Raspberry Pi. I wanted them to discover each other automatically. No central server, no config files listing IP addresses, no "go edit this YAML and restart."

Just start it and it should find its peers.

I ended up layering three approaches: mDNS, a custom UDP broadcast protocol I basically stole from 1990s Novell NetWare, and a brute-force subnet scanner as the last resort. The whole thing is ~300 lines of TypeScript.

mDNS: great on paper

First attempt was pure mDNS. Typical setup:

Laptop (Wi-Fi) → office AP → corporate router → server running the daemon

In theory mDNS handles this fine. In practice it worked on my MacBook at home, worked in a small office, then fell apart on real networks. Routers silently dropping multicast. Access points with client isolation. One corporate network had multicast disabled and nobody in IT could tell me why.

Node.js mDNS annoyances:

No built-in support — you need the multicast-dns npm package
macOS handles it natively (Bonjour), Linux needs avahi, Windows is a coin flip
Port 5353 conflicts if avahi-daemon is already running
Fails silently. Your queries just go into the void.

Stealing from 1990s NetWare

Novell NetWare had SAP — Service Advertising Protocol. Every server periodically broadcasts "hey, I exist, here's what I do" to the entire subnet. No multicast groups, no special infrastructure.

I built the same thing on UDP/IP. Two message types:

interface DiscoverMessage {
  type: "discover";
  version: 1;
}

interface AnnounceMessage {
  type: "announce";
  version: 1;
  instance_id: string;
  display_name: string;
  port: number;
  tls: boolean;
}

Node starts up, broadcasts discover. Running instances reply with announce. Periodic announce on a timer so late joiners get found too.

Socket setup

import * as dgram from "node:dgram";
import * as os from "node:os";

const DISCOVERY_PORT = 17891;

async function startDiscovery(): Promise<dgram.Socket> {
  return new Promise((resolve, reject) => {
    const socket = dgram.createSocket("udp4");

    socket.on("error", (err) => {
      if (!socket.address()) {
        reject(err); // failed during bind
      }
    });

    socket.bind(DISCOVERY_PORT, () => {
      socket.setBroadcast(true); // DO NOT FORGET THIS
      resolve(socket);
    });
  });
}

I wasted two hours on this. Ran tcpdump, stared at Wireshark captures, swapped ports, blamed the firewall.

Turns out I forgot socket.setBroadcast(true). Node.js won't send to broadcast addresses without it — doesn't throw, doesn't warn. The packets just vanish.

Broadcast addresses

On a machine with multiple NICs you need the per-subnet broadcast address, not just 255.255.255.255:

function getBroadcastTargets(): string[] {
  const targets: string[] = [];
  const ifaces = os.networkInterfaces();

  for (const [name, addrs] of Object.entries(ifaces)) {
    if (!addrs || isVirtualInterface(name)) continue;
    for (const addr of addrs) {
      if (addr.family === "IPv4" && !addr.internal) {
        targets.push(calcBroadcast(addr.address, addr.netmask));
      }
    }
  }
  return targets;
}

// broadcast = IP | ~mask, nothing fancy
function calcBroadcast(ip: string, mask: string): string {
  const ipParts = ip.split(".").map(Number);
  const maskParts = mask.split(".").map(Number);
  return ipParts
    .map((b, i) => (b | (~maskParts[i]! & 0xff)))
    .join(".");
}

192.168.1.42 with mask 255.255.255.0 → 192.168.1.255. Bitwise OR with inverted netmask.

Filtering virtual interfaces

Docker running + WireGuard VPN = my daemon "discovering" phantom peers. Broadcasting over Docker bridges and VPN tunnels, getting garbage back.

function isVirtualInterface(name: string): boolean {
  const n = name.toLowerCase();
  return (
    n.startsWith("wg") ||       // WireGuard
    n.startsWith("tun") ||      // TUN (OpenVPN)
    n.startsWith("tap") ||      // TAP
    n.startsWith("docker") ||   // Docker
    n.startsWith("br-") ||      // Docker bridge
    n.startsWith("veth") ||     // Docker veth
    n.startsWith("virbr") ||    // KVM
    n.startsWith("vmnet") ||    // VMware
    n.startsWith("vboxnet")     // VirtualBox
  );
}

Hardcoded prefix list. os.networkInterfaces() gives you zero info about physical vs virtual. On Linux you could parse sysfs but that doesn't help on macOS or Windows.

TCP probe before registering

UDP is unverified. Before registering a peer:

async function handleAnnounce(
  data: AnnounceMessage,
  rinfo: dgram.RemoteInfo,
): Promise<void> {
  // own broadcast echoing back — yes this happens
  if (isSelfIp(rinfo.address)) return;

  // not on our subnet? reject
  if (!isInLocalSubnet(rinfo.address)) return;

  // actually try to connect before believing it
  const alive = await tcpProbe(rinfo.address, data.port);
  if (!alive) return;

  registerPeer({
    id: data.instance_id,
    address: rinfo.address,
    port: data.port,
    source: "broadcast",
  });
}

You might get an announce from a service that crashed 30 seconds ago. TCP health check with 2-second timeout filters out the ghosts.

Your own broadcasts also echo back. Found that when my daemon kept discovering itself.

Announce jitter

Without randomization, 10 simultaneous instances all broadcast at the exact same second forever:

const ANNOUNCE_INTERVAL = 60_000;
const ANNOUNCE_JITTER = 10_000;

function scheduleNextAnnounce(socket: dgram.Socket): void {
  const delay = ANNOUNCE_INTERVAL + Math.random() * ANNOUNCE_JITTER;
  setTimeout(() => {
    broadcastAnnounce(socket);
    scheduleNextAnnounce(socket);
  }, delay);
}

60s base + 0-10s random jitter. Same thing NTP does.

I still use mDNS though

Despite everything above, I run mDNS alongside the broadcast protocol. On home networks it works fine, and other programs can discover your service too.

const mDNS = require("multicast-dns");
const mdns = mDNS();

const SERVICE_TYPE = "_my-service._tcp.local";

setInterval(() => {
  mdns.query({
    questions: [{ name: SERVICE_TYPE, type: "PTR" }],
  });
}, 30_000);

mdns.on("response", (response, rinfo) => {
  const allRecords = [
    ...response.answers,
    ...response.additionals,
  ];

  const srvRecords = allRecords.filter((r) => r.type === "SRV");
  const aRecords = allRecords.filter((r) => r.type === "A");

  for (const srv of srvRecords) {
    const port = srv.data.port;
    const target = srv.data.target;

    const aRecord = aRecords.find((r) => r.name === target);
    const ip = aRecord ? String(aRecord.data) : rinfo.address;

    console.log(`Found peer: ${ip}:${port}`);
  }
});

One gotcha: lots of services bind to 127.0.0.1 and advertise that via mDNS. You connect to the A record and hit yourself.

Fix — use the UDP packet's source IP when the A record is loopback:

const aRecordIp = aRecord ? String(aRecord.data) : "";
const isLoopback =
  aRecordIp === "127.0.0.1" ||
  aRecordIp === "::1" ||
  aRecordIp.startsWith("127.");

const address = aRecordIp && !isLoopback
  ? aRecordIp
  : rinfo.address;

I made mDNS a soft dependency — if multicast-dns isn't installed, broadcast-only mode:

let mdns = null;
try {
  const mDNS = require("multicast-dns");
  mdns = mDNS();
} catch {
  console.warn("mDNS unavailable, broadcast-only mode");
}

Last resort: subnet scan

When passive discovery doesn't cut it:

const CONCURRENCY = 50;
const TIMEOUT = 2_000;

async function scanSubnet(subnet: string, port: number) {
  const targets = [];
  for (let i = 1; i <= 254; i++) {
    targets.push({ host: `${subnet}.${i}`, port });
  }


---

*Part of [ClawNexus](https://silverstream.tech/clawnexus), an open-source identity registry for AI agents.*
  const queue = [...targets];

  const worker = async () => {
    while (queue.length > 0) {
      const target = queue.shift()!;
      const alive = await probeHost(target.host, target.port);
      if (alive) console.log(`Found: ${target.host}:${target.port}`);
    }
  };

  // 50 workers chewing through the queue
  const workers = Array.from(
    { length: Math.min(CONCURRENCY, targets.length) },
    () => worker(),
  );
  await Promise.allSettled(workers);
}

50 concurrent probes, 2-second timeout. Full /24 in about 10 seconds. I expose it as an on-demand API endpoint.

What I'd change

Should have filtered virtual interfaces on day one. That one wasted an entire evening — I kept thinking my broadcast protocol was broken until I noticed Docker bridges were happily replying to my packets.

Should have used a binary header for UDP instead of raw JSON. Would have prevented the JSON.parse crash when a broken mDNS responder sent invalid data. I knew it could throw. Didn't wrap it. Classic.

Test on Windows earlier. The firewall silently blocks UDP on custom ports. Ended up adding netsh advfirewall rule creation.

Only dependencies are dgram and os from Node, plus multicast-dns if you want it.

Fought with mDNS on corporate networks? What ended up working?

Find me on Substack | StratCraft

DEV Community: Whetlan

I Asked an LLM to Generate 20 Trading Strategies. 14 Were the Same Thing.

People are giving these things real money now

It's not the model. It's what the model sees.

Three generations of trading software

Two failure modes

Strategy Hallucination

Backtest Overfitting Blindness

What seems to work so far

Don't let the LLM pick parameters

Treat outputs as hypotheses

Check for actual diversity

What I keep thinking about

Why AI Code Needs the Same Rigor We Should Have Been Using All Along

Where things actually go wrong

You don't know what sticks around

What I've Settled On: Test Everything, Then Test It Again

Where It Gets Iterative

When the code quietly changes meaning on you

What Else Changed

The part that's actually hard

Your AI Agents Can Talk. They Just Can't Find Each Other.

Protocols assume you already know where to look

Agents aren't microservices

What I built

After discovery, things get fuzzy

I'm not sure this is the right abstraction

The Hardest Part of Modern C++ Isn't the Language.

C with classes

50,000 lines by hand

The pipe organ

The Swiss watch inside

1,000 lines to 10

The planet I couldn't reach

The registrant

What actually changed

Rewriting a FIX Engine in C++23: What Got Simpler (and What Didn't)

The pool that disappeared

consteval tag lookup

SIMD: the scenic route

What didn't get simpler

Benchmarks

Where I am with it

The Trident and The Green Ox

From C to modern C++

50,000 lines of C++, then 3.5 million lines of Python

A 14-day dead end

Cutting 90%

Claude 3.5 to 4.6

A domain name and a new project

AI picked the popular choice. The popular choice didn't work.

300 unit tests weren't enough

Three days after launch, a copycat appeared

The Trident and The Green Ox

npm Supply Chain Security: Mistakes I Made Publishing My First Packages

postinstall

Provenance (the silent failure that got me)

2FA doesn't protect what you think it does

Someone ran npm install in my pnpm repo

I almost shipped a .env

npm audit is noisy

--ignore-scripts

Running a Node.js Daemon with Fastify (No PM2, No systemd)

Fork, detach, forget

PID files and stale processes

Fastify as the daemon core

Health check after fork

Shutdown ordering matters

start / stop / status / restart

Interval callbacks will kill your daemon

Why not PM2 / systemd / Docker

Log rotation

Building E2E Encryption in Node.js Without libsodium

Two key pairs (blame Node)

Persisting the identity

ECDH

The actual encryption

Over the wire

DER vs PEM

What this doesn't do

Someone ran `npm install` in my pnpm repo

I almost shipped a `.env`

`npm audit` is noisy

`--ignore-scripts`