DEV Community: Bill Tu

Adding IOC, FOK, and Stop Orders to a Matching Engine

Bill Tu — Fri, 24 Apr 2026 06:23:51 +0000

A matching engine that only supports limit and market orders is a toy. Real exchanges need orders that expire, orders that demand all-or-nothing execution, and orders that sleep until a price condition is met. This article covers the implementation of time-in-force policies (GTC, IOC, FOK) and conditional stop orders (Stop-Market, Stop-Limit) in MatchEngine.

Time-in-Force: How Long Does an Order Live?

Time-in-force (TIF) controls what happens to an order's unfilled remainder after the matching attempt.

GTC: Good-Till-Cancelled

This is the default behavior that already existed. A limit order rests in the book until it is filled by a future incoming order or explicitly cancelled. Nothing changed here — we just gave it a name.

IOC: Immediate-or-Cancel

An IOC order fills as much as possible immediately, then the remainder is discarded. It never rests in the book.

Use case: a trader wants to buy up to 10 BTC at 50,000, but only if liquidity is available right now. If only 3 BTC are available, they get 3 and the remaining 7 are cancelled — not left in the book to be filled later at a potentially worse time.

The implementation is two lines in SubmitOrder, after the matching loop:

// IOC: discard any remaining quantity (never rests in book)
if order.TIF == model.IOC {
    order.Remaining = decimal.Zero
}

// Only GTC limit orders rest in the book
if !order.IsFilled() && order.Type == model.Limit {
    book.AddOrder(order)
    e.orderIndex[order.ID] = symbol
}

By setting Remaining to zero, the subsequent IsFilled() check returns true, and the order is never added to the book. The trades that were already produced are returned normally.

FOK: Fill-or-Kill

A FOK order must be filled entirely in a single matching attempt, or it is rejected completely. No partial fills, no resting.

Use case: a trader needs exactly 100 shares to complete a hedge. Getting 50 is worse than getting 0, because a partial hedge creates unbalanced risk.

FOK requires a pre-check before matching. We cannot run the matching loop and then "undo" partial fills — the resting orders have already been modified. Instead, we check available liquidity first:

if order.TIF == model.FOK {
    if !e.canFillCompletely(book, order) {
        return nil, nil // reject silently
    }
}

The canFillCompletely method walks the opposite side of the book, summing available quantity at compatible prices:

func (e *Engine) canFillCompletely(book *orderbook.OrderBook, order *model.Order) bool {
    available := decimal.Zero
    if order.Side == model.Buy {
        for _, ask := range book.Asks() {
            if order.Type == model.Limit && ask.Price.GreaterThan(order.Price) {
                break // past the price limit
            }
            available = available.Add(ask.Remaining)
            if available.GreaterThanOrEqual(order.Remaining) {
                return true // enough liquidity
            }
        }
    } else {
        for _, bid := range book.Bids() {
            if order.Type == model.Limit && bid.Price.LessThan(order.Price) {
                break
            }
            available = available.Add(bid.Remaining)
            if available.GreaterThanOrEqual(order.Remaining) {
                return true
            }
        }
    }
    return false
}

Key detail: the pre-check respects price limits. A FOK buy at 100 won't count liquidity at 101 — even though a market FOK would. The check mirrors the matching loop's price constraint exactly.

If the pre-check passes, the matching loop runs normally and is guaranteed to fill the entire order (assuming no concurrent modification, which the mutex prevents).

The TIF Enum

type TimeInForce int

const (
    GTC TimeInForce = iota // default
    IOC
    FOK
)

The zero value is GTC, which means all existing orders that don't set TIF behave exactly as before. Backward compatible by default.

Stop Orders: Sleeping Until a Price Trigger

Stop orders are fundamentally different from limit and market orders. They don't participate in matching immediately. Instead, they wait for a price condition to be met, then convert into an active order.

The Trigger Mechanism

The engine tracks the last trade price per symbol:

lastPrices map[string]decimal.Decimal

Every time a trade is recorded, the last price is updated:

func (e *Engine) recordTrades(symbol string, trades []model.Trade) {
    for _, t := range trades {
        e.lastPrices[symbol] = t.Price
        // ... record to log, invoke callback
    }
}

After each SubmitOrder that produces trades, the engine checks if any pending stop orders should be triggered:

if len(trades) > 0 {
    triggered := e.triggerStopOrders(symbol)
    trades = append(trades, triggered...)
}

Stop-Market

A stop-market order becomes a market order when triggered. A buy stop triggers when the last price rises to or above the stop price. A sell stop triggers when the last price falls to or below the stop price.

func (e *Engine) isStopTriggered(stop *model.Order, lastPrice decimal.Decimal) bool {
    if stop.Side == model.Buy {
        return lastPrice.GreaterThanOrEqual(stop.StopPrice)
    }
    return lastPrice.LessThanOrEqual(stop.StopPrice)
}

When triggered, the stop order is converted to a market order and submitted to the matching loop:

if stop.Type == model.StopMarket {
    active = model.NewMarketOrder(stop.ID, stop.Side, stop.Remaining)
}

Stop-Limit

A stop-limit order becomes a limit order when triggered. It has both a stop price (the trigger) and a limit price (the price of the resulting limit order).

if stop.Type == model.StopLimit {
    active = model.NewLimitOrder(stop.ID, stop.Side, stop.Price, stop.Remaining)
}

The difference matters: a stop-market order will fill at whatever price is available (potentially with slippage). A stop-limit order will only fill at the limit price or better — but it might not fill at all if the market moves past the limit.

Storage

Stop orders are stored separately from the order book:

stopOrders map[string][]*model.Order // symbol -> pending stops

They are not in the order book because they should not be visible to other participants and should not be matchable until triggered. They are in the order index (for duplicate ID detection and cancellation).

Cancellation

Stop orders can be cancelled before they trigger:

func (e *Engine) CancelOrder(symbol, orderID string) bool {
    // Try order book first
    if ok && book.RemoveOrder(orderID) { ... }

    // Try stop orders
    stops := e.stopOrders[symbol]
    for i, s := range stops {
        if s.ID == orderID {
            e.stopOrders[symbol] = append(stops[:i], stops[i+1:]...)
            delete(e.orderIndex, orderID)
            return true
        }
    }
    return false
}

The Trigger Loop

When stop orders are triggered, they can produce trades, which update the last price, which could trigger more stop orders. This is handled by the triggerStopOrders method processing all triggered stops in a single pass:

func (e *Engine) triggerStopOrders(symbol string) []model.Trade {
    lastPrice := e.lastPrices[symbol]
    var remaining []*model.Order
    var allTrades []model.Trade

    for _, stop := range e.stopOrders[symbol] {
        if e.isStopTriggered(stop, lastPrice) {
            // Convert to active order and match
            active := convertStop(stop)
            trades := e.match(book, active)
            e.recordTrades(symbol, trades)
            allTrades = append(allTrades, trades...)
        } else {
            remaining = append(remaining, stop)
        }
    }

    e.stopOrders[symbol] = remaining
    return allTrades
}

Note: triggered stop orders that produce trades will update lastPrices, but we don't recursively re-check stops in the same call. This prevents cascading trigger loops. The next SubmitOrder that produces trades will check stops again.

The Request API

New convenience constructors make it easy to create all order types:

// IOC limit order
req := model.NewIOCOrderRequest(model.Buy, price, qty)

// FOK limit order
req := model.NewFOKOrderRequest(model.Buy, price, qty)

// Stop-market
req := model.NewStopMarketRequest(model.Buy, stopPrice, qty)

// Stop-limit
req := model.NewStopLimitRequest(model.Buy, stopPrice, limitPrice, qty)

// Any request can chain builders
req := model.NewLimitOrderRequest(model.Buy, price, qty).
    WithOwner("alice").
    WithTIF(model.IOC)

How It All Fits in SubmitOrder

The updated SubmitOrder flow:

SubmitOrder(symbol, order)
│
├── Validate (nil, qty, price, stop price)
├── Normalize symbol
├── Check duplicate ID
│
├── Is stop order?
│   └── Yes → store in stopOrders, return
│
├── Is FOK?
│   └── Yes → canFillCompletely? No → return empty
│
├── Match against book
│
├── Is IOC?
│   └── Yes → set Remaining = 0
│
├── Has unfilled remainder + is Limit?
│   └── Yes → add to book (GTC only, IOC already zeroed)
│
├── Record trades, update lastPrice
│
└── Any trades produced?
    └── Yes → triggerStopOrders → append triggered trades

Each new feature is a small branch in the flow. The matching loop itself (matchBuy, matchSell) is completely unchanged — TIF and stop logic live outside the loop.

Test Coverage

Test	What It Verifies
`TestIOCFillsAndDiscards`	IOC fills partial, discards rest
`TestIOCNoMatchDiscards`	IOC with no liquidity produces 0 trades, 0 resting
`TestIOCViaRequest`	IOC through SubmitRequest API
`TestFOKFullFill`	FOK fills when enough liquidity exists
`TestFOKRejectInsufficientLiquidity`	FOK rejected, resting orders untouched
`TestFOKViaRequest`	FOK through SubmitRequest API
`TestStopMarketBuy`	Buy stop triggers on price rise
`TestStopMarketSell`	Sell stop triggers on price drop
`TestStopLimitOrder`	Stop-limit converts to limit and matches
`TestStopOrderCancel`	Stop cancelled before trigger
`TestStopViaRequest`	Stop through SubmitRequest API
`TestGTCIsDefault`	Default TIF is GTC, order rests in book

Summary

Feature	Implementation
GTC	Default `TimeInForce` zero value. No code change needed.
IOC	Set `Remaining = 0` after matching. 2 lines.
FOK	Pre-check liquidity with `canFillCompletely`. ~20 lines.
Stop-Market	Store in `stopOrders`, trigger on last price, convert to market.
Stop-Limit	Same trigger mechanism, convert to limit instead.
Last price tracking	Updated in `recordTrades`. 1 line.

The matching loop (matchBuy/matchSell) was not modified. All new behavior lives in SubmitOrder (before and after the match) and in triggerStopOrders (after trade recording). This keeps the core algorithm clean and the new features isolated.

GitHub: https://github.com/iwtxokhtd83/MatchEngine
Release: v0.6.0
Issue: #8 — Support IOC, FOK, GTC, and Stop order types

Implementing Self-Trade Prevention in a Matching Engine

Bill Tu — Tue, 21 Apr 2026 06:06:55 +0000

When a trader's buy order matches against their own sell order, the result is a self-trade — also known as a wash trade. No economic value is exchanged. The trader is both buyer and seller. On regulated exchanges, this is prohibited because it artificially inflates volume and can be used for market manipulation.

This article walks through the implementation of self-trade prevention (STP) in MatchEngine, an open-source order matching engine written in Go. We added four configurable STP modes, each with different behavior when a self-trade is detected.

The Problem

Before this change, the engine had no concept of order ownership. Every order was anonymous:

type Order struct {
    ID        string
    Side      Side
    Type      OrderType
    Price     decimal.Decimal
    Quantity  decimal.Decimal
    Remaining decimal.Decimal
    Timestamp time.Time
}

If Alice placed a sell at 100 and then a buy at 100, the engine would happily match them. Alice would be both the maker and the taker. The trade would appear in the trade log, inflate the volume counter, and potentially trigger downstream systems (risk checks, fee calculations, market data feeds) — all for a transaction that moved nothing between parties.

The Solution: OwnerID + STP Modes

Step 1: Add Ownership

We added an OwnerID field to the Order struct:

type Order struct {
    ID        string
    OwnerID   string          // identifies the trader
    Side      Side
    Type      OrderType
    Price     decimal.Decimal
    Quantity  decimal.Decimal
    Remaining decimal.Decimal
    Timestamp time.Time
}

And to OrderRequest for the auto-ID API:

type OrderRequest struct {
    OwnerID  string
    Side     Side
    Type     OrderType
    Price    decimal.Decimal
    Quantity decimal.Decimal
}

// Builder method for fluent API
func (r OrderRequest) WithOwner(ownerID string) OrderRequest {
    r.OwnerID = ownerID
    return r
}

The OwnerID is a plain string. It could be a user ID, an account number, a firm identifier — whatever the caller uses to distinguish participants. The engine does not interpret it; it only checks equality.

Step 2: Define STP Modes

Different exchanges handle self-trades differently. We implemented the four most common modes:

type STPMode int

const (
    STPNone           STPMode = iota // disabled (default)
    STPCancelResting                  // cancel the resting order
    STPCancelIncoming                 // cancel the incoming order
    STPCancelBoth                     // cancel both
    STPDecrement                      // reduce both by overlap qty
)

Each mode answers the question: "When a self-trade is about to happen, what do we do with the two orders?"

Mode	Resting Order	Incoming Order	Trade Produced?
CancelResting	Removed from book	Continues matching	No
CancelIncoming	Stays in book	Cancelled entirely	No
CancelBoth	Removed from book	Cancelled entirely	No
Decrement	Reduced by overlap	Reduced by overlap	No

The mode is configured at the engine level:

e := engine.New(engine.WithSTPMode(model.STPCancelResting))

Step 3: Detection

Self-trade detection is a simple equality check:

func isSelfTrade(a, b *model.Order) bool {
    return a.OwnerID != "" && a.OwnerID == b.OwnerID
}

Two important details:

Empty OwnerID bypasses STP. If either order has no owner, they always match. This preserves backward compatibility — existing code that doesn't set OwnerID continues to work exactly as before.
The check is string equality. No normalization, no case folding. The caller is responsible for consistent owner identifiers. This is deliberate — the engine should not guess what "alice" vs "Alice" means in the caller's domain.

Step 4: Integration Into the Matching Loop

The STP check is inserted into the matching loop, right after the price check and before trade execution:

func (e *Engine) matchBuy(book *orderbook.OrderBook, order *model.Order) []model.Trade {
    var trades []model.Trade

    for !order.IsFilled() {
        bestAsk := book.BestAsk()
        if bestAsk == nil {
            break
        }
        if order.Type == model.Limit && bestAsk.Price.GreaterThan(order.Price) {
            break
        }

        // Self-trade prevention
        if e.stpMode != model.STPNone && isSelfTrade(order, bestAsk) {
            if e.handleSTP(book, order, bestAsk) {
                continue // resting removed, try next level
            }
            break // incoming cancelled
        }

        trade := executeTrade(order, bestAsk, bestAsk.Price)
        trades = append(trades, trade)
    }

    return trades
}

The handleSTP method returns a boolean: true means "continue matching" (the resting order was removed, try the next one), false means "stop" (the incoming order was cancelled).

This control flow is the key design decision. When a self-trade is detected at the best price, the engine doesn't just skip that one order — it applies a policy that may affect both orders and determines whether matching should continue.

Step 5: The STP Handler

func (e *Engine) handleSTP(book *orderbook.OrderBook, incoming, resting *model.Order) bool {
    switch e.stpMode {
    case model.STPCancelResting:
        book.RemoveOrder(resting.ID)
        delete(e.orderIndex, resting.ID)
        return true

    case model.STPCancelIncoming:
        incoming.Remaining = decimal.Zero
        return false

    case model.STPCancelBoth:
        book.RemoveOrder(resting.ID)
        delete(e.orderIndex, resting.ID)
        incoming.Remaining = decimal.Zero
        return false

    case model.STPDecrement:
        overlap := decimal.Min(incoming.Remaining, resting.Remaining)
        incoming.Remaining = incoming.Remaining.Sub(overlap)
        resting.Remaining = resting.Remaining.Sub(overlap)
        if resting.IsFilled() {
            book.RemoveOrder(resting.ID)
            delete(e.orderIndex, resting.ID)
        }
        if incoming.IsFilled() {
            return false
        }
        return true

    default:
        return true
    }
}

Let's trace through each mode with a concrete example.

Walkthrough: Alice Sells 5, Then Buys 7

Setup: Alice has a resting sell order for 5 units at price 100. Bob has a resting sell order for 3 units at price 100 (placed after Alice). Alice submits a buy for 7 units at price 100.

Book before:
  Asks: [alice: SELL 5 @ 100] [bob: SELL 3 @ 100]

Incoming: alice: BUY 7 @ 100

Mode: CancelResting

Best ask is Alice's sell (5 @ 100). Self-trade detected.
Alice's sell is removed from the book. Incoming buy continues (remaining: 7).
Best ask is now Bob's sell (3 @ 100). Different owner. Trade: 3 @ 100.
Buy has 4 remaining. No more asks. Buy rests in book.

Result: 1 trade (alice buys 3 from bob). Alice's sell cancelled. Alice's buy rests with 4 remaining.

Mode: CancelIncoming

Best ask is Alice's sell (5 @ 100). Self-trade detected.
Alice's buy is cancelled entirely (remaining set to 0).
Matching stops.

Result: 0 trades. Alice's sell stays in book. Alice's buy is gone.

Mode: CancelBoth

Best ask is Alice's sell (5 @ 100). Self-trade detected.
Alice's sell is removed. Alice's buy is cancelled.
Matching stops.

Result: 0 trades. Both orders gone. Bob's sell still in book.

Mode: Decrement

Best ask is Alice's sell (5 @ 100). Self-trade detected.
Overlap = min(7, 5) = 5. Both reduced by 5.
Alice's sell: 5 - 5 = 0 (filled, removed from book).
Alice's buy: 7 - 5 = 2 (continues matching).
Best ask is Bob's sell (3 @ 100). Different owner. Trade: 2 @ 100.
Buy is filled.

Result: 1 trade (alice buys 2 from bob). No self-trade produced. Alice's sell silently decremented to zero.

Design Decisions

Why Engine-Level, Not Per-Order?

The STP mode is set on the engine, not on individual orders. This is a simplification. Some exchanges allow per-order STP modes (e.g., one order uses CancelResting while another uses CancelIncoming). We chose engine-level for two reasons:

Simplicity. One mode for the entire engine is easier to reason about and test.
Consistency. Mixed modes within the same book can produce surprising behavior. If order A uses CancelResting and order B uses CancelIncoming, and they self-trade, which mode wins?

If per-order modes are needed later, the handleSTP method can be extended to read the mode from the order itself. The matching loop doesn't need to change.

Why Not Skip Instead of Cancel?

An alternative to cancelling is skipping — leave the resting order in the book and move to the next one. This sounds simpler but creates a problem: the skipped order is still at the best price. The next time the same owner submits a matching order, the same self-trade check fires again. The order becomes permanently unmatchable by its owner but still visible to everyone else.

Cancelling (or decrementing) is cleaner. It removes the conflict and lets the matching loop proceed to the next price level without leaving orphaned orders.

Why Empty OwnerID Bypasses STP?

Backward compatibility. Before this feature, no orders had an OwnerID. All existing code that creates orders without setting OwnerID should continue to work identically. Making empty-owner orders exempt from STP achieves this without any migration.

It also serves a practical purpose: anonymous or system-generated orders (e.g., liquidation orders) that should match against anything can simply omit the OwnerID.

Testing

Nine test cases cover the feature:

Test	What It Verifies
`TestSTPDisabledByDefault`	Self-trades happen when STP is off
`TestSTPCancelResting`	Resting cancelled, incoming matches next level
`TestSTPCancelIncoming`	Incoming cancelled, resting preserved
`TestSTPCancelBoth`	Both cancelled, book empty
`TestSTPDecrement`	Both reduced, no trade produced
`TestSTPDecrementPartial`	Decrement + match against different owner
`TestSTPDifferentOwnersMatch`	Different owners match normally with STP on
`TestSTPEmptyOwnerIDAlwaysMatches`	Empty owner bypasses STP
`TestSTPWithAutoGeneratedIDs`	STP works with SubmitRequest + WithOwner

The TestSTPDecrementPartial test is the most complex — it verifies that after decrementing against a same-owner resting order, the incoming order continues to match against a different owner's order at the same price level.

Usage

// Create engine with STP enabled
e := engine.New(
    engine.WithSTPMode(model.STPCancelResting),
    engine.WithIDPrefix("ORD-"),
)

// Using SubmitOrder (legacy API)
sell := model.NewLimitOrder("s1", model.Sell, price, qty)
sell.OwnerID = "alice"
e.SubmitOrder("BTC/USD", sell)

// Using SubmitRequest (recommended API)
req := model.NewLimitOrderRequest(model.Buy, price, qty).WithOwner("alice")
id, trades, err := e.SubmitRequest("BTC/USD", req)
// trades will be empty — self-trade prevented

Summary

Aspect	Detail
New types	`STPMode` enum, `OwnerID` field on Order/OrderRequest
Configuration	`WithSTPMode(mode)` engine option
Modes	CancelResting, CancelIncoming, CancelBoth, Decrement
Default	`STPNone` (disabled, backward compatible)
Detection	String equality on `OwnerID`
Bypass	Empty `OwnerID` always matches
Tests	9 new test cases
Files changed	`model/stp.go` (new), `model/order.go`, `model/request.go`, `engine/engine.go`

The matching algorithm gained exactly two new lines in the hot path: an isSelfTrade check and a handleSTP call. When STP is disabled (the default), the stpMode != STPNone check short-circuits immediately — zero overhead for engines that don't need it.

GitHub: https://github.com/iwtxokhtd83/MatchEngine
Release: v0.5.0
Issue: #7 — Add self-trade prevention

Did Your Fix Actually Work? Comparing Profiling Reports Before and After

Bill Tu — Mon, 20 Apr 2026 06:40:10 +0000

You found the bug. heavyComputation at /app/server.js:42 was consuming 62% of CPU. You refactored it to use a worker thread. You deployed. The team celebrates.

Two hours later, latency is back. Not from heavyComputation — that's fixed. But the refactoring introduced a new bottleneck in the message serialization between the main thread and the worker. You didn't notice because you were looking at the old function, not the new one.

This is the verification gap. You profile, you fix, you deploy — but you never systematically compare "before" and "after" to confirm the fix worked and nothing else regressed.

With node-loop-detective v2.2.0, you can now diff two profiling reports:

# Before the fix
loop-detective 12345 --json > before.json

# After the fix
loop-detective 12345 --compare before.json

What the Comparison Shows

The diff report has five sections:

Pattern Changes

The most important section. Did the blocking pattern go away?

  ✔ Resolved issues:
    - cpu-hog (high)

  ✖ New issues:
    + json-heavy: JSON operations took 1200ms (15% of profile)

Three categories:

Resolved: was in the baseline, not in the current report. Your fix worked.
New: wasn't in the baseline, appeared in the current report. Your fix introduced a new problem.
Persistent: present in both. The issue wasn't addressed (or the fix didn't help).

Function Changes

Which functions got faster? Which got slower? Which are new?

  Function Changes
────────────────────────────────────────────────────────────
  ▲ serializeMessage  0ms → 450ms (+450ms)  NEW
    /app/worker-bridge.js:23
  ▼ heavyComputation  6245ms → 120ms  (-6125ms)
    /app/server.js:42

Functions are classified as:

Regressed (▲ red): self time increased by >1ms
New (+ red): appeared in the current report but not in the baseline
Improved (▼ green): self time decreased by >1ms
Removed: was in the baseline, gone from the current report
Unchanged: delta within ±1ms

Sorted with regressions first — the things you need to look at.

Lag Comparison

  Event Loop Lag
────────────────────────────────────────────────────────────
  Events: 12 → 1 (-11)
  Max:    312ms → 45ms
  Avg:    156ms → 45ms

Count, max, and average compared with deltas. Green if improved, red if regressed.

I/O Comparison

  Slow I/O
────────────────────────────────────────────────────────────
  Slow ops: 5 → 2 (-3)
  Max dur:  2340ms → 800ms

Verdict

A one-line summary:

  ✔ Overall: IMPROVED (3 improvements)

Four possible verdicts:

IMPROVED: regressions = 0, improvements > 0
REGRESSED: regressions > 0, improvements = 0
MIXED: both regressions and improvements
NO SIGNIFICANT CHANGE: nothing moved

The Workflow

Basic: Before and After

# 1. Capture baseline
loop-detective 12345 --json > baseline.json

# 2. Deploy your fix

# 3. Compare
loop-detective 12345 --compare baseline.json

With Full Output

The comparison works alongside all other flags:

# Compare + HTML report + CPU profile
loop-detective 12345 --compare baseline.json --html after-fix.html --save-profile after.cpuprofile

You get the normal terminal report, the comparison report, the HTML report, and the CPU profile — all from one command.

In CI

# In your test pipeline
loop-detective --port 9229 --json > current.json
loop-detective --port 9229 --compare baseline.json --json > diff.json

# Check the verdict
REGRESSED=$(node -e "const d=require('./diff.json'); const r=d.comparison.functions.filter(f=>f.status==='regressed').length; process.exit(r > 0 ? 1 : 0)")
if [ $? -ne 0 ]; then
  echo "Performance regression detected!"
  exit 1
fi

How the Comparison Works

The comparator (src/comparator.js) is a pure function that takes two report objects and produces a structured diff:

function compareReports(baseline, current) {
  return {
    summary: compareSummaries(baseline.summary, current.summary),
    functions: compareFunctions(baseline.heavyFunctions, current.heavyFunctions),
    patterns: comparePatterns(baseline.blockingPatterns, current.blockingPatterns),
    lagEvents: compareLag(baseline.lagEvents, current.lagEvents),
    slowIO: compareIO(baseline.slowIOEvents, current.slowIOEvents),
  };
}

Function Matching

Functions are matched by a composite key: functionName + url + lineNumber. This handles the common case where the same function exists in both reports at the same location.

const key = (f) => f.functionName + '|' + f.url + ':' + f.lineNumber;

If a function moved to a different line (e.g., you added code above it), it shows up as "removed" at the old line and "new" at the new line. This is a known limitation — line-level matching is imperfect when code changes significantly. But for the typical "fix one function, verify it improved" workflow, it works well.

Threshold for Change

A function is "improved" or "regressed" only if the self time changed by more than 1ms. This avoids noise from statistical sampling variation. A function that went from 5.2ms to 4.8ms is "unchanged" — the 0.4ms difference is within normal sampling variance.

Pattern Comparison

Pattern comparison is set-based. If cpu-hog was in the baseline but not in the current report, it's "resolved." If json-heavy is in the current report but not the baseline, it's "new." The comparison doesn't try to compare severity levels or thresholds within the same pattern type — it's a binary present/absent check.

Programmatic API

The comparator is exported for use in custom tooling:

const { compareReports, formatComparison } = require('node-loop-detective');

// Load two reports
const baseline = JSON.parse(fs.readFileSync('before.json'));
const current = JSON.parse(fs.readFileSync('after.json'));

// Compare
const diff = compareReports(baseline, current);

// Structured access
console.log('Resolved:', diff.patterns.resolved.map(p => p.type));
console.log('Regressed functions:', diff.functions.filter(f => f.status === 'regressed'));
console.log('Lag delta:', diff.lagEvents.delta);

// Formatted terminal output
console.log(formatComparison(diff));

// Or use in an API
app.get('/perf/compare', (req, res) => {
  res.json(diff);
});

Why This Matters

Performance work without measurement is guesswork. You think the fix helped, but you don't know by how much. You think nothing else regressed, but you didn't check every function.

The comparison report makes verification systematic:

Did the target function improve? Check the function changes.
Did the blocking pattern resolve? Check the pattern changes.
Did anything else regress? Check for new issues and regressed functions.
Did lag improve? Check the lag delta.
Did I/O improve? Check the I/O delta.

Five questions, one command, one report.

Try It

npm install -g node-loop-detective@2.2.0

# Save baseline
loop-detective <pid> --json > before.json

# After changes
loop-detective <pid> --compare before.json

Source: github.com/iwtxokhtd83/node-loop-detective

The hardest part of performance work isn't finding the problem. It's proving the fix worked.

Completing the Picture: Adding Memory Diagnostics to a CPU Profiler

Bill Tu — Mon, 13 Apr 2026 06:38:56 +0000

loop-detective started as a CPU profiler. It told you which functions were blocking the event loop. Then we added I/O tracking — which network calls were slow. But there was always a gap in the story.

The analyzer would detect gc-pressure — garbage collection consuming 8% of CPU time — and suggest "Reduce object allocations. Reuse buffers. Check for memory leaks." Good advice. But then what? The user knows GC is a problem, but has no way to see what's actually in memory. They'd have to switch to a completely different tool, reconnect to the process, and start over.

With v2.1.0, loop-detective can now capture heap snapshots and track memory usage. The diagnostic workflow stays in one tool, from "something is slow" to "here's the object causing the memory leak."

The Two New Capabilities

--heap-stats: The Quick Check

loop-detective 12345 --heap-stats -d 30

This samples process.memoryUsage() before and after the profiling period:

  Memory (before profiling)
    RSS: 85.2MB  Heap: 42.1MB / 65.0MB  External: 1.2MB
  Memory (after profiling)
    RSS: 92.8MB  Heap: 58.3MB / 75.0MB  External: 1.2MB
  Memory delta
    Heap: +16.2MB  RSS: +7.6MB

Four numbers tell you a lot:

RSS (Resident Set Size): total memory the OS has allocated to the process
Heap Used / Heap Total: V8's JavaScript heap — where your objects live
External: memory used by C++ objects bound to JavaScript (Buffers, etc.)

The delta is color-coded: green for <1MB growth (normal), yellow for 1-10MB (worth investigating), red for >10MB (likely a leak or large allocation during the profiling window).

This is the "should I worry?" check. If heap grew 16MB during a 30-second profile, something is allocating a lot of objects. If it grew 0.2MB, memory is probably fine and the performance issue is elsewhere.

--heap-snapshot: The Deep Dive

loop-detective 12345 --heap-snapshot ./heap.heapsnapshot

This captures a full V8 heap snapshot — a complete inventory of every JavaScript object in memory, their sizes, and their reference chains. The .heapsnapshot file can be loaded in Chrome DevTools:

Open Chrome DevTools → Memory tab
Click "Load" and select the file
Explore:
- Summary view: objects grouped by constructor, sorted by retained size
- Comparison view: diff two snapshots to find what grew
- Containment view: see the reference chain from GC roots to any object
- Statistics: pie chart of memory by type

The snapshot is captured after CPU profiling completes, so you get both the CPU analysis and the memory snapshot from the same session.

How It Works Under the Hood

Heap Stats

The simplest possible implementation. We use Runtime.evaluate to call process.memoryUsage() in the target process:

async getHeapStats() {
  const result = await this.inspector.send('Runtime.evaluate', {
    expression: `(function(){
      const m = process.memoryUsage();
      return {
        rss: m.rss,
        heapTotal: m.heapTotal,
        heapUsed: m.heapUsed,
        external: m.external,
        arrayBuffers: m.arrayBuffers || 0
      };
    })()`,
    returnByValue: true,
  });
  return result.result?.value || null;
}

We call this twice — once before profiling starts, once after it ends — and emit both values:

async _singleRun() {
  let heapBefore = null;
  try { heapBefore = await this.getHeapStats(); } catch {}

  // ... profiling happens here ...

  let heapAfter = null;
  try { heapAfter = await this.getHeapStats(); } catch {}

  if (heapBefore || heapAfter) {
    this.emit('heapStats', { before: heapBefore, after: heapAfter });
  }
}

Both calls are wrapped in try/catch because heap stats are non-essential — if they fail (target exited, inspector disconnected), the CPU profile and I/O data are still valid.

Heap Snapshot

The CDP HeapProfiler domain handles snapshot capture. The snapshot is streamed in chunks via events:

async captureHeapSnapshot() {
  let chunks = [];
  const onChunk = (msg) => {
    if (msg.method === 'HeapProfiler.addHeapSnapshotChunk') {
      chunks.push(msg.params.chunk);
    }
  };
  this.inspector.on('event', onChunk);

  try {
    await this.inspector.send('HeapProfiler.enable');
    await this.inspector.send('HeapProfiler.takeHeapSnapshot', {
      reportProgress: false
    });
    await this.inspector.send('HeapProfiler.disable');
  } finally {
    this.inspector.removeListener('event', onChunk);
  }

  return chunks.join('');
}

The V8 heap snapshot format is JSON, but it can be large — 10MB to 200MB+ depending on heap size. The CDP protocol streams it in chunks to avoid sending one massive message. We collect all chunks and join them into the final string.

A few important details:

reportProgress: false — we skip progress events to keep the implementation simple. For very large heaps, you could listen for HeapProfiler.reportHeapSnapshotProgress to show a progress bar.
The snapshot is captured after CPU profiling, not during. Taking a heap snapshot pauses the target process briefly (it needs to walk the entire object graph), so we don't want it interfering with the CPU profile.
The finally block ensures we remove the event listener even if the snapshot fails.

The Diagnostic Workflow

Here's how the pieces fit together in a real debugging session:

Step 1: Quick diagnosis

loop-detective 12345 -d 10

The report shows gc-pressure — GC is consuming 8% of CPU. The event loop isn't blocked by user code, but GC pauses are adding latency.

Step 2: Confirm with memory stats

loop-detective 12345 --heap-stats -d 30

  Memory delta
    Heap: +24.3MB  RSS: +18.1MB

Red flag. The heap grew 24MB in 30 seconds. Something is allocating objects faster than GC can collect them.

Step 3: Capture the evidence

loop-detective 12345 --heap-snapshot ./before-fix.heapsnapshot --heap-stats

Open before-fix.heapsnapshot in Chrome DevTools. The Summary view shows 500,000 Object instances retained by a Map in /app/cache.js. The cache has no eviction policy — it grows forever.

Step 4: Fix and verify

After adding an LRU eviction policy to the cache:

loop-detective 12345 --heap-stats -d 30

  Memory delta
    Heap: +0.8MB  RSS: +0.3MB

Green. Memory is stable. The gc-pressure pattern is gone from the CPU profile.

When to Use Each

Situation	Use
Quick health check	`--heap-stats`
Analyzer reports `gc-pressure`	`--heap-stats` first, then `--heap-snapshot` if growth is high
Suspected memory leak	`--heap-snapshot` (capture two snapshots minutes apart, compare in DevTools)
Full diagnostic session	`--heap-stats --save-profile ./cpu.cpuprofile --html report.html`
Memory-only investigation	`--heap-snapshot ./heap.heapsnapshot --no-io -d 1` (minimal profiling, just get the snapshot)

Performance Impact

--heap-stats: negligible. Two process.memoryUsage() calls, each taking <1ms.
--heap-snapshot: moderate. The V8 heap walker pauses the target process while it traverses the object graph. For a 100MB heap, this takes 1-5 seconds. For a 1GB heap, it could take 10-30 seconds. The process is unresponsive during this time.

This is why the snapshot is captured after profiling, not during. And it's why it's opt-in (--heap-snapshot) rather than default.

Programmatic API

For custom tooling:

const { Detective } = require('node-loop-detective');
const fs = require('fs');

const detective = new Detective({ pid: 12345, duration: 10000 });

// Memory stats before/after profiling
detective.on('heapStats', (data) => {
  const growth = data.after.heapUsed - data.before.heapUsed;
  if (growth > 50 * 1024 * 1024) {
    alert('Heap grew ' + (growth / 1024 / 1024).toFixed(0) + 'MB during profiling!');
  }
});

detective.on('profile', async (analysis) => {
  // If GC pressure detected, auto-capture heap snapshot
  const hasGcPressure = analysis.blockingPatterns.some(p => p.type === 'gc-pressure');
  if (hasGcPressure) {
    const snapshot = await detective.captureHeapSnapshot();
    fs.writeFileSync('auto-heap-' + Date.now() + '.heapsnapshot', snapshot);
  }
});

await detective.start();

This pattern — auto-capture a heap snapshot when GC pressure is detected — is particularly powerful for automated monitoring. You get the snapshot exactly when it matters, without human intervention.

What's Next

The heap snapshot is a point-in-time view. It tells you what's in memory right now, but not how it got there. Future improvements could include:

Allocation tracking via HeapProfiler.startTrackingHeapObjects — shows which functions are allocating the most objects
Memory timeline — sample process.memoryUsage() every second during profiling and show a trend line in the HTML report
Automatic leak detection — compare heap stats at the start and end of each watch cycle, flag monotonic growth

For now, the combination of --heap-stats (quick check) and --heap-snapshot (deep dive) covers the most common memory debugging workflow.

Try It

npm install -g node-loop-detective@2.1.0

# Quick memory check alongside CPU profiling
loop-detective <pid> --heap-stats

# Full memory diagnosis
loop-detective <pid> --heap-stats --heap-snapshot ./heap.heapsnapshot

# Minimal: just grab the snapshot
loop-detective <pid> --heap-snapshot ./heap.heapsnapshot --no-io -d 1

Source: github.com/iwtxokhtd83/node-loop-detective

CPU tells you what's slow. I/O tells you what's waiting. Memory tells you what's accumulating. Now you have all three in one tool.

Shareable Diagnostics: Generating HTML Reports From Production Profiling

Bill Tu — Fri, 10 Apr 2026 07:22:11 +0000

You've just diagnosed a production performance issue. loop-detective told you that processPayload at /app/handlers/payment.js:127 consumed 54% of CPU, that three HTTP calls to the payment gateway averaged 1.8 seconds, and that the event loop lagged 12 times during the 30-second capture.

Now you need to share this with your team. You copy-paste the terminal output into Slack. The ANSI color codes turn into garbage. The bar charts become meaningless characters. The carefully formatted report becomes an unreadable wall of text.

This is why we built HTML report output for node-loop-detective v2.0.0.

loop-detective 12345 --html report.html

One flag. One file. Open it in any browser. Share it anywhere.

The Problem With Terminal Output

Terminal output is perfect for the person running the diagnostic. It's immediate, it's colorful, it's right there. But it has three fundamental limitations:

It doesn't travel well. ANSI escape codes render as [31m in Slack, email, Jira, and most text editors. You lose all formatting.
It's not interactive. You can't collapse a call stack you don't care about. You can't hover over a lag event to see its timestamp. You can't sort the function table by percentage.
It's ephemeral. Once the terminal scrolls past, it's gone. You can redirect to a file, but then you have a plain text file with escape codes.

JSON output (--json) solves the data portability problem but creates a readability problem. Nobody wants to read a 500-line JSON blob in a Slack thread.

What the HTML Report Looks Like

The report is a single self-contained HTML file with a dark theme inspired by GitHub's UI. No external CSS, no JavaScript CDN links, no images to load. It works offline, it works on any device, and it's about 15-20KB for a typical report.

It has six sections:

Summary Cards

Five cards at the top showing the key numbers at a glance: profiling duration, CPU sample count, number of hot functions, lag event count, and slow I/O operation count. You can tell in one second whether this report is interesting.

Diagnosis

Each detected pattern gets a colored severity badge (HIGH in red, MED in yellow, LOW in green), a description, the code location, and a suggested fix. This is the same information as the terminal output, but formatted for readability.

CPU-Heavy Functions Table

A sortable table with visual percentage bars. The bars are color-coded: red for >50% CPU, yellow for >20%, green for the rest. Each row shows the function name, a visual bar, self time in milliseconds, and the file location.

Collapsible Call Stacks

The top blocking functions have expandable call stacks. Click to expand, click again to collapse. This keeps the report compact by default while making the full detail available on demand. The target function is highlighted in yellow.

Slow I/O Summary

Grouped by type (HTTP, FETCH, DNS, TCP), then by target. Each group shows call count, total duration, average, and max. Caller stack traces are shown inline. Error counts get a red badge.

Event Loop Lag Timeline

A visual timeline of lag events represented as colored dots. The dot size scales with lag severity. Hover over any dot to see the exact lag duration and timestamp. Below the timeline, a table aggregates lag events by code location.

Design Decisions

Why self-contained?

The report must work when attached to a Jira ticket, emailed to a colleague, or opened six months later during a post-mortem. External dependencies (CDN-hosted CSS, JavaScript libraries, web fonts) break in all these scenarios. Corporate firewalls block CDNs. Offline access fails. Links rot.

Every byte of CSS and JavaScript is inline in the HTML file. You can put it on a USB drive and it works.

Why dark theme?

Developers spend their days looking at dark-themed editors and terminals. A bright white report is jarring. The GitHub-inspired dark theme (#0d1117 background, #c9d1d9 text) feels native to the developer workflow.

Why collapsible call stacks?

A report with 5 expanded call stacks, each 10-15 frames deep, is overwhelming. Most of the time you only care about one or two stacks. Collapsible sections let you scan the targets quickly and expand only what's relevant.

The implementation is vanilla JavaScript — no framework, no build step:

document.querySelectorAll('.collapsible').forEach(el => {
  el.addEventListener('click', () => {
    el.classList.toggle('open');
    el.nextElementSibling.classList.toggle('show');
  });
});

Why not a full dashboard?

We considered generating a report with charts (line graphs for lag over time, pie charts for CPU distribution). But charts require a charting library (Chart.js, D3, etc.), which would either bloat the file or require a CDN. The current approach — colored bars, sized dots, tables — conveys the same information with pure HTML and CSS.

If you need charts, use --json and feed the data to your preferred visualization tool.

How It Works

The HTML report generator (src/html-report.js) is a pure function that takes analysis data and returns an HTML string:

function generateHtmlReport(data) {
  const { analysis, lagEvents, slowIOEvents, config, timestamp } = data;
  return `<!DOCTYPE html>
<html>
<head>
  <style>/* all CSS inline */</style>
</head>
<body>
  ${renderSummary(analysis.summary, lagEvents, slowIOEvents)}
  ${renderPatterns(analysis.blockingPatterns)}
  ${renderHeavyFunctions(analysis.heavyFunctions)}
  ${renderCallStacks(analysis.callStacks)}
  ${renderSlowIO(slowIOEvents)}
  ${renderLagEvents(lagEvents)}
  <script>/* collapsible toggle */</script>
</body>
</html>`;
}

Each section is rendered by a dedicated function. All user-provided strings (function names, file paths, error messages) are HTML-escaped to prevent XSS — even though the report is generated locally, it's good practice.

In the CLI, the events are captured before the reporter clears them:

detective.on('profile', (analysis, rawProfile) => {
  // Capture before onProfile clears the arrays
  const capturedLags = [...reporter.lagEvents];
  const capturedIO = [...reporter.slowIOEvents];

  reporter.onProfile(analysis); // prints to terminal + clears arrays

  if (config.html) {
    const html = generateHtmlReport({
      analysis, lagEvents: capturedLags, slowIOEvents: capturedIO,
      config, timestamp: analysis.timestamp,
    });
    fs.writeFileSync(path.resolve(config.html), html);
  }
});

Combining Output Formats

The HTML report works alongside all other output options:

# Terminal output + HTML report
loop-detective 12345 --html report.html

# Terminal + HTML + CPU profile for flame graphs
loop-detective 12345 -d 60 --html report.html --save-profile profile.cpuprofile

# JSON + HTML (JSON goes to stdout, HTML to file)
loop-detective 12345 --json --html report.html > data.json

Each format serves a different purpose:

Terminal: immediate diagnosis while you're at the keyboard
HTML: sharing with team, attaching to tickets, archiving
JSON: feeding to monitoring systems, custom analysis scripts
.cpuprofile: deep visual analysis in Chrome DevTools or speedscope

Programmatic API

The HTML generator is exported for use in custom tooling:

const { generateHtmlReport } = require('node-loop-detective');

// Generate from your own data
const html = generateHtmlReport({
  analysis: myAnalysisResult,
  lagEvents: myLagEvents,
  slowIOEvents: mySlowIOEvents,
  config: { duration: 30000, threshold: 50 },
  timestamp: Date.now(),
});

// Serve it from an Express endpoint
app.get('/debug/report', (req, res) => {
  res.type('html').send(html);
});

// Or save to S3 for the incident timeline
await s3.putObject({
  Bucket: 'incident-reports',
  Key: `diagnostics/${Date.now()}.html`,
  Body: html,
  ContentType: 'text/html',
});

Real-World Usage

Incident Post-Mortems

During an incident, run loop-detective with --html:

loop-detective 12345 -d 60 --html incident-2025-03-15.html

Attach the HTML file to the post-mortem document. Six months later, anyone can open it and see exactly what was happening: which functions were hot, which I/O was slow, when the lag events occurred.

Code Review Evidence

Found a performance regression? Profile the old and new versions, generate HTML reports for both, and attach them to the pull request. The reviewer can open both files side by side and compare.

Automated Reporting

In a CI/CD pipeline or scheduled job:

loop-detective --port 9229 -d 30 --html /reports/daily-$(date +%Y%m%d).html --json > /reports/daily-$(date +%Y%m%d).json

Build a simple file server over the reports directory and you have a performance history dashboard.

Try It

npm install -g node-loop-detective@2.0.0

loop-detective <pid> --html report.html

Open report.html in your browser. Share it with your team. Attach it to the ticket. The diagnostic data is no longer trapped in your terminal.

Source: github.com/iwtxokhtd83/node-loop-detective

Profiling the Threads You Forgot About: Worker Thread Diagnostics in Node.js

Bill Tu — Fri, 10 Apr 2026 06:57:05 +0000

You've profiled your Node.js server. The main thread looks healthy. CPU usage is low. Event loop lag is minimal. But users are still reporting slow responses.

Then you remember: half your request processing happens in worker threads.

Node.js worker_threads have their own V8 instances, their own event loops, and their own performance characteristics. A worker thread can be completely blocked — running a tight loop, parsing a massive JSON payload, executing a catastrophic regex — and the main thread's profiler won't see any of it.

With node-loop-detective v1.9.0, you can now list all inspector targets and profile individual worker threads. Two new flags: --list-targets and --target.

The Invisible Threads

Worker threads are increasingly common in Node.js applications. They're used for:

CPU-intensive computation (image processing, data transformation, ML inference)
Parallel request processing in frameworks like Piscina
Background jobs (report generation, data export, batch processing)
Offloading blocking operations from the main thread

The irony is that worker threads are often created specifically to handle heavy work — which makes them the most likely threads to have performance problems. But until now, most diagnostic tools only profiled the main thread.

Here's what happens when you profile a Node.js app that uses worker threads:

loop-detective 12345 -d 30

────────────────────────────────────────────────────────────
  Event Loop Detective Report
────────────────────────────────────────────────────────────
  Duration:  30012ms
  Samples:   13521
  Hot funcs: 3

  Diagnosis
────────────────────────────────────────────────────────────
   LOW  healthy
        No obvious event loop blocking patterns detected
        → Try profiling for a longer duration or during peak load

The main thread is healthy. But the workers are drowning. You just can't see them.

How the V8 Inspector Handles Threads

When Node.js opens its inspector (via --inspect or SIGUSR1), it exposes a /json/list HTTP endpoint that returns all available debugging targets. Each target is a separate V8 isolate with its own WebSocket URL:

[
  {
    "id": "1a2b3c",
    "title": "Main thread",
    "url": "file:///app/server.js",
    "webSocketDebuggerUrl": "ws://127.0.0.1:9229/1a2b3c"
  },
  {
    "id": "4d5e6f",
    "title": "Worker #1",
    "url": "file:///app/worker.js",
    "webSocketDebuggerUrl": "ws://127.0.0.1:9229/4d5e6f"
  },
  {
    "id": "7g8h9i",
    "title": "Worker #2",
    "url": "file:///app/worker.js",
    "webSocketDebuggerUrl": "ws://127.0.0.1:9229/7g8h9i"
  }
]

Previously, loop-detective always connected to targets[0] — the main thread. The worker threads were right there in the list, but we never looked at them.

Discovering Targets

The first step is knowing what's available:

loop-detective --port 9229 --list-targets

  Available inspector targets:

  [0] Main thread
      file:///app/server.js
  [1] Worker #1
      file:///app/worker.js
  [2] Worker #2
      file:///app/worker.js

  Use --target <index> to connect to a specific target.

Each target has an index, a title, and a URL (the script that created it). The main thread is always index 0.

For automation, --json gives you structured output:

loop-detective --port 9229 --list-targets --json

[
  { "index": 0, "id": "1a2b3c", "title": "Main thread", "url": "file:///app/server.js" },
  { "index": 1, "id": "4d5e6f", "title": "Worker #1", "url": "file:///app/worker.js" },
  { "index": 2, "id": "7g8h9i", "title": "Worker #2", "url": "file:///app/worker.js" }
]

Profiling a Worker Thread

Once you know the target index, profile it like any other process:

loop-detective --port 9229 --target 1 -d 30

✔ Connected to Node.js process
  Profiling for 30s with 50ms lag threshold...

⚠ Event loop lag: 1245ms at 2025-03-15T14:23:45.123Z

────────────────────────────────────────────────────────────
  Event Loop Detective Report
────────────────────────────────────────────────────────────
  Duration:  30008ms
  Samples:   8234
  Hot funcs: 7

  Diagnosis
────────────────────────────────────────────────────────────
   HIGH  cpu-hog
         Function "transformData" consumed 72.1% of CPU time (21630ms)
         at /app/worker.js:45
         → Consider breaking this into smaller async chunks

   1. transformData
      ██████████████████░░ 21630ms (72.1%)
      /app/worker.js:45:1

There it is. Worker #1 has a transformData function consuming 72% of CPU. The main thread never saw this because the work was offloaded to the worker.

A Real-World Debugging Session

Here's a typical workflow for diagnosing a worker thread issue:

# Step 1: Profile the main thread — looks healthy
loop-detective 12345 -d 10
# Result: "healthy" — no blocking detected

# Step 2: List all targets
loop-detective --port 9229 --list-targets
# Shows: [0] Main, [1] Worker #1, [2] Worker #2, [3] Worker #3

# Step 3: Profile each worker
loop-detective --port 9229 --target 1 -d 10
# Worker #1: healthy

loop-detective --port 9229 --target 2 -d 10
# Worker #2: cpu-hog on transformData — 72% CPU!

loop-detective --port 9229 --target 3 -d 10
# Worker #3: healthy

# Step 4: Deep dive on Worker #2
loop-detective --port 9229 --target 2 -d 60 --save-profile ./worker2.cpuprofile
# Open worker2.cpuprofile in Chrome DevTools for flame graph

In a script:

#!/bin/bash
# Profile all worker threads
TARGETS=$(loop-detective --port 9229 --list-targets --json)
COUNT=$(echo "$TARGETS" | node -e "process.stdin.on('data',d=>console.log(JSON.parse(d).length))")

for i in $(seq 0 $((COUNT - 1))); do
  echo "=== Profiling target $i ==="
  loop-detective --port 9229 --target $i -d 10 --json > "target-${i}.json"
done

How It Works Internally

The implementation touches two layers:

Inspector: Target Selection

The Inspector class now accepts a targetIndex parameter and uses it to select the WebSocket URL:

class Inspector {
  constructor({ host, port, targetIndex = 0 }) {
    this.targetIndex = targetIndex;
    // ...
  }

  async getWebSocketUrl() {
    const targets = await this.getTargets();
    if (this.targetIndex >= targets.length) {
      throw new Error('Target index ' + this.targetIndex + ' out of range');
    }
    return targets[this.targetIndex].webSocketDebuggerUrl;
  }
}

Detective: Target Discovery

The Detective class exposes a listTargets() method that uses the retry logic (same exponential backoff as regular connections) to handle the case where the inspector is still starting:

async listTargets() {
  this._activateInspector();
  const port = this._getInspectorPort();
  const host = this.config.inspectorHost || '127.0.0.1';

  // Retry with backoff (inspector may still be starting)
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      const inspector = new Inspector({ host, port });
      return await inspector.getTargets();
    } catch (err) {
      if (attempt === maxRetries) throw err;
      await this._sleep(delay);
    }
  }
}

What Works and What Doesn't

Everything that works on the main thread works on worker threads:

✅ CPU profiling and heavy function detection
✅ Event loop lag detection
✅ Blocking pattern analysis (cpu-hog, json-heavy, regex, gc, sync-io, crypto)
✅ --save-profile for flame graphs
✅ --watch for continuous monitoring
✅ --json for structured output

I/O tracking (http, fetch, dns, net patches) also works, but with a caveat: worker threads typically don't make HTTP requests directly. They receive data from the main thread via postMessage and send results back. If a worker does make network calls, the I/O tracker will catch them.

When Workers Don't Show Up

A few situations where --list-targets might not show your workers:

Workers created after inspector activation. If a worker is spawned after you send SIGUSR1, it may not appear in the target list. Re-run --list-targets to refresh.
Workers that have already exited. Short-lived workers (process a task, then terminate) may not be visible when you list targets. Use --watch on the main thread to catch the pattern, then target long-lived workers.
Workers without inspector support. Workers created with { execArgv: [] } explicitly disable the inspector. They won't appear in the target list.

Try It

npm install -g node-loop-detective@1.9.0

# List all targets
loop-detective --port 9229 --list-targets

# Profile a worker thread
loop-detective --port 9229 --target 1 -d 30

The main thread is just one thread. In modern Node.js applications, the interesting performance problems are often hiding in the workers.

Source: github.com/iwtxokhtd83/node-loop-detective

Why One Second Wasn't Enough: Adding Retry Logic to a Diagnostic Tool

Bill Tu — Fri, 10 Apr 2026 06:25:27 +0000

For the first seven releases of node-loop-detective, connecting to a target process looked like this:

async _findInspectorPort() {
  await this._sleep(1000); // wait 1 second
  return 9229;
}

Send SIGUSR1. Wait one second. Try to connect. If it fails, give up.

This worked on our development machines. It worked in CI. It worked on lightly loaded staging servers. Then users started running it in production — on machines with 95% CPU utilization, inside containers with throttled resources, on servers handling 50,000 requests per second.

One second wasn't enough.

What Happens After SIGUSR1

When you send SIGUSR1 to a Node.js process, it triggers the V8 Inspector to start. This involves:

The signal handler fires on the next event loop tick
Node.js initializes the inspector agent
A TCP server starts listening on port 9229
The /json/list HTTP endpoint becomes available
WebSocket connections are accepted

On an idle machine, this takes 10-50 milliseconds. On a machine where the event loop is already blocked (which is exactly when you're trying to diagnose it), step 1 might not happen for several seconds. The signal is queued, but the event loop has to process it.

This creates a paradox: the more you need the tool, the longer it takes to connect.

The Failure Mode

With a fixed 1-second wait, users on loaded systems saw:

✖ Error: Cannot connect to inspector at 127.0.0.1:9229.
  Is the Node.js inspector active? (connect ECONNREFUSED 127.0.0.1:9229)

The inspector was starting. It just wasn't ready yet. The user would run the command again, and it would work — because by then the inspector had finished initializing. But "run it again" is not an acceptable UX for a production diagnostic tool.

The Fix: Exponential Backoff

The solution is retry with exponential backoff. Instead of one attempt after a fixed delay, we make up to 5 attempts with increasing wait times:

Attempt	Delay Before	Cumulative Wait
1	500ms	500ms
2	1000ms	1.5s
3	2000ms	3.5s
4	4000ms	7.5s
5	4000ms	11.5s

The delay doubles each time, capped at 4 seconds. Total maximum wait before giving up: about 11.5 seconds.

async _connectWithRetry(host, port) {
  const maxRetries = this.config.inspectorPort ? 1 : 5;
  const baseDelay = 500;
  const maxDelay = 4000;

  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      this.inspector = new Inspector({ host, port });
      // ... set up event listeners
      await this.inspector.connect();
      return; // success
    } catch (err) {
      if (attempt === maxRetries) {
        throw new Error(
          `Failed to connect to inspector at ${host}:${port} ` +
          `after ${maxRetries} attempts. Last error: ${err.message}`
        );
      }
      // Clean up failed inspector
      if (this.inspector) {
        this.inspector.removeAllListeners();
        this.inspector = null;
      }
      const delay = Math.min(baseDelay * Math.pow(2, attempt - 1), maxDelay);
      this.emit('retry', { attempt, maxRetries, delay, error: err.message });
      await this._sleep(delay);
    }
  }
}

Design Decisions

Why not just wait longer?

We could have changed _sleep(1000) to _sleep(5000). Problem solved, right?

No. On a fast machine, you'd wait 5 seconds every time for something that takes 50ms. The whole point of a diagnostic tool is speed — you're in the middle of an incident, every second counts. Exponential backoff gives you the best of both worlds: fast connection on healthy systems, patient retry on loaded ones.

On a typical machine, the first attempt at 500ms succeeds. The user never sees a retry. On a loaded machine, the second or third attempt usually succeeds. The user sees one or two retry messages and then gets connected.

Why 5 attempts?

Five attempts with our backoff schedule gives a maximum wait of ~11.5 seconds. This is long enough to handle even severely loaded systems (where the event loop might be blocked for 5-10 seconds), but short enough that a genuine failure (wrong PID, process already exited) doesn't leave the user waiting forever.

Why skip retry for --port?

When the user specifies --port, they're connecting to an inspector that's already open. There's no SIGUSR1, no startup delay. If the connection fails, it's because the inspector isn't there — retrying won't help.

const maxRetries = this.config.inspectorPort ? 1 : 5;

One attempt for --port. Five attempts for PID-based connections.

Why emit a retry event?

The CLI shows retry progress:

  Connecting to inspector... attempt 1/5 (retry in 500ms)
  Connecting to inspector... attempt 2/5 (retry in 1000ms)
✔ Connected to Node.js process

Without this feedback, the user would see nothing for up to 11 seconds. In an incident, silence is anxiety. The retry messages tell the user "I'm working on it, the system is just slow."

For the programmatic API, the retry event lets integrators log, alert, or implement their own timeout logic:

detective.on('retry', (data) => {
  logger.warn('Inspector connection retry', {
    attempt: data.attempt,
    maxRetries: data.maxRetries,
    delay: data.delay,
    error: data.error,
  });
});

The Cleanup Problem

There's a subtle issue with retry: each failed attempt creates an Inspector instance with event listeners. If we don't clean up, we accumulate orphaned listeners:

// Each attempt creates a new Inspector
this.inspector = new Inspector({ host, port });
this.inspector.on('disconnected', () => { ... });

// If connect() fails, we need to clean up before retrying
if (this.inspector) {
  this.inspector.removeAllListeners();
  this.inspector = null;
}

Without removeAllListeners(), the disconnected handler from attempt 1 would still be attached when attempt 2 creates a new Inspector. If the target later exits, both handlers would fire, causing duplicate targetExit events.

What Users See Now

On a fast machine (most cases):

✔ Connected to Node.js process
  Profiling for 10s with 50ms lag threshold...

No retry messages. Connection happens on the first attempt at 500ms — faster than the old fixed 1-second wait.

On a loaded machine:

  Connecting to inspector... attempt 1/5 (retry in 500ms)
  Connecting to inspector... attempt 2/5 (retry in 1000ms)
✔ Connected to Node.js process
  Profiling for 10s with 50ms lag threshold...

Two retries, then success. Total wait: about 1.5 seconds.

On a very loaded machine or wrong PID:

  Connecting to inspector... attempt 1/5 (retry in 500ms)
  Connecting to inspector... attempt 2/5 (retry in 1000ms)
  Connecting to inspector... attempt 3/5 (retry in 2000ms)
  Connecting to inspector... attempt 4/5 (retry in 4000ms)

✖ Error: Failed to connect to inspector at 127.0.0.1:9229 after 5 attempts.
  Last error: Cannot connect to inspector — Is the Node.js inspector active?

Clear feedback at every step. The final error message includes the attempt count and the specific failure reason.

The Broader Pattern

Retry with exponential backoff is one of the most well-known patterns in distributed systems. But it's easy to forget when building CLI tools. We think of CLI tools as synchronous, immediate, local. "Run command, get result."

But a diagnostic tool that connects to another process over a network protocol (even localhost) is a distributed system. The target process is an independent actor with its own timing. The inspector startup is an asynchronous operation we don't control. Network connections can fail transiently.

The same principles apply:

Don't assume the first attempt will succeed. Especially under load.
Back off exponentially. Linear retry hammers a system that's already struggling.
Cap the backoff. Waiting 32 seconds between attempts is too long for an interactive tool.
Give feedback. The user needs to know the tool is working, not frozen.
Know when to stop. Five attempts is enough. If the inspector isn't up after 11 seconds, something else is wrong.

Try It

npm install -g node-loop-detective@1.8.0

loop-detective <pid>

On most machines, you won't notice the change — it just connects slightly faster than before (500ms vs 1000ms). On loaded machines, it now works where it previously failed.

Source: github.com/iwtxokhtd83/node-loop-detective

Giving Kafka-Style Topics the Routing Power of RabbitMQ

Bill Tu — Fri, 10 Apr 2026 06:11:56 +0000

One of the most common complaints about Kafka is its routing model. You have topics. You have partitions. That's it. If you want to route order events differently based on region, priority, or event type, you're stuck with two bad options: create a separate topic for every permutation (orders-us-high, orders-eu-low, ...) or have every consumer receive everything and filter client-side.

RabbitMQ solved this years ago with its exchange model. But RabbitMQ trades routing flexibility for throughput — it can't match Kafka's append-only log performance.

TitanMQ has had both systems from the start: a Kafka-style commit log for storage and RabbitMQ-style exchanges for routing. The problem was they weren't connected. The exchange implementations sat in titanmq-routing, fully tested, doing nothing. Every PRODUCE request went straight to TopicManager.append(). The routing engine was dead code.

We just fixed that. Here's how.

The Design Problem

The naive approach would be: every message goes through an exchange, always. But that breaks backward compatibility and adds latency for the common case where you just want topic-partition semantics.

The approach we chose: exchanges are opt-in and overlay the existing topic model. The broker checks if the target topic name matches a declared exchange. If it does, route through the exchange. If it doesn't, write directly to the topic. Zero overhead for users who don't use exchanges.

Producer sends to "order-events"
         │
         ▼
  ┌──────────────────┐
  │ Exchange exists?  │
  │ "order-events"   │
  └──────┬───────────┘
     yes │        no
         │         │
         ▼         ▼
  ┌────────────┐  ┌──────────────┐
  │ Route via  │  │ Direct write │
  │ exchange   │  │ to topic     │
  └────┬───────┘  └──────────────┘
       │
       ▼
  Write to each destination topic
  (fulfillment, analytics, ...)

This means existing applications that use plain topics continue to work without any changes. Exchanges are a layer you add when you need them.

ExchangeManager

The central piece is ExchangeManager, which manages the lifecycle of named exchanges:

public class ExchangeManager {

    private final ConcurrentHashMap<String, Exchange> exchanges = new ConcurrentHashMap<>();

    public void declareExchange(String name, ExchangeType type) {
        Exchange exchange = switch (type) {
            case DIRECT -> new DirectExchange(name);
            case TOPIC -> new TopicExchange(name);
            case FANOUT -> new FanoutExchange(name);
            case CONTENT_BASED -> new ContentBasedExchange(name);
        };
        exchanges.put(name, exchange);
    }

    public void bind(String exchangeName, String destinationTopic, String routingKey) {
        exchanges.get(exchangeName).bind(destinationTopic, routingKey);
    }

    public List<String> route(String exchangeName, TitanMessage message, String routingKey) {
        Exchange exchange = exchanges.get(exchangeName);
        if (exchange == null) return List.of();
        return exchange.route(message, routingKey);
    }
}

Declaration is idempotent — declaring the same exchange with the same type twice is a no-op. Declaring with a different type throws an error. This matches RabbitMQ's behavior and prevents accidental misconfiguration.

The Four Exchange Types

Direct

Exact routing key match. The simplest and fastest.

mgr.declareExchange("order-events", ExchangeType.DIRECT);
mgr.bind("order-events", "fulfillment", "order.created");
mgr.bind("order-events", "billing", "order.paid");

// Message with routingKey="order.created" → fulfillment only
// Message with routingKey="order.paid" → billing only
// Message with routingKey="order.cancelled" → nowhere (error returned)

Topic

Wildcard pattern matching. * matches one word, # matches zero or more.

mgr.declareExchange("events", ExchangeType.TOPIC);
mgr.bind("events", "us-handler", "order.us.*");
mgr.bind("events", "all-errors", "#.error");

// "order.us.created" → us-handler
// "payment.processing.error" → all-errors
// "error" → all-errors (# matches zero words too)

The pattern matching uses a recursive algorithm that handles the # wildcard's zero-or-more semantics:

private static boolean matchParts(String[] routing, int ri, String[] pattern, int pi) {
    if (ri == routing.length && pi == pattern.length) return true;
    if (pi == pattern.length) return false;

    if (pattern[pi].equals("#")) {
        if (pi == pattern.length - 1) return true;
        for (int i = ri; i <= routing.length; i++) {
            if (matchParts(routing, i, pattern, pi + 1)) return true;
        }
        return false;
    }

    if (ri == routing.length) return false;
    if (pattern[pi].equals("*") || pattern[pi].equals(routing[ri])) {
        return matchParts(routing, ri + 1, pattern, pi + 1);
    }
    return false;
}

Fanout

Broadcast to all bound destinations. Routing key is ignored.

mgr.declareExchange("notifications", ExchangeType.FANOUT);
mgr.bind("notifications", "email-service", "");
mgr.bind("notifications", "sms-service", "");
mgr.bind("notifications", "push-service", "");

// Any message → all three services

Content-Based

This is where TitanMQ goes beyond RabbitMQ. Instead of matching routing keys, it evaluates predicates against message headers.

mgr.declareExchange("smart-router", ExchangeType.CONTENT_BASED);
mgr.bind("smart-router", "us-queue", "region=us");
mgr.bind("smart-router", "eu-priority", "region=eu,priority=high");

// Message with header region=us → us-queue
// Message with headers region=eu, priority=high → eu-priority
// Message with header region=eu, priority=low → nowhere

The wire protocol uses a simple key=value,key=value format for binding rules. But programmatically, you can pass arbitrary Predicate<Map<String, String>> lambdas:

ContentBasedExchange exchange = (ContentBasedExchange) mgr.getExchange("smart-router");
exchange.bind("vip-queue", headers ->
    Integer.parseInt(headers.getOrDefault("priority", "0")) > 8 &&
    "premium".equals(headers.get("tier"))
);

Integration into the Broker

The key change is in BrokerRequestHandler.handleProduce(). The original code was a straight write:

// Before: direct write, no routing
TitanMessage message = TitanMessage.deserialize(buffer);
long offset = topicManager.append(message);

The new code checks for an exchange first:

if (exchangeManager.hasExchange(message.topic())) {
    String routingKey = message.headers().getOrDefault("routingKey", message.topic());
    List<String> destinations = exchangeManager.route(message.topic(), message, routingKey);

    for (String destTopic : destinations) {
        TitanMessage routed = TitanMessage.builder()
                .id(message.id())
                .topic(destTopic)
                .key(message.key())
                .payload(message.payload())
                .headers(message.headers())
                .timestamp(message.timestamp())
                .build();
        topicManager.append(routed);
    }
} else {
    topicManager.append(message);
}

The routing key comes from the message header routingKey. If the header isn't set, the topic name itself is used as the routing key. This keeps the producer API simple — you don't need a separate routing key parameter.

Each destination topic gets its own copy of the message with the topic field rewritten. The message ID, key, payload, headers, and timestamp are preserved. This means consumers on different destination topics see the same message content.

Wire Protocol

Three new commands were added:

Command	Code	Payload
`DECLARE_EXCHANGE`	`0x24`	`[4B nameLen][NB name][1B type]`
`BIND_EXCHANGE`	`0x25`	`[4B exchangeLen][NB exchange][4B destLen][NB dest][4B keyLen][NB key]`
`UNBIND_EXCHANGE`	`0x26`	Same as BIND

The exchange type is encoded as a single byte: 0=DIRECT, 1=TOPIC, 2=FANOUT, 3=CONTENT_BASED.

All three commands return ADMIN_RESPONSE with a success byte on completion.

What This Enables

With the routing engine live, TitanMQ can now handle patterns that previously required either multiple topics or client-side filtering:

Event-driven microservices: Declare a topic exchange for domain events. Each service binds to the patterns it cares about. The order service gets order.*, the analytics service gets # (everything), the error handler gets #.error.

Multi-region routing: Declare a content-based exchange. Bind regional queues based on the region header. Messages are routed at the broker — no wasted bandwidth sending US events to EU consumers.

Notification fanout: Declare a fanout exchange. Every notification channel (email, SMS, push) gets every message. Add a new channel by adding a binding — no producer changes needed.

Priority routing: Use content-based exchange with priority header predicates. High-priority messages go to a fast-path queue with more consumer instances. Low-priority messages go to a batch queue.

All of this on top of Kafka-style append-only logs with offset tracking, consumer groups, and replay capability. The routing happens at write time, so consumers still get the full commit log semantics — they can seek, replay, and track offsets on their destination topics.

Trade-offs

Routing adds a small amount of work per PRODUCE request: one ConcurrentHashMap.get() to check if an exchange exists, plus the routing logic itself. For direct and fanout exchanges, this is O(1). For topic exchanges, it's O(n) where n is the number of bindings (pattern matching against each binding). For content-based exchanges, it's O(n) predicate evaluations.

Messages routed to multiple destinations are written multiple times — once per destination topic. This increases disk I/O proportionally. If a fanout exchange has 10 bindings, each message is written 10 times. This is the same trade-off RabbitMQ makes, and it's the correct one: consumers on different topics need independent offset tracking and retention.

The routing engine is entirely in-memory. Exchange declarations and bindings are not persisted to disk yet. A broker restart loses all exchange configuration. This is a known limitation tracked in a separate issue — the fix will persist exchange state to the Raft log so it survives restarts and is replicated across the cluster.

GitHub: https://github.com/iwtxokhtd83/TitanMQ
Release: v1.2.0
Issue: #10

When the Patient Dies on the Table: Handling Target Process Exit During Profiling

Bill Tu — Thu, 09 Apr 2026 14:03:43 +0000

You're profiling a Node.js process that's been crashing intermittently. You attach loop-detective, start a 60-second capture, and wait. Thirty seconds in, the process crashes again.

What should happen next?

Before v1.7.0, the answer was: nothing good. loop-detective would hang waiting for a CDP response that would never come, or exit with a cryptic WebSocket error. Any lag events and slow I/O data collected during those first 30 seconds? Gone.

This is the story of how we made loop-detective handle the worst-case scenario — the target process dying mid-profiling — and why the solution required changes across every layer of the tool.

The Failure Mode

To understand the problem, you need to know how profiling works internally:

1. Connect to inspector WebSocket
2. Send Profiler.start
3. Sleep for <duration> seconds     ← target can exit here
4. Send Profiler.stop               ← this never gets a response
5. Analyze and report

Step 3 is a setTimeout wrapped in a Promise. Step 4 is a CDP command sent over the WebSocket. If the target process exits during step 3, the WebSocket closes. When step 4 tries to send Profiler.stop, it throws "Inspector not connected." The error bubbles up, and the tool exits without reporting anything useful.

But the real loss is the data. During those 30 seconds before the crash, the lag detector was running. The I/O tracker was recording slow HTTP calls. That data exists in memory — it just never gets reported because the profiling pipeline assumes it will complete normally.

The Fix: Three Layers

Layer 1: Detect the Exit Immediately

The WebSocket close event fires when the target process exits. Previously, this just emitted a disconnected event. Now it also rejects all pending CDP callbacks and emits a targetExit event:

// inspector.js
this.ws.on('close', () => {
  // Reject all pending callbacks — target is gone
  for (const { reject, timer } of this._callbacks.values()) {
    clearTimeout(timer);
    try { reject(new Error('Target process exited')); } catch {}
  }
  this._callbacks.clear();
  this.emit('disconnected');
});

This is critical. Without rejecting pending callbacks, any in-flight CDP command (like Profiler.stop) would hang until its 30-second timeout. That's 30 seconds of the user staring at a frozen terminal.

Layer 2: Cancel the Sleep

The profiling duration is implemented as a sleep:

await this._sleep(duration);

If the target exits 10 seconds into a 60-second profile, we don't want to wait the remaining 50 seconds. The sleep needs to be cancellable.

_sleep(ms) {
  return new Promise((resolve, reject) => {
    this._sleepReject = reject;
    const timer = setTimeout(() => {
      this._sleepReject = null;
      resolve();
    }, ms);
    if (timer.unref) timer.unref();
  });
}

When the disconnected event fires, the Detective cancels the sleep:

this.inspector.on('disconnected', () => {
  if (!this._stopping) {
    this._targetExited = true;
    this.emit('targetExit', { pid, message: `Target process (PID ${pid}) exited during profiling` });

    // Cancel the sleep so _captureProfile can finish
    if (this._sleepReject) {
      this._sleepReject(new Error('Target process exited'));
      this._sleepReject = null;
    }
  }
});

Layer 3: Salvage What You Can

_captureProfile now handles the case where the sleep is cancelled and the profiler can't be stopped:

async _captureProfile(duration) {
  await this.inspector.send('Profiler.enable');
  await this.inspector.send('Profiler.setSamplingInterval', { interval: 100 });
  await this.inspector.send('Profiler.start');

  try {
    await this._sleep(duration);
  } catch {
    // Sleep was cancelled (target exited) — try to stop the profiler anyway
  }

  try {
    const { profile } = await this.inspector.send('Profiler.stop');
    await this.inspector.send('Profiler.disable');
    return profile;
  } catch {
    // Target exited — no profile data available
    return null;
  }
}

The callers check for null:

const profile = await this._captureProfile(this.config.duration);
if (profile) {
  const analysis = this.analyzer.analyzeProfile(profile);
  this.emit('profile', analysis, profile);
}
// If null, targetExit event was already emitted

The key insight: even though we can't get the CPU profile (it's inside the dead process), the lag events and slow I/O events were already emitted in real-time during the profiling period. The user has already seen them scroll by in the terminal. They're not lost.

What the User Sees

Before v1.7.0:

✔ Connected to Node.js process
  Profiling for 60s with 50ms lag threshold...

⚠ Event loop lag: 245ms at 2025-03-15T14:23:45.123Z
🌐 Slow HTTP: 1820ms GET api.example.com/users → 200

<hangs for 30 seconds, then>

✖ Error: Inspector not connected

After v1.7.0:

✔ Connected to Node.js process
  Profiling for 60s with 50ms lag threshold...

⚠ Event loop lag: 245ms at 2025-03-15T14:23:45.123Z
🌐 Slow HTTP: 1820ms GET api.example.com/users → 200

  ✖ Target process (PID 12345) exited during profiling
    Any lag or I/O events collected before the exit are shown above.

✔ Disconnected cleanly

No hang. Clear message. The partial data (lag event + slow HTTP call) is visible above the exit message.

Exit Codes

We introduced a three-code system:

Code	Meaning
0	Success — profiling completed normally
1	Error — connection failed, invalid arguments, etc.
2	Target exited — process died during profiling

This matters for automation. A monitoring script can distinguish between "profiling worked" (0), "loop-detective itself had a problem" (1), and "the target crashed" (2):

loop-detective 12345 -d 60
EXIT_CODE=$?

if [ $EXIT_CODE -eq 2 ]; then
  echo "Target process crashed during profiling — check partial output"
  # Trigger incident response
elif [ $EXIT_CODE -eq 1 ]; then
  echo "loop-detective failed to connect"
elif [ $EXIT_CODE -eq 0 ]; then
  echo "Profiling completed successfully"
fi

The Irony of Profiling Crashing Processes

There's an irony here: the processes most likely to exit during profiling are the ones you most need to profile. They're crashing. They're running out of memory. They're hitting unhandled exceptions. You attached loop-detective precisely because something is wrong.

This means the "target exits during profiling" scenario isn't an edge case — it's a primary use case. If your tool can't handle it gracefully, it fails exactly when it's needed most.

The partial data is often enough. If you see three slow HTTP calls to the same endpoint right before the crash, that's a strong signal. If you see event loop lag spiking to 2 seconds, that tells you the process was CPU-bound before it died. You don't always need the full CPU profile to understand what happened.

Watch Mode Behavior

In --watch mode, a target exit stops the watch loop instead of starting a new cycle:

if (this._running && !this._targetExited) {
  setTimeout(runCycle, 1000);
}

There's no point retrying — the process is gone. The tool reports what it has and exits cleanly.

Programmatic API

For tool builders integrating loop-detective:

const { Detective } = require('node-loop-detective');

const detective = new Detective({ pid: 12345, duration: 60000 });

// Collect events as they arrive
const events = { lags: [], slowIO: [] };
detective.on('lag', (data) => events.lags.push(data));
detective.on('slowIO', (data) => events.slowIO.push(data));

// Handle target exit
detective.on('targetExit', (data) => {
  console.log(data.message);
  console.log(`Collected ${events.lags.length} lag events and ${events.slowIO.length} slow I/O events before exit`);
  // Send partial data to your monitoring system
  sendToMonitoring({ partial: true, ...events });
});

detective.on('profile', (analysis) => {
  // Full profile available — target didn't crash
  sendToMonitoring({ partial: false, analysis, ...events });
});

await detective.start();

The Broader Lesson

Diagnostic tools need to be more resilient than the systems they diagnose. If your profiler crashes when the target crashes, you've lost the most valuable debugging moment. Every diagnostic tool that attaches to external processes should answer three questions:

What happens when the target exits? Don't hang. Don't lose data. Report what you have.
How does the user know what happened? Clear messages, distinct exit codes, structured output.
What data survives? Real-time events (lag, I/O) survive because they were already emitted. Batch results (CPU profile) may be lost. Design accordingly.

The fix in v1.7.0 is about 60 lines of code across three files. But it transforms the tool from one that fails at the worst possible moment to one that handles it gracefully.

npm install -g node-loop-detective@1.7.0

Source: github.com/iwtxokhtd83/node-loop-detective

Not All Requests Are Equal: Adding Variable Cost to a Node.js Rate Limiter

Bill Tu — Thu, 09 Apr 2026 13:51:48 +0000

Rate limiters typically treat every request the same. One request, one token. But in the real world, a request that fetches a single user profile and a request that exports 10,000 records are not the same thing. Charging them the same rate limit token is like charging the same toll for a bicycle and a semi-truck.

We just shipped variable cost support in node-rate-limiter-pro v1.2.0. This post covers why it matters, how we implemented it across two algorithms and two storage backends, and the one subtle bug that almost shipped with it.

The Problem

Consider a batch API endpoint:

POST /api/users/export
Body: { "ids": ["u1", "u2", ..., "u500"] }

With a flat rate limiter set to 100 requests per minute, a client can export 50,000 users per minute by sending 100 requests of 500 IDs each. But a client making simple single-user lookups gets the same 100 requests per minute — 100 users total.

That's a 500x difference in actual resource consumption for the same rate limit.

Variable cost fixes this. Instead of consume('user:123') always deducting 1 token, you can now write:

// Each ID in the batch costs one token
const result = await limiter.consume('user:123', ids.length);

The batch export of 500 IDs now costs 500 tokens. The single lookup costs 1. The rate limiter finally reflects reality.

The API

The change is minimal — one optional parameter:

// Before (still works, backward compatible)
await limiter.consume('user:123');

// After: consume 5 tokens
await limiter.consume('user:123', 5);

For Express middleware, we added a costFn option:

app.use(limiter.middleware({
  costFn: (req) => req.body?.items?.length || 1,
}));

Implementation: Four Layers Deep

The cost parameter touches every layer of the stack: the public API, two algorithm implementations, two store backends, and two Redis Lua scripts. Here's how each one changed.

Token Bucket

The Token Bucket was the easy one. The algorithm already works with fractional tokens — we just replaced the hardcoded 1 with cost:

// Before
if (bucket.tokens >= 1) {
  bucket.tokens -= 1;
  // ...
}
const retryAfter = Math.ceil(
  ((1 - bucket.tokens) / this.maxTokens) * this.refillIntervalMs
);

// After
if (bucket.tokens >= cost) {
  bucket.tokens -= cost;
  // ...
}
const retryAfter = Math.ceil(
  ((cost - bucket.tokens) / this.maxTokens) * this.refillIntervalMs
);

The retryAfter calculation scales naturally. If you need 5 tokens and only have 2, you need to wait for 3 tokens to refill. The math is (cost - currentTokens) / refillRate.

Sliding Window — Where It Got Interesting

Our sliding window uses a weighted counter approach. Instead of storing every timestamp, it keeps two sub-window counters and calculates a weighted estimate:

estimatedCount = prevWindowCount × weight + currentWindowCount

where weight decays linearly from 1 to 0 as time progresses through the current window.

The original admission check was:

if (estimatedCount < this.maxRequests) {
  state.currCount++;
  // allowed
}

The naive variable-cost version would be:

if (estimatedCount + cost <= this.maxRequests) {
  state.currCount += cost;
  // allowed
}

This is where the bug hid. More on that in a moment. The correct implementation:

const used = Math.floor(estimatedCount);
const remaining = Math.max(0, this.maxRequests - used);

if (cost <= remaining) {
  state.currCount += cost;
  // allowed
}

We floor the estimated count to get integer remaining capacity, then check if the cost fits. The remaining field in the response now accurately reflects how many tokens are available — even when the request is denied:

// Denied response now shows actual remaining capacity
return {
  allowed: false,
  remaining,  // e.g., 2 tokens left, but you asked for 5
  // ...
};

This is a meaningful improvement over the old code, which always returned remaining: 0 on denial regardless of actual capacity.

Redis Lua Scripts

Both Redis Lua scripts needed the same treatment. The cost arrives as an additional ARGV parameter.

Token Bucket in Lua — straightforward substitution:

local cost = tonumber(ARGV[4])

if tokens >= cost then
  tokens = tokens - cost
  -- ...
  return {1, math.floor(tokens), 0}
end

local retryAfter = math.ceil(((cost - tokens) / maxTokens) * refillInterval)
return {0, math.floor(tokens), retryAfter}

Sliding Window in Lua — the ZSET approach needs to add cost members:

local cost = tonumber(ARGV[4])

if count + cost <= limit then
  for i = 1, cost do
    local seq = redis.call('INCR', key .. ':seq')
    redis.call('ZADD', key, now, now .. '-' .. seq)
  end
  return {1, limit - count - cost, 0}
end

return {0, limit - count, retryAfter}

The loop adds cost individual entries to the sorted set. Each entry gets a unique member via the atomic INCR counter (which we introduced in v1.1.0 to fix a collision bug — see our previous post). This keeps the ZSET accurate for future window calculations.

Express Middleware

The middleware layer adds a costFn option that mirrors the existing keyFn pattern:

middleware(options?: {
  keyFn?: (req: any) => string;
  costFn?: (req: any) => number;
  onLimited?: (req: any, res: any) => void;
}) {
  const costFn = options?.costFn;

  return async (req, res, next) => {
    const key = keyFn(req);
    const cost = costFn ? costFn(req) : 1;
    const result = await this.consume(key, cost);
    // ...
  };
}

The Floating Point Bug That Almost Shipped

Remember the sliding window admission check? Here's the bug we caught in testing.

Setup: limit = 3, window = 1000ms. Consume 3 tokens, wait 1100ms (just past the window boundary), then try to consume 1 more.

After 1100ms, the algorithm advances the sub-windows:

prevCount = 3, currCount = 0
elapsedInCurrent ≈ 100ms
weight = 1 - 100/1000 = 0.9
estimatedCount = 3 × 0.9 + 0 = 2.7

The old check estimatedCount < maxRequests → 2.7 < 3 → true ✅

The naive variable-cost check estimatedCount + cost <= maxRequests → 2.7 + 1 = 3.7 <= 3 → false ❌

Same scenario, different result. The request that should be allowed gets blocked.

The root cause: the weighted counter produces fractional values. The old code compared a float against an integer with <, which implicitly gave ~0.99 tokens of headroom. The new code with <= eliminated that headroom.

The fix: floor the estimate before comparing, converting the float back to integer capacity:

const used = Math.floor(estimatedCount);     // floor(2.7) = 2
const remaining = Math.max(0, limit - used); // 3 - 2 = 1
if (cost <= remaining)                       // 1 <= 1 → true ✅

This preserves the original behavior for cost = 1 while correctly handling higher costs. If estimatedCount were 2.3 and cost were 2, we'd get remaining = 1, and 2 <= 1 → false — correctly blocking a request that would exceed the limit.

Real-World Usage Patterns

Here are some patterns we've seen (or expect to see) with variable cost:

Batch APIs:

app.post('/api/batch', limiter.middleware({
  costFn: (req) => req.body?.operations?.length || 1,
}));

File uploads by size:

app.post('/upload', limiter.middleware({
  costFn: (req) => Math.ceil(Number(req.headers['content-length'] || 1) / 1_000_000),
}));

Tiered pricing (premium users get cheaper requests):

app.use(limiter.middleware({
  costFn: (req) => req.user?.plan === 'premium' ? 1 : 3,
}));

GraphQL complexity:

app.use('/graphql', limiter.middleware({
  costFn: (req) => calculateQueryComplexity(req.body.query),
}));

Design Decisions

A few choices we made along the way:

Why cost as a parameter, not a constructor option? Because cost varies per request, not per limiter. A single limiter instance might handle requests with different costs. Making it a per-call parameter is the only design that works.

Why default to 1? Backward compatibility. Every existing consume('key') call continues to work without changes. The cost parameter is purely additive.

Why floor the sliding window estimate? The weighted counter is an approximation. Flooring converts it to a conservative integer count, which is the right behavior for admission control — when in doubt, allow rather than block. This matches the original behavior and avoids surprising users with stricter-than-expected limits.

Why not validate cost > 0? We considered it, but decided against adding runtime validation on the hot path. If you pass cost = 0, you get a free peek at the current state (which is actually useful). If you pass a negative number, you're doing something wrong, but we're not going to add a branch to every consume() call to catch it.

What's Next

Variable cost was the most requested feature in our issue tracker. With it shipped, we're looking at:

get(key) — peek at rate limit state without consuming (#6)
Fixed Window Counter algorithm (#7)
IETF standard rate limit headers (#8)

The full implementation is in v1.2.0. MIT licensed, zero dependencies for in-memory mode, optional Redis for distributed deployments.

Writing 89 Tests for a Quantitative Trading Framework: Strategy and Trade-offs

Bill Tu — Thu, 09 Apr 2026 13:25:06 +0000

Adding a test suite to an existing codebase is a different exercise than writing tests alongside new code. You're reverse-engineering the implicit contracts, discovering which behaviors are intentional and which are accidental, and deciding what's worth testing versus what's noise. This article covers how we built the test suite for QuantFlow, an open-source quantitative trading framework in Python, and the decisions behind each layer of tests.

The Testing Problem in Quant Systems

Quantitative trading code has a testing problem that most software doesn't: the outputs are floating-point numbers derived from financial time series, and "correct" is often a matter of degree. An SMA of [1, 2, 3, 4, 5] with period 5 should be exactly 3.0 — that's easy. But what should the RSI of a 50-bar synthetic price series be? You can't hardcode an expected value without coupling the test to the random seed and the exact implementation.

This creates a tension between two testing philosophies:

Test against known reference values (precise but brittle)
Test invariants and contracts (robust but less specific)

We use both, choosing based on the indicator.

Test Architecture

The suite is organized into five files, each targeting a different layer:

tests/
├── conftest.py          # Shared fixtures
├── test_indicators.py   # 46 tests — all 16 indicators
├── test_risk.py         # 12 tests — risk manager rules
├── test_portfolio.py    # 16 tests — execution and position tracking
├── test_strategies.py   # 8 tests  — built-in strategy signals
└── test_engine.py       # 9 tests  — end-to-end backtest integration

The dependency flows downward: indicators have no dependencies, risk and portfolio depend on core models, strategies depend on indicators, and the engine depends on everything. Tests follow the same order — if indicator tests fail, strategy and engine tests are meaningless.

Fixtures: Deterministic Randomness

The conftest.py provides four shared fixtures that generate synthetic market data with a fixed random seed:

@pytest.fixture
def sample_prices() -> np.ndarray:
    np.random.seed(42)
    returns = np.random.normal(0.001, 0.02, 50)
    prices = 100.0 * np.cumprod(1 + returns)
    return prices

The seed 42 makes tests deterministic — same data every run. The parameters (0.1% daily drift, 2% daily volatility) produce realistic-looking price series without being tied to any real market data. This matters because tests that depend on real market data break when the data source changes, and they can't be run offline.

The sample_ohlcv fixture generates correlated OHLCV data where high > close > low holds approximately (with small random noise). This is important for indicators like ATR and Stochastic that use all four price fields — feeding them random uncorrelated data would test the math but not the real-world behavior.

Indicator Tests: Three Categories

The 46 indicator tests fall into three categories.

Warm-up Guards

Every indicator should return None when given insufficient data. This is the most important contract — it prevents strategies from acting on garbage values during the first few bars of a backtest.

def test_insufficient_data_returns_none(self):
    sma = SMA(period=20)
    assert sma.calculate(np.array([1.0, 2.0, 3.0])) is None

We test this for every indicator. It's repetitive, but it catches a common class of bugs: an indicator that returns 0.0 or NaN instead of None when it doesn't have enough data.

Known-Value Tests

For indicators with simple formulas, we test against hand-calculated values:

def test_known_value(self):
    prices = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
    sma = SMA(period=5)
    assert sma.calculate(prices) == pytest.approx(3.0)

def test_known_value(self):
    prices = np.array([100.0] * 13)
    prices[-1] = 110.0
    roc = ROC(period=12)
    assert roc.calculate(prices) == pytest.approx(10.0)

These are the strongest tests — they verify the actual computation. But they're only practical for indicators where you can compute the expected value by hand. For complex indicators like ADX or Ichimoku, hand-calculation is error-prone, so we use invariant tests instead.

Invariant Tests

For indicators where exact values are hard to predict, we test mathematical invariants that must always hold:

def test_range_0_to_100(self, sample_prices):
    rsi = RSI(period=14)
    val = rsi.calculate(sample_prices)
    assert 0.0 <= val <= 100.0

def test_all_gains_returns_100(self):
    prices = np.arange(1.0, 20.0)  # monotonically increasing
    rsi = RSI(period=14)
    assert rsi.calculate(prices) == pytest.approx(100.0)

def test_all_losses_returns_0(self):
    prices = np.arange(20.0, 1.0, -1.0)  # monotonically decreasing
    assert rsi.calculate(prices) == pytest.approx(0.0, abs=0.01)

RSI must be between 0 and 100. All-gain series must produce RSI of 100. All-loss series must produce RSI near 0. These invariants hold regardless of the implementation details.

For Bollinger Bands: upper > middle > lower (when prices have non-zero variance). For Stochastic: %K between 0 and 100. For Williams %R: between -100 and 0. These are mathematical properties of the formulas, not implementation details.

Consistency Tests

One particularly valuable test pattern verifies that calculate() and series() agree:

def test_calculate_matches_series_last(self, sample_prices):
    ema = EMA(period=12)
    calc_val = ema.calculate(sample_prices)
    series_val = ema.series(sample_prices)[-1]
    assert calc_val == pytest.approx(series_val, rel=1e-10)

After the v0.2.0 optimization where EMA.calculate() was rewritten to avoid building the full series, this test ensures the optimized path produces the same result as the reference implementation. It's a regression test disguised as a unit test.

The NaN Leakage Test

One test specifically targets a bug we fixed in v0.2.0:

def test_no_nan_leakage(self):
    prices = np.arange(1.0, 40.0)
    macd = MACD()
    m, s, h = macd.calculate(prices)
    if m is not None:
        assert not np.isnan(m)
        assert not np.isnan(s)
        assert not np.isnan(h)

The contract is: calculate() returns either valid floats or None. Never NaN. This test exists because NaN is insidious — it passes through arithmetic silently and makes comparisons return False, causing strategies to silently skip signals without any error.

The Stochastic Alignment Test

Another test targets the %D alignment bug from v0.2.0:

def test_d_alignment(self, sample_ohlcv):
    stoch = Stochastic(k_period=5, d_period=3)
    k_series, d_series = stoch.series(...)
    first_k = np.where(~np.isnan(k_series))[0][0]
    first_d = np.where(~np.isnan(d_series))[0][0]
    assert first_d == first_k + stoch.d_period - 1

%D is a moving average of %K, so it must start exactly d_period - 1 bars after %K begins. This is a structural invariant — if the alignment is wrong, the %D values are computed from the wrong %K windows.

Risk Manager Tests: State Machine Behavior

The risk manager is a stateful component — once the drawdown circuit breaker triggers, it stays triggered until explicitly reset. The tests model this as a state machine:

def test_drawdown_halts_trading(self):
    signal = Signal(SignalType.BUY, size=10)
    result = self.rm.check(signal, 100.0, 75000.0, 100000.0, 0)
    assert result.type == SignalType.HOLD

def test_drawdown_halt_is_sticky(self):
    self.rm.check(Signal(SignalType.BUY, size=10), 100.0, 75000.0, 100000.0, 0)
    # Even if equity recovers, still halted
    result = self.rm.check(Signal(SignalType.BUY, size=10), 100.0, 95000.0, 100000.0, 0)
    assert result.type == SignalType.HOLD

def test_drawdown_reset(self):
    self.rm.check(Signal(SignalType.BUY, size=10), 100.0, 75000.0, 100000.0, 0)
    self.rm.reset()
    result = self.rm.check(Signal(SignalType.BUY, size=10), 100.0, 95000.0, 100000.0, 0)
    assert result.type == SignalType.BUY

Three tests, three states: normal → halted → reset. The "sticky" test is critical — it verifies that the halt persists even when equity recovers. Without this test, a bug that resets the halt flag on every call would go undetected.

The position sizing test uses exact arithmetic:

def test_position_sizing_caps_size(self):
    # equity=100000, max_pct=10% → max $10000 → at $100/share → max 100 shares
    signal = Signal(SignalType.BUY, size=500)
    result = self.rm.check(signal, 100.0, 100000.0, 100000.0, 0)
    assert result.size <= 100

The comment documents the math. This is intentional — risk management tests should be readable by someone who isn't a developer, because the rules they encode are business logic that a portfolio manager or risk officer should be able to verify.

Portfolio Tests: Isolating Execution Mechanics

Portfolio tests use zero slippage to make assertions predictable:

def setup_method(self):
    self.portfolio = Portfolio(
        initial_capital=100000.0,
        commission_rate=0.001,
        slippage_rate=0.0,  # zero slippage for predictable tests
    )

This is a deliberate choice. Slippage adds randomness to fill prices, which makes exact assertions impossible. By setting it to zero, we can test the execution logic in isolation. Slippage behavior is tested separately at the engine level.

The most important portfolio test verifies the v0.2.0 fix for equity calculation:

def test_equity_reflects_current_price(self):
    signal = Signal(SignalType.BUY, size=100)
    self.portfolio.execute_signal(signal, "TEST", 100.0, self.ts)
    self.portfolio.update_market_prices("TEST", 120.0)
    assert self.portfolio.equity > self.portfolio.cash + 100 * 100.0

If equity used entry price (the old bug), equity would equal cash + 100 * 100. With the fix, it equals cash + 100 * 120, which is strictly greater. The test doesn't check the exact value — it checks the relationship, which is more robust.

The short margin test verifies the v0.2.0 safety fix:

def test_short_margin_check(self):
    self.portfolio.short_margin_rate = 1.0
    signal = Signal(SignalType.SELL, size=2000)  # $200k margin > $100k cash
    result = self.portfolio.execute_signal(signal, "TEST", 100.0, self.ts)
    assert result is None

This test exists because the absence of a margin check was a real bug. The test documents the expected behavior: shorts that exceed available margin are rejected.

Strategy Tests: Signal Generation

Strategy tests face a unique challenge: strategies are stateful (they track previous indicator values for crossover detection), and their outputs depend on the full price history. Testing them requires constructing price series that produce known indicator states.

def test_buy_on_oversold(self):
    strategy = RSIMeanReversion(period=14, oversold=30, size=100)
    portfolio = Portfolio()
    prices = np.linspace(200, 100, 50)  # strong downtrend
    bar = make_bar(100.0, prices)
    signal = strategy.on_bar(bar, portfolio)
    assert signal.type == SignalType.BUY

A monotonically decreasing price series guarantees RSI will be very low (near 0), which is well below the oversold threshold of 30. We don't need to know the exact RSI value — we just need to know it's below 30, which is guaranteed by the price pattern.

The SMA crossover test is more involved because it requires a crossover event:

def test_generates_buy_on_golden_cross(self):
    strategy = SMACrossover(fast_period=3, slow_period=5, size=100)
    portfolio = Portfolio()
    prices_down = np.linspace(120, 100, 20)
    prices_up = np.linspace(100, 130, 10)
    prices = np.concatenate([prices_down, prices_up])
    signals = []
    for i in range(len(prices)):
        bar = make_bar(prices[i], prices[:i + 1])
        signals.append(strategy.on_bar(bar, portfolio))
    buy_signals = [s for s in signals if s.type == SignalType.BUY]
    assert len(buy_signals) > 0

The price series goes down then sharply up. At some point during the upturn, the fast SMA (3-period) will cross above the slow SMA (5-period). We don't assert exactly when — just that it happens at least once. This makes the test resilient to minor changes in the crossover detection logic.

Engine Tests: End-to-End Integration

Engine tests verify that all components work together. They use an InMemoryDataFeed to avoid file I/O:

class InMemoryDataFeed(DataFeed):
    def __init__(self, df: pd.DataFrame):
        self._df = df

    def load(self) -> pd.DataFrame:
        return self._df

The most valuable engine test verifies that commission has a measurable effect:

def test_commission_reduces_equity(self, data_feed):
    r1 = BacktestEngine(..., commission=0.0, ...).run()
    r2 = BacktestEngine(..., commission=0.01, ...).run()
    assert r2.equity_curve[-1] <= r1.equity_curve[-1]

This is an invariant test at the system level: higher commission should never increase final equity. It doesn't test the exact commission calculation — that's covered in portfolio tests — but it verifies that commission flows through the entire pipeline correctly.

Edge case tests verify the system doesn't crash on degenerate inputs:

def test_single_bar_no_crash(self):
    df = pd.DataFrame({...}, index=pd.DatetimeIndex([datetime(2024, 1, 1)]))
    result = BacktestEngine(data_feed=InMemoryDataFeed(df), ...).run()
    assert len(result.equity_curve) == 1

def test_zero_volume_no_crash(self):
    df = pd.DataFrame({..., "volume": np.zeros(n), ...})
    result = BacktestEngine(data_feed=InMemoryDataFeed(df), ...).run()
    assert result.total_return == pytest.approx(0.0)

These tests exist because real market data contains edge cases: single-bar datasets from API errors, zero-volume bars from illiquid instruments, gaps in timestamps. The system should handle all of these without raising exceptions.

What We Chose Not to Test

A few deliberate omissions:

We don't test the YahooDataFeed because it makes HTTP requests to an external API. Testing it would require either mocking the HTTP layer (which tests the mock, not the code) or hitting the real API (which is slow, flaky, and rate-limited). The CSVDataFeed is tested indirectly through the engine tests.

We don't test the LiveEngine because it runs an infinite loop with time.sleep(). Testing it properly would require threading, timeouts, and mock brokers — significant infrastructure for a component that's architecturally identical to the backtest engine but with a different data source.

We don't test the plot() method on PerformanceReport because visual output is best verified by humans. We could assert that it doesn't raise an exception, but that's a weak test that adds maintenance burden.

Running the Suite

pip install pytest
python -m pytest tests/ -v

============================= 89 passed in 4.87s ==============================

All 89 tests run in under 5 seconds. No external dependencies, no network calls, no file I/O (except the fixtures that generate in-memory DataFrames). This matters for CI — fast tests get run; slow tests get skipped.

The test suite is part of QuantFlow v0.3.0. Contributions and additional test cases are welcome.

Who Should Own the Order ID? Moving ID Generation Inside the Matching Engine

Bill Tu — Thu, 09 Apr 2026 13:04:07 +0000

Order IDs seem trivial. Every order needs one, they need to be unique, and they show up in every trade record. In the first version of MatchEngine, the caller provided the ID as a string. It worked. Then it didn't.

This article explains why we moved ID generation inside the engine, the design choices behind the implementation, and the three new API methods that replaced the old approach.

The Problem With Caller-Provided IDs

The original API looked like this:

order := model.NewLimitOrder("my-order-1", model.Buy, price, quantity)
trades, err := e.SubmitOrder("BTC/USD", order)

The caller chose the ID. The engine accepted it. This created three problems:

1. Uniqueness is the caller's problem. The engine added duplicate detection in v0.2.0, but that just turned a silent corruption into an error. The caller still had to generate unique IDs — and every caller had to solve the same problem independently. In a system with multiple order sources (REST API, WebSocket, FIX gateway), coordinating unique IDs across all of them is a real engineering challenge.

2. No ordering guarantee. Caller-provided IDs are arbitrary strings. There is no way to determine which order came first by looking at the IDs. For debugging, auditing, and replay, monotonically increasing IDs are invaluable. They give you a total ordering of events without consulting timestamps.

3. The API is noisy. Every call site has to construct an Order struct with an ID, a side, a type, a price, a quantity, and a timestamp. The ID and timestamp are boilerplate — the caller doesn't care about them. They care about "sell 5 BTC at 50,000."

The Design: Atomic Counter

We chose the simplest possible ID generator: an atomic uint64 counter.

type Engine struct {
    nextID   uint64  // atomic counter
    idPrefix string  // optional prefix
    // ...
}

func (e *Engine) generateID() string {
    id := atomic.AddUint64(&e.nextID, 1)
    return e.idPrefix + strconv.FormatUint(id, 10)
}

atomic.AddUint64 increments the counter and returns the new value in a single atomic operation. No mutex needed. No allocation. The result is converted to a string with strconv.FormatUint, which is the fastest integer-to-string conversion in Go's standard library.

Why Not UUID?

UUIDs (google/uuid) are globally unique, which sounds appealing. But they have drawbacks for this use case:

Property	Atomic Counter	UUID v4
Uniqueness	Unique within engine instance	Globally unique
Ordering	Monotonically increasing	Random (no ordering)
Size	1-20 chars (`"1"` to `"18446744073709551615"`)	36 chars (`"550e8400-e29b-41d4-a716-446655440000"`)
Speed	~1ns (atomic add + format)	~200ns (crypto/rand + format)
Readability	`"ORD-42"`	`"550e8400-e29b-41d4-a716-446655440000"`

For a single-instance matching engine, the atomic counter is strictly better. It is faster, smaller, ordered, and human-readable. If the engine were distributed across multiple nodes, UUIDs or a distributed ID service (like Twitter's Snowflake) would be necessary. But that is a different problem for a different architecture.

Why Not Inside the Mutex?

The generateID method uses atomic.AddUint64, which is lock-free. It runs before the engine acquires its mutex in SubmitOrder. This is deliberate:

func (e *Engine) SubmitLimitOrder(symbol string, side model.Side, price, quantity decimal.Decimal) (string, []model.Trade, error) {
    id := e.generateID()  // lock-free, outside mutex
    order := model.NewLimitOrder(id, side, price, quantity)
    trades, err := e.SubmitOrder(symbol, order)  // acquires mutex internally
    return id, trades, err
}

If ID generation were inside the mutex, it would add contention for no reason. The atomic operation is already thread-safe on its own. Keeping it outside the mutex means the critical section is no longer than before.

The Prefix Option

IDs can be prefixed for readability or multi-engine disambiguation:

e := engine.New(engine.WithIDPrefix("ORD-"))
// Generates: "ORD-1", "ORD-2", "ORD-3", ...

e := engine.New(engine.WithIDPrefix("ENGINE-A-"))
// Generates: "ENGINE-A-1", "ENGINE-A-2", ...

The prefix is concatenated at generation time. No extra allocation — strconv.FormatUint returns a string, and + on two strings is a single allocation.

The New API

Three new methods replace the old pattern:

SubmitLimitOrder

func (e *Engine) SubmitLimitOrder(
    symbol string,
    side model.Side,
    price, quantity decimal.Decimal,
) (string, []model.Trade, error)

The caller provides only what they care about: symbol, side, price, quantity. The engine generates the ID and returns it.

id, trades, err := e.SubmitLimitOrder("BTC/USD", model.Sell, price, qty)
// id = "1" (or "ORD-1" with prefix)

SubmitMarketOrder

func (e *Engine) SubmitMarketOrder(
    symbol string,
    side model.Side,
    quantity decimal.Decimal,
) (string, []model.Trade, error)

Same pattern, no price parameter (market orders don't have one).

id, trades, err := e.SubmitMarketOrder("BTC/USD", model.Buy, qty)

SubmitRequest

For cases where the order type is determined at runtime (e.g., from a JSON payload), SubmitRequest accepts an OrderRequest struct:

type OrderRequest struct {
    Side     Side
    Type     OrderType
    Price    decimal.Decimal
    Quantity decimal.Decimal
}

No ID field. No timestamp. Just the intent.

req := model.NewLimitOrderRequest(model.Buy, price, qty)
id, trades, err := e.SubmitRequest("BTC/USD", req)

This is particularly useful when deserializing from external input:

// From a REST API handler
var req model.OrderRequest
json.NewDecoder(r.Body).Decode(&req)
id, trades, err := e.SubmitRequest(symbol, req)

The caller never touches an order ID. The engine owns the lifecycle.

Backward Compatibility

The original SubmitOrder method is unchanged:

func (e *Engine) SubmitOrder(symbol string, order *model.Order) ([]model.Trade, error)

It still accepts caller-provided IDs, still checks for duplicates, and still works exactly as before. The new methods are built on top of it — they generate an ID, construct an Order, and call SubmitOrder internally.

This means existing code continues to work without modification. Migration is opt-in.

How the Pieces Fit Together

The call chain for SubmitLimitOrder:

SubmitLimitOrder("BTC/USD", Sell, 50000, 1)
    │
    ├── generateID()              ← atomic, lock-free
    │   └── "ORD-7"
    │
    ├── NewLimitOrder("ORD-7", Sell, 50000, 1)
    │   └── Order{ID: "ORD-7", ...}
    │
    └── SubmitOrder("BTC/USD", order)
        ├── normalize symbol      ← "BTC/USD"
        ├── acquire mutex
        ├── validate symbol
        ├── check duplicate ID    ← "ORD-7" not in index
        ├── match against book
        ├── add to book if unfilled
        ├── record trades
        └── release mutex

The ID is generated before the mutex is acquired. If SubmitOrder returns an error (invalid price, bad symbol, etc.), the ID is "wasted" — the counter has already incremented. This is fine. Gaps in the ID sequence are harmless, and avoiding them would require either rolling back the atomic counter (unsafe) or generating the ID inside the mutex (unnecessary contention).

Testing Concurrency

The most important property of the ID generator is uniqueness under concurrent access. We test this with 10 goroutines each submitting 20 orders simultaneously:

func TestIDsUniqueAcrossGoroutines(t *testing.T) {
    e := New()

    idCh := make(chan string, 200)
    var wg sync.WaitGroup
    for g := 0; g < 10; g++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for i := 0; i < 20; i++ {
                id, _, _ := e.SubmitLimitOrder("BTC", model.Sell, d("100"), d("1"))
                idCh <- id
            }
        }()
    }
    wg.Wait()
    close(idCh)

    seen := make(map[string]bool)
    for id := range idCh {
        if seen[id] {
            t.Errorf("duplicate ID generated: %s", id)
        }
        seen[id] = true
    }
    if len(seen) != 200 {
        t.Errorf("expected 200 unique IDs, got %d", len(seen))
    }
}

200 orders, 10 goroutines, zero duplicates. The atomic counter guarantees this without any locking.

We also verify monotonic ordering:

func TestIDsAreMonotonicallyIncreasing(t *testing.T) {
    e := New()

    var ids []string
    for i := 0; i < 10; i++ {
        id, _, _ := e.SubmitLimitOrder("BTC", model.Sell, d("100"), d("1"))
        ids = append(ids, id)
    }

    for i := 1; i < len(ids); i++ {
        if ids[i] <= ids[i-1] {
            t.Errorf("IDs not monotonically increasing: %s <= %s", ids[i], ids[i-1])
        }
    }
}

Note: monotonic ordering is guaranteed only within a single goroutine. Across goroutines, the IDs are unique but the observation order depends on scheduling. Goroutine A might generate ID 5 before goroutine B generates ID 4, but B might write to the channel first. The IDs themselves are still strictly ordered by their numeric value.

Before and After

Before (caller manages IDs)

e := engine.New()

// Caller must generate unique IDs
sellID := fmt.Sprintf("sell-%d", time.Now().UnixNano())
sell := model.NewLimitOrder(sellID, model.Sell, price, qty)
trades, err := e.SubmitOrder("BTC/USD", sell)

buyID := fmt.Sprintf("buy-%d", time.Now().UnixNano())
buy := model.NewLimitOrder(buyID, model.Buy, price, qty)
trades, err = e.SubmitOrder("BTC/USD", buy)

// Hope the IDs are unique...
e.CancelOrder("BTC/USD", sellID)

After (engine manages IDs)

e := engine.New(engine.WithIDPrefix("ORD-"))

sellID, _, _ := e.SubmitLimitOrder("BTC/USD", model.Sell, price, qty)
buyID, trades, _ := e.SubmitLimitOrder("BTC/USD", model.Buy, price, qty)

// IDs are guaranteed unique
e.CancelOrder("BTC/USD", sellID)

Less code, no uniqueness bugs, and the returned IDs are useful for logging, cancellation, and client responses.

Summary

Aspect	Before	After
ID source	Caller-provided string	Engine-generated atomic counter
Uniqueness	Caller's responsibility (error on duplicate)	Guaranteed by engine
Ordering	Arbitrary	Monotonically increasing
Thread safety	N/A (caller's problem)	Lock-free atomic operation
API surface	`SubmitOrder(symbol, *Order)`	`SubmitLimitOrder`, `SubmitMarketOrder`, `SubmitRequest`
Backward compat	N/A	`SubmitOrder` still works

The change is small — one new field on the engine, one three-line method, three thin wrapper methods, and one new OrderRequest type. But it shifts the responsibility for a correctness-critical property (ID uniqueness) from every caller to a single, tested, atomic implementation inside the engine. That is the right place for it.

GitHub: https://github.com/iwtxokhtd83/MatchEngine
Release: v0.4.0
Issue: #6 — Engine should generate unique order IDs internally