DEV Community: Gabor Koos

Decorating Promises Without Breaking Them

Gabor Koos — Fri, 10 Apr 2026 14:18:20 +0000

I wanted .get().json().

This came up while building convenience plugins for ffetch, a lightweight fetch wrapper focused on keeping native semantics intact. Libraries like ky solve the ergonomics problem by introducing a custom Response-like object, which works great until something outside the library expects a plain Response. I wanted a different path.

Not because I needed it, strictly speaking. await fetch('/api/todos/1') followed by await response.json() works perfectly fine. But after the hundredth time writing that two-step dance across a codebase, you start reaching for something cleaner.

The usual answer is a wrapper class or a custom Promise subclass. Both work, but both carry a hidden cost: you are now responsible for whatever happens when you swap out the native Response for your own abstraction. instanceof checks break. Framework integrations that inspect the response directly can behave unexpectedly. And the moment someone passes your custom object into something that expected a plain Response, you have a problem.

I wanted a different answer. This is about that.

The Goal

The cleaner call site I was after looks like this:

const todo = await client.get('/api/todos/1').json()

Two requirements in tension. On one hand, .json() should be reachable without a separate await and variable assignment. On the other hand, await client.get('/api/todos/1') should still resolve to a genuine, unmodified Response — not a wrapper, not a subclass, not a Proxy.

Most approaches collapse this tension by picking one side. Either you get ergonomics and lose native semantics, or you keep native semantics and write the two-liner. The question is whether you can actually have both.

The Mechanism

A Promise in JavaScript is an object. Like any object, you can assign properties to it at runtime.

That is the whole trick. Instead of wrapping the Promise or replacing it with something else, you decorate it in place: attach the convenience methods directly as properties on the Promise instance returned by the fetch call.

Here is what that looks like in practice:

function attachResponseShortcuts(promise: Promise<Response>) {
  const descriptor = (fn: (r: Response) => unknown) => ({
    value: function (this: Promise<Response>) {
      return this.then(fn)
    },
    enumerable: false,
    writable: false,
    configurable: false,
  })

  Object.defineProperties(promise, {
    json:        descriptor((r) => r.json()),
    text:        descriptor((r) => r.text()),
    blob:        descriptor((r) => r.blob()),
    arrayBuffer: descriptor((r) => r.arrayBuffer()),
    formData:    descriptor((r) => r.formData()),
  })

  return promise
}

A few things are happening here that are worth unpacking.

Property descriptors, not assignment. Using Object.defineProperties instead of promise.json = fn gives explicit control over the property attributes. Each method is enumerable: false which means it stays invisible to for...in loops, Object.keys, and JSON serialization. (They will still show up in browser DevTools when you expand the object and in Object.getOwnPropertyDescriptors(), but that is useful for debugging anyway — the point is they do not pollute standard iteration or JSON output.) It is writable: false and configurable: false, so it cannot be accidentally overwritten or deleted at runtime. This is intentional: without these locks, a careless reassignment (promise.json = myMock) would silently break the convenience layer for everyone holding that promise. The tradeoff is that it also prevents intentional overrides — if you need to mock .json() in a test, you cannot. This is a deliberate choice favoring safety over flexibility.

Forwarding, not reimplementing. Each method is a one-liner that calls .then() on the Promise itself (via this) and delegates immediately to the native Response method. The parsing behavior, error handling, and body consumption rules all come from the browser or runtime. We are not reimplementing anything. The methods are thin pass-throughs.

The Promise remains a Promise. await client.get('/api/todos/1') still resolves to the same native Response it always did. The added methods live on the instance itself, not on the prototype chain like native methods do — they are invisible properties on the Promise object. They do not affect the resolution value, the prototype chain, or any standard Promise behavior. (This is a meaningful difference: calls to promise.constructor or Object.getPrototypeOf(promise) see an untouched Promise, not a subclass or wrapper.)

Idempotency

If multiple plugins or hooks might touch the same promise — which is the case in a plugin-based architecture — you need to guard against decorating the same object twice. Object.defineProperties will throw if you try to redefine a non-configurable property.

A marker handles this, and this is one of the rare cases where a Symbol is genuinely useful: it provides collision-free identity that no other code can accidentally claim.

const DECORATED = Symbol('ffetch.responseShortcutsDecorated')

function attachResponseShortcuts(promise: Promise<Response>) {
  if ((promise as any)[DECORATED]) return promise

  // ... defineProperties ...

  Object.defineProperty(promise, DECORATED, { value: true, enumerable: false })
  return promise
}

The marker is invisible to Object.keys, Object.getOwnPropertyNames, and iteration — only findable via Object.getOwnPropertySymbols if you explicitly look for it. Decoration becomes a safe, idempotent operation regardless of call order, with zero risk of collision with any third-party code or browser internals.

Typing It

TypeScript does not know about properties you attach at runtime, so you have to tell it. The cleanest model here is an intersection type: the call site return type is Promise<Response> intersected with the shortcut interface.

interface ResponseShortcuts {
  json<T = unknown>(): Promise<T>
  text(): Promise<string>
  blob(): Promise<Blob>
  arrayBuffer(): Promise<ArrayBuffer>
  formData(): Promise<FormData>
}

type DecoratedPromise = Promise<Response> & ResponseShortcuts

This is honest. DecoratedPromise really is both things simultaneously: a standard Promise that resolves to Response, and an object that happens to have five extra methods. The intersection expresses both without hiding either.

When the library does not have the plugin installed, the return type is Promise<Response> with no extras. When it does, it is Promise<Response> & ResponseShortcuts. TypeScript catches you if you try to call .json() without the plugin, and it autocompletes when you have it. No runtime cost either way.

Tradeoffs Worth Naming

This technique is additive, not transformative. That is its strength and its limit.

It cannot change what Response contains. If you call .json() on a response that came back with a text/html body, you get the same parse error you would have got with the two-liner. The shortcut is a convenience, not a type-safe schema layer.

Body consumption rules are also unchanged. Response bodies can only be read once — calling .json() and then separately awaiting the response and calling .json() again will fail, exactly as native fetch would. Decoration does not change the underlying object.

The TypeScript types also do not capture body consumption state. If the response body was already read (e.g., by an upstream handler or middleware), calling .json() will throw at runtime. TypeScript will not catch this — the types express the structural shape of the methods, not the preconditions for their success. This is a general limitation of modeling Response state in TypeScript, not specific to this technique, but it is worth knowing: the intersection type Promise<Response> & ResponseShortcuts is a shape guarantee, not a behavioral one.

And if you or your team prefer strict explicitness — no augmented promise objects, all parsing explicit — then this pattern is probably not the right call. It is a style choice. The native two-liner is perfectly readable, just longer.

Where this genuinely shines is in a plugin or middleware architecture where you want to offer ergonomics as opt-in behavior. The baseline remains untouched, native fetch-compatible, and requires zero knowledge of the convenience layer to work with.

The Broader Point

What I find interesting about this technique is that it demonstrates a property of JavaScript that is easy to forget: objects are open. A Promise is not a sealed system. You can extend it in flight without wrapping or subclassing, and without disturbing the contract anyone else has with it.

Preserve native behavior first. Layer ergonomics second, explicitly, and as close to invisibly as possible.

If the shortcut is there and you use it, you gain a line. If the shortcut is there and you do not use it, nothing changes. That is the shape of a good opt-in.

The full implementation lives in ffetch if you want to see it in context. But the technique itself applies anywhere you need to decorate promises with convenience methods.

Introducing the Fetch Client Chaos Arena

Gabor Koos — Sun, 05 Apr 2026 13:40:02 +0000

There are many HTTP clients in the JavaScript ecosystem, and while they all solve similar problems, they can behave very differently under stress, retries, and failures. Picking the right one is not always straightforward.

Introducing ffetch-demo: a live browser arena for benchmarking JavaScript HTTP clients under controlled network chaos. The idea is simple: run the same request workload through different clients and compare how they behave when conditions get rough.

In the demo, you can configure chaos scenarios such as:

latency injection
random failures and drops
status-code spikes
retry pressure and timeout stress

Built With chaos-fetch

The chaos layer in the arena is powered by @fetchkit/chaos-fetch, which makes it easy to apply deterministic and randomized network stressors through middleware-style configuration.

npm: @fetchkit/chaos-fetch
GitHub: fetch-kit/chaos-fetch

Current clients in the arena:

native fetch
axios
ky
@fetchkit/ffetch

The output focuses on practical reliability signals (success/failure rates, error patterns, and latency distributions) so you can quickly see behavioral differences between clients.

Live demo: fetch-kit.github.io/ffetch-demo

GitHub: fetch-kit/ffetch-demo

Your Debounce Is Lying to You

Gabor Koos — Sat, 28 Mar 2026 06:52:27 +0000

Debounce is one of those patterns every frontend developer learns early and keeps using forever.

At its core, debouncing does one thing well: it coalesces a burst of calls into one invocation after a quiet window. That is a great fit for noisy UI signals.

Its most familiar use case is autocomplete, but the same pattern applies to resize handlers, scroll listeners, live validation, filter controls, and telemetry hooks.

A typical implementation looks like this:

function debounce(fn, delay) {
    let timer;
    return (...args) => {
        clearTimeout(timer);
        timer = setTimeout(() => fn(...args), delay);
    };
}

const search = debounce(async (q) => {
    const res = await fetch(`/api/search?q=${q}`);
    const data = await res.json();
    render(data);
}, 300);

It looks disciplined. It feels efficient. It ships fast.

And this is where the title comes from.

The issue is not debounce itself. The issue is this vanilla debounce + fetch pattern once real network behavior enters the picture.

It gives the feeling that requests are "under control," but it does not control request lifecycle: response ordering, cancellation of stale work, or failure behavior.

That is why it feels like debounce is "lying" in production: the UI looks smoothed, while the network layer is still fragile.

In this article, we will keep debounce for what it is good at (UI smoothing), then harden the request path with cancellation, retries, and better error handling.

The Illusion of "Fixed" Behavior

Debounce is convincing: you type quickly, the UI triggers fewer calls, and the network tab looks quieter. It feels like the system is now stable. But in production, under real network conditions, many things can go wrong. You will experience stale data, wasted requests, silent failures and other unexpected behaviors.

This is true for any network request, but debounce adds another layer of complexity: it makes the UI look smooth while the network can still be unpredictable. This mismatch can create a false sense of security.

Debounce itself only guarantees one thing:

"I won't call this function too often."

Debounce smooths input frequency, not request lifecycle. It does not guarantee:

Responses arrive in order.
Stale requests stop running.
Failures are handled consistently.

In other words: it is a UI pattern, not a network pattern. So you have to make sure the underlying network layer is robust enough to handle real-world conditions. Before we examine what can go wrong, let's set the stage!

The Companion Code

To make these problems visible, we have a companion demo app with a single text input where every keystroke triggers a debounced request to /api/echo?q=<input>. The backend is an Express server that returns { query, timestamp }, and the frontend appends each response to a div as query@timestamp. The stack is minimal: Node.js + Express on the backend, and plain HTML/CSS/JavaScript in the browser.

Clone the repo and install dependencies:

git clone https://github.com/gkoos/article-debouncing.git
cd article-debouncing
npm install

Then start the app:

npm start

And navigate to http://localhost:3000 in your browser. You will see something like this:

Now, as you type in the input field, you will see the responses coming back in the list below:

The UI also shows toast notifications for request success and failure, which will become relevant in later sections.

This is our baseline setup, it demonstrates the basic pattern. There is a 300ms debounce on the input, and the backend immediately responds with the query and a timestamp. The UI appends each response to the list.

Problem 1: Race Conditions (aka Stale UI)

On your local machine, everything is fast and smooth. But in production, network conditions are unpredictable. Requests can take varying amounts of time to complete, so there is no guarantee that responses will arrive in the same order they were sent. Let's see what happens if we add random delays to the server response to simulate real network conditions.

Check out the 01-stale-requests branch and restart the server:

git checkout 01-stale-requests
npm start

We added a middleware that introduces a random delay of 0–1000ms for each request. Now, when you type quickly, you might see responses arriving out of order:

We typed 12345678, but the UI shows 1234567! The response for 7 came back after 8, so the UI is now stale. This is a classic race condition, and debounce itself does not prevent it. The UI is showing results for an older query, which can lead to confusion and errors in a real application.

How to fix this? We need to ensure that only the latest request's response is processed, and any previous requests are either cancelled or ignored. We could implement a simple version of this by keeping track of the latest query and ignoring responses that don't match it. But that would still allow all requests to run, which is inefficient. A better approach is to use the AbortController API to cancel stale requests, so they don't consume resources or trigger side effects when they complete.

AbortController is a browser-native API. You create a controller, pass its signal to fetch, and call abort() whenever you want to cancel the request. The fetch will throw an AbortError, which you can catch and ignore since it's expected.

Here is the updated debounce callback with cancellation:

let controller;

const debouncedFetch = debounce(async (q) => {
  if (!q) return;

  if (controller) controller.abort();
  controller = new AbortController();

  try {
    const response = await fetch(`/api/echo?q=${encodeURIComponent(q)}`, {
      signal: controller.signal
    });
    const data = await response.json();
    // render data...
  } catch (err) {
    if (err.name === 'AbortError') return;
    // handle real errors...
  }
}, 300);

Two things changed:

Before each request, we abort any in-flight request from the previous call and create a fresh controller.
After catching an error, we check if it's an AbortError and return early: these are expected and not real failures.

The result: in normal flow, only the last request in a typing burst ever completes. Previous ones are cancelled at the network level, not just ignored after the fact. (For absolute safety, you can also guard the render step with a request ID check. This covers the tiny edge window where a response resolves right before a newer request aborts the previous one.)

Problem 2: Network Failures

The network is not only unpredictable in terms of latency, but also in terms of reliability. Sometimes a request can fail that would have succeeded if retried. This can be due to transient server issues like network congestion, temporary spikes in load, or database timeouts. If we want a more robust user experience, we need to handle these failures gracefully.

Let's simulate random failures in our backend. Check out the 02-failures branch and restart the server:

git checkout 02-failures
npm start

This version of the server adds a random failure mechanism: each request has a 40% chance to fail with a 500 error. Now this may be too aggressive, but it will help us see the problem clearly. When you type in the input field, you will see some requests fail:

Well, that's not good. First, our UI shows undefined@undefined when a request fails. That happens because the server returns { error: 'Internal Server Error' } on a 500, so data.query and data.timestamp are both undefined. Vanilla fetch doesn't throw on HTTP error status codes: it only rejects on network failures. So we need to check response.ok ourselves:

const response = await fetch(`/api/echo?q=${encodeURIComponent(q)}`, {
  signal: controller.signal
});
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const data = await response.json();

Now a 500 throws before we ever touch the body, the catch handles it, and the error toast shows instead of broken output.

But that's just the start. In a real app, you would want to implement some retry logic for transient failures. For example, you could automatically retry a failed request up to 3 times with exponential backoff. This way, if a request fails due to a temporary issue, it has a chance to succeed without the user having to do anything.

To implement this manually you'd need to write retry loops, track attempt counts, implement backoff timing, and make sure none of it fires after a cancellation. That's not trivial, and it's not the interesting part of your app, so let's use a library for that.

@fetchkit/ffetch is a thin wrapper around fetch that handles exactly this. We'll use it for the retry behavior in our demo.

There are good alternatives in this space (for example ky, axios, or a custom wrapper). I chose ffetch here because it keeps a fetch-compatible API surface and handles abort-aware retries cleanly.

Since this is a minimal demo with no build step, we load it directly from a CDN rather than installing it. In a real project you'd npm install @fetchkit/ffetch and import it normally, but here a single import line is enough:

import { createClient } from 'https://esm.sh/@fetchkit/ffetch';

const api = createClient({
  retries: 3, // retry up to 3 times on failure
  shouldRetry: (ctx) => ctx.response?.status >= 500, // only retry on 5xx errors
});

api has the exact same call signature as fetch: same arguments, same return type. You drop it in as a replacement:

const response = await api(`/api/echo?q=${encodeURIComponent(q)}`, {
  signal: controller.signal
});

What this buys us in this specific scenario:

retries: 3 — if the server returns a 500, ffetch retries up to 3 more times before giving up
shouldRetry — we only retry on 5xx; anything else (network error, abort) propagates immediately
abort-aware backoff — if controller.abort() fires during the delay between retries, the wait exits immediately and the abort propagates; no stale work keeps running in the background

That last point is quite convenient here. Without it, aborting a request mid-retry would cut the active fetch but leave the backoff timer running, which would then fire another fetch attempt that instantly aborts. Now we handle this correctly.

There's one more thing we can clean up. Because the native fetch does not throw on HTTP error status codes (one of the pain points of the API), we had to add a manual check:

if (!response.ok) throw new Error(`HTTP ${response.status}`);

ffetch can handle this too. With the throwOnHttpError: true config option, any HTTP error response throws automatically, no manual check needed.

const api = createClient({
  retries: 3,
  shouldRetry: (ctx) => ctx.response?.status >= 500,
  throwOnHttpError: true,
});

Now the fetch call is just:

const response = await api(`/api/echo?q=${encodeURIComponent(q)}`, {
  signal: controller.signal
});
const data = await response.json();

The catch block still handles everything - HTTP errors, network failures, real errors - without any extra branching in the happy path.

The final implementation can be found in the 03-fixed branch:

git checkout 03-fixed
npm start

For reference, the full code for the debounce callback with ffetch can be found here.

Conclusion

Debounce is not the problem. The problem is treating it as a complete solution for network control when it only handles one dimension of it: call frequency. It is a very useful pattern for smoothing out noisy UI signals, but it does not handle the complexities of real network behavior. To build a robust application, you need to complement debounce with proper request lifecycle management: cancellation of stale requests, retries with backoff for transient failures, and consistent error handling. This way, you can ensure that your UI remains not only responsive but also accurate even under unpredictable network conditions.

Developing and Benchmarking the Same Feature in Node and Go

Gabor Koos — Thu, 19 Mar 2026 19:43:56 +0000

When I started building chaos-proxy, the initial goal was simple: make API chaos testing practical for JavaScript and TypeScript teams. I wanted something that could sit between an app and its upstream API and introduce realistic turbulence on demand: latency spikes, intermittent failures, and other behavior that makes integration tests feel closer to production.

Node.js was the obvious first runtime for that because the ecosystem, tooling, and middleware ergonomics are excellent for rapid iteration. It is hard to overstate how productive that setup is when the main audience is already living in npm, TypeScript, and JavaScript test runners.

Later, I rewrote the same proxy in Go to push raw proxy performance further and support higher throughput under load. The intent was not to replace one with the other philosophically, but to explore a different optimization frontier with the same product idea.

This post documents what happened when I implemented the same non-trivial feature in both runtimes: hot config reload. Then I reran the benchmark from my previous article to see how the newer versions compare.

The interesting part is not only the final numbers. It is also how two mature runtimes guide you toward different internal designs, even when you are enforcing the same external behavior contract.

Old benchmark post:
https://blog.gaborkoos.com/posts/2025-10-11-Nodejs-vs-Go-in_Practice-Performance-Comparison-of-chaos-proxy-And-chaos-proxy-go/

Repositories:

Node implementation: https://github.com/fetch-kit/chaos-proxy
Go implementation: https://github.com/fetch-kit/chaos-proxy-go

Implementing Hot Config Reload in Two Runtimes

The goal of hot config reload was to allow users to update the proxy's behavior without downtime. This means that when a new config is posted to the /reload endpoint, the proxy should parse, validate, and apply the new configuration atomically, without interrupting in-flight requests. This enables advanced testing scenarios where you can change the chaos behavior on the fly to model dynamic production conditions like feature rollouts, traffic shifts, or evolving failure modes.

Both implementations follow the same external contract:

POST /reload accepts a full config snapshot
Parse -> validate -> build -> swap, all-or-nothing
Deterministic in-flight behavior (request-start snapshot semantics)
Reject concurrent reload requests
Consistent status model (400, 409, 415, success returns version and reload duration)

So the user-facing behavior is aligned. Clients see the same API and guarantees. The internal shape is where Node and Go felt very different.

Runtime model

Node leaned toward a dynamic runtime object: rebuild middleware/router chain, then swap the active runtime. That style maps naturally to the way Node applications are often composed. Rebuilds are straightforward to express, and the overall control flow stays compact.

Go leaned toward immutable runtime snapshots: config + router + version behind an atomic pointer. In practice, this makes the runtime feel more explicit. You can point to exactly what a request observed and exactly when a new version became active.

Concurrency model

In Node, most complexity is around making reload writes serialized and safe while requests continue flowing.

In Go, the read/write split is explicit: request path loads one snapshot at request start, reload path builds fresh state under lock, then atomically swaps.

Behaviorally both approaches are equivalent from a user perspective. The difference is mostly in how obvious the invariants are when you revisit the code weeks later.

In-flight guarantees

Both versions guarantee request-start snapshot semantics.

In Node, this is easier to accidentally violate if mutable shared state leaks into request handling.

In Go, the pointer-load-at-entry pattern makes this guarantee structurally harder to violate.

That was one of the strongest practical contrasts for me: same requirement, different default safety profile.

Router lifecycle and rebuild mechanics

Node composition is lightweight and ergonomic for rebuilds.

Go rebuilds a fresh router and re-registers middleware/routes on each reload. Behavior is explicit and predictable at the snapshot level, with middleware execution order deterministic only when config uses ordered list elements (not multiple keys in one map). It can look verbose at first, but this explicitness pays off when debugging edge cases around reload timing.

Validation and rollback boundaries

Both use the same pipeline: parse -> validate -> build -> swap.

Node gives more dynamic flexibility but needs stricter guard discipline.

Go's type-driven pipeline made failure paths and rollback behavior cleaner to reason about.

In both runtimes, treating build and swap as separate phases was the key to keeping rollback semantics simple.

Stateful middleware behavior

Both implementations rebuild middleware instances on reload. That means in-memory middleware state (for example counters or local token buckets) resets by design after a successful reload. This is intentional and worth calling out to users because it is product behavior, not an implementation accident.

Benchmark Rerun

After adding hot config reload support, I reran the old benchmark setup.

The goal here was not to produce an absolute, universal number for every environment. The goal was to keep methodology stable enough to compare the old and new versions and see whether the relative shape changed.

System and Test Environment (Same Machine as the Old Article)

This rerun was executed on the same machine as the benchmark in the previous article, with the same local topology (Caddy backend on localhost, proxy on localhost, load generated by hey on the same host).

Machine characteristics:

CPU: AMD Ryzen 7 5800H with Radeon Graphics
Cores/Threads: 8 cores / 16 threads
Base clock: 3.2 GHz
RAM: 16 GB DDR4
OS: Windows 10 Home 22H2 64-bit

Benchmark setup characteristics:

Backend: Caddy serving /api/hello on localhost:8080
Proxy target: localhost:5000
Load generator: hey
Command pattern: hey -n 1000 -c 50 http://localhost:/api/hello
Runs per scenario: 3 (median reported)

Reproducibility command block (same pattern used for this article):

# 1) Start Caddy backend
./caddy.exe run --config Caddyfile

# 2) Baseline (direct Caddy)
for i in 1 2 3; do ./hey -n 1000 -c 50 http://localhost:8080/api/hello | tee -a baseline-caddy-runs.txt; done

# 3) Node proxy benchmark (in another terminal, start proxy first)
npx chaos-proxy --config chaos.yaml
for i in 1 2 3; do ./hey -n 1000 -c 50 http://localhost:5000/api/hello | tee -a node-3.0.1-runs.txt; done

# Stop the Node proxy process before running the Go proxy benchmark (both use port 5000)

# 4) Go proxy benchmark (in another terminal, start proxy first)
./chaos-proxy-go.exe --config chaos.yaml
for i in 1 2 3; do ./hey -n 1000 -c 50 http://localhost:5000/api/hello | tee -a go-0.2.1-runs.txt; done

Versions in this rerun:

chaos-proxy (Node): 3.0.1
chaos-proxy-go (Go): 0.2.1

I also verified response-size parity for fairness:

Caddy: 94 bytes/request
Node 3.0.1: 94 bytes/request
Go 0.2.1: 94 bytes/request

This check mattered because an earlier Node run returned compacted JSON (smaller payload), which could bias throughput. The final numbers below use matched response sizes.

Current Rerun (Median of 3)

Scenario	Requests/sec	Avg Latency (s)	P99 Latency (s)
Direct Caddy	24,912.1845	0.0018	0.0156
chaos-proxy Node 3.0.1	3,788.0065	0.0129	0.0318
chaos-proxy-go 0.2.1	7,286.8293	0.0062	0.0248

Old Benchmark Reference (from previous post)

Scenario	Requests/sec	Avg Latency (s)	P99 Latency (s)
Direct Caddy	28,383.8519	0.0016	0.0116
chaos-proxy Node 2.0.0	4,262.3420	0.0115	0.0417
chaos-proxy-go 0.0.5	8,828.0577	0.0053	0.0140

What changed?

1) Go vs Node in current versions

Go is still clearly ahead.
Throughput: Go is about 1.92x higher than Node (7286.8 vs 3788.0 req/sec).
Average latency: Node is about 2.08x slower than Go (0.0129s vs 0.0062s).

2) Go old vs Go new

Throughput decreased from 8828.1 to 7286.8 req/sec (~17.5% lower).
Average latency increased from 0.0053s to 0.0062s (~17.0% higher).
P99 increased from 0.0140s to 0.0248s.

3) Node old vs Node new

Throughput decreased from 4262.3 to 3788.0 req/sec (~11.1% lower).
Average latency increased from 0.0115s to 0.0129s (~12.2% higher).
P99 improved from 0.0417s to 0.0318s.

Adding hot-reload-safe runtime mechanics introduces measurable overhead even in steady-state forwarding paths, which is why both implementations are slower than their previous versions in this benchmark shape.

I did not trigger reloads during benchmark traffic, so this should be interpreted as structural overhead from the runtime architecture needed to guarantee safe reload semantics, not reload execution cost itself.

Why There Is Overhead Even Without Calling /reload

Even if reload is never triggered during the benchmark request stream, the hot reload feature still changes the steady-state architecture:

Requests now run through runtime indirection designed for safe snapshot semantics.
Runtime objects and routing/middleware composition are organized around swap-ready boundaries.
Concurrency guards and state-boundary discipline are now part of the normal request path design.

In other words, the cost is not from running /reload repeatedly during the test. The cost comes from maintaining reload-safe invariants all the time.

Conclusion

Implementing the same feature in Node and Go was one of the most useful engineering exercises I have done in a while.

The final behavior contract can be identical across runtimes, but the implementation pressure points are very different:

Node emphasizes dynamic composition and careful mutation control.
Go emphasizes snapshot immutability and explicit concurrency boundaries.

Performance-wise, the high-level outcome still holds: the Go proxy remains roughly 2x faster than the Node proxy in this benchmark shape. At the same time, both implementations are now better specified in terms of live reconfiguration semantics, which was the actual feature goal. The implementations are likely not fully performance-tuned yet. For now, that trade-off is acceptable for the feature guarantees we wanted.

And yes, it was genuinely fun to build.

From the Database Zoo to the Database Safari

Gabor Koos — Tue, 10 Mar 2026 01:46:38 +0000

Over the past year I've been writing a series called The Database Zoo, exploring the growing ecosystem of modern databases. The idea behind the series was simple: instead of treating "the database" as a single category, look at the different species that exist today - probabilistic databases, time-series systems, vector databases, and more - and understand why they were built and what problems they solve.

While working on the series, it became clear that the topic deserved a more structured and expanded treatment.

That work eventually turned into a book.

I'm currently writing The Database Safari, to be published by Apress (Springer Nature). The book grows out of the ideas in the Database Zoo series, but develops them into a more cohesive guide to specialized databases: how they work internally, what trade-offs they make, and when they make sense in real systems.

The book is already listed on SpringerLink:

https://link.springer.com/book/9798868827082

I'll share more updates as the writing progresses.

Using Pagination to Improve GraphQL Performance

Gabor Koos — Thu, 19 Feb 2026 22:42:13 +0000

GraphQL makes it easy to request exactly the data you need, but that flexibility can quickly turn into a performance problem when queries return large result sets. A single field that returns "all items" may work fine during development, yet silently degrade into slow responses, high memory usage, or even process crashes as data volume grows.

This is especially relevant in Node.js backends, where resolvers often materialize entire result sets in memory before returning a response. Fetching a large number of records in a single GraphQL query doesn't just increase response time, it can put sustained pressure on the event loop, garbage collector, and overall process stability.

Pagination is the standard solution to this problem, but not all pagination strategies behave the same under load.

In this article, we'll look at three common approaches to pagination in a Node.js GraphQL API: fetching everything at once, offset-based pagination, and cursor-based pagination. Rather than treating pagination as a purely theoretical concern, we'll instrument each approach and observe how it affects response times and memory usage.

We'll build a minimal GraphQL API using Express and Apollo Server, backed by a SQLite database seeded with 500,000 products. You'll see how naive queries show up as slow requests and memory spikes, how offset-based pagination improves things but still has hidden costs, and why cursor-based pagination is - spoiler alert! - the recommended pattern for stable, scalable GraphQL APIs.

Setting Up the Project

To keep things simple, the article is accompanied by a runnable demo repository. The project is a small Node.js GraphQL API, with a SQLite backend populated with 500,000 products.

Prerequisites

To follow along, you'll only need:

Node.js 18+
npm
A basic understanding of GraphQL and Node.js development

Installation

Start by cloning the repository and installing dependencies:

git clone https://github.com/gkoos/article-graphql-pagination
cd article-graphql-pagination
npm install

The project uses Prisma with SQLite and includes a seed script that creates 500,000 product records to demonstrate performance differences between pagination strategies.

Run the following commands to initialize and seed the database:

npx prisma generate
npm run prisma:migrate
npm run prisma:seed

This will create the database schema and populate it with realistic-looking product data.

Running the Server

Once setup is complete, start the server:

npm start

The GraphQL API will be available at http://localhost:4000, where you can explore the schema and run queries using the Apollo Sandbox.

The Problem: Fetching All Data at Once

Before introducing pagination, let's start with the simplest approach: returning all records from a GraphQL field in a single request.

In our app, the allProducts query does exactly that. It loads all 500,000 products from the database and returns them as a single response. This kind of query is easy to write, easy to understand, and surprisingly common in early GraphQL schemas.

Here's the resolver behind it:

// src/resolvers.js
...
allProducts: async () => {
    console.time('allProducts');
    const products = await prisma.product.findMany({
    orderBy: { id: 'asc' }
    });
    console.timeEnd('allProducts');
    console.log(`Fetched ${products.length} products (ALL)`);
    return products;
},

There's nothing technically wrong with this resolver. It does exactly what it promises. The problem is scale.

Running the Fetch-All Query

Open the GraphQL Sandbox at http://localhost:4000 and run the following query:

query allProducts {
  allProducts {
    id
    name
    price
    category
  }
}

Depending on your machine, the query may take several seconds to complete. You'll also notice that the response payload is very large: hundreds of thousands of objects serialized into JSON and sent over the wire in one go.

In your terminal, you should see a log message like this:

allProducts: 2.817s
Fetched 500000 products (ALL)

A single request:

Executes a large database query
Allocates memory for all 500,000 rows
Serializes the entire result set before responding

In a real production API, this kind of request can quickly become problematic under concurrent load.

Why This Pattern Breaks Down

Even in this local setup, you can observe:

High response times (often several seconds)
Significant memory usage spikes during request processing

Because the resolver loads the entire dataset eagerly, the cost of this query scales linearly with the number of rows in the table. As the dataset grows, so does response time, memory pressure, and GC activity in the Node.js process.

This is one of those things that often goes unnoticed during development, but becomes very visible once real data and real traffic hit the system.

Fetching all data at once has a few fundamental problems:

Unbounded results: There's no upper limit on how much data a client can request.
Poor memory characteristics: Large result sets must be held in memory until the response is sent.
Unpredictable performance: Response time grows with dataset size, not request intent.
Easy to abuse: A single client can unintentionally (or intentionally) stress the backend.

Pagination exists to put boundaries around this behavior.

Naive Offset-Based Pagination

A natural first step after realizing that fetching everything at once doesn't scale is to introduce offset-based pagination. This approach limits the number of records returned per request and allows clients to "page through" results using a combination of limit and offset.

Offset-based pagination is simple to implement and easy to reason about, which makes it a common choice in REST APIs and an equally common first attempt in GraphQL.

Implementing Offset Pagination

In our demo project, the productsOffset query exposes this pattern:

// src/resolvers.js
...
productsOffset: async (_, { limit, offset }) => {
    console.time('productsOffset');
    if (limit > 100) {
        limit = 100; // enforce a maximum limit to prevent abuse
    }
    const products = await prisma.product.findMany({
    take: limit,
    skip: offset,
    orderBy: { id: 'asc' }
    });
    console.timeEnd('productsOffset');
    console.log(`Fetched ${products.length} products (offset: ${offset}, limit: ${limit})`);
    return products;
},

One thing that's important to note here is if we don't limit the number of records returned, we could still end up fetching everything at once. Always implement a server-side maximum limit to prevent abuse.

The resolver uses Prisma's take and skip options to implement limit and offset behavior. Clients can specify how many records they want (limit) and where to start (offset).

The corresponding GraphQL query looks like this:

query productsOffset {
  productsOffset(limit: 20, offset: 0) {
    id
    name
    price
    category
  }
}

Instead of returning all 500,000 products, this query fetches just a small window of results. Clients can request subsequent pages by increasing the offset value.

Observing the Improvement

Run the offset-based query a few times from the GraphQL Sandbox, changing the offset to simulate paging through the dataset.

In your terminal, you should see logs like this:

productsOffset: 17.44ms
Fetched 20 products (offset: 0, limit: 20)

Compared to the fetch-all approach, you should immediately notice:

Much faster response times
Shorter database query time
Lower overall memory usage per request

By limiting how many records are loaded and serialized, offset-based pagination dramatically reduces the per-request cost. Even under load, this approach is far more stable than returning everything at once.

The Hidden Cost of Offsets

While offset-based pagination is a clear improvement, it comes with a less obvious downside.

As the offset value increases, the database still needs to scan past the skipped rows to reach the requested page. For small offsets this isn't a problem, but deeper pages can become increasingly expensive, especially on large tables.

Let's query the last page of products:

query productsOffset {
  productsOffset(limit: 20, offset: 499980) {
    id
    name
    price
    category
  }
}

Run this query and observe the terminal logs:

productsOffset: 1.055s
Fetched 20 products (offset: 499980, limit: 20)

In this particular case, the first query took 17ms, while the last page took more than a second!

From the client's perspective, this query looks almost identical to fetching the first page, but from the database's perspective, it may involve scanning hundreds of thousands of rows before returning just 20.

Why This Matters in GraphQL APIs

Offset-based pagination also has semantic issues in GraphQL:

Unstable pagination: Inserts or deletes can shift offsets, causing clients to skip or duplicate items.
No natural continuation: Clients must manage offsets manually.
Poor fit for infinite scrolling: Large offsets become increasingly inefficient.

These limitations are why offset-based pagination is generally considered a transitional solution in GraphQL APIs.

Cursor-Based Pagination

Offset-based pagination improves performance by limiting result size, but it still becomes less efficient as clients paginate deeper into a dataset. In GraphQL APIs, the recommended alternative is cursor-based pagination, where each page starts from a known position instead of skipping an arbitrary number of rows.

Cursor-based pagination is a better fit for large datasets because its performance depends on page size, not page number.

Implementing Cursor-Based Pagination

In this project, cursor-based pagination is implemented using Prisma’s native cursor support. Each product's id is encoded into an opaque cursor, which the client passes back when requesting the next page.

At a high level, the resolver:

Decodes the after cursor (if present)
Uses it as a database cursor
Fetches first + 1 records to determine if another page exists
Builds a connection-style response with edges and pageInfo

Here is the resolver implementation:

// src/resolvers.js
// Helper function to encode cursor
function encodeCursor(id) {
  return Buffer.from(id.toString()).toString('base64');
}

// Helper function to decode cursor
function decodeCursor(cursor) {
  return parseInt(Buffer.from(cursor, 'base64').toString('ascii'));
}

...

productsCursor: async (_, { first = 20, after }) => {
    console.time('productsCursor');

    const cursor = after ? { id: decodeCursor(after) } : undefined;

    // Fetch one extra to determine if there's a next page
    const products = await prisma.product.findMany({
    take: first + 1,
    ...(cursor && {
        skip: 1, // Skip the cursor itself
        cursor: cursor
    }),
    orderBy: { id: 'asc' }
    });

    const hasNextPage = products.length > first;
    const edges = products.slice(0, first).map(product => ({
    cursor: encodeCursor(product.id),
    node: product
    }));

    const pageInfo = {
    hasNextPage,
    hasPreviousPage: !!after,
    startCursor: edges.length > 0 ? edges[0].cursor : null,
    endCursor: edges.length > 0 ? edges[edges.length - 1].cursor : null
    };

    const totalCount = await prisma.product.count();

    console.timeEnd('productsCursor');
    console.log(`Fetched ${edges.length} products (cursor-based, after: ${after || 'start'})`);

    return {
    edges,
    pageInfo,
    totalCount
    };
},

This approach ensures that each query resumes from a precise position in the dataset rather than scanning past thousands of rows.

Querying with Cursors

To fetch the first page of products:

query cursorProductsFirst {
  productsCursor(first: 20) {
    edges {
      cursor
      node {
        id
        name
        price
        category
      }
    }
    pageInfo {
      hasNextPage
      hasPreviousPage
      startCursor
      endCursor
    }
    totalCount
  }
}

In the terminal, you'll see something like:

productsCursor: 39.993ms
Fetched 20 products (cursor-based, after: start)

And the response will be:

{
  "data": {
    "productsCursor": {
      "edges": [
        {
          "cursor": "MQ==",
          "node": {
            "id": 1,
            "name": "Product 1",
            "price": 581.7240166646505,
            "category": "Clothing"
          }
        },
        ...
        {
          "cursor": "MjA=",
          "node": {
            "id": 20,
            "name": "Product 20",
            "price": 979.7302196981608,
            "category": "Sports"
          }
        }
      ],
      "pageInfo": {
        "hasNextPage": true,
        "hasPreviousPage": false,
        "startCursor": "MQ==",
        "endCursor": "MjA="
      },
      "totalCount": 500000
    }
  }
}

The format is slightly different from our offset-based implementation, but all 20 products are returned as expected, plus some useful pagination metadata. To fetch the next page, the client simply uses the endCursor from the previous response:

query cursorProductsNext {
  productsCursor(first: 20, after: "MjA=") {
    edges {
      cursor
      node {
        id
        name
        price
        category
      }
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

And the response will contain products 21-40:

{
  "data": {
    "productsCursor": {
      "edges": [
        {
          "cursor": "MjE=",
          "node": {
            "id": 21,
            "name": "Product 21",
            "price": 194.5758511706771,
            "category": "Toys"
          }
        },
        ...
        {
          "cursor": "NDA=",
          "node": {
            "id": 40,
            "name": "Product 40",
            "price": 527.7330156641641,
            "category": "Electronics"
          }
        }
      ],
      "pageInfo": {
        "hasNextPage": true,
        "hasPreviousPage": true,
        "startCursor": "MjE=",
        "endCursor": "NDA="
      },
      "totalCount": 500000
    }
  }
}

And for the next page, we would use the new endCursor value of "NDA=" and so on.

The cursor itself is opaque to the client and should be treated as an implementation detail. If the client can "guess" cursor values, it may lead to unintended behavior.

Now let's try to fetch the last page using the cursor! To do this on the client-side, we should keep following the endCursor values until we reach the end. However, for demonstration purposes, we will cheat a little and directly encode the 499980th product's ID and create a cursor for it. In resolvers.js, the encodeCursor() function does this. What we need is Buffer.from("499980").toString("base64"), which results in NDk5OTgw, therefore our query to fetch the last page looks like this:

query cursorProductsNext {
  productsCursor(first: 20, after: "NDk5OTgw") {
    edges {
      cursor
      node {
        id
        name
        price
        category
      }
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

Check your terminal logs again:

productsCursor: 35.197ms
Fetched 20 products (cursor-based, after: NDk5OTgw)

As you can see, the response times remain consistent regardless of how deep we paginate into the dataset!

Compared to offset-based pagination, you should observe:

Consistent database execution time, even for later pages
Uniform request duration across pages
Stable memory usage per request

Because each query starts from a known position, the database does not need to scan past large numbers of rows.

Why Cursor-Based Pagination Scales Better

Cursor-based pagination avoids the main pitfalls of offset-based pagination:

Performance does not degrade as clients paginate deeper
Pagination remains stable when records are inserted or deleted
Works naturally with infinite scrolling or stream-like UIs
Produces predictable, easy-to-compare timings/measurements in observability tools

Although cursor-based pagination requires slightly more setup than offset-based pagination, it provides far more reliable performance characteristics and is the preferred pattern for production GraphQL APIs.

Conclusion

Pagination is often treated as a schema design detail in GraphQL, but as shown earlier, it has a direct and measurable impact on performance, memory usage, and system stability.

Fetching all data at once may be convenient, but it quickly becomes a liability as datasets grow. Offset-based pagination improves the situation by limiting result size, yet still introduces hidden costs that surface as users paginate deeper. Cursor-based pagination, on the other hand, provides consistent performance characteristics regardless of dataset size, making it the most reliable choice for production GraphQL APIs.

More importantly, this article highlights the value of observability-driven decisions. Without instrumentation, all three approaches can appear to "work". But with proper profiling in place, the differences become clear, allowing you to make informed choices about how to design your API for real-world usage patterns.

If you're building or maintaining a GraphQL API in Node.js, cursor-based pagination should be your default (unless your dataset is small and unlikely to grow). And whatever approach you choose, instrument it early. Pagination is not just about shaping responses: it's about shaping how your system behaves under real-world load.

Lessons Learned from Running a Privacy-First Disposable Email Service: Insights from nullmail.cc

Gabor Koos — Wed, 18 Feb 2026 22:00:21 +0000

A few days ago, I got an unexpected email from Cloudflare: my domain, maildock.store, had stopped using their nameservers and was at risk of being deleted. This was unexpected, as I hadn't made any changes to the DNS settings. After some investigation, I discovered that the domain had been flagged for abuse, likely due to its association with disposable email services.

For context, maildock.store powers nullmail.cc, a privacy-first disposable email service. Users can create addresses, receive emails, and have them automatically deleted after expiration - all without signing up, tracking, or sending any outgoing mail. The frontend is a SvelteKit app on Vercel, emails are forwarded via forwardemail.net into a Supabase database, and the system automatically cleans up expired content. It's minimal by design, but robust enough to provide a fully functional disposable inbox - mostly in free tiers of various services.

Despite this simplicity, domains like maildock.store can trigger automated abuse flags. What followed was a whirlwind of DNS checks, WHOIS lookups, blacklist verification, and conversations with the .store registry, Radix. In this post, I'll walk through the story, how (I think) I resolved the issue, and the architectural choices that made it possible to recover safely, while keeping the service privacy-first.

Design Philosophy

At its core, Nullmail is built around privacy, simplicity, and minimalism. The goal is straightforward: users should be able to receive emails without giving away personal information, signing up for accounts, or being tracked. There's no analytics, no logging beyond what's necessary to deliver emails, and no outgoing SMTP - the service is strictly receive-only.

Every design choice reflects this philosophy. Addresses are ephemeral and automatically expire, keeping the system lean and reducing the risk of abuse. The database only stores what's necessary: the address itself and the emails sent to it. No unnecessary metadata, no IP logs, no behavioral tracking. Even the UI is stripped down to essentials, giving users just enough functionality to check their inbox and manage addresses.

This minimal, privacy-first approach has a practical benefit as well: it reduces the attack surface and limits what can go wrong. There's no complicated backend for sending mail, no rate-limiting infrastructure, and no analytics that could trigger false positives with anti-spam systems.

Architecture Overview

The simplicity of Nullmail's design is mirrored in its architecture. The service combines a few lightweight, well-chosen components to deliver a fully functional disposable email system while staying mostly in free tiers:

Frontend: A SvelteKit app hosted on Vercel. It handles the user interface for reading emails and managing addresses, with minimal JavaScript and no tracking.
Backend/API: The same SvelteKit app provides serverless API routes on Vercel for core operations: creating new addresses, listing inboxes, fetching email bodies, and extending expiry timestamps.
Database: A Supabase Postgres instance stores addresses and emails. The database schema is minimal, consisting of:
- addresses table: stores the address and its expiry timestamp.
- emails table: stores sender, recipient, subject, body, and delivery timestamp. Each email references an address, and expired addresses (and their emails) are automatically deleted via a scheduled cron job every five minutes.
Email ingestion: All incoming emails are routed through forwardemail.net. ForwardEmail posts inbound mail to a Vercel API endpoint, which inserts it into the Supabase database.
Domains & DNS: The service operates under the domains maildock.store and nullmail.cc. DNS is managed through Cloudflare, which provides:
- Nameserver hosting
- MX, SPF, DMARC, TLSRPT, and _security TXT records
- Proxying for web traffic (though Nullmail is mostly static)
- Protection against accidental misconfiguration or abuse flags
Optional UX: Browser extensions for Chrome and Firefox open the site with a fromExtension=1 flag for minor interface tweaks.

The codebase can be found on GitHub: gkoos/nullmail.

This architecture is lightweight but resilient, perfectly aligned with the privacy-first philosophy: no outgoing SMTP, minimal logging, and automated cleanup. It also meant that when the domain was flagged by Radix, most of the infrastructure was unaffected: the problem was isolated to DNS and registry-level issues.

The Incident

Even with a minimal, receive-only setup, disposable email domains can attract attention from automated abuse systems. As I mentioned earlier, I got an email from Cloudflare warning that maildock.store had stopped using their nameservers and was at risk of being deleted. At first, it felt like a false alarm — I hadn't touched any DNS settings.

Digging deeper, I discovered that the domain had been placed on ServerHold by Radix, the registry that manages .store domains. ServerHold is usually reserved for domains flagged for abuse, spam, or other policy violations. In my case, the likely trigger was the domain's association with my disposable email service, even though Nullmail is strictly receive-only and doesn't send outbound mail.

To investigate, I ran WHOIS checks, TXT and MX lookups, and verified DNS settings through Cloudflare. I also checked common spam/blacklist sources like SURBL: initially, the domain appeared flagged, though later it was cleared. While the frontend and database remained fully functional, the suspension meant that any new email delivery could fail and the domain risked being removed entirely.

This incident highlighted the harsh reality: even a minimal, privacy-first service with just a few users can run into registry-level issues. Fortunately, the architecture - isolated frontend, serverless backend, and Cloudflare-managed DNS - meant that the problem was largely contained to the domain itself, and recovery would be possible with the right steps.

The Resolution

Once I confirmed that maildock.store was on ServerHold, the next step was to contact Radix directly via their unsuspension form. I explained the service, emphasized it's receive-only nature, and detailed the privacy-first safeguards in place.

Radix responded positively and removed the ServerHold, reinstating the domain. To further prevent future issues and provide a point of contact for abuse reports, I created a dedicated abuse mailbox (abuse@maildock.store). This mailbox is stored in the Supabase database, has no public access, and is checked only via the Supabase UI as needed.

Next, I verified and cleaned up the DNS records in Cloudflare:

Removed old registrar NS entries that were no longer needed.
Deleted test or placeholder A records that could confuse DNS checks.
Confirmed MX records pointing to forwardemail.net were correct.
Ensured SPF, DMARC, TLSRPT, and _security TXT records were properly configured, with the abuse mailbox as the contact.

After these steps, the domain was fully operational again: emails were being received reliably, and the system continued to enforce automatic cleanup of expired content.

This experience reinforced the importance of clear abuse contact channels, proper DNS hygiene, and documenting a simple, minimal architecture that isolates potential issues. Even a small, receive-only service can be flagged, but thoughtful design makes recovery straightforward.

Honestly, I should have anticipated this sooner. Luckily, the solution was simple, and the service is back up without any lasting damage. The incident also provided valuable insights into how registry-level abuse flags work and how to design a service that can recover gracefully from them.

Lessons Learned

The ServerHold incident with maildock.store offered several insights about running a minimal disposable email service:

Despite being receive-only, with no outgoing mail, maildock.store was still flagged for potential abuse. utomated systems at the registry level tend to be cautious and sometimes overly conservative. Also, because no logs are kept, there's no way to tell what triggered the flag, which is a risk of a privacy-first approach.
DNS and registry configurations matter more than expected.
Inconsistencies in nameservers or leftover records may trigger alerts. While the system itself was unaffected, the domain's reachability depends on clear, correct DNS entries.
Direct communication with the registry is crucial - and they were responsive and helpful in resolving the issue once I provided context about the service and its safeguards.
Automated abuse flags are often resolvable with context. Having a clear explanation and a point of contact (abuse@maildock.store) allowed me to restore the domain quickly.
Minimalism has trade-offs: a simple architecture isolates most operations from failures like this, but no logging and lack of outbound monitoring means we rely on external feedback to detect issues.
The "why" remains uncertain: we still don't know exactly what triggered the abuse flag. This leaves open questions about how disposable domains are evaluated, and whether additional safeguards could reduce false positives. And also makes me wonder if the domain was targeted by a malicious actor who reported it, or tried to exploit the service, or if it was just an automated flag based on the domain's history or traffic patterns.

Overall, the experience reinforced that running a privacy-first service comes with uncertainties and edge cases. While minimalism and privacy help in some areas, external systems (registries, DNS providers, abuse monitors) can still impact availability in ways that are outside the service's direct control. In a larger context, this highlights the need to do your due diligence and understand the challenges of running a service that interacts with the broader internet ecosystem, even if it's designed to be as simple and private as possible. Also, value your users, but don't trust them to not try to abuse the service, and design with that in mind.

Advanced Asynchronous Patterns in JavaScript

Gabor Koos — Fri, 30 Jan 2026 00:08:08 +0000

async/await made asynchronous JavaScript far more readable, but readability isn't the same as control. Once async operations span multiple tasks, layers, or services, managing their coordination becomes the real challenge.
Patterns like cancellation propagation, timeouts, bounded concurrency, and controlled error handling repeatedly surface in production systems, yet they rarely get grouped together in a single, practical discussion.
This article explores these patterns, showing how they interact and the subtle but nasty pitfalls that often go unnoticed when building real-world asynchronous systems in JavaScript.

Cancellation Is the Missing Primitive

Once you move beyond async/await, cancellation quickly becomes a core concern. Promises represent results, not running work, so they cannot be forcibly stopped once started. This can lead to resource leaks or orphaned operations - a challenge explored in another article on this blog.

AbortController addresses this by providing a cooperative signal: it doesn't preempt execution, but APIs that observe the signal can stop work, clean up resources, or reject appropriately. This fits naturally with patterns from modern JavaScript concurrency, where explicit coordination is necessary to avoid unpredictable behavior.

Cancellation is distinct from other common async concerns:

Timeouts stop waiting, but not the underlying operation.
Failures indicate errors, not a cancellation decision.
Cancellation communicates that a result is no longer needed.

A practical example:

const controller = new AbortController();
const signal = controller.signal;

fetch(url, { signal })
  .then(response => { /* … */ })
  .catch(err => {
    if (err.name === 'AbortError') {
      console.log('Fetch was aborted');
    }
  });

// later
controller.abort();

Here, the fetch operation can be aborted, and the promise will reject with an AbortError. This allows the caller to handle cancellation explicitly.

The key is cooperation: only operations that check the signal will respond. Long-running loops or promises that ignore it continue running.

Effective patterns include:

Accepting an AbortSignal consistently at all layers.
Propagating it through call chains.
Periodically checking signal.aborted in compute-heavy tasks.

These approaches form a foundation for predictable asynchronous systems and connect naturally to coordination and flow-control patterns.

Timeouts Are a Form of Cancellation

In asynchronous systems, a timeout is essentially a signal that the result is no longer needed. Unlike synchronous code, where a function returns immediately, async operations continue running unless explicitly told to stop. Historically, developers used Promise.race() to enforce timeouts, but modern JavaScript provides first-class signal-based primitives that are more composable and predictable.

Historical Approach: `Promise.race()`

Before modern signal combinators, developers often used Promise.race() to enforce timeouts:

function fetchWithTimeout(url, ms) {
  const controller = new AbortController();
  const signal = controller.signal;

  const timeout = new Promise((_, reject) =>
    setTimeout(() => {
      controller.abort();
      reject(new Error('Timeout exceeded'));
    }, ms)
  );

  return Promise.race([
    fetch(url, { signal }),
    timeout
  ]);
}

This pattern works, but it quickly becomes verbose when multiple layers of asynchronous operations need timeouts. Each layer must handle the race, duplicating logic and introducing potential inconsistencies.

Modern Approach: `AbortSignal.timeout()`

With AbortSignal.timeout(), timeouts can be expressed declaratively:

const timeoutSignal = AbortSignal.timeout(5000); // signal that aborts after 5 seconds

fetch(url, { signal: timeoutSignal }) // adds timeout behavior
  .then(res => console.log('Success', res))
  .catch(err => {
    if (err.name === 'AbortError') {
      console.log('Cancelled or timed out');
    }
  });

Here, the timeout is a signal that automatically aborts after the given time. This pattern scales better across layers. Each function can accept an AbortSignal, and timeouts can be composed naturally without duplicating logic. No extra Promise.race(), no manual timers: the timeout is a signal that aborts automatically. Each function can accept an AbortSignal, making timeouts composable across layers without duplicating logic.

Composing Multiple Signals: `AbortSignal.any()`

Often, multiple cancellation sources exist simultaneously: user aborts, parent signals, or timeouts. AbortSignal.any() lets you combine them into a single signal:

const userController = new AbortController(); // user-initiated abort
const timeoutSignal  = AbortSignal.timeout(5000); // timeout abort

const combinedSignal = AbortSignal.any([ // combines both signals
  userController.signal,
  timeoutSignal
]);

fetch(url, { signal: combinedSignal })
  .catch(err => {
    if (err.name === 'AbortError') {
      console.log('Cancelled by user or timeout');
    }
  });

The fetch operation above will abort if any of the constituent signals fire. This declarative approach makes complex workflows predictable and composable.

The evolution from Promise.race() to AbortSignal.timeout() and AbortSignal.any() illustrates a key principle: timeouts and cancellation should be expressed declaratively, not imperatively. Modern APIs treat signals as first-class primitives that are composable, predictable, and safe to propagate across multiple async operations.

Composing Async Work Without Losing Control

Once cancellation and timeouts are handled, the next challenge is composing multiple asynchronous operations in a way that remains predictable and controllable. In production systems, tasks rarely run in isolation: you may need to fetch multiple resources concurrently, process streams in parallel, or coordinate nested services. Without a principled approach, these operations quickly become brittle, leaking resources or leaving partially completed work.

Theory: Why Composition Is Hard

The difficulty arises because each async operation can fail, cancel, or timeout independently. Naively combining promises with Promise.all or nested await calls often leads to:

Unhandled rejections if one task fails.
Stranded operations if one task is cancelled but others keep running.
Hard-to-maintain coordination logic as the number of tasks grows.

A robust solution treats each operation as a cancellable unit, and propagates cancellation, timeouts, and errors through a structured concurrency model. Conceptually, this is similar to having a "parent scope" that owns all child tasks: abort the parent, and all children stop automatically.

Running Multiple Tasks Concurrently

With modern signal-based patterns, you can combine multiple tasks while preserving cancellation:

const controller = new AbortController();
const signal = controller.signal;

async function fetchAll(urls, signal) {
  const tasks = urls.map(url => fetch(url, { signal }));
  return Promise.all(tasks); // aborting signal stops all fetches
}

const urls = ['/data1', '/data2', '/data3'];

fetchAll(urls, signal)
  .then(results => console.log('All fetched', results))
  .catch(err => {
    if (err.name === 'AbortError') console.log('Operation cancelled');
  });

// later
controller.abort(); // stops all ongoing fetches

Here, fetchAll accepts an AbortSignal that propagates to all fetch operations. If the signal is aborted, all fetches stop cleanly.

This pattern keeps the composition declarative: each function only observes a single signal, and higher-level logic defines how signals combine.

Handling Partial Failures

Sometimes, you want to continue other tasks even if one fails. You can wrap individual tasks to handle their errors independently:

const tasks = urls.map(url =>
  fetch(url, { signal: combinedSignal })
    .catch(err => ({ error: err, url }))
);

const results = await Promise.all(tasks);
console.log('Results with individual error handling', results);

In this example, each fetch handles its own errors, allowing the overall operation to complete even if some tasks fail. The results array contains either successful responses or error objects, enabling fine-grained handling.

This approach separates task coordination from task error handling, making complex asynchronous flows easier to reason about.

Patterns for Predictable Composition

To summarize, effective asynchronous composition in JavaScript relies on a handful of key patterns:

Treat all tasks as cancellable units.
Propagate signals from parent to children.
Combine signals declaratively (AbortSignal.any) for multiple abort sources.
Separate failure handling from orchestration when partial completion is acceptable.
Use structured concurrency principles: a parent scope owns all child operations.

By following these patterns, asynchronous operations remain predictable, composable, and maintainable — even in deep call stacks or large-scale applications.

Bounded Concurrency

In large-scale asynchronous systems, running all tasks at once can be as dangerous as running none. Fetching hundreds of URLs, processing large streams, or spawning compute-heavy operations simultaneously can overwhelm network, memory, or CPU resources. Bounded concurrency enforces a limit on the number of tasks running in parallel, allowing systems to remain responsive and predictable.

Theory: Why Concurrency Needs Bounds

Resources are always finite. Without limits, uncontrolled concurrency can lead to:

Memory usage can spike, network bandwidth can be saturated, or connection pools can be exhausted.
Downstream services may become overloaded.
Errors and cancellations can cascade unpredictably.

Bounded concurrency treats tasks as a pool: only a fixed number run at any given time. Additional tasks wait for a slot to free up. When combined with cancellation signals, this model allows controlled, safe, and abortable parallelism.

Implementing Bounded Concurrency

A simple pattern uses a queue and Promise.all:

async function runWithConcurrency(tasks, limit, signal) {
  const results = [];
  const executing = new Set();

  for (const task of tasks) {
    // Wait for a slot if limit is reached
    while (executing.size >= limit) {
      await Promise.race(executing);
    }

    // Start task
    const p = task(signal).finally(() => executing.delete(p));
    executing.add(p);
    results.push(p);
  }

  return Promise.all(results);
}

Here, runWithConcurrency accepts an array of task functions, a concurrency limit, and an AbortSignal. It ensures that only limit tasks run simultaneously. When a task completes, it frees up a slot for the next task.

Each task receives a signal, allowing cancellation or timeouts to propagate. The concurrency limit ensures that only limit tasks are active simultaneously, preventing resource overload.

Example: Fetching Multiple URLs with Limits

Let's see how to use runWithConcurrency to fetch multiple URLs with a concurrency limit:

const urls = ['/data1', '/data2', '/data3', '/data4', '/data5'];
const controller = new AbortController();
const signal = controller.signal;

async function fetchTask(url, signal) {
  return fetch(url, { signal }).then(r => r.json());
}

runWithConcurrency(
  urls.map(url => async (signal) => fetchTask(url, signal)),
  2, // max 2 concurrent fetches
  signal
).then(results => console.log('Fetched all with concurrency limit', results))
 .catch(err => {
   if (err.name === 'AbortError') console.log('Operation cancelled');
 });

In this example, only two fetches run at a time. Cancelling the signal aborts all ongoing tasks immediately, and queued tasks never start.

Integrating Timeouts and User Cancellation

The concurrency pool integrates seamlessly with signal combinators:

const timeoutSignal = AbortSignal.timeout(5000);
const userController = new AbortController();
const combinedSignal = AbortSignal.any([timeoutSignal, userController.signal]);

runWithConcurrency(
  urls.map(url => async (signal) => fetchTask(url, combinedSignal)),
  2,
  combinedSignal
);

Now the pool respects both user-initiated aborts and timeouts without additional wiring. Each task observes a single, combined signal, keeping the orchestration declarative.

Key Patterns

Limit active tasks to prevent resource overload.
Pass cancellation signals to every task for cooperative termination.
Combine multiple abort sources with AbortSignal.any().
Queue excess tasks for later execution rather than failing them.

Bounded concurrency turns an otherwise chaotic async workflow into a controlled, predictable system, and when combined with cancellation and timeouts, it gives developers precise control over both execution and resource usage.

Controlled Error Handling

In real-world asynchronous systems, errors are inevitable. Tasks may fail due to network issues, timeouts, user cancellations, or unexpected exceptions. The challenge is to handle these failures without undermining the coordination patterns established in previous sections: cancellation, timeouts, and bounded concurrency.

Theory: Separation of Concerns

A key principle is separating error handling from orchestration. Orchestration controls how tasks run and interact, while error handling decides what to do when they fail. Mixing these concerns can lead to brittle systems:

Cancelling a parent task should stop children without forcing global failures.
Individual failures should not automatically crash the entire workflow if partial results are acceptable.
Errors should propagate predictably and consistently.

Treating orchestration and error handling as separate layers makes it easier to reason about large-scale async systems.

Handling Partial Failures

Often, it is acceptable for some tasks to fail while others succeed. You can wrap individual tasks to capture errors without breaking the overall orchestration. Here's how to handle cases where partial success is acceptable:

const tasks = urls.map(url =>
  fetch(url, { signal: combinedSignal })
    .then(res => res.json())
    .catch(err => ({ error: err, url }))
);

const results = await Promise.all(tasks);
console.log('Results with partial error handling', results);

Each task handles its own errors, while the orchestration layer (Promise.all) continues to wait for all tasks. This preserves the bounded concurrency and cancellation guarantees while avoiding premature failure propagation.

Propagating Critical Failures

Some errors, however, are unrecoverable or require aborting the entire operation. You can propagate these selectively:

const tasks = urls.map(url =>
  async () => {
    const res = await fetch(url, { signal: combinedSignal });
    if (!res.ok) throw new Error(`Critical failure fetching ${url}`);
    return res.json();
  }
);

try {
  const results = await runWithConcurrency(tasks, 2, combinedSignal);
  console.log('All tasks completed', results);
} catch (err) {
  console.error('Critical failure, operation aborted:', err);
  // signal propagation ensures other tasks are aborted
}

Here, the orchestration respects cancellation signals, so aborting due to a critical failure stops all remaining tasks cleanly.

Patterns for Predictable Error Handling

As a summary, effective error handling in asynchronous workflows involves:

Wrap individual tasks to capture recoverable errors without stopping the workflow.
Use signals consistently so that cancellations propagate even in error scenarios.
Distinguish recoverable vs critical failures; abort the parent signal only when necessary.
Keep orchestration logic separate from task-level error handling to avoid coupling and duplication.
Compose with bounded concurrency and timeouts to maintain control even under partial failure.

By following these patterns, asynchronous workflows remain robust, composable, and predictable. Errors, cancellations, and timeouts coexist cleanly, giving developers full control over execution and failure modes in complex JavaScript systems.

Practical Observations

The patterns we've explored - cancellation, timeouts, bounded concurrency, and error handling - provide the building blocks for predictable asynchronous workflows. In practice, however, applying them correctly is often more subtle than just following the APIs. Let's highlight a few lessons learned from real systems, including common pitfalls, trade-offs, and heuristics that can make the difference between robust async code and fragile, hard-to-debug workflows.

Early Cancellation Is Only Half the Battle

Passing an AbortSignal to your function is necessary, but not sufficient. Tasks can still continue running if they hold internal state, perform long loops, or retry operations without checking the signal. In production, failing to check signal.aborted regularly or clean up resources can lead to "orphaned tasks" that quietly consume memory, network connections, or CPU, sometimes surfacing as mysterious failures hours later.

Concurrency Limits Are Contextual

A limit that works for one workload can fail for another. CPU-bound tasks may need a smaller limit than network-bound tasks. Backpressure (the concept of controlling the flow of data to prevent overwhelming a system) isn't just for streams - it matters whenever many promises compete for resources. Developers often set arbitrary limits without profiling, which leads to subtle latency spikes or cascading timeouts under load.

Timeouts Are Negotiation Points

Timeouts aren't just an implementation detail, they reflect expectations between system layers. Too short, and you create false failures; too long, and tasks tie up resources. In layered architectures, each layer must respect global policies. Ignoring this often leads to confusing bugs where some layers time out while others keep running indefinitely.

Errors Are Multi-Dimensional

Partial failures, retries, and network flakiness mean that error handling must be decoupled from orchestration. In practice, developers mix these concerns, leading to workflows where retries are applied inconsistently, cancellations are ignored, or critical errors propagate incorrectly. Observing patterns in production shows that failure semantics need to be explicit and layered. Always ask: "Is this error recoverable? Should it abort the whole operation? Can other tasks continue?"

Composability Breaks Without Discipline

It's tempting to hard-code concurrency or cancellation inside functions for simplicity. The real-world cost appears when tasks are reused in multiple workflows: suddenly signals clash, timeouts multiply, and debugging becomes hard. Composable APIs require consistent signal propagation, clean separation of orchestration, and predictable side effects. Skipping this discipline makes scaling async systems painful.

Cleanup Is Always Trickier Than You Think

Timers, network handles, database cursors all are easy to forget when aborting a task. In simple scripts it's harmless, but in long-running services it accumulates as memory leaks or stalled connections. Observing production systems shows that tying cleanup to the signal itself is the only reliable approach.

Observability Matters

Async patterns are tricky: cancellations, timeouts, and partial failures can silently affect results. Logging or metrics that expose which tasks were aborted, which timed out, and which failed partially make debugging tractable. Without this, even correct patterns become almost impossible to reason about when things go wrong.

Patterns Are Tools, Not Rules

Finally, none of these patterns are universal laws. The right choice depends on task criticality, resource constraints, and workflow semantics. Observing systems in production shows that developers who rigidly apply patterns without considering context often introduce complexity without benefit.

Composing Complex Pipelines

Real-world asynchronous workflows rarely consist of a single task. Often, multiple operations must run concurrently, sequentially, or in a mix, with cancellation, timeouts, concurrency limits, and error handling coordinated across stages. Understanding how these primitives interact is crucial for building robust pipelines.

Design Patterns for Pipelines

Parent-Child Ownership: Treat the pipeline itself as the "parent" task. Child operations inherit signals and timeouts. Aborting the parent stops all children consistently, preventing orphaned tasks - just like we saw earlier.
Stage Isolation: Separate logically distinct stages (e.g., fetching, processing, saving) to apply different concurrency limits or error-handling policies. This avoids one stage monopolizing resources or propagating failures unnecessarily.
Error Scope Management: Decide per stage whether errors should propagate or be contained. Some stages can tolerate partial failures, others must enforce strict all-or-nothing semantics.
Backpressure Awareness: Design the pipeline so downstream stages can signal upstream tasks to slow down. Pull-based iteration or explicit queues help maintain system stability under load.

Common Pitfalls

Overlapping concurrency pools that exceed system capacity: each stage must respect global limits.
Nested or hidden cancellations that lead to silent task leaks: watch out for tasks that never complete because their signals were aborted without proper handling.
Layered timeouts that conflict, causing confusing early failures or runaway tasks.: the timeout strategy must be coherent across the pipeline.
Coupled orchestration and error handling that make the pipeline fragile or hard to reason about.: careful separation of concerns is essential to maintain clarity and correctness.

By thinking in terms of pipeline structure, stage policies, and signal propagation, developers can design workflows that remain predictable and maintainable, even as complexity grows.

Frameworks & Libraries

Many libraries and frameworks implement or wrap some of these asynchronous patterns. These tools implement common patterns effectively, but mastering the underlying primitives ensures you can use them safely and predictably.

Concurrency & Queuing Libraries

p-limit / p-queue: Lightweight tools for bounding concurrency in promise-based workflows. They let you enforce parallelism limits per stage or globally.

Observation: These libraries handle execution limits but don’t propagate cancellation signals automatically, so you still need to integrate AbortSignal manually for clean task abortion.

Reactive & Stream-Based Libraries

RxJS, most.js, Highland.js: Functional reactive libraries that represent async operations as streams or observables. They provide composable pipelines, backpressure support, and declarative error handling.

Observation: They excel at structuring complex flows, but cancellation semantics may differ from native AbortSignal, and timeouts often need explicit operators. Understanding the underlying primitives helps bridge these gaps.

Framework-Level APIs

Node.js APIs (like undici, stream.pipeline, or EventEmitter patterns) increasingly support AbortSignal for cooperative cancellation.

Observation: Using these APIs effectively requires propagating signals consistently across layers. Libraries make common patterns easier but do not eliminate the need for orchestration discipline.

Takeaways

Libraries can reduce boilerplate, enforce concurrency, or structure pipelines declaratively.
None automatically solve all aspects of async coordination: cancellation, error propagation, backpressure, and timeouts still require developer attention.
Understanding the primitive patterns ensures that library usage remains safe and predictable in production.

Conclusion

Building robust asynchronous systems in JavaScript requires more than just async/await. Cancellation, timeouts, bounded concurrency, and controlled error handling are essential patterns that interact in subtle ways. By treating cancellation as a first-class primitive, expressing timeouts declaratively, composing tasks with clear ownership, and separating error handling from orchestration, developers can create predictable, maintainable workflows. These patterns are not just theoretical: they reflect real-world challenges observed in production systems. Mastering them helps developers to build scalable, resilient applications that handle the complexities of modern asynchronous programming.

The Go Build System: Optimised for Humans and Machines

Gabor Koos — Thu, 08 Jan 2026 22:47:24 +0000

Introduction: The Illusion of Simplicity

You probably type go build or go run dozens of times every week without thinking much about what happens under the hood. On the surface, these commands feel almost magical: you press Enter, and suddenly your code is compiled, linked, and - sometimes - executed. But beneath that simplicity lies a carefully orchestrated system, optimized to make your life as a developer easier while also being fast and predictable for machines.

Understanding how Go handles building, running, and caching code isn't just an academic exercise. It explains why incremental builds are so fast, why CI pipelines behave consistently, and why sometimes a seemingly trivial change can trigger a full recompilation. This article walks through the modern Go toolchain as it exists today, presenting a mental model you can trust.

By the end, you'll have a clear picture of:

how Go resolves dependencies and structures your code into packages,
how compilation and linking work behind the scenes,
why the build cache is so reliable, and
what actually happens when you type go build or go run.

If you've ever been curious about why Go builds "just work" or why your temporary go run binaries seem almost instantaneous, this is the deep dive that connects the dots - for humans and machines alike.

The Go Toolchain Mental Model

At first glance, go build, go run, and go test look like separate commands, each with its own behavior. In reality, they are just frontends for the same underlying pipeline. Every Go command goes through a predictable sequence: it loads modules, resolves package dependencies, compiles packages, optionally links them into an executable, and sometimes executes the result. The differences between commands mostly come down to what happens to the final artifact, not the mechanics of building it.

A key concept to internalize is that Go builds packages, not individual files. Every .go file in a package is treated collectively, and the package itself is the unit that the compiler and build cache track. This has several consequences:

Modifying any single file in a package can trigger a rebuild of the entire package.
Packages become the natural boundaries for caching and parallel compilation.
Small, focused packages tend to scale better in large codebases because the compiler can reuse more cached results.

The pipeline is conceptually simple, but highly optimized: Go knows exactly what needs recompilation and what can be reused, which is why incremental builds feel almost instantaneous. You can think of the toolchain as a smart coordinator: it orchestrates compiling, linking, caching, and execution so you rarely have to worry about the details. Once you internalize this mental model, the behavior of go build and go run stops feeling like magic and starts making predictable sense.

From `go.mod` to a Build Plan

Before Go ever touches your source files, it needs to figure out what to build and in what order. This begins with the module system, centered around your go.mod and go.sum files. These files define the module graph, which is the full dependency tree of your project, along with precise versions for every module. By reading these files, the Go toolchain knows exactly which packages are part of your build and which external code to fetch, verify, and incorporate.

Once the module graph is loaded, Go evaluates each package to determine its source set. This includes every .go file that belongs to the package, filtered by build tags, operating system, architecture, and any constraints you've specified. Only after this evaluation does the compiler know what code it actually needs to process. This ensures that your builds are deterministic: the same go build command run on different machines produces identical results, assuming the same module versions.

An important aspect of modern Go is the role of the go directive in go.mod. This directive declares the minimum Go version your module is designed for. It influences several characteristics of the build: language semantics, compiler behavior, and even static analysis. Depending on the go directive, language semantics, compiler behavior, and checks can differ - the toolchain enforces these during compilation. This is part of Go's focus on reproducibility, ensuring that your code behaves consistently across environments.

By the end of this stage, the toolchain has a complete, ordered build plan: it knows which packages to compile, in what sequence, and which files belong to each package. With this information in hand, it moves on to the next step: compiling packages and linking them into binaries, confident that nothing will be missed or miscompiled.

Compilation and Linking in Practice

Once Go has the build plan from the module system, it begins turning your code into something the machine can execute. This happens in two distinct stages: compilation and linking. Understanding these stages is key to appreciating why Go builds are fast, deterministic, and scalable.

Compilation Is Per-Package

Go compiles one package at a time. Each package - whether it's part of your project or an external dependency - is treated as an independent unit. The compiler produces intermediate artifacts for every package, which are stored in the build cache. This means that if a package hasn't changed since the last build, Go can skip recompiling it entirely, even if other packages that depend on it are being rebuilt.

Parallelism is another advantage of this per-package approach: since the compiler knows the dependency graph, it can compile multiple independent packages concurrently, fully leveraging multi-core CPUs. This is why large Go projects often feel surprisingly fast to build: a lot of work is done in parallel, and nothing is recompiled unnecessarily.

Linking Is Selective

Linking is the process of combining compiled packages into a single executable. Go only links main packages into binaries. Library packages never get linked on their own, they exist purely as reusable artifacts for other packages. This distinction is important: when you run go build ./... on a project, Go may compile dozens of packages but produce zero binaries if none of the packages are main!

Linking is often the most expensive step in a build because it involves combining all dependencies into a single executable, resolving symbols, and embedding metadata. By keeping linking selective, and relying on cached package compilation, builds remain efficient.

What Ends Up in the Binary

The final binary is more than just your compiled code. It includes:

All dependent packages that are reachable from main
Build metadata, including the module version and commit information
Machine-level instructions optimized for the target platform

This combination is why Go binaries are self-contained and reproducible: they include everything needed to run without relying on external libraries or runtime environments. From a human perspective, this makes deployment straightforward. From a machine perspective, the build system can verify and cache everything efficiently, ensuring that repeated builds are fast and deterministic.

The Build Cache: The Center of Gravity

At the heart of Go's speed and predictability is its build cache. Every compiled package, every intermediate artifact, and even some tool outputs are stored in a content-addressed cache, which allows Go to reuse work across builds, commands, and even go run invocations. Understanding how the cache works is essential to grasping why Go builds feel almost instantaneous, even for large projects.

What the Cache Stores

The build cache is more than just compiled binaries. It contains:

Compiled package artifacts (.a files) for all packages in the build graph
Test results, including cached success information
Temporary tool outputs needed for execution by go run or go test

The cache lives on disk (by default in $GOCACHE) and is fully deterministic, meaning the same package compiled with the same inputs will always produce the same cache entry. This ensures that repeated builds, or builds across different machines, produce identical results.

Content-Addressed, Not Timestamp-Based

Unlike traditional build systems that rely on file timestamps, Go uses content-based hashing to determine cache keys. Each cache key is a function of:

the source code content
the compiler version
any build flags
the target platform (GOOS/GOARCH)
relevant environment variables

This design guarantees that builds are reproducible and avoids false cache misses due to innocuous changes like timestamps or file order.

Cache Invalidation Explained

Even with a robust cache, Go will sometimes recompile packages. Common causes include:

Modifying source code or build tags
Changing compiler flags or environment variables
Renaming files within a package

Go's caching system is smart: it only rebuilds what actually needs rebuilding. Even small, non-semantic changes can trigger recompilation if they affect the package’s build hash, but otherwise, the cache is trusted implicitly.

Why the Cache Is Safe to Trust

The build cache is designed to be transparent and reliable:

You rarely need to manually clear it
Rebuilding from scratch produces identical artifacts
go run, go test, and go build all leverage it consistently

This is why Go's incremental builds are so fast: the compiler never does more work than necessary. From a developer perspective, it feels magical. From a systems perspective, it's simply an optimized pipeline that treats package artifacts as first-class citizens.

`go build`: Producing Artifacts

The go build command is the workhorse of the Go toolchain. Its job is simple to describe but sophisticated in execution: compile packages, link them if necessary, and produce a binary that is correct and reproducible. Understanding what go build actually does helps you predict its behavior and avoid common surprises.

How go build Handles Packages

When you run go build on a module or package, the tool first examines the dependency graph derived from your go.mod. Every package in the graph is checked against the build cache: if the cache contains a valid compiled artifact for a package, Go reuses it instead of recompiling. Only packages that have changed - or whose dependencies changed - are rebuilt.

Because Go operates at the package level, touching a single file inside a package can trigger a rebuild of the entire package. Conversely, if a dependency hasn't changed, it's never rebuilt, even if other packages rely on it. This per-package granularity is one of the reasons Go's incremental builds scale so well, even for large projects.

Linking and the Final Binary

As we mentioned earlier, go build only produces an executable for main packages. Library packages are compiled into intermediate artifacts but never linked on their own. When linking a main package, Go combines all compiled packages into a single binary. This process also embeds metadata into the executable, including:

module version information
commit hashes (if available)
platform-specific build metadata

By default, inclusion of version control details is governed by the -buildvcs flag, which defaults to "auto" and stamps VCS information when the repository context allows (use -buildvcs=false to omit or -buildvcs=true to require it). More details can be found in the documentation here.

This makes Go binaries self-contained and highly reproducible, allowing you to deploy them confidently without worrying about missing dependencies.

Where the Artifacts Go (Sorry 😀)

By default, go build writes the binary in the current directory, named after the package. If the package is a library, go build doesn't produce a binary at all, it only ensures that the package and its dependencies are compiled. You can control output locations with the -o flag or use ./... to build multiple packages in one go.

On Windows, executables have a .exe suffix. When building multiple main packages at once (for example, ./cmd/...) without -o, Go writes one binary per main package into the current directory.

Predictable and Reliable Builds

The combination of per-package compilation, caching, and selective linking ensures that go build is predictable. You can trust that:

builds are reproducible across machines
unchanged code is never rebuilt unnecessarily
intermediate artifacts are reused to optimize build time

In short, go build is not just compiling code, it's orchestrating a deterministic pipeline that balances human convenience with machine efficiency.

`go run`: Convenience Without Special Privileges

If go build is the workhorse that produces artifacts you can deploy, go run is the fast lane for experimenting and executing code immediately. Many developers think of it as "compiling and running in one step", but it's not: under the hood, it leverages the same build system as go build, it's just optimized for convenience rather than artifact persistence.

What `go run` Actually Does

When you type go run main.go (or a list of files), Go first evaluates the package and its dependencies just as it would for go build. Any cached compiled packages are reused, so the compiler does minimal work for unchanged code. Then, Go links the main package into a temporary binary, executes it, and deletes the binary once the program finishes.

From a caching perspective, go run is not a special path, it fully participates in the build cache. This explains why repeated invocations of the same program often feel instantaneous: the heavy lifting has already been done, and only linking or changed packages may trigger compilation.

Why `go run` Feels Different

Despite sharing the same underlying pipeline, go run can feel slower in certain scenarios. Because it produces a temporary binary every time, linking is repeated, even if all dependencies are cached. For small programs, this overhead is negligible, but for projects with large dependency graphs, it can be noticeable.

Another difference is that go run does not leave a persistent artifact. This is exactly the point: it trades binary reuse for ease of execution. You don't need to think about where to place the binary or what to call it, the tool handles it automatically.

When `go run` Is the Right Tool - and When It Isn't

go run is ideal for:

quick experiments or scripts
running one-off programs without cluttering the filesystem
testing small programs interactively

It's less suitable for:

production builds or deployment
long-running servers where repeated linking adds overhead
CI pipelines where caching persistent binaries is more efficient

For these cases, the recommended pattern is go build && ./binary, which gives you the benefits of caching, reproducibility, and a persistent artifact without sacrificing performance.

`go test` and Cached Correctness

The go test command builds on the same principles as go build and go run, but adds a layer of test-specific caching and execution logic. Understanding how tests interact with the build system helps explain why some tests run instantly while others trigger a rebuild, and why Go's approach feels both fast and predictable.

Compilation Reuse in Tests

When you run go test, Go first determines the dependency graph for the test package, including any imported packages. Packages that haven't changed are reused from the build cache, just as with go build or go run. This means that large test suites can often start executing almost immediately, because most of the compilation work has already been done.

Even when multiple packages are involved, Go only rebuilds the packages that actually changed. The combination of per-package compilation and caching ensures that incremental test runs are fast, even in large projects.

Test Result Caching

In addition to caching compiled packages, Go also caches test results. If a test passes and none of its dependencies or relevant flags have changed, Go can skip re-running the test entirely.

Test result caching applies only in package list mode (e.g., go test . or go test ./...). In local directory mode (go test with no package args), caching is disabled.

This behavior is controlled by the -count flag. For example, go test -count=1 forces execution regardless of cached results. (-count repeats tests/benchmarks. -count=1 is the idiomatic way to bypass cached results. See the documentation for further details.)

Caching test results improves developer productivity and CI efficiency, especially for large projects with extensive test coverage. It also reinforces Go's philosophy: the system should avoid unnecessary work while preserving correctness.

Cache Invalidation in Testing

A test may be re-run automatically if:

The test code itself has changed.
Any dependency of the test has changed.
Flags affecting the test have changed.
Non-cacheable flags or changed files/env also invalidate reuse.

Otherwise, Go trusts the cached result, knowing it is deterministic and reproducible. This approach reduces "flaky" builds caused by unnecessary rebuilds and emphasizes predictability over blind convenience.

Optional Handy Snippets

Here are some useful go test invocations that leverage caching behavior:

Fresh run: go test -count=1 ./... - as we saw earlier, this disables test result caching.
Stress a test: go test -run '^TestFoo$' -count=100 ./pkg - runs TestFoo 100 times to check for flakiness.
Bench stability: go test -bench . -count=3 - runs all benchmarks 3 times to get stable measurements.

Why This Matters for Developers

From a developer's perspective, the combination of build caching and test result caching creates a workflow that feels instantaneous and reliable:

Small changes trigger only the necessary compilation steps.
Passing tests rarely run again unless something changes.
Developers can iterate rapidly without worrying about hidden state.

By treating both packages and test results as first-class cacheable artifacts, Go makes testing fast and predictable, reinforcing the same "human + machine" optimization that underlies go build and go run.

Observing and Debugging the Build System

Most of the time, Go's build system does exactly what you expect, quietly and efficiently. When something feels off, though, the toolchain gives you direct, low-level visibility into what it's doing. The key is knowing which switches to flip and how to interpret what you see.

Making the Toolchain Talk

Go provides a small set of flags that expose the build pipeline without changing its behavior:

-x prints the actual commands executed during the build. This includes compiler invocations, linker steps, and tool executions. It’s the fastest way to answer the question: "What is Go actually doing right now?"
-n shows what would be executed, without running the commands. This is useful when you want to understand the build plan without triggering a rebuild.
-work preserves the temporary build directory instead of deleting it. This lets you inspect intermediate files, generated code, and temporary artifacts produced during compilation or linking.

These flags turn the Go toolchain from a black box into a transparent pipeline. Importantly, they don't disable caching, they simply make cache hits and misses visible.

Understanding Why a Package Rebuilt

One of the most common sources of confusion is a package rebuilding "for no apparent reason". With the right mental model, this becomes easier to diagnose:

A package rebuilds when any input to its cache key changes.
Inputs include source code, build tags, compiler flags, target platform, and relevant environment variables.
Dependency changes propagate upward through the package graph.

Using -x, you can often see whether Go reused a cached artifact or recompiled a package, and infer why from the context. This removes the temptation to reach for blunt tools like go clean -cache as a first response.

Forcing Rebuilds (When You Actually Mean It)

Sometimes you really do want to bypass the cache. For example, when validating a clean build or debugging toolchain issues. Go supports this explicitly:

-a forces rebuilding of packages, ignoring cached compiled artifacts
go clean -cache clears the entire build cache

These options are intentionally explicit and slightly inconvenient. Go is designed to make correct reuse the default, and manual cache invalidation the exception. If you find yourself clearing the cache regularly, it's often a sign that something else in the build setup needs attention.

Avoiding Superstition-Driven Fixes

Because Go's build system is deterministic, guessing rarely helps. Flags like -x, -n, and -work give you concrete evidence of what's happening, which is almost always enough to explain surprising behavior.

Once you trust that:

builds are content-addressed,
packages are the unit of work,
and the cache is safe to reuse,

debugging build behavior becomes a matter of observation rather than trial and error.

Implications for Real Projects

The design choices behind Go's build system aren't accidental. They show up most clearly once you move beyond small examples and start working on real codebases: continuous integration pipelines, large repositories, and editor-driven workflows. The same principles that make go build feel fast locally are what make Go scale so well in production environments.

CI Pipelines and Reproducibility

Go's emphasis on deterministic, content-addressed builds makes it particularly well-suited for CI. Because build outputs are derived entirely from source content, module versions, and explicit configuration, CI builds behave consistently across machines and environments. There's no reliance on filesystem timestamps, hidden state, or global configuration.

This predictability also makes Go builds highly cache-friendly. Whether you're using a shared build cache, container layers, or remote caching infrastructure, Go's package-level compilation model fits naturally. When a build is slow in CI, it's usually because something actually changed, not because the system decided to do extra work.

Monorepos and Large Codebases

In large repositories, the build cache becomes a performance boundary. Because Go caches compiled packages independently, small, well-defined packages can be reused across many builds with minimal overhead. This encourages a code structure where dependencies are explicit and packages remain focused.

The flip side is that overly large or tightly coupled packages can become bottlenecks. A small change in a heavily used package can invalidate a large portion of the cache, increasing build times across the entire repository. Go doesn't hide this cost though, it makes package boundaries visible and meaningful, rewarding good structure and exposing poor separation early.

Editors, Tooling, and Automation

The same build model powers Go's tooling ecosystem. Code editors, language servers, linters, and code generators all rely on the same package-level understanding of your code. Because the toolchain exposes a clear, deterministic build pipeline, tools can integrate deeply without guessing or reimplementing build logic.

This is one reason Go tooling feels unusually consistent: editors and CI systems see your code the same way the compiler does. From autocomplete to refactoring to automated testing, everything builds on the same assumptions about packages, dependencies, and caching.

Conclusion: Trust the Model

Go's build system succeeds because it makes a clear trade-off: it optimizes for predictability over cleverness, and for explicit structure over implicit behavior. At the surface, this looks like simplicity. Underneath, it's a carefully engineered pipeline that treats packages as the unit of work, content as the source of truth, and caching as a correctness feature rather than a performance hack.

Once you internalize this model, many everyday behaviors start to make sense. Builds are fast not because Go is doing less work, but because it avoids doing unnecessary work. go run feels convenient because it reuses the same machinery as go build, not because it shortcuts correctness. Test execution is reliable because test results are cached using the same deterministic rules as compiled packages.

For humans, this means fewer surprises, faster feedback loops, and tooling that behaves consistently across code editors, machines, and CI systems. For machines, it means reproducible builds, cache-friendly artifacts, and a system that scales naturally as codebases grow. The same design choices serve both audiences.

If there's one takeaway, it's this: Go's build system isn't something to fight or work around. It's an API in its own right - one that rewards understanding. Once you trust the model, the toolchain stops feeling magical and starts feeling dependable, which is exactly what you want from the infrastructure that builds your code.

Backpressure in JavaScript: The Hidden Force Behind Streams, Fetch, and Async Code

Gabor Koos — Tue, 06 Jan 2026 22:29:47 +0000

We all know JavaScript's asynchronous model. async/await, Promises, and streams give the illusion that code runs sequentially while magically handling heavy work in the background. But if you've ever processed a large file, streamed data from an API, or handled bursts of network requests, you've probably run into a familiar problem: memory usage spikes, CPU sits idle, or your server crashes under a sudden load. "Everything is async", so what is going on?

The answer lies in a concept many developers have never heard by name: backpressure. Backpressure is the system-level feedback mechanism that allows a consumer to slow down a producer when it's producing data faster than the consumer can handle. Without it, your asynchronous tasks wouldn't just run concurrently, they'd pile up, creating unbounded queues in memory and ultimately breaking your application.

In JavaScript, backpressure exists in multiple places: Node.js streams, the Fetch API, Web Streams, and even async loops over large datasets. But it can be tricky. The language gives you the tools: ReadableStream, WritableStream, stream events like drain - but it doesn't enforce correct usage. And many developers end up ignoring these signals, mostly because the code "just works" on small datasets. Then the data grows, the load increases, and suddenly your app is struggling to keep up: crashes, OOMs, and latency spikes seem to come out of nowhere.

This article will unpack what backpressure really is, why it matters in JavaScript, and how to write async code that respects it. By the end, you'll see that backpressure isn't a limitation, it's a feature of well-behaved systems, and understanding it can save you from countless production headaches.

What Backpressure Actually Is (and Isn't)

Backpressure is one of those concepts that feels obvious once you see it, but most developers only realize it happening when their app starts breaking under load. Let’s unpack it carefully.

Producer vs Consumer

At its core, backpressure is about communication between a producer and a consumer:

Producer: anything that generates data. Examples in JavaScript include a network request, a file reader, or an async generator.
Consumer: anything that processes data. This could be parsing JSON, writing to disk, or sending data over a WebSocket.

Problems arise when the producer generates data faster than the consumer can handle. Without a way to slow down the producer, data starts piling up in memory, creating unbounded queues that eventually crash your app. For example:

async function processData(generator) {
  for await (const chunk of generator()) {
    heavyProcessing(chunk) // slow consumer
  }
}

Even though for await looks sequential, the generator might produce chunks faster than heavyProcessing can handle, resulting in memory bloat, asynchronous CPU spikes, and eventual crashes.

What Backpressure Means

Backpressure is the mechanism that lets the consumer signal the producer to slow down. In JavaScript, this often happens implicitly in streams:

When writable.write(chunk) returns false, it tells the producer to stop writing temporarily.
When using readable.pipe(writable), the pipe manages flow automatically.
In web streams, the pull() method only asks for more data when the consumer is ready.

Key point: backpressure is about rate control, not order of execution or batching. Simply buffering all incoming data is not backpressure, it just postpones the problem!

How Ignoring It Breaks Things

Ignoring backpressure can lead to a few familiar symptoms:

Memory spikes: Data piles up in memory faster than it can be processed.
Latency collapse: Requests slow down unpredictably as queues grow.
Crashes / OOMs: Eventually, the process runs out of memory.

Buffers and queues can hide the problem temporarily, but they don't solve it. True backpressure is about coordination, ensuring that the producer never overwhelms the consumer.

In the next section, we'll briefly look at how backpressure appears outside JavaScript, and why it's a problem every system-level programmer has had to solve, even before JS existed.

Backpressure Before JavaScript

Backpressure didn't start with JavaScript. It's a fundamental concept in computing systems: something developers have been dealing with long before ReadableStream or Node.js existed. Understanding its history helps explain why it exists in JS today and why it matters.

Pipes and Streams in Unix

In Unix, the classic example is a pipeline of processes:

cat largefile.txt | grep "error" | sort | uniq

Each process is a consumer of the previous process's output and a producer for the next. If one process reads slower than its predecessor writes, Unix automatically pauses the faster process until the slower one catches up. That's backpressure in action: a natural flow-control mechanism built into the system.

TCP Flow Control

At the network level, TCP also relies on backpressure. If a receiver cannot process incoming packets fast enough, it tells the sender to slow down via windowing and acknowledgment mechanisms. Without this feedback, network buffers could overflow, leading to dropped packets and retransmissions.

Messaging Systems

Message queues, like RabbitMQ or Kafka, implement backpressure as well. Producers either block or receive signals when queues are full, ensuring consumers aren't overwhelmed. Systems that ignore this risk data loss or memory exhaustion.

Why It Matters for JS Developers

These examples show that backpressure is a property of any system where work is produced faster than it can be consumed. JavaScript inherits the same problem in streams, async iterators, fetch, and beyond. What's different in JS is the language gives you the primitives, but not the enforcement: if you ignore the signals, your memory grows and your app breaks.

Backpressure in Node.js Streams

Node.js popularized backpressure through its streams API, which provides a robust mechanism for controlling data flow between producers and consumers. Understanding streams is essential for writing high-performance, memory-safe Node applications.

Readable Streams and `highWaterMark`

A Readable Stream is a source of data: like a file, HTTP request, or socket. Internally, Node buffers data in memory. The key parameter controlling backpressure is highWaterMark, which sets the soft limit of the internal buffer:

const fs = require('fs');
const stream = fs.createReadStream('largefile.txt', { highWaterMark: 16 * 1024 });

Here, highWaterMark is 16 KB. When the buffer reaches this limit, the stream stops reading from the underlying source until the buffer is drained. This is the first layer of backpressure: the producer slows down when the consumer cannot keep up.

Writable Streams and the `write()` Return Value

A Writable Stream consumes data. The most common mistake is ignoring the return value of write(). This boolean tells you whether the internal buffer is full:

const fs = require('fs');
const writable = fs.createWriteStream('output.txt');

function writeData(data) {
  if (!writable.write(data)) {
    // backpressure signal: wait for 'drain'
    writable.once('drain', () => {
      console.log('Buffer drained, continue writing');
    });
  }
}

If you ignore false and keep writing, Node will buffer everything in memory, eventually causing your app to run out of memory. The drain event signals that it's safe to resume writing.

Using `pipe()` for Automatic Backpressure

Node streams also support automatic backpressure management through pipe(). When you pipe a readable to a writable, Node internally listens for the consumer's signals and pauses/resumes the producer accordingly:

const fs = require('fs');

const readable = fs.createReadStream('largefile.txt');
const writable = fs.createWriteStream('copy.txt');

readable.pipe(writable);

Here, the readable stream automatically pauses when the writable's buffer is full and resumes when the drain event fires. This makes pipe() one of the simplest and safest ways to handle backpressure.

Common Pitfalls

Even with streams, it's easy to break backpressure:

Ignoring write() return values: queues grow unchecked.
Using Promise.all() on chunks: creates unbounded concurrency. Many writes may happen simultaneously, overwhelming the writable stream.
Reading everything into memory: readFileSync or fs.promises.readFile may crash on large files.

Streams exist because they provide flow control by design. Learning to respect the signals (write() return value, drain, pipe()) is how you implement real backpressure in Node.js.

Node streams expose a built-in contract between producer and consumer. If you ignore it, your memory grows - if you respect it, your application handles large or fast data sources safely.

How `async/await` Can Accidentally Destroy Backpressure

async/await is one of JavaScript's greatest abstractions for writing readable asynchronous code. But it can also mask backpressure problems, making you think your consumer is keeping up when it isn't. Understanding this is crucial for building reliable, memory-safe applications.

The Illusion of Sequential Safety

It's easy to assume that wrapping work in await naturally enforces proper flow control:

for await (const chunk of stream) {
  process(chunk); // heavy CPU work
}

At first glance, this seems safe: each chunk is processed before moving to the next. But if process(chunk) launches asynchronous tasks internally - like database writes or network requests - the actual concurrency may be much higher than it appears. The producer continues to deliver new chunks to your loop while earlier tasks are still pending, causing memory growth.

The `Promise.all()` Trap

A common pattern is to process multiple chunks concurrently using Promise.all():

const chunks = await getAllChunks();
await Promise.all(chunks.map(processChunk));

This eagerly starts all chunk processing in parallel. For small datasets, this works fine, but with large streams, you're effectively removing any backpressure, because the producer's work is no longer paced by the consumer! Memory usage spikes, and your process may crash.

Why Await ≠ Flow Control

Even for await loops don't inherently enforce backpressure if the work inside the loop is asynchronous:

for await (const chunk of readableStream) {
  someAsyncTask(chunk); // fire-and-forget
}

Here, the loop awaits only the next chunk, not the completion of someAsyncTask. The readable stream continues producing new chunks, and your memory usage grows unbounded.

Rule of thumb: backpressure requires the consumer to signal readiness. Just awaiting the next item in a loop does not automatically create that signal if your processing is asynchronous.

Patterns That Preserve Backpressure

To maintain backpressure with async/await, consider:

Sequential processing: await each async task before moving to the next.
Bounded concurrency: limit the number of in-flight promises with a small worker pool.
Respect stream signals: combine await with the writable's write() return value or drain event.

Example using bounded concurrency:

import pMap from 'p-map';

const mapper = async (chunk) => await processChunk(chunk);

await pMap(readableStream, mapper, { concurrency: 5 });

Here, p-map ensures at most 5 chunks are processed concurrently, preventing runaway memory growth while still allowing parallelism.

Remember, async/await is syntactic sugar, not a flow-control mechanism. If your asynchronous work inside a loop or Promise.all() is unbounded, you break backpressure and risk crashes or latency spikes.

Backpressure in Fetch, Web Streams, and the Browser

Backpressure of course isn't limited to Node.js. In the browser, modern APIs like fetch and Web Streams expose similar flow-control mechanisms, though they can be even subtler because of the single-threaded UI environment.

Fetch + Streams

When you call fetch, the response body can be accessed as a stream:

const response = await fetch('/large-file');
const reader = response.body.getReader();

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  processChunk(value);
}

Here, the read() call implicitly applies backpressure. The browser will not deliver the next chunk until the previous one has been consumed. If your processChunk function is slow or CPU-intensive, the stream naturally slows down the network reading, preventing memory overload.

However, if you accidentally read the entire response at once using response.text() or response.arrayBuffer(), you bypass backpressure entirely, forcing the browser to allocate memory for the whole payload at once.

Web Streams API

The Web Streams API generalizes this pattern. Streams in the browser support two key mechanisms for backpressure:

Pull-based reading

Consumers request more data when ready using a pull() method in a custom ReadableStream:

const stream = new ReadableStream({
  start(controller) { /* optional setup */ },
  pull(controller) {
    controller.enqueue(generateChunk());
  },
  cancel(reason) { console.log('Stream cancelled', reason); }
});

Here, the browser calls pull() only when the consumer is ready for more data, creating natural backpressure.

WritableStream signaling

When writing to a WritableStream, the write() promise only resolves when the consumer has processed the chunk. If the consumer is slow, write() automatically pauses the producer (the promise will stay pending):

const writable = new WritableStream({
  write(chunk) {
    return processChunk(chunk); // returns a promise
  }
});

Where Browser Backpressure Can Break Down

Even with these APIs, there are common pitfalls:

UI thread blocking: Long synchronous work can starve the main thread, causing latency even if streams are correctly used.
Fire-and-forget async operations: Like in Node, launching many promises inside a pull() method can overwhelm the consumer.
Ignoring transfer costs: Passing large objects between threads (e.g., with postMessage) can trigger copying overhead if you don't use Transferables.

As we can see, backpressure in the browser works similarly to Node.js streams: the consumer drives the pace of the producer. Properly used, it prevents memory spikes and keeps your app responsive. Ignoring these mechanisms - by reading entire responses at once, launching unbounded promises, or blocking the UI - defeats backpressure, creating systems that can crash or become unresponsive under load.

It's still about signaling readiness, not just awaiting asynchronous operations. JavaScript provides the primitives in both Node and the browser, but developers must respect them.

Buffers: The Double-Edged Sword

Buffers are everywhere in JavaScript streams. They act as shock absorbers, temporarily storing data when the producer is faster than the consumer. While buffers are essential for smooth streaming, they can also mask backpressure problems if misused.

What Buffers Do

A buffer's main purpose is to decouple producer speed from consumer speed. By holding onto data temporarily, buffers allow small variations in processing time without immediately stalling the producer. In the example earlier:

const fs = require('fs');
const readable = fs.createReadStream('largefile.txt', { highWaterMark: 64 * 1024 });

highWaterMark sets the buffer size. The readable stream can accumulate up to 64 KB of data before signaling the producer to pause. This allows small variations in consumer speed without immediately blocking the producer.

Buffers exist in both Node streams and Web Streams, and their behavior is similar: they let the system manage short-term fluctuations in throughput.

When Buffers Hide Problems

Problems arise when buffers are unbounded or ignored:

Memory growth: If the consumer can't keep up and the buffer grows beyond expectations, your app can exhaust memory.
Latency spikes: Large buffers introduce additional delay before the consumer sees new data.
Delayed failure: Buffers can postpone a crash, making the problem harder to detect until traffic spikes dramatically.

Take this example:

// Reading entire file into memory
const data = await fs.promises.readFile('hugefile.txt');
process(data); // instantaneous, but memory-heavy

Even though this "works" for small files, it completely ignores backpressure. The buffer (memory) absorbs all data at once, leaving no flow control.

How to Use Buffers Wisely

Buffers are powerful when bounded and intentional:

Set reasonable highWaterMark values.
Respect writable return values and drain events.
Use streaming APIs instead of reading everything at once.
Combine with bounded concurrency for async tasks to avoid hidden buildup.

Buffers should support backpressure, not replace it. Think of them as a cushion: they smooth out short-term spikes, but the consumer must still be able to handle the flow long-term.

Buffers are not a cure-all. They are a tool to make backpressure effective, not a substitute for it. Understanding their limits ensures that your Node.js and browser applications remain responsive, memory-safe, and resilient under load.

Recognizing Backpressure Problems in Real Apps

Backpressure problems usually don't announce themselves with clear errors: they creep in slowly, manifesting as memory growth, latency spikes, or unpredictable behavior. Perceiving these symptoms early is key to building robust asynchronous applications.

Common Symptoms

Memory Growth Over Time

The app's memory usage steadily increases under load, even when requests are processed asynchronously.
Often caused by unbounded buffers or producers generating data faster than consumers can handle.

Latency Collapse

Requests start taking longer as the system processes more data.
Queues form behind slow consumers, delaying new tasks.

Crashes or Out-of-Memory Errors

Eventually, excessive buffering leads to process termination or browser tab crashes.

High CPU with Low Throughput

A symptom of inefficient flow: the CPU is busy juggling many small tasks, but actual work completion lags behind.

Diagnostic Questions

When backpressure issues appear, ask:

Where does data queue? Are producers creating more work than consumers can handle?
Does your code respect the backpressure signals provided by streams or async iterators?
Are you launching too many concurrent promises (e.g., with Promise.all() or unbounded async loops)?
Are buffers growing unbounded in Node streams, fetch requests, or Web Streams?

Early Warning Tips

Monitor memory usage in development under realistic load.
Test streams with intentionally slow consumers to observe backpressure behavior.
Use small bounded buffers and gradually scale them up.

Backpressure issues are often subtle but predictable. By watching for memory growth, latency spikes, and unbounded concurrency, you can identify potential problems before they hit production and design your streams and async flows to respect the natural pace of your consumers.

Designing Backpressure-Friendly JavaScript Code

Understanding backpressure conceptually is important, but the real benefit comes from writing code that respects it. In JavaScript, both Node.js and the browser provide primitives for flow control—but it's up to the developer to use them correctly.

This section focuses on patterns and strategies for designing JavaScript applications that handle high-volume or fast data streams safely, without repeating low-level stream API details.

Think in Terms of Flow, Not Tasks

Backpressure is about coordinating producer and consumer rates. Instead of thinking in terms of "launch tasks as fast as possible", design your system around how much work can actually be handled at a time.

Identify natural boundaries: buffers, streams, network requests, or event loops.
Avoid unbounded queues of work (e.g., infinite Promise.all() or uncontrolled event handlers).

Use Pull-Based or Demand-Driven Designs

Producer-driven: Traditional model where the producer pushes data. Requires careful monitoring of buffers and signals.
Consumer-driven: Better pattern for JavaScript: consumers pull data when ready. This naturally enforces backpressure, especially with Web Streams or async iterators.

The guiding principle: the consumer should control the pace.

Bound Concurrency

Even when using async/await, unbounded parallelism is dangerous. Instead of letting every task run simultaneously:

Use worker pools for CPU-heavy tasks.
Use limited async queues for I/O-heavy tasks.
Measure the "sweet spot" for concurrency empirically, considering memory, CPU, and network.

This ensures your system scales without crashing, even if the producer is fast.

Monitor and React

Design systems to observe flow in real time:

Track buffer lengths, memory growth, and queue sizes.
Detect when consumers lag and temporarily slow producers if possible.
Introduce graceful degradation rather than letting memory explode or requests fail silently.

Prefer Declarative Coordination

Instead of manually juggling streams and buffers:

Use high-level libraries that implement flow control primitives.
Prefer iterators, async generators, and pull-based streams to abstract away low-level buffering logic.
Focus on designing pipelines that express intentional flow control rather than ad-hoc buffering.

Backpressure-friendly design is system thinking applied in JavaScript: coordinate producers and consumers, limit concurrency, and observe flow continuously. By applying these principles, your applications can handle large datasets, fast streams, or bursts of requests without depending on trial-and-error or unbounded buffers.

Conclusion: Respect the Flow

Backpressure isn't an optional detail in asynchronous JavaScript, it's a fundamental property of any system where producers can generate data faster than consumers can handle. From Node.js streams to fetch and Web Streams in the browser, JavaScript provides primitives that allow consumers to signal readiness and prevent runaway memory growth or latency spikes.

The key lessons are:

Identify producers and consumers. Understand where data is generated and where it's processed.
Respect the signals. Streams provide built-in backpressure mechanisms (write() return values, drain events, pull() in Web Streams), and async iterators can enforce flow when used correctly.
Bound concurrency. Avoid unbounded Promise.all() or fire-and-forget loops. Use worker pools, limited queues, or libraries for controlled parallelism.
Use buffers wisely. Buffers smooth temporary spikes but are not a substitute for proper flow control. Always keep them bounded.
Monitor and diagnose: watch memory, queue lengths, and latency to catch hidden backpressure problems before they impact production.

By designing systems that respect the natural pace of their consumers, JavaScript developers can handle large datasets, high-throughput streams, or bursty network traffic safely and efficiently. Backpressure is not a limitation, it's a feature that enables robust, scalable, and maintainable asynchronous code.

Cancellation In JavaScript: Why It's Harder Than It Looks

Gabor Koos — Tue, 23 Dec 2025 17:51:20 +0000

At some point, every JavaScript developer asks the same question: why can't I just cancel this async operation?

A user navigates away, a component unmounts, a newer request supersedes an older one - surely there must be a way to stop work that's no longer needed!

In practice, we reach for familiar patterns: Promise.race() with a timeout, ignoring the result when it eventually arrives, or wiring up an AbortController and assuming the problem is solved. Often this appears to work, until the application starts leaking resources, performing late side effects, or behaving inconsistently under load.

The underlying, fundamental issue is: JavaScript does not provide task cancellation as a primitive. Once asynchronous work has been scheduled, there is no general mechanism to forcibly stop it. Promises, callbacks, and async functions represent results and continuations, not ownership of the underlying execution.

This creates a mismatch between intent and reality: developers think in terms of "stopping work", but the language operates in terms of letting work run to completion and optionally reacting to its outcome. As a result, many so-called cancellation techniques merely stop waiting for a result rather than stopping the work itself.

Understanding this gap is essential, because it explains much of JavaScript's async behavior: why promises can't be cancelled, why timeouts don't halt execution, and why AbortController is designed as a signaling mechanism instead of a kill switch. Once that model is clear, the limitations around cancellation stop feeling accidental - they follow directly from how JavaScript executes code.

Cancellation vs Timeout vs Failure

One reason cancellation is so often misunderstood in JavaScript is that it gets conflated with two very different concepts: timeouts and failures. All three may result in "this operation didn't produce a value", but they describe fundamentally different situations.

Cancellation: "I no longer want this"

Cancellation is an external decision. The operation itself may be perfectly healthy and capable of completing, but something outside of it - user input, application state, navigation, or a newer request - has made the result irrelevant.

Importantly, cancellation says nothing about correctness. The operation did not fail. It was simply asked to stop because its result is no longer needed.

In well-designed systems, cancellation is expected and routine, not exceptional.

Timeout: "I stopped waiting"

A timeout does not cancel work. It only limits how long a caller is willing to wait for a result.

In JavaScript, timeouts are commonly implemented using Promise.race():

await Promise.race([
  doWork(),
  timeout(1000)
]);

When the timeout wins the race, the awaiting code resumes, but doWork() continues running. Any side effects it performs will still happen. Any resources it holds will remain allocated until it finishes or cleans up on its own.

Today, most modern APIs accept an AbortSignal instead. This improves resource cleanup and intent signaling, but it does not change the fundamental model: aborting is still cooperative, and only affects code that opts in.

This distinction is easy to miss because the caller regains control, creating the illusion that the work has stopped. In reality, the timeout merely stopped observing the result.

Failure: "Something went wrong"

Failures describe internal problems: network errors, invalid input, logic bugs, unavailable resources. They are usually represented as rejected promises or thrown errors.

Unlike cancellation, failures are not intentional. They indicate that the operation could not complete successfully even if its result was still desired.

Treating cancellation as a failure often leads to awkward error handling. Code starts catching “errors” that are not errors at all, or suppressing failures because they might just be cancellations. Over time, real failures become harder to distinguish from normal control flow.

Why this distinction matters

In JavaScript APIs, timeouts and failures are frequently overloaded to stand in for cancellation. This works superficially, but it obscures intent and pushes responsibility onto the caller to guess what actually happened.

Once you separate these concepts, a pattern emerges: JavaScript is good at expressing waiting and failure, but it has no built-in notion of stopping work. Everything that looks like cancellation is either a timeout, an ignored result, or a cooperative protocol layered on top.

Why Promises Can't Be Cancelled

When developers ask why cancellation is hard in JavaScript, what they usually mean is: why can't I cancel a Promise? After all, promises are the foundation of async/await, and most asynchronous work is expressed in terms of them. If promises represented "tasks", cancellation would seem straightforward.

But promises were never designed to model tasks.

Promises represent results, not execution

A promise is a placeholder for a value that will be available in the future. It says nothing about how that value is produced, or even whether there is ongoing work associated with it. By the time you have a promise, the underlying operation may already be finished, in progress, or shared with other consumers.

This distinction is subtle but crucial: a promise does not own the work that led to it.

Once created, a promise must eventually settle - either fulfilled or rejected. There is no third state for "abandoned" or "cancelled", because that would break the core guarantee that promises make: if you have a reference to one, you can reliably attach handlers and eventually observe an outcome.

The "cancel a promise" fallacy

Imagine a hypothetical .cancel() method on promises. What would it actually do?

Consider this:

const p = fetchData();

p.then(render);
p.then(cacheResult);

If one consumer calls p.cancel(), what happens to the others? Should their handlers stop running? Should the promise reject? With what error? And what if a third consumer attaches a .then() after cancellation?

These questions don't have consistent answers without introducing global side effects. Promises are intentionally shareable and composable, cancellation would make their behavior depend on who else is observing them.

This is why cancellation doesn't fit as a method on the promise itself. Cancellation is about controlling work, while promises are about observing outcomes.

What would break if promises were cancellable

Making promises cancellable would ripple through the entire async ecosystem:

Shared promises would become fragile, since any consumer could affect others.
Memoization and caching would be unsafe - cached promises could be cancelled by accident.
async/await would lose its simple mental model, because awaiting a promise would no longer guarantee eventual completion.

In other words, cancellation would introduce hidden coupling between otherwise independent pieces of code.

Why cancellation had to live elsewhere

Earlier libraries experimented with cancellable promises, and the idea even surfaced during early standardization discussions. The conclusion was consistent: cancellation is not a property of the promise, but a protocol between the caller and the callee.

That protocol needs a separate channel: something that can be passed around, observed, and acted upon - without undermining the semantics of promises themselves. This is why modern JavaScript models cancellation as a signal, not as an operation on the promise.

Once you see promises as immutable views over future values rather than handles to running tasks, their lack of cancellation stops looking like an omission. It's a boundary that keeps asynchronous code predictable and composable.

What AbortController Really Is

If promises can't be cancelled, how do we actually stop or control asynchronous work in JavaScript? That's where AbortController comes in. Understanding what it really does - and what it cannot do - is key to designing cancellation-aware code.

AbortController as a signaling mechanism

AbortController is essentially a messenger. It allows one piece of code to notify others that a task should no longer continue. It does this via an AbortSignal:

const controller = new AbortController();
const signal = controller.signal;

fetch(url, { signal })
  .then(response => console.log('Fetched!', response))
  .catch(err => {
    if (err.name === 'AbortError') {
      console.log('Fetch was aborted');
    } else {
      console.error(err);
    }
  });

// Later, trigger abort
controller.abort();

Here, controller.abort() doesn't magically stop every line of JavaScript. Instead, it informs any cooperating API - in this case, fetch - that the work is no longer desired. fetch responds by rejecting its promise with an AbortError and closing the underlying network connection. That's all that happens automatically.

What AbortController can do

Signal intent: Any consumer that observes the signal can react.
Enable resource cleanup: APIs like fetch or streams can close connections, release handles, or stop producing data.
Propagate cancellation: Signals can be passed down through multiple layers of an API call chain, allowing higher-level code to request termination of lower-level operations.

Essentially, AbortController provides a cooperative cancellation protocol. Consumers must opt in and decide how to respond.

What AbortController cannot do

Stop arbitrary JavaScript execution: CPU-bound loops, synchronous functions, or other work will continue running until completion unless they explicitly check the signal.
Enforce cleanup automatically: Only the code that responds to the signal can free resources or terminate tasks.
Cancel promises generically: It does not magically cancel the underlying promise, it only signals intent to abort.

Abort is cooperative by design

The cooperative nature of AbortController is intentional:

It avoids breaking shared state or running code unexpectedly.
It preserves the run-to-completion semantics of JavaScript.
It gives API authors flexibility in how they respond to abort signals, rather than imposing one-size-fits-all behavior.

For example, consider a long-running computation:

async function compute(signal) {
  let i = 0;
  while (i < 1e9) {
    if (signal.aborted) {
      console.log('Computation aborted');
      return;
    }
    i++;
  }
  return i;
}

Without explicitly checking signal.aborted, there's no way to stop this computation. The signal doesn't “kill” the function, it merely provides a way for the function to notice it should exit early.

Resource Cleanup vs Task Termination

A common misconception in JavaScript cancellation is thinking that signalling a task to abort automatically stops all work. In reality, there's a crucial distinction between stopping a task and cleaning up resources, and understanding it is essential to writing robust asynchronous code.

Stopping work vs cleaning up

When you call controller.abort() on an AbortController, the APIs that observe the signal typically release resources:

fetch closes the underlying network connection.
Streams stop producing data and can free buffers.
Database or file handles may be closed if the API supports abort signals.

This is what "resource cleanup" means: the system ensures that things like sockets, memory buffers, or file descriptors are not left dangling. Cleanup is essential to prevent memory leaks, connection exhaustion, or other subtle bugs.

However, resource cleanup does not automatically stop all ongoing work. Any CPU-bound computation, synchronous logic, or code outside cooperative APIs continues running until it naturally completes.

Why JavaScript focuses on cleanup, not termination

JavaScript's execution model enforces run-to-completion: once a function begins, it will run to the end of its current synchronous block. The event loop does not allow preemptive interruption. As a result:

Forcefully killing a function mid-execution would risk leaving shared state inconsistent.
Partial side effects (like partially updated DOM or partially written files) could corrupt the system.
Memory safety and predictable execution would be compromised.

Instead, JavaScript emphasizes cooperative patterns, where code voluntarily checks for cancellation and exits cleanly. AbortController fits this model: it signals intent, and APIs or functions decide how to respond.

AbortController as a cleanup trigger

Most modern APIs that support AbortSignal focus on clean termination of resources:

const controller = new AbortController();
const signal = controller.signal;

const stream = someStreamAPI({ signal });

controller.abort(); // triggers cleanup

Here, stream may stop producing data, close internal buffers, and release file descriptors. Any consuming code can then notice the abort and stop processing further. The work is not forcibly terminated: instead, the API and the caller cooperate to exit safely.

To stop CPU-intensive tasks or custom computations, developers must check signal.aborted periodically, see the earlier example in the Abort is cooperative by design section.

This combination of cleanup + cooperative exit is the pattern JavaScript provides for cancellation. It preserves safety while allowing developers to reclaim resources and stop long-running operations gracefully.

Why JavaScript Cannot Forcefully Stop Code

One of the reasons cancellation in JavaScript works differently than in other languages is how the language executes code. Understanding this is key to realizing why AbortController cannot magically "kill" a function or promise.

No preemption in JavaScript

JavaScript runs on a single-threaded event loop. Each function runs to completion before the next task is executed:

function busyLoop() {
  for (let i = 0; i < 1e9; i++) {
    // CPU-bound work
  }
  console.log('Done!');
}

busyLoop();
console.log('This runs only after busyLoop finishes');

While busyLoop() is running, the event loop cannot interrupt it. There is no mechanism to inject code that forcibly stops execution mid-block. This design makes JavaScript predictable, but it also means cancellation must be cooperative.

Why forceful termination would be unsafe

Imagine if JavaScript allowed arbitrary termination:

Shared mutable state could be left inconsistent:

obj.count++;
// terminated here -> obj.count never incremented properly

Partial updates could corrupt data:

arr.push(newItem);
// terminated here -> arr in inconsistent state

Promises could never be reliably observed:

Consumers expecting a value might never get notified if the underlying task disappears mid-execution.

Because JavaScript encourages shared objects and composable async code, preemptive termination is inherently unsafe.

Why Web Workers don't fundamentally change this

Some developers think: "I can just run CPU work in a Web Worker and terminate it." Technically, you can:

const worker = new Worker('worker.js');
worker.terminate(); // kills the worker thread

But this is process-level termination, not task-level cancellation:

terminate() stops all code in the worker, regardless of what it's doing.
There is no granular control over individual tasks or promises inside the worker.
Messages in transit may be lost, leaving partially processed data.

Web Workers provide a way to isolate tasks that might need to be forcibly killed, but inside the main thread, JavaScript still cannot preempt code safely. This is why cooperative signals like AbortController are the preferred pattern: they let code exit voluntarily while cleaning up resources.

How Other Languages Model Cancellation

JavaScript's cooperative cancellation model can feel limiting, but looking at other languages helps explain why. Different environments make different trade-offs between safety, control, and composability.

Cooperative cancellation (Go, Rust async)

Languages like Go and Rust provide explicit mechanisms for cooperative cancellation:

Go: context propagation

ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()

select {
case <-doWork(ctx):
    fmt.Println("Completed")
case <-ctx.Done():
    fmt.Println("Cancelled")
}

ctx is passed explicitly to all functions that might need to cancel.
The work itself checks the context and exits early.
Resources can be cleaned up in a structured way.
This is conceptually similar to AbortController in JS: a signal passed down the call chain, requiring cooperation.

Rust: async cancellation

Futures in Rust can be polled with a cancellation signal.
Tasks yield control points where the runtime can stop work if the signal indicates cancellation.
Again, the task itself must check the signal, it cannot be killed mid-instruction.

The key idea is cooperative cancellation: the runtime provides a signal, and the code decides how and when to exit.

Structured concurrency (Kotlin, Swift)

Modern languages like Kotlin (coroutines) and Swift (async/await) take this further with structured concurrency:

Tasks are tied to a parent scope.
When a parent cancels, all child tasks receive a cancellation signal.
This ensures that async work is bounded, predictable, and easy to clean up.

Example in Kotlin:

val job = launch {
    val child = launch {
        repeat(1000) { i ->
            println("Working $i")
            delay(100)
        }
    }
    delay(500)
    child.cancel() // cooperative cancellation
}

The pattern enforces lifecycle and cancellation rules without unsafe preemption.

Preemptive cancellation (threads)

Other environments, like Java or C#, offer preemptive cancellation via threads: a thread can be interrupted or aborted mid-execution. But this introduces complex safety issues:

Shared mutable state may be left inconsistent.
Locks or resources may never be released.
Libraries often discourage forced thread termination for safety reasons.

JavaScript avoids this entirely on the main thread, because the language relies on shared memory and single-threaded execution. Forceful termination would compromise stability and predictability.

Takeaways for JavaScript

Cooperative signals, like AbortController, are the closest equivalent to cancellation in Go, Rust, or Kotlin.
JavaScript deliberately avoids preemption to maintain safety and simplicity.
Many "gotchas" in JS cancellation are the same trade-offs other languages have to manage when they choose safety over brute-force control.

Practical Patterns for Cancellation in JS Today

Understanding the constraints of cancellation is one thing, applying them effectively is another. Modern JavaScript provides tools and patterns to handle cancellation safely and predictably, mostly built around AbortController and cooperative design.

Passing AbortSignal everywhere

A good practice is to design APIs to accept an AbortSignal as a first-class parameter:

async function fetchWithSignal(url, signal) {
  const response = await fetch(url, { signal });
  const data = await response.json();
  return data;
}

Callers can then create a controller and abort if needed:

const controller = new AbortController();
const signal = controller.signal;

fetchWithSignal('/api/data', signal)
  .then(data => console.log(data))
  .catch(err => {
    if (err.name === 'AbortError') console.log('Request cancelled');
    else console.error(err);
  });

// Later
controller.abort();

This pattern allows cancellation to propagate through multiple layers of API calls and ensures resource cleanup where supported.

Making long-running work abortable

For CPU-bound tasks or loops, you need to check the signal explicitly. Splitting work into chunks with occasional checks allows cooperative cancellation:

async function heavyComputation(signal) {
  let result = 0;
  for (let i = 0; i < 1e9; i++) {
    if (signal.aborted) {
      console.log('Computation aborted');
      return;
    }
    result += i;
    if (i % 1e6 === 0) await Promise.resolve(); // yield to event loop
  }
  return result;
}

Checking signal.aborted lets the function exit early.
Yielding occasionally prevents blocking the event loop for too long.

This approach mirrors structured concurrency in other languages: tasks cooperate with cancellation and remain responsive.

Designing cancellation-aware APIs

When building libraries or components:

Accept an AbortSignal instead of inventing custom cancellation flags.
Document what cancellation does:
- Does it stop network requests?
- Does it free memory or file handles?
- Does it stop computation?
Avoid hidden background work:
- Ensure that cancelled tasks do not continue modifying shared state.
Propagate signals through all dependent operations:
- If a high-level operation is aborted, all sub-operations should observe the same signal.

Example:

async function processBatch(batch, signal) {
  const results = [];
  for (const item of batch) {
    if (signal.aborted) break;
    results.push(await processItem(item, signal));
  }
  return results;
}

This guarantees predictable cancellation without leaving partial operations or resources dangling.

Combining with React or Node.js

React: Pass AbortSignal to fetch or long-running operations inside useEffect, and abort in cleanup functions.
Node.js: Many APIs like fs.promises streams or fetch (via node-fetch or native support) accept signals. Use them to prevent lingering resource usage during server shutdowns or request cancellation.

By consistently using cooperative patterns, signals, and well-designed APIs, you can implement robust cancellation in JavaScript without breaking promises, leaking resources, or creating unsafe preemption.

Conclusion: Stop Trying to “Kill” Promises

Cancellation in JavaScript is fundamentally different from what developers coming from other languages might expect. Promises are immutable placeholders for future values, not handles to running tasks. There is no built-in mechanism to forcibly stop work, and trying to treat them that way leads to fragile, unpredictable code.

Instead, JavaScript provides cooperative cancellation via AbortController and AbortSignal. These tools allow code to:

Signal that work is no longer needed
Clean up resources like network connections, streams, or file handles
Enable tasks to exit early if they opt in

The key takeaway is that cancellation is intent, not enforcement. Work only stops when the code performing it checks the signal and responds. CPU-bound loops, synchronous computations, or code outside cooperative APIs continue running until they voluntarily exit.

By embracing this model:

APIs become more predictable and composable
Resource leaks and side effects are minimized
Async code can handle user-driven interruptions cleanly

Ultimately, cancellation in JavaScript is less about killing promises and more about designing your tasks to be responsive and cooperative. Understanding this distinction allows developers to write robust, maintainable asynchronous code without fighting the language's execution model.

The Database Zoo: Vector Databases and High-Dimensional Search

Gabor Koos — Tue, 25 Nov 2025 20:33:54 +0000

This post is part of The Database Zoo: Exotic Data Storage Engines , a series exploring purpose-built databases engineered for specific workloads. Each post dives into a different type of specialized engine, explaining the problem it solves, the design decisions behind its architecture, how it stores and queries data efficiently, and real-world use cases. The goal is to show not just what these databases are, but why they exist and how they work under the hood.

Introduction

Vector embeddings have quietly become one of the most important data types in modern systems. Every LLM application, recommendation engine, semantic search feature, image similarity tool, fraud detector, and "find me things like this" workflow ultimately boils down to the same operation: convert some input into a high-dimensional vector, then search for its nearest neighbours.

At small scales this is straightforward, but as the volume of data and dimensionality grow, it's the sort of problem that turns general-purpose databases into smoke.

Vector search workloads have very different characteristics from classical OLTP (Online Transaction Processing) or document-store workloads:

You're not querying for exact values, you're querying for semantic similarity.
The data lives in hundreds to thousands of dimensions, where traditional indexing breaks down.
The storage footprint is huge, and compression becomes essential.
The ingestion rate is often tied to model pipelines continuously producing new embeddings.
Queries frequently combine vector similarity with structured filters ("find the closest items, but only in category X, location Y").

This is why vector databases exist. They're not "databases that store vectors", they're purpose-built engines optimized around approximate nearest neighbour (ANN) search, distance-based retrieval, metadata filtering, high-throughput ingestion, and lifecycle management for embeddings at scale.

In this article we'll walk through how vector databases are structured, why they look the way they do, what indexing techniques they rely on, how queries are executed, what trade-offs matter, and where these systems shine or struggle in practice. By the end, you should have a mental model strong enough to reason about algorithm choice, storage design, performance tuning, and architectural decisions for any vector search workload.

Why General-Purpose Databases Struggle

Even the most robust relational and document-oriented databases stumble when faced with vector search workloads. The patterns and scale of high-dimensional embeddings expose fundamental limitations in systems designed for exact-match or low-dimensional indexing.

High-Dimensional Similarity Queries

Vector search is fundamentally about similarity, not equality. Unlike a traditional SQL query that looks for a value or range, a vector query typically asks:

Which vectors are closest to this one according to some distance metric?

General-purpose databases are optimized for exact-match or low-dimensional range queries. Indexes like B-trees or hash maps fall apart in high dimensions - a phenomenon known as the curse of dimensionality. As dimensions increase, nearly all points appear equidistant, making scans and traditional indexes increasingly ineffective.

Approximate Nearest Neighbour Workload

At scale, brute-force searches across millions or billions of embeddings are computationally infeasible:

Each query requires computing distances (e.g., cosine similarity, Euclidean distance) to every candidate vector.
For high-dimensional vectors (often 128–2048 dimensions or more), this is expensive both in CPU/GPU cycles and memory bandwidth.
General-purpose stores offer no native acceleration or pruning strategies, leaving applications to implement expensive application-side filtering.

Approximate Nearest Neighbour (ANN) algorithms solve this, but general-purpose databases do not implement them. Without ANN, even modest datasets produce query latencies measured in seconds or minutes rather than milliseconds.

Metadata Filtering and Hybrid Queries

Vector searches rarely occur in isolation. Most real-world applications require hybrid queries, such as:

"Find items similar to this embedding, but only within category X or date range Y."
"Retrieve the nearest vectors for this query, filtered by tags or user attributes."

Relational databases can filter metadata efficiently, but they cannot combine these filters with high-dimensional distance calculations without either brute-force scanning or complex application-level pipelines.

Ingestion at Scale

Modern vector pipelines can continuously produce embeddings:

Models generate embeddings in real-time for new documents, images, or user interactions.
Millions of embeddings per day can quickly saturate storage and indexing pipelines.
General-purpose databases lack optimized write paths for high-dimensional vectors, often requiring bulky serialization and losing performance at scale.

Storage and Compression Challenges

Embeddings are dense, high-dimensional floating-point vectors. Naive storage in relational tables or JSON documents results in:

Large storage footprints (hundreds of GB to TBs for millions of vectors).
Poor cache locality and memory efficiency.
Slow scan performance, especially if vectors are stored in row-major formats instead of columnar or block-aligned layouts optimized for similarity search.

Specialized vector databases implement compression, quantization, or block-oriented storage schemes to reduce disk and memory usage while maintaining query accuracy.

Summary

General-purpose relational and document stores are reliable for exact-match or low-dimensional queries, but vector search workloads present unique challenges:

High-dimensional, similarity-based queries that break traditional indexes.
Expensive distance computations across large datasets.
Hybrid queries combining vector similarity with metadata filtering.
High ingestion rates tied to embedding pipelines.
Storage and memory efficiency demands.

These challenges justify the emergence of vector databases: purpose-built engines designed to efficiently store, index, and query embeddings while supporting metadata filters, high throughput, and scalable approximate nearest neighbour algorithms.

Core Architecture

Vector databases are built to handle high-dimensional embeddings efficiently, addressing both the computational and storage challenges that general-purpose systems cannot. Their architecture revolves around optimized storage, indexing, and query execution tailored to similarity search workloads.

Storage Layouts

Unlike relational databases, vector databases adopt storage formats that prioritize both memory efficiency and fast distance computations:

Dense vector storage: Embeddings are stored as contiguous arrays of floats or quantized integers, improving cache locality and enabling SIMD or GPU acceleration.
Block-aligned layouts: Vectors are grouped in blocks to facilitate batch computation of distances, reduce I/O overhead, and leverage vectorized hardware instructions.
Hybrid memory and disk storage: Recent or frequently queried vectors may reside in RAM for low-latency access, while older or less critical vectors are persisted on disk with fast retrieval mechanisms.
Quantization & compression: Techniques like product quantization (PQ), scalar quantization, or HNSW-based pruning reduce storage size and accelerate distance calculations with minimal loss in accuracy.

These storage choices allow vector databases to scale to billions of embeddings without sacrificing query performance.

Indexing Strategies

Efficient indexing is critical for fast similarity search:

Approximate Nearest Neighbour (ANN) structures: Indexes like HNSW (Hierarchical Navigable Small Worlds), IVF (Inverted File Index), or PQ-based graphs enable sub-linear search times in high-dimensional spaces.
Metadata-aware indexes: Secondary indexes track categorical or temporal attributes, allowing hybrid queries that filter embeddings by tags before performing vector distance computations.
Multi-level indexes: Some systems maintain coarse-grained partitioning first (e.g., via clustering) and then fine-grained graph traversal within partitions, balancing query speed and memory usage.
Dynamic updates: Indexes are designed to handle real-time insertion of new vectors without full rebuilds, maintaining responsiveness under high ingestion workloads.

Together, these structures allow vector databases to perform ANN searches over millions or billions of vectors with millisecond-scale latency.

Query-Aware Compression

Vector databases often store embeddings in compressed formats, enabling efficient computation without fully decompressing:

Product quantization (PQ): Splits each vector into sub-vectors and encodes each sub-vector with a compact codebook. Distance calculations can then be approximated directly in the compressed domain.
Binary hashing / Hamming embeddings: High-dimensional vectors are converted into binary codes to allow extremely fast distance computations using Hamming distance.
Graph-aware compression: Index structures like HNSW can store edge lists and vector representations in quantized form, reducing memory footprint while preserving search quality.

These techniques reduce both RAM usage and disk I/O, critical for large-scale vector datasets.

Hybrid Filtering and Search

Real-world applications often require a combination of vector similarity and structured filtering:

Filtered ANN search: Indexes can integrate metadata constraints (e.g., category, date, owner) to prune candidate vectors before computing distances.
Multi-modal queries: Some databases support queries that combine multiple vectors or modalities (e.g., image + text embeddings) while respecting filter criteria.
Lazy evaluation: Distance computations are performed only on a subset of candidates returned from the ANN index, balancing speed and accuracy.

This hybrid approach ensures that vector databases are not just fast for raw similarity search but practical for complex application queries.

Summary

The core architecture of vector databases relies on:

Contiguous, cache-friendly storage for dense embeddings.
ANN-based indexing structures for sub-linear high-dimensional search.
Query-aware compression and quantization to reduce memory and computation costs.
Metadata integration and hybrid filtering to support real-world application requirements.

By combining these elements, vector databases achieve fast, scalable similarity search while managing storage, memory, and computational efficiency in ways that general-purpose databases cannot match.

Query Execution and Patterns

Vector databases are designed around the unique demands of similarity search in high-dimensional spaces. Queries typically involve finding the closest vectors to a given embedding, often combined with filters or aggregations. Efficient execution requires careful coordination between indexing structures, storage layouts, and distance computation strategies.

Common Query Types

k-Nearest Neighbor (k-NN) Search

Fetch the top k vectors most similar to a query embedding, according to a distance metric (e.g., cosine similarity, Euclidean distance, inner product).

Example: Finding the 10 most similar product images to a new upload.

Optimized by: ANN indexes (HNSW, IVF, PQ) that prune the search space and avoid scanning all vectors.

Range / Radius Search

Retrieve all vectors within a specified distance threshold from the query embedding.

Example: Returning all text embeddings within a similarity score > 0.8 for semantic search.

Optimized by: Multi-level index traversal with early pruning based on approximate distance bounds.

Filtered / Hybrid Queries

Combine vector similarity search with structured filters on metadata or attributes.

Example: Find the closest 5 product embeddings in the "electronics" category with a price < $500.

Optimized by: Pre-filtering candidates using secondary indexes, then performing ANN search on the reduced set.

Batch Search

Execute multiple vector queries simultaneously, often in parallel.

Example: Performing similarity searches for hundreds of user queries in a recommendation pipeline.

Optimized by: Vectorized computation leveraging SIMD or GPU acceleration, and batching index traversal.

Query Execution Strategies

Vector databases translate high-level queries into efficient execution plans tailored for high-dimensional search:

Candidate Selection via ANN Index

The index identifies a subset of promising vectors rather than scanning all embeddings.
HNSW or IVF partitions guide the search toward relevant regions in the vector space.

Distance Computation

Exact distances are computed only for candidate vectors.
Some systems perform computations directly in the compressed domain (PQ or binary embeddings) to reduce CPU cost.

Parallel and GPU Execution

Queries are often executed in parallel across index partitions, CPU cores, or GPU threads.
Large-scale search over millions of vectors benefits significantly from hardware acceleration.

Hybrid Filtering

Metadata or category filters are applied either before or during candidate selection.
Reduces unnecessary distance calculations and ensures relevance of results.

Dynamic Updates

Indices are maintained dynamically, allowing real-time insertion of new vectors without full rebuilds.
Ensures query latency remains low even as the dataset grows continuously.

Example Query Patterns

Single vector search: Find the top 10 most similar embeddings to a query image.
Filtered similarity: Return nearest neighbors for a text embedding in a specific language or category.
Batch recommendation: Compute top-N recommendations for hundreds of users simultaneously.
Hybrid multi-modal search: Retrieve the closest matches to a query vector that also meet attribute constraints (e.g., price, date, tags).

Key Takeaways

Vector database queries differ from traditional relational lookups:

Most searches rely on approximate distance computations over high-dimensional embeddings.
Efficient query execution hinges on ANN indexes, compressed storage, and hardware acceleration.
Real-world applications often combine vector similarity with structured metadata filtering.
Batch and hybrid query support is essential for scalable recommendation, search, and personalization pipelines.

By aligning execution strategies with the structure of embedding spaces and leveraging specialized indexes, vector databases achieve sub-linear search times and millisecond-scale response, even for billions of vectors.

Popular Vector Database Engines

Several purpose-built vector databases have emerged to handle the challenges of high-dimensional similarity search, each optimized for scale, query latency, and integration with other data systems. Here, we highlight a few widely adopted engines:

Milvus

Overview:

Milvus is an open-source vector database designed for large-scale similarity search. It supports multiple ANN index types, high-concurrency queries, and integration with both CPU and GPU acceleration.

Architecture Highlights:

Storage engine: Hybrid approach with in-memory and disk-based vector storage.
Indexes: Supports HNSW, IVF, PQ, and binary indexes for flexible trade-offs between speed and accuracy.
Query execution: Real-time and batch similarity search with support for filtered queries.
Scalability: Horizontal scaling with Milvus cluster and sharding support.

Trade-offs:

Excellent for large-scale, real-time vector search workloads.
Requires tuning index types and parameters to balance speed and recall.
GPU acceleration improves throughput but increases infrastructure complexity.

Use Cases:

Recommendation engines, multimedia search (images, videos), NLP semantic search.

Weaviate

Overview:

Weaviate is an open-source vector search engine with strong integration for structured data and machine learning pipelines. It provides a GraphQL interface and supports semantic search with AI models.

Architecture Highlights:

Storage engine: Combines vectors with structured objects for hybrid queries.
Indexes: HNSW-based ANN indexes optimized for low-latency retrieval.
Query execution: Integrates filtering on object properties with vector similarity search.
ML integration: Supports on-the-fly embedding generation via built-in models or external pipelines.

Trade-offs:

Excellent for applications combining vector search with structured metadata.
Less optimized for extreme-scale datasets compared to Milvus or FAISS clusters.
Query performance can depend on the complexity of combined filters.

Use Cases:

Semantic search in knowledge bases, enterprise search, AI-powered chatbots.

Pinecone

Overview:

Pinecone is a managed vector database service with a focus on operational simplicity, low-latency search, and scalability for production workloads.

Architecture Highlights:

Storage engine: Fully managed cloud infrastructure with automated replication and scaling.
Indexes: Provides multiple ANN options, abstracting complexity from users.
Query execution: Automatic vector indexing, hybrid search, and batch queries.
Monitoring & reliability: SLA-backed uptime, automatic failover, and consistency guarantees.

Trade-offs:

Fully managed, reducing operational overhead.
Less flexibility in index tuning compared to open-source engines.
Cost scales with dataset size and query volume.

Use Cases:

Real-time recommendations, personalization engines, semantic search for enterprise applications.

FAISS

Overview:

FAISS is a library for efficient similarity search over dense vectors. Unlike full database engines, it provides the building blocks to integrate ANN search into custom systems.

Architecture Highlights:

Storage engine: In-memory with optional persistence.
Indexes: Supports IVF, HNSW, PQ, and combinations for memory-efficient search.
Query execution: Highly optimized CPU and GPU kernels for fast distance computation.
Scalability: Designed for research and production pipelines with custom integrations.

Trade-offs:

Extremely fast and flexible for custom applications.
Lacks built-in metadata storage, transaction support, or full DB features.
Requires additional engineering for distributed deployment and persistence.

Use Cases:

Large-scale research experiments, AI model embeddings search, custom recommendation systems.

Other Notable Engines

VESPA: Real-time search engine with support for vector search alongside structured queries.
Qdrant: Open-source vector database optimized for hybrid search and easy integration with ML workflows.
RedisVector / RedisAI: Adds vector similarity search capabilities to Redis, allowing hybrid queries and fast in-memory search.

Key Takeaways

While each vector database has its strengths and trade-offs, they share common characteristics:

Vector-focused storage: Optimized for ANN search, often in combination with compressed or quantized representations.
Hybrid query support: Ability to combine similarity search with structured metadata filters.
Scalability: From in-memory single-node searches to distributed clusters handling billions of embeddings.
Trade-offs: Speed, accuracy, and cost must be balanced based on workload, dataset size, and latency requirements.

Selecting the right vector database depends on use case requirements: whether you need full operational simplicity, extreme scalability, hybrid queries, or tight ML integration. Understanding these distinctions allows engineers to choose the best engine for their high-dimensional search workloads, rather than relying on general-purpose databases or custom implementations.

Trade-offs and Considerations

Vector databases excel at workloads involving high-dimensional similarity search, but their optimizations come with compromises. Understanding these trade-offs is essential when selecting or designing a vector database for your application.

Accuracy vs. Latency

Approximate nearest neighbor (ANN) indexes provide sub-linear query time, enabling fast searches over billions of vectors.
However, faster indexes (like HNSW or IVF+PQ) may return approximate results, potentially missing the exact nearest neighbors.
Engineers must balance search speed with recall requirements. In some applications, slightly lower accuracy is acceptable for much faster queries, while others require near-perfect matches.

Storage Efficiency vs. Query Speed

Many vector databases use quantization, compression, or dimension reduction to reduce storage footprint.
Aggressive compression lowers disk and memory usage but can increase query latency or reduce search accuracy.
Choosing the right index type and vector representation is critical: dense embeddings may need more storage but allow higher accuracy, while compact representations reduce cost but may degrade results.

Hybrid Search Trade-offs

Modern vector databases support filtering on structured metadata alongside vector similarity search.
Hybrid queries can add complexity, increasing latency or requiring additional indexing.
Designers must weigh the benefit of richer queries against the performance impact of combining vector and structured filters.

Scalability Considerations

Some engines (e.g., Milvus, Pinecone) scale horizontally via sharding, replication, or GPU clusters.
Distributed systems add operational complexity, including network overhead, consistency management, and fault tolerance.
Smaller datasets may be efficiently handled in a single-node or in-memory setup (e.g., FAISS), avoiding the overhead of distributed clusters.

Operational Complexity

Open-source vector databases require domain knowledge for tuning index parameters, embedding storage, and query optimization.
Managed services like Pinecone reduce operational burden but limit low-level control over index configurations or hardware choices.
Backup, replication, and monitoring strategies vary across engines; engineers must plan for persistence and reliability in production workloads.

Embedding Lifecycle and Updates

Vector databases often optimize for append-heavy workloads, where vectors are rarely updated.
Frequent updates or deletions can degrade index performance or require expensive rebuilds.
Use cases with dynamic embeddings (e.g., user profiles in recommendation systems) require careful strategy to maintain query performance.

Cost vs. Performance

GPU acceleration improves throughput and lowers latency but increases infrastructure cost.
Distributed storage and indexing also add operational expense.
Decisions around performance, recall, and hardware resources must align with application requirements and budget constraints.

Key Takeaways

Vector databases excel when workloads involve high-dimensional similarity search at scale, but no single engine fits every scenario.
Engineers must balance accuracy, latency, storage efficiency, scalability, operational complexity, and cost.
Consider query patterns, update frequency, hybrid filtering, and embedding characteristics when selecting an engine. Understanding these trade-offs ensures that vector search applications deliver relevant results efficiently, while avoiding bottlenecks or excessive operational overhead.

Use Cases and Real-World Examples

Vector databases are not just theoretical tools, they solve practical, high-dimensional search problems across industries. Below are concrete scenarios illustrating why purpose-built vector search engines are indispensable:

Semantic Search and Document Retrieval

Scenario: A company wants to allow users to search large text corpora or knowledge bases by meaning rather than exact keywords.

Challenges:

High-dimensional embeddings for documents and queries
Large-scale search over millions of vectors
Low-latency responses for interactive applications

Vector Database Benefits:

ANN indexes like HNSW or IVF+PQ enable fast semantic similarity searches.
Filtering by metadata (e.g., document type, date) supports hybrid queries.
Scalable vector storage accommodates ever-growing corpora.

Example: A customer support platform uses Milvus to index millions of support tickets and FAQs. Users can ask questions in natural language, and the system retrieves semantically relevant answers in milliseconds.

Recommendation Systems

Scenario: An e-commerce platform wants to suggest products based on user behavior, item embeddings, or content features.

Challenges:

Generating embeddings for millions of users and products
Real-time retrieval of similar items for personalized recommendations
Hybrid filtering combining vector similarity and categorical constraints (e.g., in-stock, region)

Vector Database Benefits:

Efficient similarity search over large embedding spaces.
Supports filtering by metadata for contextual recommendations.
Handles dynamic updates for new items and changing user preferences.

Example: A streaming service leverages FAISS to provide real-time content recommendations, using vector embeddings for movies, shows, and user preferences to improve engagement.

Image, Audio, and Video Search

Scenario: A media platform wants users to search for images or video clips using example content instead of keywords.

Challenges:

High-dimensional embeddings for visual or audio features
Similarity search across millions of media items
Low-latency response for interactive exploration

Vector Database Benefits:

Stores and indexes embeddings from CNNs, transformers, or other feature extractors.
ANN search enables fast retrieval of visually or auditorily similar content.
Scales with GPU acceleration for massive media collections.

Example: An online fashion retailer uses Pinecone to allow users to upload photos of clothing items and find visually similar products instantly.

Fraud Detection and Anomaly Detection

Scenario: Financial institutions need to detect suspicious transactions or patterns in real-time.

Challenges:

Embeddings representing transaction patterns or user behavior
Continuous ingestion of high-dimensional data streams
Detection of anomalies or unusual similarity patterns among accounts

Vector Database Benefits:

ANN search identifies nearest neighbors in embedding space quickly.
Helps detect outliers or clusters of suspicious activity.
Can integrate metadata filters to limit searches to relevant contexts.

Example: A bank uses Milvus to monitor transaction embeddings, flagging unusual patterns that deviate from typical user behavior, enabling early fraud detection.

Conversational AI and Chatbots

Scenario: A company wants to enhance a chatbot with contextual understanding and retrieval-augmented generation.

Challenges:

Large embeddings for conversational history, documents, or FAQs
Matching user queries to the most relevant context for AI response generation
Low-latency retrieval in live interactions

Vector Database Benefits:

Fast similarity search to find relevant passages or prior interactions.
Supports hybrid filtering for domain-specific context (e.g., product manuals, policies).
Enables scalable, real-time RAG workflows.

Example: A SaaS company integrates Pinecone with a large language model to provide contextual, accurate, and fast answers to user queries, improving support efficiency and satisfaction.

Example Workflow: Building a Semantic Search Engine with Milvus

This section provides a concrete end-to-end example of a vector search workflow, using Milvus to illustrate how data moves from embedding generation to similarity search, highlighting architecture and optimizations discussed earlier.

Scenario

We want to build a semantic search engine for a knowledge base containing 1 million documents. Users will enter natural language queries, and the system will return the most semantically relevant documents.

The workflow covers:

Embedding generation
Vector storage and indexing
Query execution
Hybrid filtering
Retrieval and presentation

Following this workflow demonstrates how a vector database enables fast, accurate similarity search at scale.

Step 1: Embedding Generation

Each document is transformed into a high-dimensional vector using a transformer model (e.g., Sentence-BERT):

from sentence_transformers import SentenceTransformer

model = SentenceTransformer('all-MiniLM-L6-v2')
document_embedding = model.encode("The quick brown fox jumps over the lazy dog")

Key Concepts Illustrated:

Converts unstructured text into fixed-size numeric vectors.
Captures semantic meaning, enabling similarity-based retrieval.
Embeddings are the core data type stored in vector databases.

Step 2: Vector Storage and Indexing

Vectors are stored in Milvus with an ANN index (HNSW):

from pymilvus import connections, FieldSchema, CollectionSchema, DataType, Collection

connections.connect("default", host="localhost", port="19530")

fields = [
    FieldSchema(name="doc_id", dtype=DataType.INT64, is_primary=True),
    FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=384)
]

schema = CollectionSchema(fields, description="Knowledge Base Vectors")
collection = Collection("kb_vectors", schema)

collection.insert([list(range(1_000_000)), embeddings])
collection.create_index("embedding", {"index_type": "HNSW", "metric_type": "COSINE"})

Storage Highlights:

ANN index allows sub-linear similarity search over millions of vectors.
Supports incremental inserts for dynamic document collections.
Efficient disk and memory management for high-dimensional data.

Step 3: Query Execution

A user submits a query:

query_embedding = model.encode("How do I reset my password?")
results = collection.search([query_embedding], "embedding", param={"metric_type":"COSINE"}, limit=5)

Execution Steps:

Transform query into embedding space.
ANN search retrieves nearest neighbors efficiently using HNSW.
Results ranked by similarity score.
Only top-k results returned for low-latency response.

Step 4: Hybrid Filtering

Optionally, filter results by metadata, e.g., document category or publication date:

results = collection.search(
    [query_embedding],
    "embedding",
    expr="category == 'FAQ' && publish_date > '2025-01-01'",
    param={"metric_type":"COSINE"},
    limit=5
)

Highlights:

Combines vector similarity with traditional attribute filters.
Enables precise, context-aware retrieval.
Reduces irrelevant results while leveraging ANN efficiency.

Step 5: Retrieval and Presentation

The system returns document IDs and similarity scores, which are then mapped back to full documents:

for res in results[0]:
    print(f"Doc ID: {res.id}, Score: {res.score}")

Output:

Fast, semantically relevant results displayed to users.
Low latency enables interactive search experiences.
System can scale horizontally with additional nodes or shards for larger datasets.

Key Concepts Illustrated

End-to-end vector workflow: From raw text → embeddings → storage → similarity search → filtered results.
ANN indexes: Provide sub-linear query performance on millions of vectors.
Hybrid filtering: Combines vector similarity with traditional attributes for precise results.
Scalability: Supports incremental inserts, sharding, and distributed deployment.

By following this workflow, engineers can build production-grade semantic search engines, recommendation systems, or retrieval-augmented applications using vector databases like Milvus, Pinecone, or FAISS.

Conclusion

Vector databases are purpose-built engines designed for high-dimensional search, enabling fast and accurate similarity queries over massive datasets. By combining efficient storage, indexing structures like HNSW or IVF, and optimized query execution, they handle workloads that general-purpose databases struggle with.

Understanding the core principles: embedding generation, vector indexing, and approximate nearest neighbor search helps engineers choose the right vector database and design effective semantic search or recommendation systems.

DEV Community: Gabor Koos

Decorating Promises Without Breaking Them

The Goal

The Mechanism

Idempotency

Typing It

Tradeoffs Worth Naming

The Broader Point

Introducing the Fetch Client Chaos Arena

Built With chaos-fetch

Further Reading

Your Debounce Is Lying to You

The Illusion of "Fixed" Behavior

The Companion Code

Problem 1: Race Conditions (aka Stale UI)

Problem 2: Network Failures

Conclusion

Developing and Benchmarking the Same Feature in Node and Go

Implementing Hot Config Reload in Two Runtimes

Runtime model

Concurrency model

In-flight guarantees

Router lifecycle and rebuild mechanics

Validation and rollback boundaries

Stateful middleware behavior

Benchmark Rerun

System and Test Environment (Same Machine as the Old Article)

Current Rerun (Median of 3)

Old Benchmark Reference (from previous post)

What changed?

Why There Is Overhead Even Without Calling /reload

Conclusion

From the Database Zoo to the Database Safari

Using Pagination to Improve GraphQL Performance

Setting Up the Project

Prerequisites

Installation

Running the Server

The Problem: Fetching All Data at Once

Running the Fetch-All Query

Why This Pattern Breaks Down

Naive Offset-Based Pagination

Implementing Offset Pagination

Observing the Improvement

The Hidden Cost of Offsets

Why This Matters in GraphQL APIs

Cursor-Based Pagination

Implementing Cursor-Based Pagination

Querying with Cursors

Why Cursor-Based Pagination Scales Better

Conclusion

Lessons Learned from Running a Privacy-First Disposable Email Service: Insights from nullmail.cc

Design Philosophy

Architecture Overview

The Incident

The Resolution

Lessons Learned

Advanced Asynchronous Patterns in JavaScript

Cancellation Is the Missing Primitive

Timeouts Are a Form of Cancellation

Historical Approach: Promise.race()

Modern Approach: AbortSignal.timeout()

Composing Multiple Signals: AbortSignal.any()

Composing Async Work Without Losing Control

Theory: Why Composition Is Hard

Running Multiple Tasks Concurrently

Handling Partial Failures

Patterns for Predictable Composition

Bounded Concurrency

Theory: Why Concurrency Needs Bounds

Implementing Bounded Concurrency

Example: Fetching Multiple URLs with Limits

Integrating Timeouts and User Cancellation

Key Patterns

Controlled Error Handling

Theory: Separation of Concerns

Handling Partial Failures

Propagating Critical Failures

Patterns for Predictable Error Handling

Practical Observations

Historical Approach: `Promise.race()`

Modern Approach: `AbortSignal.timeout()`

Composing Multiple Signals: `AbortSignal.any()`

From `go.mod` to a Build Plan

`go build`: Producing Artifacts

`go run`: Convenience Without Special Privileges

What `go run` Actually Does

Why `go run` Feels Different

When `go run` Is the Right Tool - and When It Isn't

`go test` and Cached Correctness

Readable Streams and `highWaterMark`

Writable Streams and the `write()` Return Value

Using `pipe()` for Automatic Backpressure