DEV Community: Sheikh Ahnaf Hasan

How a 22-Minute Redis Blip Ate 18 GB of RAM

Sheikh Ahnaf Hasan — Thu, 09 Jul 2026 03:43:15 +0000

A production memory-leak postmortem: two well-behaved libraries, one pathological interaction, and 50 million tiny objects the garbage collector was never allowed to free.

3:36 AM

The email from Railway arrived at 3:36 AM CDT on July 2nd:

Deploy Crashed!
Uh oh. Your deployment for taskqueue crashed within the staging environment.

Annoying, but survivable — staging crashes happen. What didn't add up came later: the production instance of the same service, which never crashed, was now sitting at 14–18 GB of RAM. This is a background worker that normally idles below 100 MB. It stayed at 14–18 GB for six days, working normally the entire time, until we redeployed it — at which point it dropped back to 21 MB like nothing had ever happened.

The memory graph looked like this: flat for weeks, a near-vertical cliff up to 14 GB, a six-day plateau, then a cliff back down at the redeploy. Not a slow leak. A step function.

Slow leaks are easy to reason about — something grows a little with every request. But what allocates fourteen gigabytes in minutes and then just… stops? And why did the memory never come back down after whatever-it-was ended?

What we knew

Our setup: a Go microservices dialer system on Railway. The service in question, taskqueue, is a background worker built on asynq, a Redis-backed job queue. It processes call logs, syncs recordings, that kind of thing.

Digging through Loki gave us the trigger within the hour. At exactly 08:36:00 UTC (= 3:36 AM CDT — the same minute as the crash email), every service in the fleet started screaming the same thing:

dial tcp 10.132.244.37:6379: i/o timeout

Redis was unreachable. Not just for one service — for all of them, in both environments simultaneously, each pointing at its own Redis instance. A platform-level network blip. It lasted about 22 minutes; by 08:58 the errors stopped and everything reconnected.

Here's the thing though. Six services hit the identical outage. Five of them logged some errors, reconnected, and moved on with flat memory. Only taskqueue — in both environments — exploded. Staging hit its container limit ~9 minutes in and got OOM-killed (the 3:36 email). Production had more headroom, absorbed 14 GB, and survived.

Five services shrugged. One ballooned. The only architectural difference: taskqueue is the only service that runs an asynq server.

Ruling out our own code

Before blaming a library, we audited our own code, and it came up clean: every worker pool was semaphore-bounded and joined, every batch load was small (verified in logs), asynq's concurrency was capped at 10, mutexes released on error paths. Nothing in our code could hold gigabytes.

We also had two crucial negative results from the logs:

Memory stopped growing the moment Redis recovered. Whatever was allocating only did so while Redis was down.
The service kept logging and processing normally for six days while carrying the 14 GB. The memory wasn't in use by anything — it was dead weight. And CPU sat slightly elevated the whole time.

That elevated CPU turned out to be a beautiful clue: it's the signature of a garbage collector repeatedly scanning a huge live heap it can't shrink. Which raises the real question —

Why didn't the GC clean it up?

The rule that explains this whole incident:

Go's garbage collector can only free memory that nothing references anymore. Reachable memory is, by definition, memory the program might still need — the GC must keep it, forever if necessary.

So this wasn't "Go being lazy about returning memory to the OS." Something had allocated millions of objects during those 22 minutes and kept a reference to every one of them.

We found it by reading two libraries' source code — specifically, the paths they take when Redis is down. Failure paths are the least-exercised, least-reviewed code in any dependency, and this bug needed two of them to line up.

Ingredient 1: asynq retries with zero backoff

asynq's processor polls Redis in a loop asking "any tasks for me?" Look at its error handling (v0.25.1, processor.go, condensed):

switch {
case errors.Is(err, errors.ErrNoProcessableTask):
    // Queue is empty — normal. Sleep before polling again.
    time.Sleep(p.taskCheckInterval/2 + jitter)
    <-p.sema
    return
case err != nil:
    // ANY OTHER ERROR — e.g. "can't reach Redis"
    p.logger.Errorf("Dequeue error: %v", err)
    <-p.sema
    return   // ← no sleep. The outer for-loop calls this again IMMEDIATELY.
}

Empty queue? Sleep politely. Broken broker? Retry instantly, forever. While Redis was unreachable, asynq was hammering Dequeue in a hot loop.

On its own that's just wasted CPU — each failed attempt's memory should become garbage and get collected. Should.

Ingredient 2: go-redis leaks a "ticket" on every failed connection

go-redis maintains a connection pool. In v9.17.2, when the pool needs a connection, it creates a small bookkeeping object — a wantConn, holding a context, a cancel function, and a result channel — and appends it to an internal queue (internal/pool/pool.go):

w := &wantConn{
    ctx:       dialCtx,                        // context.WithTimeout → timer, cancelCtx
    cancelCtx: cancel,
    result:    make(chan wantConnResult, 1),
}
...
p.dialsQueue.enqueue(w)   // ← always enqueued

Now the bug. Search the package for who removes entries from dialsQueue: there is exactly one call site, and it runs only when a dial succeeds and hands its connection to a waiting requester. When a dial fails, the ticket gets marked "done"… and stays in the queue. Forever. The queue belongs to the pool; the pool belongs to the Redis client; the client lives as long as the process. Every ticket is permanently reachable. The GC is not allowed to touch it.

One more detail turns this from a trickle into a firehose: after PoolSize consecutive dial failures, go-redis stops dialing and fails fast, instantly returning the cached error:

if atomic.LoadUint32(&p.dialErrorsNum) >= uint32(p.cfg.PoolSize) {
    return nil, p.getLastDialError()
}

Sensible optimization — except the ticket is enqueued before that check triggers the failure. So the failure path becomes microsecond-fast, asynq's no-backoff loop spins at full CPU speed, and every single spin permanently leaks one ticket.

Tight loop × small leak × 22 minutes = one very large number.

The receipts: 40,000 leaked tickets per second

Theory is cheap, so we reproduced it. Taskqueue on a dev machine, Prometheus + pprof wired up, and then we simulated the outage.

First attempt: docker stop redis. Nothing happened. Memory stayed flat at 35 MB. This was our favorite wrong result of the whole investigation, because the reason is so instructive: a stopped container refuses connections instantly, and connection refused is not a timeout error — go-redis handles it with internal retries and backoff, which paces the loop. The production outage was packets silently dropped — i/o timeout — and cached timeout errors skip the retry logic entirely.

The error type mattered more than the error.

Second attempt: drop the packets like the real outage did (iptables -j DROP on the Redis port). Within seconds:

21:14  RSS:    34 MB     ← baseline
21:21  RSS: 1,703 MB
21:22  RSS: 2,082 MB
21:24  RSS: 3,950 MB
21:27  RSS: 4,906 MB     ← ~1.2 GB/minute

Same error lines as production, down to the exact file and line. And the heap profile, ~100 seconds in, is the whole story in five rows:

Type: inuse_objects
   7,167,240   pool.(*ConnPool).queuedNewConn     ← wantConn structs + channels
   4,442,693   context.WithDeadlineCause          ← one timer-context per ticket
   2,354,866   context.(*cancelCtx).Done          ← cancel channels
                 ... 98% of the heap, rooted under:
                 asynq.(*processor).exec → rdb.Dequeue → queuedNewConn

~4.4 million tickets in ~100 seconds — about 40,000 leaked tickets per second, at roughly 220 bytes of permanently-reachable heap each. Scale that to production's 22-minute outage and you get on the order of 50 million tickets ≈ 14 GB. The arithmetic closes.

The rest of the incident reproduced just as faithfully. Unblock Redis: growth freezes instantly, but live heap stays pinned — the plateau. Goroutine count flat at ~38 the whole time (this was never a goroutine leak — it was data). Restart the process: 40 MB. The only cure is the one we'd used in production without understanding it.

The twist: nobody's test matrix contained this bug

Here's the part that makes this a systems story rather than a "library X has a bug" story.

asynq v0.25.1 declares a dependency on go-redis v9.7.0 — a version that predates this entire async-dialing pool design. The wantConn queue didn't exist when asynq's error-handling loop was written. But our own go.mod pinned go-redis v9.17.2 for our other Redis usage, and Go's module resolution always picks the higher version for the whole binary. So asynq's hot error loop ran on top of a pool implementation it had never been tested against.

Neither library is "broken" in isolation. asynq's no-backoff retry was harmless on the go-redis it was built against. go-redis's ticket leak was slow-motion under normal request-driven traffic (the upstream reporter hit it gradually, under high concurrency). It took the combination — an unpaced retry loop driving a per-call leak at microsecond frequency — to produce a gigabyte a minute. The combination existed only in binaries like ours.

And it was a real upstream bug: go-redis issue #3678 reports exactly this object accumulation in v9.17.2, and PR #3680 (merged January 2026) fixes "zombie wantConn elements accumulating in the wantConnQueue" by purging done tickets on every enqueue. The fix ships in v9.20.1.

Our fix, after a week of forensics, two library source-audits, and a live reproduction:

- github.com/redis/go-redis/v9 v9.17.2
+ github.com/redis/go-redis/v9 v9.20.1

One line.

What we actually learned

1. Your dependencies' error paths are part of your system. Everything worked until Redis blinked for 22 minutes. The bug wasn't in any code we wrote — it lived in the interaction between two libraries' failure handling, a code path that only executes when infrastructure is already on fire.

2. "Go has a GC" does not mean "no memory leaks." Go leaks are almost never missing free() — they're accidental retention: an append to a process-lifetime slice, a map that only grows, a cache with no eviction. The GC faithfully preserved all 50 million of those tickets because the pool still pointed at them. Reachable is reachable.

3. Version skew between a library and its dependencies is a real risk surface. MVS silently upgraded asynq's Redis client by ten minor versions. Nobody tested that pairing, because that pairing existed only in downstream binaries.

4. The error type can matter more than the error. connection refused was harmless; i/o timeout was catastrophic — because one took the retry-with-backoff path and the other took the fail-fast path. If we'd only tested our reproduction with docker stop, we'd have wrongly exonerated the library.

5. Observability gaps decide how long incidents take to solve. We had logs (great — they found the trigger in an hour) but no /metrics and no pprof on the affected service. The root cause took days of source-code archaeology; with a heap profile it would have taken minutes — our eventual repro profile named the exact function on the first look. The service now ships pprof, Prometheus metrics (including asynq queue stats), and a dashboard whose memory panel plots RSS against live heap — the one comparison that instantly distinguishes "real leak" from "GC being lazy."

The next time a graph steps up 14 gigabytes and refuses to come down, we'll know in two minutes. Hopefully now, so will you.

Technical appendix: asynq v0.25.1 processor.go:152-193 (no-backoff error branch); go-redis v9.17.2 internal/pool/pool.go:577 (enqueue), pool.go:627 (only dequeue — success path), pool.go:370-372 (fail-fast); fixed by wantConnQueue.discardDoneAtFront() in v9.20.1. Reproduction: block the Redis port with iptables -j DROP (packet drop → dial timeout), not docker stop (→ connection refused, which is paced). Heap profiles captured via net/http/pprof, analyzed with go tool pprof -inuse_objects and -inuse_space.

I Built an MCP Server to Search Documentation from Claude (So You Don't Have to Web Search)

Sheikh Ahnaf Hasan — Thu, 05 Mar 2026 15:05:28 +0000

Ever been coding with Claude and needed to check the docs for a library? You either:

Open a browser, search, scan through pages
Ask Claude, who might hallucinate outdated info
Copy-paste docs into the chat (token explosion)

I got tired of this loop. So I built DocMCP – an MCP server that crawls documentation sites, indexes them locally, and lets Claude search them directly.

What is MCP?

Model Context Protocol (MCP) is Anthropic's open standard for connecting AI assistants to external tools and data. Think of it as "plugins for Claude" – but standardized and secure.

The Problem

Claude's knowledge has a cutoff date. When you're working with:

A new library version
Recently updated API docs
Framework-specific patterns

...Claude might not have the latest info. Web search helps, but it's slow and breaks your flow.

The Solution

DocMCP lets you:

# One-time setup
npx @pieeee/docmcp init

# Index any docs site
npx @pieeee/docmcp add https://tailwindcss.com/docs
npx @pieeee/docmcp add https://react.dev
npx @pieeee/docmcp add https://docs.astro.build

Then in Claude Code, Cursor, or Claude Desktop:

"How do I center a div in Tailwind?"

Claude searches your local indexed docs and gives you accurate, up-to-date answers.

How It Works

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  Crawl Docs │ ──► │  Chunk +    │ ──► │   SQLite    │
│  (sitemap)  │     │  Embed      │     │  FTS5 + Vec │
└─────────────┘     └─────────────┘     └─────────────┘
                                               │
                                               ▼
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Claude    │ ◄── │  MCP Server │ ◄── │   Hybrid    │
│   Answer    │     │  (stdio)    │     │   Search    │
└─────────────┘     └─────────────┘     └─────────────┘

Crawl: Follow sitemap or recursive links
Parse: Clean HTML → Markdown, preserve code blocks
Chunk: Split at headings (~512 tokens each)
Index: BM25 (keyword) + vector embeddings (semantic)
Search: Hybrid ranking with Reciprocal Rank Fusion

Why Hybrid Search?

Pure keyword search fails on:

"how to make elements wrap to next line"

(Answer: flex-wrap – but "wrap" ≠ "flex-wrap")

Pure vector search fails on:

"useState hook"

(Exact API names need exact matches)

DocMCP combines both. BM25 handles exact terms, vectors handle semantics. Results are merged using RRF (Reciprocal Rank Fusion).

Setup in 2 Minutes

1. Install

npm install -g @pieeee/docmcp

2. Initialize

docmcp init

Choose your embedding provider:

Anthropic (Voyage) – recommended for Claude users
OpenAI – if you have an OpenAI key
BM25 only – no API key needed, keyword search only

3. Index Some Docs

docmcp add https://tailwindcss.com/docs
docmcp add https://nextjs.org/docs

4. Connect to Claude

Claude Code:

claude mcp add docmcp -- docmcp serve

Cursor (~/.cursor/mcp.json):

{
  "mcpServers": {
    "docmcp": {
      "command": "docmcp",
      "args": ["serve"]
    }
  }
}

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "docmcp": {
      "command": "docmcp",
      "args": ["serve"]
    }
  }
}

Real Example

After indexing Tailwind docs:

Me: "How do I add a gradient background in Tailwind?"

Claude (using search_docs):

Based on the Tailwind documentation, you can use the bg-gradient-to-{direction} utilities:
<div class="bg-gradient-to-r from-cyan-500 to-blue-500">
  
</div>
Directions: t (top), r (right), b (bottom), l (left), tr, br, etc.

Accurate. Up-to-date. No hallucination.

What's Under the Hood

SQLite with FTS5 for BM25 full-text search
sqlite-vec for vector similarity search
Crawlee for respectful, sitemap-aware crawling
Turndown for HTML → Markdown conversion
Written in TypeScript, runs on Node.js 20+

All data stays local in ~/.docmcp/. No cloud dependencies (except embedding API calls if you use one).

Embedding Provider Comparison

Provider	API Key	Best For
Anthropic (Voyage)	`ANTHROPIC_API_KEY`	Claude users, high quality
OpenAI	`OPENAI_API_KEY`	Already have OpenAI key
BM25 only	None	Zero setup, privacy-first

BM25-only mode is surprisingly good for technical docs where you're often searching exact function names.

Roadmap

[ ] Local embeddings (ONNX, no API needed)
[ ] add_docs MCP tool (index from Claude directly)
[ ] Scheduled re-crawls for freshness
[ ] VS Code extension

Try It Out

npm install -g @pieeee/docmcp
docmcp init
docmcp add https://docs.your-favorite-library.com

GitHub: github.com/pieeee/docmcp

I Built an MCP Server to Search Documentation from Claude (So You Don't Have to Web Search)

Sheikh Ahnaf Hasan — Thu, 05 Mar 2026 15:05:28 +0000

Ever been coding with Claude and needed to check the docs for a library? You either:

Open a browser, search, scan through pages
Ask Claude, who might hallucinate outdated info
Copy-paste docs into the chat (token explosion)

I got tired of this loop. So I built DocMCP – an MCP server that crawls documentation sites, indexes them locally, and lets Claude search them directly.

What is MCP?

Model Context Protocol (MCP) is Anthropic's open standard for connecting AI assistants to external tools and data. Think of it as "plugins for Claude" – but standardized and secure.

The Problem

Claude's knowledge has a cutoff date. When you're working with:

A new library version
Recently updated API docs
Framework-specific patterns

...Claude might not have the latest info. Web search helps, but it's slow and breaks your flow.

The Solution

DocMCP lets you:

# One-time setup
npx @pieeee/docmcp init

# Index any docs site
npx @pieeee/docmcp add https://tailwindcss.com/docs
npx @pieeee/docmcp add https://react.dev
npx @pieeee/docmcp add https://docs.astro.build

Then in Claude Code, Cursor, or Claude Desktop:

"How do I center a div in Tailwind?"

Claude searches your local indexed docs and gives you accurate, up-to-date answers.

How It Works

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  Crawl Docs │ ──► │  Chunk +    │ ──► │   SQLite    │
│  (sitemap)  │     │  Embed      │     │  FTS5 + Vec │
└─────────────┘     └─────────────┘     └─────────────┘
                                               │
                                               ▼
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Claude    │ ◄── │  MCP Server │ ◄── │   Hybrid    │
│   Answer    │     │  (stdio)    │     │   Search    │
└─────────────┘     └─────────────┘     └─────────────┘

Crawl: Follow sitemap or recursive links
Parse: Clean HTML → Markdown, preserve code blocks
Chunk: Split at headings (~512 tokens each)
Index: BM25 (keyword) + vector embeddings (semantic)
Search: Hybrid ranking with Reciprocal Rank Fusion

Why Hybrid Search?

Pure keyword search fails on:

"how to make elements wrap to next line"

(Answer: flex-wrap – but "wrap" ≠ "flex-wrap")

Pure vector search fails on:

"useState hook"

(Exact API names need exact matches)

DocMCP combines both. BM25 handles exact terms, vectors handle semantics. Results are merged using RRF (Reciprocal Rank Fusion).

Setup in 2 Minutes

1. Install

npm install -g @pieeee/docmcp

2. Initialize

docmcp init

Choose your embedding provider:

Anthropic (Voyage) – recommended for Claude users
OpenAI – if you have an OpenAI key
BM25 only – no API key needed, keyword search only

3. Index Some Docs

docmcp add https://tailwindcss.com/docs
docmcp add https://nextjs.org/docs

4. Connect to Claude

Claude Code:

claude mcp add docmcp -- docmcp serve

Cursor (~/.cursor/mcp.json):

{
  "mcpServers": {
    "docmcp": {
      "command": "docmcp",
      "args": ["serve"]
    }
  }
}

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "docmcp": {
      "command": "docmcp",
      "args": ["serve"]
    }
  }
}

Real Example

After indexing Tailwind docs:

Me: "How do I add a gradient background in Tailwind?"

Claude (using search_docs):

Based on the Tailwind documentation, you can use the bg-gradient-to-{direction} utilities:
<div class="bg-gradient-to-r from-cyan-500 to-blue-500">
  
</div>
Directions: t (top), r (right), b (bottom), l (left), tr, br, etc.

Accurate. Up-to-date. No hallucination.

What's Under the Hood

SQLite with FTS5 for BM25 full-text search
sqlite-vec for vector similarity search
Crawlee for respectful, sitemap-aware crawling
Turndown for HTML → Markdown conversion
Written in TypeScript, runs on Node.js 20+

All data stays local in ~/.docmcp/. No cloud dependencies (except embedding API calls if you use one).

Embedding Provider Comparison

Provider	API Key	Best For
Anthropic (Voyage)	`ANTHROPIC_API_KEY`	Claude users, high quality
OpenAI	`OPENAI_API_KEY`	Already have OpenAI key
BM25 only	None	Zero setup, privacy-first

BM25-only mode is surprisingly good for technical docs where you're often searching exact function names.

Roadmap

[ ] Local embeddings (ONNX, no API needed)
[ ] add_docs MCP tool (index from Claude directly)
[ ] Scheduled re-crawls for freshness
[ ] VS Code extension

Try It Out

npm install -g @pieeee/docmcp
docmcp init
docmcp add https://docs.your-favorite-library.com

GitHub: github.com/pieeee/docmcp

Automatic SSL Solution for SaaS/MicroSaaS Applications with Caddy, Node.js and Docker

Sheikh Ahnaf Hasan — Thu, 29 Feb 2024 01:30:02 +0000

Hey there! Over the past couple of years, I've had my hands on quite a few SaaS applications. One thing a lot of them have in common is this nifty feature that lets users plug in their own custom domains. Now, getting a domain up and running is pretty straightforward—you just need to give your users the server IP address or a CNAME value. But here's the catch: offering free and automatic SSL? That can get a bit tricky.

At first, I gave bash scripting a go to generate SSL with Let's Encrypt and used good old Nginx to create and renew SSL certificates. But let me tell you, it was a bit of a headache to maintain. I also tried out a Cloudflare feature, SSL for SaaS, but it asked users to add a bunch of DNS records, which wasn't really what I was looking for. What I wanted was a solution where users only had to add a CNAME.

So I dug a little deeper and came across this gem: Caddy. Caddy is this fantastic, extensible, cross-platform, open-source web server that's written in Go. The best part? It comes with automatic HTTPS. It basically condenses all the work our scripts and manual maintenance were doing into just 4-5 lines of config. So, stick around and I'll walk you through how to set up an automatic SSL solution with Caddy, Docker and a Node.js server.

Let's Get Our Application Up and Running!

First things first, we're going to set up a friendly little express application that greets us with a cheery "Hello World" page. We'll need a couple of APIs for this.

/ - This one's our main star! It serves the "Hello World" page.
/tls-check - Our buddy Caddy uses this to make sure it's dealing with a valid domain. We'll chat more about this later on in our journey.

Create a new directory named caddy-saas-node-application and then change your current directory to this new one. While we'll be using pnpm, you can also use yarn or npm to install packages.

To install express, run the command pnpm add express.

Next, update the package.json file to include a start script:

{
  "name": "caddy-saas-node-application",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node index.js"
  },
  "keywords": [],
  "author": "pieeee",
  "license": "ISC",
  "dependencies": {
    "express": "^4.18.2"
  }
}

Create a main.js file, which will contain the code for our Express application. Add the following code:

/**
 * Express application for the Caddy SaaS Node Application.
 * @module index
 */

const express = require("express");

const app = express();

app.use(express.json());
app.use(express.urlencoded({ extended: true }));

/**
 * Route handler for the root URL.
 * @name GET /
 * @function
 * @param {Object} req - Express request object.
 * @param {Object} res - Express response object.
 */
app.get("/", (_, res) => {
  res.send("Hello World");
});

/**
 * Array of whitelisted domains.
 * @type {string[]}
 */
const whitelistedDomains = [];

/**
 * Handles TLS (Transport Layer Security) check for a given domain.
 * This endpoint checks if the provided domain is whitelisted for TLS connections.
 * It is accessed via a GET request and expects a domain name as a query parameter.
 *
 * @route GET /tls-check
 * @param {Object} req - The request object from Express.js.
 * @param {Object} req.query - The query string object.
 * @param {string} req.query.domain - The domain name to check for TLS whitelisting.
 * @param {Object} res - The response object from Express.js.
 * @returns {Object} The response object with a status code and a JSON body.
 *          If the domain query parameter is missing, it returns a 400 status code with an error message.
 *          If the domain is found in the whitelist, it returns a 200 status code with a success message.
 *          If the domain is not in the whitelist, it returns a 403 status code with an error message.
 */
app.get("/tls-check", (req, res) => {
  const domain = req.query.domain;

  if (!domain) {
    return res.status(400).json({
      error: "Domain is required",
    });
  }

  if (whitelistedDomains.includes(domain)) {
    return res.status(200).json({
      message: "Domain is whitelisted",
    });
  }

  return res.status(403).json({
    error: "Domain is not whitelisted",
  });
});

/**
 * Start the server and listen on port 8080.
 * @name listen
 * @function
 * @param {number} port - The port number to listen on.
 * @param {Function} callback - The callback function to execute when the server starts listening.
 */
app.listen(8080, () => {
  console.log("Server is running on port :8080\nhttp://localhost:8080/");
});

This API endpoint /tls-check validates the TLS whitelisting status of a domain. It accepts a GET request with a domain query parameter. If the domain parameter is missing, it returns HTTP 400 with an error message. If the domain is present in the whitelistedDomains array, it responds with HTTP 200, indicating the domain is whitelisted. Otherwise, it returns HTTP 403, stating the domain is not whitelisted.

The whitelistedDomains contains the domain names of allowed users. In a production environment, you might validate these from your database. Now that our app is ready, lets move on to the next part.

Docker and docker-compose setup

Create a Dockerfile at the root of your project directory and add the following configurations to keep your Express application container running.

# Use a node base image
FROM node:18

# Create app directory in container
WORKDIR /usr/src/app

# Install pnpm
RUN npm install -g pnpm

# Copy package.json and pnpm-lock.yaml files
COPY package.json pnpm-lock.yaml* ./

# Install app dependencies using pnpm
RUN pnpm install --frozen-lockfile

# Bundle app source inside Docker image
COPY . .

RUN pnpm run build

EXPOSE 8080

CMD ["pnpm", "start"]

Now, create a docker-compose.yml file and add the following configuration. We'll use docker-compose to manage both the app container and the caddy container in one location.

version: "3"
services:
    app:
        build: .
        networks:
            - webnet
    caddy:
        image: caddy:2
        ports:
            - "80:80"
            - "443:443"
        volumes:
            - ./Caddyfile:/etc/caddy/Caddyfile
            - caddy_data:/data
            - caddy_config:/config
        networks:
            - webnet
        depends_on:
            - app

networks:
    webnet:

volumes:
    caddy_data:
    caddy_config:

The Docker Compose configuration defines two services, app and caddy, within a custom network named webnet. The app service is built from a Dockerfile in the current directory, indicating the express application. The caddy service uses the official Caddy server image version 2, exposing ports 80 and 443 for HTTP and HTTPS traffic, respectively. It mounts a Caddyfile for configuration, alongside named volumes caddy_data and caddy_config for persisting data and Caddy configuration. The caddy service is configured to depend on the app service, ensuring app is started first. Both services are connected via the webnet network, facilitating inter-service communication.

Now let’s create Caddyfile at the root of the project directory and add the following configs:

{
    on_demand_tls {
        ask http://app:8080/tls-check
        burst 5
        interval 2m
    }
}

https:// {
    tls {
        on_demand
    }

    reverse_proxy app:8080
}

This little configuration does all the work. When a request from a new domain arrives, the on_demand_tls block triggers our /tls-check API to validate the domain. If the domain is validated, Caddy will generate an SSL certificate and assign it. The generation only occurs on the first request. By default, the generated SSLs expire in three months. After that, a new SSL is generated and assigned on the first request. The "burst 5" and "interval 2m" specify that a maximum of five certificates can be generated every two minutes. You can read more here: Caddy on-demand automation

The https:// block enables the reverse proxy to our application. You can learn more here: Caddy reverse proxy

That's it! All you need to do now is deploy your application on a virtual machine, install Docker, build the app, and enjoy the magic of Caddy.

Useful Resources

Essential React Custom Hooks

Sheikh Ahnaf Hasan — Tue, 20 Feb 2024 10:10:43 +0000

Hello there! Having spent several years developing React applications, I've noticed a handful of functionalities that pop up again and again. They're not just limited to a single project, but are versatile enough to be used in various contexts. Recognizing and transforming these recurring functionalities into custom hooks can save us a ton of time and effort. Plus, these custom hooks are super adaptable and can be easily integrated into almost any project, no matter how big or small, or how simple or complex.

Let's dig a little deeper into these useful custom hooks and chat about how they can be practically applied:

useDebounce: This handy hook lets us delay the processing of a function for a set amount of time. It's a lifesaver when we want to limit how often a function can trigger or run.
useLocalStorageState: With this hook, we can easily save, retrieve, and play with data in the browser's local storage. It's a gem for keeping the state consistent across sessions or tabs.
useWindowScroll: This nifty custom hook helps keep track of the window's scroll position. It's great for a range of uses like lazy loading images, infinite scrolling, or revealing/hiding elements based on the scroll position.

useDebounce

import { useEffect, useState } from "react";

export function useDebounce({
  value,
  delay,
}: {
  value: string;
  delay: number;
}) {
  // State and setters for debounced value
  const [debouncedValue, setDebouncedValue] = useState(value);

  useEffect(
    () => {
      // Update debounced value after delay
      const handler = setTimeout(() => {
        setDebouncedValue(value);
      }, delay);

      // Cancel the timeout if value changes (also on delay change or unmount)
      // This is how we prevent debounced value from updating if value is changed ...
      // .. within the delay period. Timeout gets cleared and restarted.
      return () => {
        clearTimeout(handler);
      };
    },
    [value, delay] // Only re-call effect if value or delay changes
  );

  return debouncedValue;
}

The useDebounce is a custom React hook designed to delay the update of a state value until after a specified delay. This is useful for reducing the number of updates that occur in response to rapid or frequent changes to a value, such as user input in a text field. Let's break down how it works and provide an example of its usage.

How `useDebounce` Works:

Input Parameters: It takes an object with two properties:
- value: The current value that you want to debounce.
- delay: The amount of time (in milliseconds) you want to wait after the last change before updating the debounced value.
State: It initializes a state variable debouncedValue with the initial value passed to the hook.
Effect Hook: It uses the useEffect hook to perform side effects:
- Set Timeout: Inside useEffect, it sets a timeout that waits for the specified delay before updating debouncedValue to the current value.
- Cleanup Function: Returns a cleanup function that clears the timeout if the value or delay changes before the timeout completes. This prevents the debounced value from updating if the input value changes again within the delay period.
Return: It returns the debouncedValue. This value reflects the latest value passed to the hook, but only after the specified delay has elapsed without any further changes to the value.

Example Usage:

Imagine you have a search input field where you want to fetch search results from an API, but you want to minimize the number of API calls by only fetching the results once the user has stopped typing for 500 milliseconds.

import React, { useState } from 'react';
import { useDebounce } from './useDebounce';

function SearchComponent() {
  const [inputValue, setInputValue] = useState('');
  const debouncedSearchTerm = useDebounce({ value: inputValue, delay: 500 });

  // Effect for API call
  useEffect(() => {
    if (debouncedSearchTerm) {
      // Function to fetch search results
      console.log(`Fetching search results for: ${debouncedSearchTerm}`);
      // Here you would fetch your search results using debouncedSearchTerm
    }
  }, [debouncedSearchTerm]); // This effect runs only when debouncedSearchTerm changes

  return (
    <input
      type="text"
      value={inputValue}
      onChange={(e) => setInputValue(e.target.value)}
      placeholder="Search..."
    />
  );
}

In this example:

The user types into the search input field.
The inputValue state updates with each keystroke, but the debouncedSearchTerm only updates after the user has stopped typing for 500 milliseconds.
The useEffect hook listening to debouncedSearchTerm then triggers, potentially fetching search results based on the debounced term.

This approach ensures that your application performs the search operation efficiently, reducing unnecessary API calls or processing when the user is typing fast.

useLocalStorageState

import { useState, useEffect, useMemo } from "react";

// Define a custom hook that syncs state with local storage
export const useLocalStorageState = <T>(
  key: string,
  initialValue: T
): [T, React.Dispatch<React.SetStateAction<T>>] => {
  // Get the initial value from local storage, if available
  const storedValue =
    typeof window !== "undefined" ? localStorage.getItem(key) : null;

  const initial = useMemo(() => {
    try {
      return storedValue ? JSON.parse(storedValue) : initialValue;
    } catch (error) {}
  }, [initialValue, storedValue]);

  // Create the state using useState
  const [state, setState] = useState<T>(initial);

  // Update local storage whenever the state changes
  useEffect(() => {
    localStorage.setItem(key, JSON.stringify(state));
  }, [key, state]);

  return [state, setState];
};

// Path: src/lib/hooks/use-sessionstorage-state.hook.ts
export const useSessionStorageState = <T>(
  key: string,
  initialValue: T
): [T, React.Dispatch<React.SetStateAction<T>>] => {
  const storedValue =
    typeof window !== "undefined" ? sessionStorage.getItem(key) : null;
  const initial = useMemo(() => {
    try {
      return storedValue ? JSON.parse(storedValue) : initialValue;
    } catch (error) {}
  }, [initialValue, storedValue]);
  const [state, setState] = useState<T>(initial);
  useEffect(() => {
    sessionStorage.setItem(key, JSON.stringify(state));
  }, [key, state]);
  return [state, setState];
};

The useLocalStorageState function is a custom React hook designed to synchronize a component's state with the local storage of the browser. This allows the state to persist across browser sessions. Here's a breakdown of how it works and an example of how to use it.

How `useLocalStorageState` Works:

Generics: It uses TypeScript generics (<T>) to allow for flexibility in the type of value it can handle (e.g., string, number, object).
Input Parameters:
- key: A unique string that identifies the item in local storage.
- initialValue: The initial value for the state if no value is found in local storage.
Retrieving Initial Value:
- Checks if the window object is available to ensure code doesn't break during server-side rendering (e.g., in Next.js applications).
- Attempts to get the stored value from local storage using the provided key.
- Uses useMemo to parse the stored JSON value only once or fallback to the initialValue if parsing fails or if no value is found.
State Management:
- Initializes the state with either the parsed local storage value or the provided initial value.
Effect for Syncing with Local Storage:
- Utilizes useEffect to update the local storage item whenever the state changes. It serializes the state to a JSON string and stores it under the provided key.
Return Values:
- Returns a tuple containing the current state and a setter function (setState), similar to the return value of the useState hook.

Example Usage:

Let's say you have a form input where you want to persist the user's input even if they refresh the page or close and reopen the browser.

import React from 'react';
import { useLocalStorageState } from './useLocalStorageState';

function FormComponent() {
  const [name, setName] = useLocalStorageState('userName', '');

  return (
    <div>
      <label htmlFor="name">Name:</label>
      <input
        id="name"
        type="text"
        value={name}
        onChange={(e) => setName(e.target.value)}
      />
      <p>Hello, {name}!</p>
    </div>
  );
}

In this example:

The useLocalStorageState hook is used to bind the input field's value to a piece of state that is synchronized with local storage. The key is 'userName'.
If the user enters their name and then refreshes the page, the input field will still display their name because it was saved to local storage.
The state is initially set to the value retrieved from local storage if it exists; otherwise, it falls back to an empty string ('').
Any changes to the input field update both the state and the corresponding item in local storage, ensuring the two are always in sync.

This approach enhances the user experience by making the UI state persistent across sessions without requiring any backend infrastructure to store user data.

useWindowScroll

import { useEffect, useState } from "react";

interface WindowScrollState {
  x: number;
  y: number;
}

export const useWindowScroll = (): WindowScrollState => {
  const [scrollPosition, setScrollPosition] = useState<WindowScrollState>({
    x: typeof window !== "undefined" ? window.scrollX : 0,
    y: typeof window !== "undefined" ? window.scrollY : 0,
  });

  useEffect(() => {
    const handleScroll = () => {
      setScrollPosition({
        x: typeof window !== "undefined" ? window.scrollX : 0,
        y: typeof window !== "undefined" ? window.scrollY : 0,
      });
    };

    if (typeof window !== "undefined") {
      // Attach the scroll event listener when the component mounts
      window.addEventListener("scroll", handleScroll);

      // Remove the scroll event listener when the component unmounts
      return () => {
        window.removeEventListener("scroll", handleScroll);
      };
    }
  }, []);

  return scrollPosition;
};

The useWindowScroll function is a custom React hook designed to track and provide the window's scroll position. It encapsulates the logic for listening to scroll events and updating the state accordingly to reflect the current scroll position of the window. This can be particularly useful for features like showing or hiding elements based on scroll position, implementing scroll-based animations, or simply monitoring user scroll behavior. Let's break down its workings and illustrate its usage with an example.

How `useWindowScroll` Works:

State Initialization: Initializes a state variable scrollPosition with an object that contains two properties:
- x: Represents the horizontal scroll position. It's set to window.scrollX if the window object is available, otherwise to 0.
- y: Represents the vertical scroll position. It's set to window.scrollY if the window object is available, otherwise to 0.
Effect Hook: Uses useEffect to set up side effects:
- Scroll Event Listener: Defines a function handleScroll that updates scrollPosition based on the current scroll position of the window.
- Event Listener Registration: If the window object is available, it adds handleScroll as an event listener to the window's scroll event. This ensures the scroll position is updated whenever the user scrolls.
- Cleanup: Returns a cleanup function that removes the scroll event listener when the component using this hook unmounts, preventing potential memory leaks.
Return Value: Returns the current scrollPosition state, allowing components to access the latest scroll position as { x, y }.

Example Usage:

Consider a scenario where you want to display a "Back to Top" button only when the user has scrolled down a significant amount.

import React from 'react';
import { useWindowScroll } from './useWindowScroll';

function BackToTopButton() {
  const { y } = useWindowScroll();

  const handleBackToTop = () => {
    window.scrollTo({ top: 0, behavior: 'smooth' });
  };

  return (
    <button
      style={{
        position: 'fixed',
        bottom: '20px',
        right: '20px',
        display: y > 200 ? 'block' : 'none',
      }}
      onClick={handleBackToTop}
    >
      Back to Top
    </button>
  );
}

In this example:

The useWindowScroll hook provides the vertical scroll position (y) of the window.
The "Back to Top" button is conditionally rendered based on the scroll position. It appears only when the user has scrolled down more than 200 pixels.
Clicking the button scrolls the window back to the top smoothly.

This approach enhances user navigation on long pages by providing an easily accessible means to return to the top of the page, improving the overall user experience.

How do I enable custom domain feature in a application?

Sheikh Ahnaf Hasan — Fri, 29 Apr 2022 06:04:06 +0000

I am developing an web application which has a feature to enable the users to use their own domain name for pointing to a route. The application is top to bottom written in next.js.

DEV Community: Sheikh Ahnaf Hasan

How a 22-Minute Redis Blip Ate 18 GB of RAM

3:36 AM

What we knew

Ruling out our own code

Why didn't the GC clean it up?

Ingredient 1: asynq retries with zero backoff

Ingredient 2: go-redis leaks a "ticket" on every failed connection

The receipts: 40,000 leaked tickets per second

The twist: nobody's test matrix contained this bug

What we actually learned

I Built an MCP Server to Search Documentation from Claude (So You Don't Have to Web Search)

What is MCP?

The Problem

The Solution

How It Works

Why Hybrid Search?

Setup in 2 Minutes

1. Install

2. Initialize

3. Index Some Docs

4. Connect to Claude

Real Example

What's Under the Hood

Embedding Provider Comparison

Roadmap

Try It Out

I Built an MCP Server to Search Documentation from Claude (So You Don't Have to Web Search)

What is MCP?

The Problem

The Solution

How It Works

Why Hybrid Search?

Setup in 2 Minutes

1. Install

2. Initialize

3. Index Some Docs

4. Connect to Claude

Real Example

What's Under the Hood

Embedding Provider Comparison

Roadmap

Try It Out

Automatic SSL Solution for SaaS/MicroSaaS Applications with Caddy, Node.js and Docker

Let's Get Our Application Up and Running!

Docker and docker-compose setup

Useful Resources

Essential React Custom Hooks

useDebounce

How useDebounce Works:

Example Usage:

useLocalStorageState

How useLocalStorageState Works:

Example Usage:

useWindowScroll

How useWindowScroll Works:

Example Usage:

How do I enable custom domain feature in a application?

discussion

How `useDebounce` Works:

How `useLocalStorageState` Works:

How `useWindowScroll` Works: