<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Souvick Sarkar</title>
    <description>The latest articles on DEV Community by Souvick Sarkar (@souvick_20).</description>
    <link>https://dev.to/souvick_20</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3783481%2F0fcbca64-1627-45a8-a8c0-552a794dce43.jpg</url>
      <title>DEV Community: Souvick Sarkar</title>
      <link>https://dev.to/souvick_20</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/souvick_20"/>
    <language>en</language>
    <item>
      <title>I Built a Production-Grade Auth API From Scratch — Here's Everything I Learned</title>
      <dc:creator>Souvick Sarkar</dc:creator>
      <pubDate>Fri, 31 Jul 2026 10:03:38 +0000</pubDate>
      <link>https://dev.to/souvick_20/i-built-a-production-grade-auth-api-from-scratch-heres-everything-i-learned-3on3</link>
      <guid>https://dev.to/souvick_20/i-built-a-production-grade-auth-api-from-scratch-heres-everything-i-learned-3on3</guid>
      <description>&lt;p&gt;Most tutorials teach you authentication in 15 minutes. They show you jwt.sign(), slap a middleware on a route, and call it a day.&lt;/p&gt;

&lt;p&gt;Real authentication systems are nothing like that.&lt;/p&gt;

&lt;p&gt;I spent 5 weeks building Swaraksha — a production-grade authentication API that handles everything a real system needs: JWT token pairs, refresh token rotation with theft detection, TOTP-based multi-factor authentication, role-based access control, account lockouts, rate limiting, and a fully automated deployment pipeline.&lt;/p&gt;

&lt;p&gt;This article is the deep dive. I will walk you through every feature, every design decision, and every mistake I made along the way.&lt;/p&gt;

&lt;p&gt;GitHub: &lt;a href="https://github.com/SouvickSarkar20/Swaraksha" rel="noopener noreferrer"&gt;Repo Link&lt;/a&gt;&lt;br&gt;
Live Swagger Docs: &lt;a href="http://168.144.117.195/docs" rel="noopener noreferrer"&gt;Demo Link&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Tech Stack&lt;/strong&gt;&lt;br&gt;
Before we get into the code, here is what I used and why:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyvqjdr5wthugx17nf6qv.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyvqjdr5wthugx17nf6qv.png" alt=" " width="799" height="304"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Part 1: Authentication That Actually Works&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Problem With Single Tokens&lt;/strong&gt;&lt;br&gt;
Most tutorials give you one JWT token. The user logs in, gets a token, and uses it for everything. But this creates a horrible tradeoff:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Short-lived token (15 minutes)? The user has to log in again every 15 minutes. Terrible UX.&lt;/li&gt;
&lt;li&gt;Long-lived token (7 days)? If someone steals it, they have full access for a week. Terrible security.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;The Solution&lt;/strong&gt;: Token Pairs&lt;br&gt;
Swaraksha uses two tokens:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Access Token (15 minutes) — Used for every API request. Short-lived, so even if stolen, the damage window is tiny.&lt;/li&gt;
&lt;li&gt;Refresh Token (7 days) — Stored securely by the client. Used only to get a new access token when the old one expires.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fln4r35nkxmvvtrth0q0h.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fln4r35nkxmvvtrth0q0h.png" alt=" " width="800" height="260"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Refresh Token Rotation (Theft Detection)&lt;/strong&gt;&lt;br&gt;
Here is where it gets interesting. Every time a client uses a refresh token to get a new access token, I immediately invalidate the old refresh token and issue a brand new one.&lt;/p&gt;

&lt;p&gt;Why? Imagine this scenario:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A hacker steals your refresh token.&lt;/li&gt;
&lt;li&gt;You use your (original) refresh token to get a new access token.&lt;/li&gt;
&lt;li&gt;The server gives you a new token pair and marks the old refresh token as "used."&lt;/li&gt;
&lt;li&gt;The hacker tries to use the stolen (now "used") refresh token.&lt;/li&gt;
&lt;li&gt;The server detects reuse of a consumed token → Token Theft Detected!&lt;/li&gt;
&lt;li&gt;The server instantly revokes the ENTIRE token family, logging out both you and the hacker.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is called Automatic Reuse Detection, and it is the industry standard used by Auth0 and Okta.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Part 2: Multi-Factor Authentication (MFA)&lt;/strong&gt;&lt;br&gt;
Adding a password is good. Adding a second factor is better.&lt;/p&gt;

&lt;p&gt;Swaraksha implements TOTP (Time-Based One-Time Password) — the same protocol used by Google Authenticator and Authy.&lt;/p&gt;

&lt;p&gt;How It Works&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Setup: The server generates a secret key and returns it as a QR code URI.&lt;/li&gt;
&lt;li&gt;Enable: The user scans the QR code in their authenticator app and sends back a 6-digit code to prove it works.&lt;/li&gt;
&lt;li&gt;Login: After entering their password, users must provide the current 6-digit code from their authenticator app.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F35qr7nbbmjol4z9hj14r.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F35qr7nbbmjol4z9hj14r.png" alt=" " width="705" height="410"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The beauty of TOTP is that the server never needs to send a code to the user. The authenticator app and the server independently generate the same code based on the shared secret and the current time. If they match, the user is verified.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Part 3: Role-Based Access Control (RBAC)&lt;/strong&gt;&lt;br&gt;
Not every user should have the same power. Swaraksha has three roles:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftrnxk5iozynvwqfwuqzv.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftrnxk5iozynvwqfwuqzv.png" alt=" " width="746" height="286"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The authorize Middleware&lt;/strong&gt;&lt;br&gt;
The key insight is separating authentication (who are you?) from authorization (what are you allowed to do?).&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F28tm2j5ovn8uos6llzba.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F28tm2j5ovn8uos6llzba.png" alt=" " width="718" height="512"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Now protecting a route is a single line:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fefuw0px22scw9li2tjrb.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fefuw0px22scw9li2tjrb.png" alt=" " width="477" height="127"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Part 4: Defensive Programming&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Account Lockouts&lt;/strong&gt;&lt;br&gt;
Brute-force attacks try thousands of passwords per second. To stop them, Swaraksha tracks failed login attempts:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;5 failed attempts → Account is locked for 15 minutes.&lt;/li&gt;
&lt;li&gt;Every failed attempt increments a counter in the database.&lt;/li&gt;
&lt;li&gt;A successful login resets the counter to zero.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Rate Limiting with Upstash Redis&lt;/strong&gt;&lt;br&gt;
Even with account lockouts, an attacker could target thousands of different accounts. Rate limiting stops this by restricting requests per IP address:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Login routes: 5 requests per 15 minutes per IP&lt;/li&gt;
&lt;li&gt;API routes: 100 requests per minute per IP&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fimnfzchq1azzacrfs2a3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fimnfzchq1azzacrfs2a3.png" alt=" " width="800" height="237"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Zod Validation at the Framework Level&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Instead of manually checking if (!email || !password) in every route, Swaraksha uses Zod schemas that Fastify automatically enforces:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbhskyjwy2my8i8yo7zs8.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbhskyjwy2my8i8yo7zs8.png" alt=" " width="783" height="350"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Part 5: Containerization with Docker&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why Docker?&lt;/strong&gt;&lt;br&gt;
"But it works on my machine!" — Every developer, ever.&lt;/p&gt;

&lt;p&gt;Docker packages your entire application — Node.js version, dependencies, compiled code — into a single, portable image. It will run identically on your laptop, your teammate's laptop, and a cloud server in Singapore.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Multi-Stage Builds&lt;/strong&gt;&lt;br&gt;
A naive Dockerfile would install TypeScript, compile the code, and ship everything — including the compiler. That is wasteful and insecure.&lt;/p&gt;

&lt;p&gt;Swaraksha uses a multi-stage build:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcd872cngtvylhukyxeph.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcd872cngtvylhukyxeph.png" alt=" " width="615" height="622"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Part 6: CI/CD — From Push to Production&lt;/strong&gt;&lt;br&gt;
This is the part that makes everything feel like magic.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Pipeline&lt;/strong&gt;&lt;br&gt;
Every time I push code to main, GitHub Actions automatically:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Checks out the code on a fresh Ubuntu server&lt;/li&gt;
&lt;li&gt;Installs dependencies and caches node_modules&lt;/li&gt;
&lt;li&gt;Runs TypeScript compilation to catch type errors&lt;/li&gt;
&lt;li&gt;Runs the entire Vitest test suite to catch logic errors&lt;/li&gt;
&lt;li&gt;Builds the Docker image to catch packaging errors&lt;/li&gt;
&lt;li&gt;Pushes the image to GitHub Container Registry&lt;/li&gt;
&lt;li&gt;SSHs into my DigitalOcean Droplet and pulls the new image&lt;/li&gt;
&lt;li&gt;Runs database migrations in a disposable container&lt;/li&gt;
&lt;li&gt;Swaps the live container with the new one&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;The Migration Deadlock&lt;/strong&gt;&lt;br&gt;
I hit an interesting bug during deployment. My original pipeline was:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Start the container&lt;/li&gt;
&lt;li&gt;Wait for it to be running&lt;/li&gt;
&lt;li&gt;Run database migrations inside the container&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;*&lt;em&gt;The problem? *&lt;/em&gt;&lt;br&gt;
The Fastify server tried to connect to the database on startup. If the database schema was not migrated yet, the server crashed instantly. I could never reach step 3 because the container died at step 1.&lt;/p&gt;

&lt;p&gt;The fix: Run migrations in a temporary, disposable container before starting the real server:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzbklp54azjlri3cikdj9.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzbklp54azjlri3cikdj9.png" alt=" " width="755" height="217"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Part 7: Infrastructure &amp;amp; Security&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Server Hardening&lt;br&gt;
The DigitalOcean Droplet runs Ubuntu with:&lt;/p&gt;

&lt;p&gt;UFW Firewall — Only ports 22 (SSH), 80 (HTTP), and 443 (HTTPS) are open. Port 3000 (Node.js) is blocked from the outside world.&lt;/p&gt;

&lt;p&gt;SSH Key Authentication — Password login is disabled. Only cryptographic keys can access the server.&lt;/p&gt;

&lt;p&gt;Non-root Docker User — Even if someone exploits the Node.js app, they cannot escalate to root.&lt;/p&gt;

&lt;p&gt;Caddy Reverse Proxy&lt;br&gt;
The Node.js server is never directly exposed to the internet. A Caddy reverse proxy sits in front of it:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Internet → Caddy (port 80) → Node.js (port 3000, internal only)&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Caddy handles all incoming traffic, protects against malformed requests, and forwards legitimate traffic to the Node.js app over Docker's internal network.&lt;/p&gt;

&lt;p&gt;Secrets Management&lt;br&gt;
No secret is ever committed to Git. Environment variables are stored as GitHub Secrets and injected into the server's .env file during deployment by the CI/CD pipeline.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Part 8: Observability&lt;/strong&gt;&lt;br&gt;
A deployed app without monitoring is a ticking time bomb. Swaraksha includes:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Sentry (Error Tracking)&lt;/strong&gt;&lt;br&gt;
Every unhandled exception is automatically captured and sent to Sentry with the full stack trace&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxjw98fyzwo58z5kt231d.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxjw98fyzwo58z5kt231d.png" alt=" " width="637" height="222"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Health Endpoint&lt;/strong&gt;&lt;br&gt;
A simple /health endpoint returns the server's uptime and timestamp. Uptime monitoring services (like UptimeRobot) ping this every 5 minutes and email me if the server goes down.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I Learned&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Authentication is never "done."&lt;/strong&gt; There is always another attack vector to consider — token theft, brute force, replay attacks, timing attacks.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;CI/CD is not optional.&lt;/strong&gt; Manual deployments are error-prone. Automating the pipeline saved me hours and eliminated "it worked on my machine" bugs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Docker multi-stage builds are essential.&lt;/strong&gt; Shipping development tools to production is a security risk and a waste of resources.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Database migrations in production are scary&lt;/strong&gt; The migration deadlock taught me to always migrate before the app starts, never after.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Monitoring is as important as coding.&lt;/strong&gt; If your server crashes and nobody knows, did it really crash? Yes. Yes it did.&lt;/p&gt;

&lt;p&gt;Try It Yourself&lt;br&gt;
GitHub: github.com/SouvickSarkar20/Swaraksha&lt;br&gt;
Live Swagger Docs: &lt;a href="http://168.144.117.195/docs" rel="noopener noreferrer"&gt;http://168.144.117.195/docs&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Thanks for reading. If you have questions or spot something I could improve, drop a comment below!&lt;/em&gt;&lt;/p&gt;

</description>
      <category>node</category>
    </item>
    <item>
      <title>Scaling PostgreSQL without Microservices: Lessons from Notion’s 480 Shards</title>
      <dc:creator>Souvick Sarkar</dc:creator>
      <pubDate>Sat, 21 Feb 2026 06:03:51 +0000</pubDate>
      <link>https://dev.to/souvick_20/scaling-postgresql-without-microservices-lessons-from-notions-480-shards-32id</link>
      <guid>https://dev.to/souvick_20/scaling-postgresql-without-microservices-lessons-from-notions-480-shards-32id</guid>
      <description>&lt;p&gt;I’ve been using Notion to manage my projects for a long time—it’s a faithful friend in my workflow. Recently, while studying database scaling, a thought hit me: How does the "manager" manage itself?&lt;/p&gt;

&lt;p&gt;With millions of users reading and writing data every second, the infrastructure behind the scenes must be immense. I decided to dive deep into their architecture, and here is what I learned about the scaling strategy that keeps Notion running.&lt;/p&gt;

&lt;p&gt;📝 &lt;strong&gt;TL;DR: Scaling Notion’s Monolith&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I spent the last few days deconstructing how Notion scaled their PostgreSQL database to handle billions of blocks while keeping their Node.js monolith. Here is the blueprint of what I learned:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Application-Level Sharding&lt;/em&gt;: Instead of one massive DB, they use 480 logical shards mapped to a smaller set of physical nodes.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The Shard Router&lt;/em&gt;: The logic lives in the TypeScript code, using a simple space_id % 480 math to route requests instantly.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;PgBouncer&lt;/em&gt;: They use this as a "traffic controller" to pool connections and prevent the database from choking under high load.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Zero-Downtime Migrations&lt;/em&gt;: I broke down how they moved billions of rows using a "Shadow Write" strategy to keep the app live during the transition.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Architecture at a Glance&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fpklkabrwn12k0tmvhlc2.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fpklkabrwn12k0tmvhlc2.png" alt=" " width="800" height="347"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Chapter 1: The Problem with Monolith&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;In its early days, Notion followed a simple architecture: a Node.js backend paired with a single PostgreSQL instance. But eventually, they hit the ceiling:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;CPU Saturation&lt;/em&gt;: Daily spikes were hitting 90%+.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The Vacuum Problem&lt;/em&gt;: Autovacuum couldn't keep up, risking a Transaction ID Wraparound—a state where the database stops accepting writes to prevent corruption.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Chapter 2: Why Not Microservices?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The common logic is: Split the code, split the load.&lt;br&gt;
Notion took the opposite approach. They kept the Monolithic backend to maintain operational velocity and data locality (essential for their complex "block" graph) and focused entirely on sharding the Persistence Layer.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Chapter 3: The 480-Shard Blueprint&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The "pro move" here was decoupling data from hardware using Logical Shards:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The Key&lt;/em&gt;: They used space_id as the Partition Key so all data for one workspace stays together for fast joins.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The Setup&lt;/em&gt;: They created 480 independent schemas (Logical Shards) and distributed them across 32 physical AWS RDS instances.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The Result&lt;/em&gt;: When a server gets overwhelmed, they just "pick up" a logical schema and move it to a new server. Linear Scalability.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Chapter 4: The Great "Shadow" Migration&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;How do you move 480 terabytes of data while the plane is mid-flight?&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The Backfill&lt;/em&gt;: Historical data moved to shards in the background.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Double Writing&lt;/em&gt;: Code wrote new changes to both the old DB and new shards simultaneously.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The Cutover&lt;/em&gt;: Once a comparison engine verified the data was identical, they flipped the switch.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;SOME FUTURE IMPROVEMENTS -&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Scaling to 96 Nodes&lt;br&gt;
By 2023, the original 32 servers hit their limit. Because we had 480 Logical Shards, scaling was simple: we tripled capacity to 96 nodes. We didn't change any code; we just redistributed the shards. This is the beauty of Linear Scalability.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;It’s Blocks All the Way Down&lt;br&gt;
Why the rapid growth? In Notion, Everything is a Block. A single page is actually a tree of dozens of individual units (text, toggles, images). We aren't just scaling pages; we are managing billions of atomic blocks.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Data Lakes &amp;amp; Connection Hubs&lt;br&gt;
Analytics: To run reports across 480 separate databases, we piped everything into a central Data Lake using tools like Fivetran and Snowflake.&lt;br&gt;
Networking: We used PgBouncer for connection pooling, preventing our backend from choking while trying to talk to hundreds of shards at once.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Wrapping Up&lt;/strong&gt;&lt;br&gt;
Notion’s journey proves that you don't always need to chase the latest architectural trends. By focusing on the actual bottleneck—the Persistence Layer—they scaled to billions of blocks while keeping their team lean and their code manageable.&lt;/p&gt;

&lt;p&gt;I am in the early stages of my engineering journey and would be happy to learn and contribute to more conversations like this.&lt;/p&gt;

&lt;p&gt;Do share your thoughts in the comments!&lt;/p&gt;

</description>
      <category>systemdesign</category>
      <category>postgres</category>
      <category>backend</category>
      <category>learning</category>
    </item>
  </channel>
</rss>
