<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Gouranga Das Samrat</title>
    <description>The latest articles on DEV Community by Gouranga Das Samrat (@gouranga-das-khulna).</description>
    <link>https://dev.to/gouranga-das-khulna</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F2193879%2F834a1499-2027-4355-87be-bb678e90ae5c.jpg</url>
      <title>DEV Community: Gouranga Das Samrat</title>
      <link>https://dev.to/gouranga-das-khulna</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/gouranga-das-khulna"/>
    <language>en</language>
    <item>
      <title>CDN — Content Delivery Network</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Sat, 27 Jun 2026 02:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/cdn-content-delivery-network-1nnd</link>
      <guid>https://dev.to/gouranga-das-khulna/cdn-content-delivery-network-1nnd</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;One-liner:&lt;/strong&gt; A globally distributed network of servers that caches your static content close to users, dramatically reducing latency and offloading traffic from your origin server.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  📌 The Problem
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User in Mumbai → requests image from origin server in US-East
Latency: 200–300ms (round trip across the globe)

Multiply by: every image, CSS file, JS bundle, video chunk...
Result: Slow, painful experience
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  💡 The Solution: CDN
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User in Mumbai → requests image
CDN Edge Node in Mumbai → serves cached image
Latency: 5–20ms (local)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;CDN brings content &lt;strong&gt;closer to users&lt;/strong&gt; by caching it at edge locations worldwide.&lt;/p&gt;




&lt;h2&gt;
  
  
  🗺️ CDN Architecture
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;                        [Origin Server - US]
                               │
              ┌────────────────┼────────────────┐
              │                │                │
        [Edge - Mumbai]  [Edge - London]  [Edge - Singapore]
              │
         [User - India] ← 15ms →
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Cloudflare has 300+ edge locations. AWS CloudFront has 400+ PoPs (Points of Presence).&lt;/p&gt;




&lt;h2&gt;
  
  
  📦 What CDNs Cache
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Static Assets (Always CDN-appropriate)
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Images (JPEG, PNG, WebP, SVG)&lt;/li&gt;
&lt;li&gt;JavaScript bundles (&lt;code&gt;bundle.js&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;CSS files (&lt;code&gt;styles.css&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Fonts (&lt;code&gt;.woff2&lt;/code&gt;, &lt;code&gt;.ttf&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Videos (HLS/DASH streaming chunks)&lt;/li&gt;
&lt;li&gt;HTML (for static sites)&lt;/li&gt;
&lt;li&gt;PDF documents&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Dynamic Content (Advanced CDN features)
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;API responses (short TTL — 1–5 minutes)&lt;/li&gt;
&lt;li&gt;Personalized content (with Vary headers)&lt;/li&gt;
&lt;li&gt;HTML with Edge Side Includes (ESI)&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  🔄 Cache Flow: CDN Request Lifecycle
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Cache HIT (fast path)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User → CDN Edge → Cache HIT → Serve from edge
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Cache MISS (slow path — first request)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User → CDN Edge → Cache MISS
               → Fetch from Origin Server
               → Store in edge cache
               → Serve to user
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Cache INVALIDATION (when you deploy)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;You push new bundle.js →
Signal CDN to invalidate old bundle.js →
Next request → MISS → fetches new version → caches it
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  ⏱️ Cache Control Headers
&lt;/h2&gt;

&lt;p&gt;The origin server controls CDN caching via HTTP headers:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight http"&gt;&lt;code&gt;&lt;span class="err"&gt;# Cache for 1 year (immutable assets with content hash in filename)
Cache-Control: public, max-age=31536000, immutable

# Cache for 5 minutes (dynamic-ish content)
Cache-Control: public, max-age=300

# Don't cache (private user data)
Cache-Control: private, no-store

# Cache but validate with server first
Cache-Control: no-cache
ETag: "abc123"
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Filename Hashing Strategy
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;bundle.js       → hard to cache long (changes on deploy)
bundle.a3f4c1.js → cache for 1 year (hash changes on deploy)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🔒 CDN Security Features
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Feature&lt;/th&gt;
&lt;th&gt;Description&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;DDoS Protection&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Absorb volumetric attacks at edge (Cloudflare: 150+ Tbps capacity)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;WAF&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Web Application Firewall — block SQLi, XSS at edge&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Bot Protection&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Detect and block malicious bots&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;TLS Termination&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Handle HTTPS at edge globally&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Rate Limiting&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Throttle requests at edge before they hit origin&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Geo-blocking&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Block traffic from specific countries&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  ⚙️ CDN Configuration Example (Cloudflare)
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;Page Rules&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
&lt;span class="err"&gt;*&lt;/span&gt;&lt;span class="s"&gt;.myapp.com/static/*  → Cache Everything, Edge TTL&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt; &lt;span class="s"&gt;1 year&lt;/span&gt;
&lt;span class="err"&gt;*&lt;/span&gt;&lt;span class="s"&gt;.myapp.com/api/*     → Bypass Cache (dynamic)&lt;/span&gt;
&lt;span class="err"&gt;*&lt;/span&gt;&lt;span class="s"&gt;.myapp.com/*.html    → Cache, Edge TTL&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt; &lt;span class="s"&gt;1 hour&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🔄 Push vs Pull CDN
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Pull CDN (Most Common)
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Origin server remains the source of truth&lt;/li&gt;
&lt;li&gt;CDN fetches content from origin &lt;strong&gt;on first request&lt;/strong&gt; (cache miss)&lt;/li&gt;
&lt;li&gt;Automatic — no manual upload needed&lt;/li&gt;
&lt;li&gt;Examples: Cloudflare, CloudFront&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Push CDN
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;You explicitly &lt;strong&gt;upload&lt;/strong&gt; content to CDN&lt;/li&gt;
&lt;li&gt;CDN serves it from edge; origin doesn't need to exist for every request&lt;/li&gt;
&lt;li&gt;Good for known, large files (video, software downloads)&lt;/li&gt;
&lt;li&gt;Examples: AWS S3 + CloudFront for uploads, Akamai NetStorage&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  🌍 Popular CDN Providers
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Provider&lt;/th&gt;
&lt;th&gt;Strength&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Cloudflare&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Best DDoS protection, free tier, 300+ PoPs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;AWS CloudFront&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Deep AWS integration, Lambda@Edge&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Fastly&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Real-time purging, developer-friendly&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Akamai&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Largest network, enterprise&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;BunnyCDN&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Cheap, fast, developer-friendly&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  📊 CDN Impact on Performance
&lt;/h2&gt;

&lt;p&gt;Without CDN (Mumbai user, US origin):&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Latency: ~200ms&lt;/li&gt;
&lt;li&gt;Origin bandwidth: 100GB/day&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;With CDN:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Latency: ~15ms (13× faster!)&lt;/li&gt;
&lt;li&gt;Cache hit ratio: ~85%&lt;/li&gt;
&lt;li&gt;Origin bandwidth: ~15GB/day (85% reduction)&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  🎨 Diagram
&lt;/h2&gt;

&lt;p&gt;The diagram shows:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;World map with edge nodes&lt;/li&gt;
&lt;li&gt;Cache HIT vs MISS flows&lt;/li&gt;
&lt;li&gt;Origin server → edge propagation&lt;/li&gt;
&lt;li&gt;Cache-Control header on HTTP response&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  🔑 Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;CDN = &lt;strong&gt;geographic caching&lt;/strong&gt; — serve content from the closest edge node&lt;/li&gt;
&lt;li&gt;Reduces latency, reduces origin load, handles DDoS&lt;/li&gt;
&lt;li&gt;Use long TTLs + &lt;strong&gt;content hashing&lt;/strong&gt; for static assets&lt;/li&gt;
&lt;li&gt;Most modern apps should use a CDN from day 1 (Cloudflare free tier is excellent)&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>systemdesign</category>
      <category>api</category>
      <category>backend</category>
      <category>webdev</category>
    </item>
    <item>
      <title>API Gateway</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Sun, 21 Jun 2026 02:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/api-gateway-4dok</link>
      <guid>https://dev.to/gouranga-das-khulna/api-gateway-4dok</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;One-liner:&lt;/strong&gt; A single entry point for all client requests that handles cross-cutting concerns like auth, rate limiting, routing, and logging — so your microservices don't have to.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  📌 The Problem
&lt;/h2&gt;

&lt;p&gt;In a microservices architecture, clients would need to call many services directly:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Mobile App → auth-service:3001
           → user-service:3002
           → order-service:3003
           → payment-service:3004
           → notification-service:3005
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Problems:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Client needs to know every service's address&lt;/li&gt;
&lt;li&gt;Auth logic duplicated in every service&lt;/li&gt;
&lt;li&gt;CORS, rate limiting repeated everywhere&lt;/li&gt;
&lt;li&gt;No single point for monitoring&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  💡 The Solution: API Gateway
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Mobile App ──────────────► [API Gateway]
                                │
                                ├─► auth-service
                                ├─► user-service
                                ├─► order-service
                                └─► payment-service
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The gateway is the &lt;strong&gt;single front door&lt;/strong&gt; to your backend.&lt;/p&gt;




&lt;h2&gt;
  
  
  🛠️ What API Gateways Do
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Request Routing
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;POST /auth/login      → auth-service
GET  /users/:id       → user-service
POST /orders          → order-service
GET  /products        → product-service
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  2. Authentication &amp;amp; Authorization
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Client → [Gateway]
          → Validate JWT token
          → If invalid: 401 Unauthorized (request never hits services)
          → If valid: forward with user context
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  3. Rate Limiting
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User A: 100 requests/min → allow
User A: 101st request    → 429 Too Many Requests
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  4. SSL Termination
&lt;/h3&gt;

&lt;p&gt;Handles HTTPS at the gateway; internal traffic can be HTTP.&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Request/Response Transformation
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="err"&gt;Client&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;sends:&lt;/span&gt;&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"user_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;123&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="err"&gt;Gateway&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;transforms&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;to:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"userId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"user_123"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"timestamp"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"..."&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  6. Load Balancing
&lt;/h3&gt;

&lt;p&gt;Routes to healthy instances of each microservice.&lt;/p&gt;

&lt;h3&gt;
  
  
  7. Caching
&lt;/h3&gt;

&lt;p&gt;Cache responses at the gateway level for frequently accessed data.&lt;/p&gt;

&lt;h3&gt;
  
  
  8. Logging &amp;amp; Monitoring
&lt;/h3&gt;

&lt;p&gt;Single place to log all incoming requests, response times, error rates.&lt;/p&gt;

&lt;h3&gt;
  
  
  9. Circuit Breaking
&lt;/h3&gt;

&lt;p&gt;If a downstream service is failing, stop sending requests → return cached/fallback response.&lt;/p&gt;

&lt;h3&gt;
  
  
  10. API Versioning
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;/v1/users → old-user-service (v1)
/v2/users → new-user-service (v2)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🔄 Request Lifecycle Through Gateway
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Client Request
    │
    ▼
[1] SSL Termination
    │
    ▼
[2] Authentication (JWT/OAuth validation)
    │
    ▼
[3] Rate Limiting Check
    │
    ▼
[4] Request Routing (which service?)
    │
    ▼
[5] Request Transformation (headers, body)
    │
    ▼
[6] Forward to Microservice
    │
    ▼
[7] Response Transformation
    │
    ▼
[8] Logging &amp;amp; Metrics
    │
    ▼
Client Response
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  ⚖️ API Gateway vs Load Balancer
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Feature&lt;/th&gt;
&lt;th&gt;Load Balancer&lt;/th&gt;
&lt;th&gt;API Gateway&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Primary role&lt;/td&gt;
&lt;td&gt;Distribute traffic&lt;/td&gt;
&lt;td&gt;Smart routing + cross-cutting concerns&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Layer&lt;/td&gt;
&lt;td&gt;L4 or L7&lt;/td&gt;
&lt;td&gt;L7 (always)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Authentication&lt;/td&gt;
&lt;td&gt;❌&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Rate limiting&lt;/td&gt;
&lt;td&gt;Limited&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Request transformation&lt;/td&gt;
&lt;td&gt;❌&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Circuit breaking&lt;/td&gt;
&lt;td&gt;❌&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Routing by path&lt;/td&gt;
&lt;td&gt;L7 only&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Overhead&lt;/td&gt;
&lt;td&gt;Low&lt;/td&gt;
&lt;td&gt;Higher&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;blockquote&gt;
&lt;p&gt;Often used together: LB in front of API Gateway for the gateway's own HA.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  ⚖️ API Gateway vs Reverse Proxy
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Feature&lt;/th&gt;
&lt;th&gt;Reverse Proxy&lt;/th&gt;
&lt;th&gt;API Gateway&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Purpose&lt;/td&gt;
&lt;td&gt;Forward requests, hide servers&lt;/td&gt;
&lt;td&gt;Orchestrate microservices&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Auth&lt;/td&gt;
&lt;td&gt;❌&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Business logic&lt;/td&gt;
&lt;td&gt;Minimal&lt;/td&gt;
&lt;td&gt;Yes (routing rules, transforms)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Example&lt;/td&gt;
&lt;td&gt;Nginx, HAProxy&lt;/td&gt;
&lt;td&gt;Kong, AWS API Gateway&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  🌍 Popular API Gateway Products
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Product&lt;/th&gt;
&lt;th&gt;Type&lt;/th&gt;
&lt;th&gt;Best For&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Kong&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Open source&lt;/td&gt;
&lt;td&gt;Self-hosted, plugin ecosystem&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;AWS API Gateway&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Managed&lt;/td&gt;
&lt;td&gt;AWS ecosystem, serverless&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Nginx&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Open source&lt;/td&gt;
&lt;td&gt;Can be configured as gateway&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Traefik&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Open source&lt;/td&gt;
&lt;td&gt;Docker/Kubernetes native&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Apigee&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Enterprise&lt;/td&gt;
&lt;td&gt;Google Cloud, enterprise features&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Azure API Management&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Managed&lt;/td&gt;
&lt;td&gt;Azure ecosystem&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  ⚠️ Gateway Pitfalls
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Pitfall&lt;/th&gt;
&lt;th&gt;Impact&lt;/th&gt;
&lt;th&gt;Fix&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Gateway is SPOF&lt;/td&gt;
&lt;td&gt;If gateway dies, everything dies&lt;/td&gt;
&lt;td&gt;Run multiple instances behind LB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Too much logic in gateway&lt;/td&gt;
&lt;td&gt;Gateway becomes a monolith&lt;/td&gt;
&lt;td&gt;Keep it thin — only cross-cutting concerns&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;High latency&lt;/td&gt;
&lt;td&gt;Every request passes through gateway&lt;/td&gt;
&lt;td&gt;Optimize, cache at gateway, keep logic light&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Tight coupling&lt;/td&gt;
&lt;td&gt;Gateway knows too much about services&lt;/td&gt;
&lt;td&gt;Use routing rules, not business logic&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  🎨 Diagram
&lt;/h2&gt;

&lt;p&gt;The diagram shows:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Client → Gateway → multiple microservices&lt;/li&gt;
&lt;li&gt;Auth, Rate Limiting, Logging modules inside gateway&lt;/li&gt;
&lt;li&gt;Gateway cluster (multiple instances) behind a load balancer&lt;/li&gt;
&lt;li&gt;Circuit breaker to a failing service&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  🔑 Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;API Gateway = &lt;strong&gt;single entry point&lt;/strong&gt; + cross-cutting concerns&lt;/li&gt;
&lt;li&gt;Move auth, rate limiting, logging &lt;strong&gt;out of services&lt;/strong&gt; and into the gateway&lt;/li&gt;
&lt;li&gt;The gateway can become a bottleneck/SPOF — make it HA and keep it lightweight&lt;/li&gt;
&lt;li&gt;Don't put business logic in the gateway — that belongs in services&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>systemdesign</category>
      <category>api</category>
      <category>backend</category>
    </item>
    <item>
      <title>SQL vs NoSQL</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Sat, 20 Jun 2026 02:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/sql-vs-nosql-f69</link>
      <guid>https://dev.to/gouranga-das-khulna/sql-vs-nosql-f69</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;One-liner:&lt;/strong&gt; SQL databases store data in structured tables with strict schemas; NoSQL databases trade strict consistency for flexibility and horizontal scalability.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  📌 SQL (Relational Databases)
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Core Properties
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Structured&lt;/strong&gt; — Data in tables with rows and columns&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Schema&lt;/strong&gt; — Fixed structure, must be defined upfront&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;ACID&lt;/strong&gt; — Atomicity, Consistency, Isolation, Durability&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Relations&lt;/strong&gt; — Foreign keys, JOINs across tables&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Query Language&lt;/strong&gt; — SQL (standardized)&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Examples
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Database&lt;/th&gt;
&lt;th&gt;Best For&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;PostgreSQL&lt;/td&gt;
&lt;td&gt;Complex queries, JSONB support, general purpose&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;MySQL&lt;/td&gt;
&lt;td&gt;Web apps, e-commerce&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SQLite&lt;/td&gt;
&lt;td&gt;Local/embedded, mobile apps&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oracle&lt;/td&gt;
&lt;td&gt;Enterprise, banking&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;MS SQL Server&lt;/td&gt;
&lt;td&gt;Microsoft ecosystem&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  When to Use SQL
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;✅ Complex queries with JOINs&lt;/li&gt;
&lt;li&gt;✅ Strong consistency required (banking, payments)&lt;/li&gt;
&lt;li&gt;✅ Data is relational by nature (users → orders → products)&lt;/li&gt;
&lt;li&gt;✅ ACID transactions needed&lt;/li&gt;
&lt;li&gt;✅ Data structure is well-known and unlikely to change&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  📌 NoSQL (Non-Relational Databases)
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Core Properties
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Flexible schema&lt;/strong&gt; — Add/remove fields without migrations&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Horizontally scalable&lt;/strong&gt; — Designed to scale out&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Eventual consistency&lt;/strong&gt; — Usually (some offer strong consistency)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Optimized for specific access patterns&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Types of NoSQL
&lt;/h3&gt;

&lt;h4&gt;
  
  
  1. Document Store
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="err"&gt;//&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;MongoDB,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;CouchDB,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;Firestore&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"user_123"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Rahul"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"address"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"city"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Delhi"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"pin"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"110001"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"tags"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"premium"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"verified"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Best for: User profiles, product catalogs, CMS&lt;/p&gt;

&lt;h4&gt;
  
  
  2. Key-Value Store
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Redis, DynamoDB, Memcached
"session:abc123" → { userId: 42, expiry: ... }
"counter:pageviews" → 1000000
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Best for: Sessions, caching, real-time counters, leaderboards&lt;/p&gt;

&lt;h4&gt;
  
  
  3. Column-Family Store
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Cassandra, HBase
Row key → { column1: val, column2: val, column3: val }
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Best for: Time-series data, IoT, write-heavy workloads, Netflix viewing history&lt;/p&gt;

&lt;h4&gt;
  
  
  4. Graph Database
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Neo4j, Amazon Neptune
Nodes (Person) → Edges (FOLLOWS) → Nodes (Person)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Best for: Social networks, fraud detection, recommendation engines&lt;/p&gt;




&lt;h2&gt;
  
  
  ⚖️ The Comparison
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Dimension&lt;/th&gt;
&lt;th&gt;SQL&lt;/th&gt;
&lt;th&gt;NoSQL&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Schema&lt;/td&gt;
&lt;td&gt;Fixed&lt;/td&gt;
&lt;td&gt;Flexible&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Horizontal scaling&lt;/td&gt;
&lt;td&gt;Hard&lt;/td&gt;
&lt;td&gt;Easy&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Transactions&lt;/td&gt;
&lt;td&gt;Full ACID&lt;/td&gt;
&lt;td&gt;Often eventual consistency&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;JOINs&lt;/td&gt;
&lt;td&gt;Easy&lt;/td&gt;
&lt;td&gt;Avoid (denormalize)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Query flexibility&lt;/td&gt;
&lt;td&gt;High (SQL)&lt;/td&gt;
&lt;td&gt;Limited to access patterns&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Learning curve&lt;/td&gt;
&lt;td&gt;Moderate&lt;/td&gt;
&lt;td&gt;Varies by type&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Maturity&lt;/td&gt;
&lt;td&gt;50+ years&lt;/td&gt;
&lt;td&gt;15–20 years&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  🔄 ACID vs BASE
&lt;/h2&gt;

&lt;h3&gt;
  
  
  ACID (SQL)
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Property&lt;/th&gt;
&lt;th&gt;Meaning&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Atomicity&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;All or nothing — transaction either fully succeeds or fully fails&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Consistency&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;DB always goes from one valid state to another&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Isolation&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Concurrent transactions don't interfere&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Durability&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Once committed, data survives crashes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  BASE (NoSQL)
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Property&lt;/th&gt;
&lt;th&gt;Meaning&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Basically Available&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;System always responds (maybe stale data)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Soft state&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;State can change over time, even without input&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Eventually consistent&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;System will eventually be consistent&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  🏗️ Choosing Between SQL and NoSQL
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Decision Tree:

Is the data relational?
├── Yes → SQL (start here)
└── No ↓
    Do you need flexible schema?
    ├── Yes → Document (MongoDB)
    └── No ↓
        Is it write-heavy time-series?
        ├── Yes → Column (Cassandra)
        └── No ↓
            Is it graph-traversal heavy?
            ├── Yes → Graph (Neo4j)
            └── No → Key-Value (Redis/DynamoDB)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🌍 Real-World Usage Patterns
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Company&lt;/th&gt;
&lt;th&gt;SQL&lt;/th&gt;
&lt;th&gt;NoSQL&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Uber&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;PostgreSQL (trips)&lt;/td&gt;
&lt;td&gt;Cassandra (location updates)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Netflix&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;MySQL (billing)&lt;/td&gt;
&lt;td&gt;Cassandra (viewing history)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Instagram&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;PostgreSQL (user data)&lt;/td&gt;
&lt;td&gt;Redis (feed cache)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Twitter&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;MySQL&lt;/td&gt;
&lt;td&gt;Redis (timeline), Cassandra (tweets)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Airbnb&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;MySQL&lt;/td&gt;
&lt;td&gt;ElasticSearch (search)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;blockquote&gt;
&lt;p&gt;Most large systems use &lt;strong&gt;both&lt;/strong&gt; — polyglot persistence!&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  🔑 Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Default to SQL&lt;/strong&gt; for new projects — better tooling, stronger guarantees&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Move to NoSQL&lt;/strong&gt; when you hit specific scale/flexibility walls&lt;/li&gt;
&lt;li&gt;Most production systems use &lt;strong&gt;both&lt;/strong&gt; (polyglot persistence)&lt;/li&gt;
&lt;li&gt;NoSQL doesn't mean "no schema" — you still have implicit structure&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>systemdesign</category>
      <category>database</category>
      <category>sql</category>
      <category>nosql</category>
    </item>
    <item>
      <title>Read Replicas</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Sun, 14 Jun 2026 02:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/read-replicas-48ng</link>
      <guid>https://dev.to/gouranga-das-khulna/read-replicas-48ng</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;One-liner:&lt;/strong&gt; Copies of your primary database that handle read queries, freeing the primary to focus on writes.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  📌 The Problem
&lt;/h2&gt;

&lt;p&gt;Most web applications are &lt;strong&gt;read-heavy&lt;/strong&gt; (80-90% reads, 10-20% writes).&lt;/p&gt;

&lt;p&gt;A single database server handles both reads and writes → becomes a bottleneck.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;All traffic → [Primary DB]
              CPU: 95% 🔥
              Connections: maxed out 🔥
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  💡 The Solution: Read Replicas
&lt;/h2&gt;

&lt;p&gt;Add secondary copies of the database:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Primary&lt;/strong&gt; (Master) → handles all &lt;strong&gt;writes&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Replicas&lt;/strong&gt; (Slaves) → handle all &lt;strong&gt;reads&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[App Server 1] ──write──► [Primary DB] ──replicates──► [Replica 1]
[App Server 2] ──read──►  [Replica 1]                 [Replica 2]
[App Server 3] ──read──►  [Replica 2]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🔄 How Replication Works
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Synchronous Replication
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Write → Primary → waits for Replica ACK → confirms to client
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;✅ Zero data loss — Replica always has latest data&lt;/li&gt;
&lt;li&gt;❌ Higher write latency — must wait for replica confirmation&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Asynchronous Replication
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Write → Primary → confirms to client → replicates to Replica in background
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;✅ Lower write latency&lt;/li&gt;
&lt;li&gt;❌ Replica lag — reads might return slightly stale data&lt;/li&gt;
&lt;li&gt;❌ If primary crashes before replication, data loss possible&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Semi-Synchronous
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Write → Primary → waits for ACK from at least 1 replica → confirms
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;Balance between safety and latency (MySQL's default option)&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  📊 Replication Lag
&lt;/h2&gt;

&lt;p&gt;Replica lag is the delay between a write on primary and it being visible on replica.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Lag&lt;/th&gt;
&lt;th&gt;Impact&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&amp;lt; 100ms&lt;/td&gt;
&lt;td&gt;Usually fine — imperceptible to users&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100ms – 1s&lt;/td&gt;
&lt;td&gt;Acceptable for most use cases&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&amp;gt; 1s&lt;/td&gt;
&lt;td&gt;Noticeable — "I just posted but I can't see it" bug&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Minutes&lt;/td&gt;
&lt;td&gt;Data consistency issue — investigate urgently&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Handling Lag in Code
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Read-after-write consistency:&lt;/span&gt;
&lt;span class="c1"&gt;// After a user writes, route their next read to primary&lt;/span&gt;

&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;getUserProfile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;userId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;justUpdated&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;justUpdated&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;primaryDB&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;query&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;SELECT * FROM users WHERE id = ?&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;userId&lt;/span&gt;&lt;span class="p"&gt;]);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;replicaDB&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;query&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;SELECT * FROM users WHERE id = ?&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;userId&lt;/span&gt;&lt;span class="p"&gt;]);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🏗️ Architecture Patterns
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Single Replica (Simple)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Writes → [Primary] → [Replica 1] ← Reads
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Multiple Replicas (Common)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;         [Primary] ──► [Replica 1]  \
Writes →      │    ──► [Replica 2]  ──► Read traffic distributed
              └────► [Replica 3]  /
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Cascading Replicas
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Primary] → [Replica 1] → [Replica 2] → [Replica 3]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Reduces load on primary but increases replica lag.&lt;/p&gt;

&lt;h3&gt;
  
  
  Regional Replicas (Multi-region)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Primary - Mumbai] ──► [Replica - Singapore]
                   ──► [Replica - US-East]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Serve reads from the closest region to the user.&lt;/p&gt;




&lt;h2&gt;
  
  
  🔄 Failover: When Primary Dies
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Detect failure&lt;/strong&gt; — health check fails&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Elect new primary&lt;/strong&gt; — replica with least lag is promoted&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Redirect writes&lt;/strong&gt; — app/load balancer routes writes to new primary&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Old primary rejoins&lt;/strong&gt; — becomes a replica when it comes back
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Before: App → Primary(A) → Replica(B)
Failure: Primary(A) dies
After:  App → Primary(B)   [B promoted]
              ↑
           Replica(A) [A rejoins as replica]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  ⚠️ Read Replica Caveats
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Caveat&lt;/th&gt;
&lt;th&gt;Detail&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Eventual consistency&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Reads may return stale data&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Write bottleneck&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Replicas help reads; writes still limited to primary&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Complex routing&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Need to distinguish read vs write queries&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Cost&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Each replica = another database server&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  🌍 When to Add Read Replicas
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;DB CPU &amp;gt; 70% sustained → add replica
Read:Write ratio &amp;gt; 5:1 → add replica
Reporting queries slowing down app → add dedicated analytics replica
Multi-region users → add geo-replica
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🎨 Diagram
&lt;/h2&gt;

&lt;p&gt;The diagram shows:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Primary DB receiving writes&lt;/li&gt;
&lt;li&gt;Replication arrows to 2-3 replicas&lt;/li&gt;
&lt;li&gt;App servers routing reads to replicas&lt;/li&gt;
&lt;li&gt;Failover promotion flow&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  🔑 Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Read replicas solve &lt;strong&gt;read scalability&lt;/strong&gt; — writes still bottleneck on primary&lt;/li&gt;
&lt;li&gt;Use &lt;strong&gt;async replication&lt;/strong&gt; for performance, &lt;strong&gt;sync&lt;/strong&gt; for zero data loss&lt;/li&gt;
&lt;li&gt;Always handle &lt;strong&gt;replica lag&lt;/strong&gt; in your application code&lt;/li&gt;
&lt;li&gt;Read replicas are the first step; sharding is the next if writes bottleneck&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>systemdesign</category>
      <category>database</category>
    </item>
    <item>
      <title>Database Sharding</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Sat, 13 Jun 2026 02:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/database-sharding-2ihl</link>
      <guid>https://dev.to/gouranga-das-khulna/database-sharding-2ihl</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;One-liner:&lt;/strong&gt; Splitting a large database into smaller, independent pieces (shards) spread across multiple machines — each shard holds a subset of the data.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  📌 Why Sharding?
&lt;/h2&gt;

&lt;p&gt;Read replicas help with reads. But what when &lt;strong&gt;writes&lt;/strong&gt; bottleneck?&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;10M writes/day → 1 Primary DB can't keep up
Data: 100 TB → 1 server can't store it all
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Sharding splits both storage AND write load.&lt;/p&gt;




&lt;h2&gt;
  
  
  💡 How Sharding Works
&lt;/h2&gt;

&lt;p&gt;Imagine a &lt;code&gt;users&lt;/code&gt; table with 100M rows.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Without sharding:&lt;/strong&gt; All 100M rows on one server.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;With sharding (4 shards):&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Shard 0: users where userId % 4 = 0  (25M rows)
Shard 1: users where userId % 4 = 1  (25M rows)
Shard 2: users where userId % 4 = 2  (25M rows)
Shard 3: users where userId % 4 = 3  (25M rows)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🔑 Sharding Strategies
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Hash-Based Sharding
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;shard&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;hash&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;shardKey&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;%&lt;/span&gt; &lt;span class="n"&gt;numShards&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;✅ Even distribution&lt;br&gt;&lt;br&gt;
❌ Adding shards → massive rebalancing (use consistent hashing!)&lt;br&gt;&lt;br&gt;
Best for: User IDs, random keys&lt;/p&gt;
&lt;h3&gt;
  
  
  2. Range-Based Sharding
&lt;/h3&gt;


&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Shard 0: userId 1–1,000,000
Shard 1: userId 1,000,001–2,000,000
...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;✅ Simple, range queries efficient&lt;br&gt;&lt;br&gt;
❌ Hot spots if data isn't evenly distributed (new users pile into last shard)&lt;br&gt;&lt;br&gt;
Best for: Time-series data (date ranges), ordered data&lt;/p&gt;
&lt;h3&gt;
  
  
  3. Directory-Based Sharding
&lt;/h3&gt;


&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Lookup Service / Mapping Table]
userId 1234 → Shard 3
userId 5678 → Shard 1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;✅ Most flexible — can move data between shards&lt;br&gt;&lt;br&gt;
❌ Lookup service becomes bottleneck / SPOF&lt;br&gt;&lt;br&gt;
Best for: When data movement between shards is needed&lt;/p&gt;
&lt;h3&gt;
  
  
  4. Geographic Sharding
&lt;/h3&gt;


&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Users in India    → Shard-India  (Mumbai)
Users in US       → Shard-US     (Virginia)
Users in Europe   → Shard-EU     (Frankfurt)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;✅ Low latency for users&lt;br&gt;&lt;br&gt;
❌ Cross-shard queries for global reports&lt;br&gt;&lt;br&gt;
Best for: Multi-region products with data residency requirements&lt;/p&gt;


&lt;h2&gt;
  
  
  🔑 Choosing a Shard Key
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;The most critical decision in sharding.&lt;/strong&gt; A bad shard key causes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Hot spots&lt;/strong&gt; — all traffic goes to one shard&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Uneven data&lt;/strong&gt; — one shard is 10× larger&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cross-shard joins&lt;/strong&gt; — expensive queries&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Good Shard Key Properties
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Property&lt;/th&gt;
&lt;th&gt;Why&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;High cardinality&lt;/td&gt;
&lt;td&gt;Many unique values → even distribution&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Low correlation&lt;/td&gt;
&lt;td&gt;Shouldn't correlate with access patterns&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Immutable&lt;/td&gt;
&lt;td&gt;Changing it means moving data between shards&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Frequently queried&lt;/td&gt;
&lt;td&gt;Queries can be routed to one shard&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;
&lt;h3&gt;
  
  
  Common Shard Keys
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;System&lt;/th&gt;
&lt;th&gt;Shard Key&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;User data&lt;/td&gt;
&lt;td&gt;&lt;code&gt;userId&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Multi-tenant SaaS&lt;/td&gt;
&lt;td&gt;&lt;code&gt;tenantId&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Messages&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;conversationId&lt;/code&gt; or &lt;code&gt;userId&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Orders&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;customerId&lt;/code&gt; or &lt;code&gt;orderId&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Time-series&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;timestamp&lt;/code&gt; + &lt;code&gt;deviceId&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;


&lt;h2&gt;
  
  
  ⚠️ Sharding Challenges
&lt;/h2&gt;
&lt;h3&gt;
  
  
  Cross-Shard Queries
&lt;/h3&gt;


&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- This is easy on 1 DB:&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;orders&lt;/span&gt; &lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="n"&gt;users&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;orders&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;userId&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;users&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt;

&lt;span class="c1"&gt;-- With sharding: users on Shard 0, orders on Shard 1 → expensive cross-shard join!&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Solutions:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Denormalize data (store redundant user info in orders table)&lt;/li&gt;
&lt;li&gt;Application-level joins (fetch from each shard, merge in code)&lt;/li&gt;
&lt;li&gt;Co-locate related data on same shard (order and user on same shard)&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Rebalancing
&lt;/h3&gt;

&lt;p&gt;Adding a new shard → need to move data → expensive operation&lt;/p&gt;

&lt;p&gt;Solution: &lt;strong&gt;Consistent hashing&lt;/strong&gt; minimizes data movement&lt;/p&gt;
&lt;h3&gt;
  
  
  Hot Spots
&lt;/h3&gt;

&lt;p&gt;Celebrity/viral content — 90% of requests go to one shard.&lt;/p&gt;

&lt;p&gt;Solution: &lt;strong&gt;Key splitting&lt;/strong&gt; — split the hot key across multiple shards with a suffix:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;tweet:viral_id_0, tweet:viral_id_1, tweet:viral_id_2 ...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🏗️ Sharding Architecture
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[App Server] → [Shard Router / Proxy Layer]
                    │
          ┌────────┼────────┐
       [Shard 0] [Shard 1] [Shard 2]
          │          │         │
       [Replica] [Replica] [Replica]  (each shard also has replicas!)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🆚 Sharding vs Partitioning
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Term&lt;/th&gt;
&lt;th&gt;Meaning&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Sharding&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Horizontal partitioning across &lt;strong&gt;different machines&lt;/strong&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Partitioning&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Splitting data within &lt;strong&gt;one machine&lt;/strong&gt; (e.g., PostgreSQL table partitions)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Start with partitioning (within one DB), then shard when you need cross-machine scale.&lt;/p&gt;




&lt;h2&gt;
  
  
  🎨 Diagram
&lt;/h2&gt;

&lt;p&gt;The diagram shows:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Shard router receiving queries&lt;/li&gt;
&lt;li&gt;Hash-based routing to 3 shards&lt;/li&gt;
&lt;li&gt;Each shard with its own replica&lt;/li&gt;
&lt;li&gt;Cross-shard join problem highlighted in red&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  🔑 Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Sharding splits &lt;strong&gt;both data and write load&lt;/strong&gt; across machines&lt;/li&gt;
&lt;li&gt;Shard key choice is &lt;strong&gt;irreversible&lt;/strong&gt; — choose carefully&lt;/li&gt;
&lt;li&gt;Start with &lt;strong&gt;vertical scaling → read replicas → sharding&lt;/strong&gt; (don't jump to sharding early)&lt;/li&gt;
&lt;li&gt;Every large database (DynamoDB, Cassandra, MongoDB) shards internally for you&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>database</category>
      <category>systemdesign</category>
      <category>sql</category>
      <category>postgres</category>
    </item>
    <item>
      <title>🥧 314 Trillion Digits of Pi: The Software Engineering Secrets Behind y-cruncher</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Thu, 11 Jun 2026 04:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/314-trillion-digits-of-pi-the-software-engineering-secrets-behind-y-cruncher-5ddp</link>
      <guid>https://dev.to/gouranga-das-khulna/314-trillion-digits-of-pi-the-software-engineering-secrets-behind-y-cruncher-5ddp</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;How a high school project became the most dominant Pi-computing benchmark in the world — and what every software engineer can learn from it.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;If someone told you a single program could stress-test your CPU, RAM, and storage simultaneously, recover from hardware failures mid-computation, run for &lt;strong&gt;110 days straight&lt;/strong&gt;, and spit out 314 trillion digits of Pi at the end — you'd probably assume it was built by a team of PhDs at a national lab.&lt;/p&gt;

&lt;p&gt;It was built by one person. It started as a high school project. And it's been setting world records since 2009.&lt;/p&gt;

&lt;p&gt;This is y-cruncher. Let's talk about it.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Even Is y-cruncher?
&lt;/h2&gt;

&lt;p&gt;&lt;a href="http://www.numberworld.org/y-cruncher/" rel="noopener noreferrer"&gt;y-cruncher&lt;/a&gt; is a multi-threaded, SIMD-vectorized program that computes mathematical constants — Pi, e, square roots, and more — to trillions of decimal digits. It's the tool of choice for:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;World record Pi computations&lt;/strong&gt; (every record since 2009 has used it)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;CPU stress testing and overclocking validation&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Memory subsystem benchmarking&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hardware stability detection&lt;/strong&gt; (it'll find flaws that Prime95 and AIDA64 miss)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;As of November 2025, the current world record stands at &lt;strong&gt;314 trillion digits&lt;/strong&gt;, computed in a single uninterrupted 110-day run on a 384-core AMD EPYC server. The verification took just 4.37 hours.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why Should a Software Engineer Care?
&lt;/h2&gt;

&lt;p&gt;Fair question. You're probably not computing Pi for a living. But y-cruncher is a goldmine of fascinating engineering decisions:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;It exploits &lt;strong&gt;SIMD instruction sets&lt;/strong&gt; (SSE, AVX, AVX-512) at a level most production software never touches&lt;/li&gt;
&lt;li&gt;Its &lt;strong&gt;checkpoint-restart system&lt;/strong&gt; is a masterclass in fault-tolerant distributed computation&lt;/li&gt;
&lt;li&gt;It implements &lt;strong&gt;custom memory allocators&lt;/strong&gt; that outperform the OS for specific access patterns&lt;/li&gt;
&lt;li&gt;It demonstrates how &lt;strong&gt;multi-socket NUMA topology&lt;/strong&gt; wreaks havoc on parallel performance — and how to fight back&lt;/li&gt;
&lt;li&gt;Its benchmark results expose the &lt;strong&gt;memory bandwidth ceiling&lt;/strong&gt; that most workloads never hit but y-cruncher constantly runs into&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In short: reading about y-cruncher will make you a better systems programmer, even if you never run it.&lt;/p&gt;




&lt;h2&gt;
  
  
  Getting Started: Installation in Under 2 Minutes
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Windows
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;Download &lt;code&gt;y-cruncher v0.8.7.9547b.zip&lt;/code&gt; from the &lt;a href="http://www.numberworld.org/y-cruncher/" rel="noopener noreferrer"&gt;official site&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Extract and run &lt;code&gt;y-cruncher.exe&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;You may need the &lt;a href="https://aka.ms/vs/17/release/vc_redist.x64.exe" rel="noopener noreferrer"&gt;MSVC redistributable&lt;/a&gt; if you see DLL errors&lt;/li&gt;
&lt;/ol&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Antivirus false positives are common due to the low-level SIMD code. The binary is safe — but the static-linked version was reworked specifically to reduce false positives.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3&gt;
  
  
  Linux
&lt;/h3&gt;

&lt;p&gt;Choose between two variants based on your needs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Static — most portable, works on nearly any distro, no TBB/NUMA binding&lt;/span&gt;
wget http://www.numberworld.org/y-cruncher/y-cruncher&lt;span class="se"&gt;\ &lt;/span&gt;v0.8.7.9547-static.tar.xz
&lt;span class="nb"&gt;tar&lt;/span&gt; &lt;span class="nt"&gt;-xf&lt;/span&gt; &lt;span class="s2"&gt;"y-cruncher v0.8.7.9547-static.tar.xz"&lt;/span&gt;
&lt;span class="nb"&gt;cd &lt;/span&gt;y-cruncher_v0.8.7.9547-static
./y-cruncher

&lt;span class="c"&gt;# Dynamic — full features (NUMA binding, TBB) but requires Ubuntu 24.04+ or compatible&lt;/span&gt;
wget http://www.numberworld.org/y-cruncher/y-cruncher&lt;span class="se"&gt;\ &lt;/span&gt;v0.8.7.9547-dynamic.tar.xz
&lt;span class="nb"&gt;tar&lt;/span&gt; &lt;span class="nt"&gt;-xf&lt;/span&gt; &lt;span class="s2"&gt;"y-cruncher v0.8.7.9547-dynamic.tar.xz"&lt;/span&gt;
./y-cruncher
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;System requirements:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;64-bit x86/x64 processor&lt;/li&gt;
&lt;li&gt;Windows 8+ or any 64-bit Linux distro&lt;/li&gt;
&lt;li&gt;RAM: as much as you can get — more is almost always better&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Running Your First Benchmark
&lt;/h2&gt;

&lt;p&gt;When you launch y-cruncher, you'll get a console menu. For benchmarking:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Select &lt;strong&gt;"Benchmark"&lt;/strong&gt; from the main menu&lt;/li&gt;
&lt;li&gt;Choose a size (start with &lt;strong&gt;250 million&lt;/strong&gt; or &lt;strong&gt;1 billion&lt;/strong&gt; digits — comfortable for most modern desktops)&lt;/li&gt;
&lt;li&gt;Watch it go&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;What you'll see reported:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Computation mode : Ram Only
Decimal Digits   : 1,000,000,000
Hexadecimal Digits: 830,482,023

Start Date       : ...
End Date         : ...

Total Computation Time : 14.670 seconds
Total Verification Time: 10.421 seconds
Total Time             : 25.091 seconds
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Tip:&lt;/strong&gt; "Total Computation Time" is the relevant benchmark number. "Total Time" includes verification, which is a separate algorithmic pass.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3&gt;
  
  
  What are "good" numbers?
&lt;/h3&gt;

&lt;p&gt;Here's a quick reference for 1 billion digits on common hardware (lower = better):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Hardware&lt;/th&gt;
&lt;th&gt;Time (seconds)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Ryzen 9 9950X (16C, DDR5-6000)&lt;/td&gt;
&lt;td&gt;~14.7s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Intel Core i9-13900KS&lt;/td&gt;
&lt;td&gt;~15.9s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Ryzen 9 7950X (16C, DDR5-5200)&lt;/td&gt;
&lt;td&gt;~16.8s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Ryzen 9 3950X (16C, DDR4-3200)&lt;/td&gt;
&lt;td&gt;~29.5s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Core i7-11800H (laptop, 60W)&lt;/td&gt;
&lt;td&gt;~32.3s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;If your number is significantly higher than expected for your hardware, it's usually a memory configuration issue (see below).&lt;/p&gt;




&lt;h2&gt;
  
  
  The Memory Bandwidth Trap: Why Your Expensive CPU Might Be Underperforming
&lt;/h2&gt;

&lt;p&gt;This is one of the most practically useful things y-cruncher teaches.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;y-cruncher is memory-bound.&lt;/strong&gt; Almost completely. On every high-end desktop since ~2012, the CPU sits and waits for data. This means:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;GHz doesn't matter as much as memory bandwidth&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Unpopulated DIMM slots hurt you more than you think&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Memory frequency matters enormously&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Real benchmark showing this on a Core i9-7940X:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Memory speed&lt;/th&gt;
&lt;th&gt;10 billion digit time&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;DDR4-2666&lt;/td&gt;
&lt;td&gt;365 seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;DDR4-3466&lt;/td&gt;
&lt;td&gt;322 seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;That's a &lt;strong&gt;~12% speedup&lt;/strong&gt; just from faster RAM on the &lt;em&gt;same CPU&lt;/em&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  How to maximize memory bandwidth for y-cruncher
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Checklist:&lt;/span&gt;
&lt;span class="c"&gt;# 1. Populate ALL memory channels&lt;/span&gt;
&lt;span class="c"&gt;#    (4-channel platform = use 4 DIMMs, not 2)&lt;/span&gt;

&lt;span class="c"&gt;# 2. Enable XMP/EXPO in BIOS&lt;/span&gt;
&lt;span class="c"&gt;#    Most DDR4/DDR5 kits ship at JEDEC defaults (3200/4800)&lt;/span&gt;
&lt;span class="c"&gt;#    XMP can push to 6000+ MT/s on DDR5&lt;/span&gt;

&lt;span class="c"&gt;# 3. On Skylake-X specifically: also overclock L3 cache&lt;/span&gt;
&lt;span class="c"&gt;#    (L3 bandwidth is an additional bottleneck on that architecture)&lt;/span&gt;

&lt;span class="c"&gt;# 4. Enable Large Pages (Windows)&lt;/span&gt;
&lt;span class="c"&gt;#    Run y-cruncher as Administrator for this to work&lt;/span&gt;
&lt;span class="c"&gt;#    Post-Spectre/Meltdown mitigations cause up to 5% overhead&lt;/span&gt;
&lt;span class="c"&gt;#    Large pages bypass the problematic page table walk&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  The Engineering Marvel: Checkpoint-Restart
&lt;/h2&gt;

&lt;p&gt;Here's what separates y-cruncher from just being a fast calculator.&lt;/p&gt;

&lt;p&gt;Computing 300 trillion digits of Pi takes &lt;strong&gt;months&lt;/strong&gt;. On commodity hardware. With power outages, kernel panics, memory errors, and cosmic ray bit flips all waiting to destroy your work.&lt;/p&gt;

&lt;p&gt;y-cruncher's solution is a robust checkpoint-restart system that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Periodically snapshots computation state to disk&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Verifies checkpoints with redundancy checks&lt;/strong&gt; (catching hardware bit errors before they propagate)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Resumes automatically after any interruption&lt;/strong&gt; — even after software bugs in y-cruncher itself have been fixed and the computation re-started&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;Several world record computations have &lt;em&gt;survived bugs in y-cruncher itself&lt;/em&gt; because the checkpoint infrastructure caught the error, allowed a fix, and resumed from the last valid state. That's some serious fault tolerance engineering.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The 314 trillion digit record in November 2025 is remarkable specifically because it was the first recent record achieved &lt;em&gt;without&lt;/em&gt; checkpointing — a single uninterrupted 110-day run. This is described as Storage Review's third attempt; the previous two were stopped by hardware/software issues.&lt;/p&gt;

&lt;p&gt;For the software engineers in the room: this is a production-grade distributed systems problem solved elegantly in a desktop application.&lt;/p&gt;




&lt;h2&gt;
  
  
  Advanced: The NUMA Problem (And Why Multi-Socket Is Hard)
&lt;/h2&gt;

&lt;p&gt;If you're running y-cruncher on a workstation with two CPUs — or benchmarking cloud instances — pay attention.&lt;/p&gt;

&lt;p&gt;On multi-socket (NUMA) systems, memory access latency and bandwidth are &lt;em&gt;asymmetric&lt;/em&gt;. Core 0 on Socket 0 reaches Socket 0's RAM in ~80ns. It reaches Socket 1's RAM in ~140ns. If your threads are spread across both sockets but chasing the same data, you'll hit contention that tanks throughput.&lt;/p&gt;

&lt;p&gt;y-cruncher's documentation is explicit about this:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Benchmark numbers on multi-socket machines "may not be entirely representative of what the hardware is capable of"&lt;/li&gt;
&lt;li&gt;The Push Pool vs Cilk Plus scheduler choice matters for machines with &amp;gt;64 cores&lt;/li&gt;
&lt;li&gt;Node-interleaving in the BIOS should be &lt;em&gt;disabled&lt;/em&gt; on Windows systems with &amp;gt;64 logical cores (otherwise you get imbalanced processor groups)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The load imbalance symptoms to watch for:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Core count is not a power of two&lt;/li&gt;
&lt;li&gt;Cores are heterogeneous (hybrid architectures like Intel's P+E core designs)&lt;/li&gt;
&lt;li&gt;Background processes stealing cycles from any single thread&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Using y-cruncher for Stress Testing (The Real Killer App)
&lt;/h2&gt;

&lt;p&gt;Many overclockers and hardware enthusiasts use y-cruncher &lt;em&gt;specifically&lt;/em&gt; because it's uniquely good at exposing instability:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;It simultaneously maxes out CPU computation AND the entire memory subsystem.&lt;/strong&gt; Most stress tests only hit one or the other. y-cruncher hits both at the same time.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The telltale error you'll see on unstable hardware:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Redundancy Check Failed: Coefficient is too large
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This means the two independent algorithmic passes (compute + verify) produced different results — which means either the CPU computed something wrong, or memory delivered corrupted data.&lt;/p&gt;

&lt;p&gt;If you see this on a machine you thought was stable: &lt;strong&gt;don't ignore it.&lt;/strong&gt; This is y-cruncher doing exactly what it's designed to do.&lt;/p&gt;

&lt;p&gt;Common causes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;RAM running at XMP/EXPO speeds without sufficient voltage&lt;/li&gt;
&lt;li&gt;CPU overclocked too aggressively (especially with AVX offsets not set)&lt;/li&gt;
&lt;li&gt;Thermal throttling corrupting in-flight computation&lt;/li&gt;
&lt;li&gt;Subtimings too tight on the memory controller&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Understanding the Algorithms (For the Curious)
&lt;/h2&gt;

&lt;p&gt;y-cruncher uses two independent algorithms for most constants:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Computation pass:&lt;/strong&gt; Chudnovsky algorithm for Pi (exponentially converging series). Each term adds ~14.18 digits of Pi. The challenge is computing this in arbitrary precision — which requires implementing arithmetic on numbers with &lt;em&gt;trillions of digits&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Verification pass:&lt;/strong&gt; A different formula (e.g., Ramanujan's formula, or Bailey-Borwein-Plouffe variants) to independently confirm the result. If both passes agree to the last digit, the result is almost certainly correct.&lt;/p&gt;

&lt;p&gt;The interesting engineering is in the big number arithmetic:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Uses &lt;strong&gt;Number Theoretic Transforms&lt;/strong&gt; (NTTs) — the modular arithmetic equivalent of FFTs — for multiplication&lt;/li&gt;
&lt;li&gt;Exploits &lt;strong&gt;SIMD vector units&lt;/strong&gt; (AVX-512 on modern hardware) to parallelize the transform butterflies&lt;/li&gt;
&lt;li&gt;Implements &lt;strong&gt;cache-oblivious algorithms&lt;/strong&gt; for better memory access patterns during the transform&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The result: multiplication of two N-digit numbers in O(N log N) time instead of O(N²). At a trillion digits, this difference is the gap between "computationally feasible" and "computationally impossible."&lt;/p&gt;




&lt;h2&gt;
  
  
  Practical Takeaways for Software Engineers
&lt;/h2&gt;

&lt;p&gt;Whether you run y-cruncher or never touch it, here's what to take away:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Memory bandwidth is often your real bottleneck.&lt;/strong&gt; Profile for it. Don't just assume your algorithm is CPU-bound.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Fault tolerance is a first-class feature.&lt;/strong&gt; The world record wouldn't exist without checkpoint-restart. Think about what your long-running jobs do when a node dies.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. NUMA topology changes everything in parallel code.&lt;/strong&gt; Thread affinity and memory locality matter more than raw core count on multi-socket systems.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. SIMD is still a performance multiplier worth understanding.&lt;/strong&gt; A 4× speedup from AVX-2 or 8× from AVX-512 is not unusual for data-parallel numerical code. Compilers help, but hand-tuning helps more.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. Stress test with something that actually stresses.&lt;/strong&gt; y-cruncher's simultaneous CPU + memory load finds problems that single-threaded or DRAM-only tests miss entirely.&lt;/p&gt;




&lt;h2&gt;
  
  
  Where to Go From Here
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Download y-cruncher:&lt;/strong&gt; &lt;a href="http://www.numberworld.org/y-cruncher/" rel="noopener noreferrer"&gt;numberworld.org/y-cruncher&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GitHub mirror:&lt;/strong&gt; Available for HTTPS downloads (linked from the main site)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Benchmarks leaderboard:&lt;/strong&gt; Rankings from 25 million to 1 trillion digits are published on the site&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Advanced docs:&lt;/strong&gt; The site has deep documentation on multi-threading internals, memory allocation strategies, and swap mode configuration&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Mersenneforum subforum:&lt;/strong&gt; For discussion with the community doing record-level runs&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;The fact that a program started as a high school project now drives the world's most demanding computational records — and teaches systems programmers lessons about memory bandwidth, fault tolerance, NUMA, and SIMD — is genuinely inspiring.&lt;/p&gt;

&lt;p&gt;Go run it. Break your overclock. Then fix it and run it again.&lt;/p&gt;




</description>
      <category>cpp</category>
      <category>algorithms</category>
      <category>computerscience</category>
      <category>performance</category>
    </item>
    <item>
      <title>pkg.go.dev Finally Has an Official API — No More Web Scraping</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Wed, 10 Jun 2026 04:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/pkggodev-finally-has-an-official-api-no-more-web-scraping-25cj</link>
      <guid>https://dev.to/gouranga-das-khulna/pkggodev-finally-has-an-official-api-no-more-web-scraping-25cj</guid>
      <description>&lt;p&gt;If you've ever built a Go tool that needed package metadata — documentation, versions, symbols, import counts — you already know the pain. You were either scraping the pkg.go.dev HTML (fragile, bad, and you knew it), or you were guessing from local filesystem data and hoping for the best.&lt;/p&gt;

&lt;p&gt;The Go team just shipped a real fix for this: an official pkg.go.dev API.&lt;/p&gt;

&lt;p&gt;Let's look at what it actually does.&lt;/p&gt;




&lt;h2&gt;
  
  
  What's Available
&lt;/h2&gt;

&lt;p&gt;The API lives under &lt;code&gt;/v1beta&lt;/code&gt; and covers the main things you'd actually need:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Endpoint&lt;/th&gt;
&lt;th&gt;What it gives you&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/v1beta/package/{path}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Package metadata&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/v1beta/module/{path}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Module metadata&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/v1beta/versions/{path}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;All versions of a module&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/v1beta/packages/{path}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;All packages in a module&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/v1beta/search?q={query}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Search results&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/v1beta/symbols/{path}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Exported symbols&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/v1beta/imported-by/{path}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;What imports this package&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/v1beta/vulns/{path}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Known vulnerabilities&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The &lt;code&gt;v1beta&lt;/code&gt; label is honest — it's not finalized yet. But the team has committed to moving toward a stable &lt;code&gt;v1&lt;/code&gt; after a feedback period, and they're promising backward compatibility going forward.&lt;/p&gt;




&lt;h2&gt;
  
  
  Quick Start: Hit It With curl
&lt;/h2&gt;

&lt;p&gt;No auth, no setup. Just GET requests:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl https://pkg.go.dev/v1beta/package/github.com/google/go-cmp/cmp | jq &lt;span class="nb"&gt;.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Response:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"modulePath"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"github.com/google/go-cmp"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"version"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"v0.7.0"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"isLatest"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"isStandardLibrary"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"goos"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"all"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"goarch"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"all"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"path"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"github.com/google/go-cmp/cmp"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"cmp"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"synopsis"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Package cmp determines equality of values."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"isRedistributable"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You can pin a specific version:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="s2"&gt;"https://pkg.go.dev/v1beta/package/github.com/google/go-cmp/cmp?version=v0.6.0"&lt;/span&gt; | jq &lt;span class="nb"&gt;.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Or use &lt;code&gt;master&lt;/code&gt; / &lt;code&gt;main&lt;/code&gt; branch names, which resolve automatically to their pseudo-version:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="s2"&gt;"https://pkg.go.dev/v1beta/package/github.com/google/go-cmp/cmp?version=master"&lt;/span&gt; | jq &lt;span class="s1"&gt;'{path, version}'&lt;/span&gt;
&lt;span class="c"&gt;# {&lt;/span&gt;
&lt;span class="c"&gt;#   "path": "github.com/google/go-cmp/cmp",&lt;/span&gt;
&lt;span class="c"&gt;#   "version": "v0.7.1-0.20260310220054-34c9473539b8"&lt;/span&gt;
&lt;span class="c"&gt;# }&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  One Gotcha: Module Paths Must Be Unambiguous
&lt;/h2&gt;

&lt;p&gt;This tripped me up when I first read the docs. The API follows a "precision over convenience" rule that differs from the web UI.&lt;/p&gt;

&lt;p&gt;On the website, if you type &lt;code&gt;example.com/a/b/c&lt;/code&gt;, it resolves to the longest matching module path automatically. The API won't do that. If &lt;code&gt;example.com/a/b/c&lt;/code&gt; could live in either &lt;code&gt;example.com/a&lt;/code&gt; or &lt;code&gt;example.com/a/b&lt;/code&gt;, the API returns an error and asks you to be more specific.&lt;/p&gt;

&lt;p&gt;Makes sense for programmatic use — you don't want silent resolution surprises in a tool. Just something to plan for when you're building integrations.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Reference CLI: &lt;code&gt;pkgsite-cli&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;The Go team also shipped a reference client you can install and start using immediately:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;go &lt;span class="nb"&gt;install &lt;/span&gt;golang.org/x/pkgsite/cmd/internal/pkgsite-cli@latest
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Search for packages:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pkgsite-cli search &lt;span class="s2"&gt;"uuid"&lt;/span&gt;
&lt;span class="c"&gt;# github.com/google/uuid&lt;/span&gt;
&lt;span class="c"&gt;#   Module:   github.com/google/uuid@v1.6.0&lt;/span&gt;
&lt;span class="c"&gt;#   Synopsis: Package uuid generates and inspects UUIDs.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Inspect a package:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pkgsite-cli package github.com/google/go-cmp/cmp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;See what imports a package:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pkgsite-cli package &lt;span class="nt"&gt;--imported-by&lt;/span&gt; github.com/google/go-cmp/cmp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;List exported symbols:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pkgsite-cli package &lt;span class="nt"&gt;--symbols&lt;/span&gt; github.com/google/go-cmp/cmp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;List all versions of a module:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pkgsite-cli module &lt;span class="nt"&gt;-versions&lt;/span&gt; github.com/google/go-cmp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Worth noting: the CLI interface is explicitly marked as unstable — it's a reference implementation, not a finished product. Use it to understand the API, not as a dependency in scripts you care about.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why This Matters Now
&lt;/h2&gt;

&lt;p&gt;The "years of community feedback" line in the announcement is underselling it. Go tooling has had a real data access gap for a long time. Custom linters, documentation generators, dependency analyzers, IDE plugins — all of them were either scraping or going without.&lt;/p&gt;

&lt;p&gt;The announcement also mentions AI-assisted coding tools specifically, and that's probably the real driver here. Tools that reason about your dependencies need structured, reliable data about packages. Scraping doesn't cut it when you want a language server to tell you something accurate about a module it's never seen before.&lt;/p&gt;

&lt;p&gt;An OpenAPI spec is published too, so generating clients in any language is straightforward.&lt;/p&gt;




&lt;h2&gt;
  
  
  What I'd Use This For
&lt;/h2&gt;

&lt;p&gt;A few things come to mind immediately:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Dependency audits&lt;/strong&gt; — pull vulnerability data via &lt;code&gt;/v1beta/vulns/{path}&lt;/code&gt; for every module in your &lt;code&gt;go.sum&lt;/code&gt; without spinning up a full &lt;code&gt;govulncheck&lt;/code&gt; run.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Import graph tooling&lt;/strong&gt; — the &lt;code&gt;/v1beta/imported-by&lt;/code&gt; endpoint makes it practical to build tools that understand downstream impact of an API change.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Documentation bots&lt;/strong&gt; — fetch synopsis and symbol data for packages your team uses, feed it wherever you need it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Version tracking&lt;/strong&gt; — watch specific modules for new releases without polling GitHub.&lt;/p&gt;




&lt;h2&gt;
  
  
  The State of It
&lt;/h2&gt;

&lt;p&gt;It's a &lt;code&gt;v1beta&lt;/code&gt;. The endpoints work, the data is good, and the team is clearly committed to making this a real stable API. But I wouldn't bet a production pipeline on the exact response shape staying identical until &lt;code&gt;v1&lt;/code&gt; lands.&lt;/p&gt;

&lt;p&gt;For internal tooling and experiments? Use it today. For anything you're shipping to other people? Keep an eye on the issue tracker and wait for &lt;code&gt;v1&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Go give it a curl. The &lt;a href="https://pkg.go.dev/api" rel="noopener noreferrer"&gt;full API spec&lt;/a&gt; and &lt;a href="https://pkg.go.dev/api" rel="noopener noreferrer"&gt;OpenAPI definition&lt;/a&gt; are live now.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Feedback and bug reports go to the &lt;a href="https://github.com/golang/pkgsite/issues" rel="noopener noreferrer"&gt;pkgsite issue tracker&lt;/a&gt;. The team specifically asked for community feedback before the v1 freeze, so this is a good time to poke at edge cases.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>go</category>
      <category>opensource</category>
      <category>programming</category>
      <category>api</category>
    </item>
    <item>
      <title>Database Scaling</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Sun, 07 Jun 2026 02:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/database-scaling-56fa</link>
      <guid>https://dev.to/gouranga-das-khulna/database-scaling-56fa</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;One-liner:&lt;/strong&gt; Databases are the hardest part to scale — the strategy is: vertical first → read replicas → caching → sharding.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  📌 Scaling Ladder
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Stage 1: Single DB (&amp;lt; 10K users)
Stage 2: Separate DB server from app server
Stage 3: Add caching (Redis) — reduce DB reads by 80%
Stage 4: Read Replicas — distribute read traffic
Stage 5: Database Sharding — distribute write traffic + data
Stage 6: CQRS + Event Sourcing (advanced)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Never jump to Stage 5 without exhausting Stages 3 and 4.&lt;/strong&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  📊 The Read/Write Split
&lt;/h2&gt;

&lt;p&gt;Most web apps: ~90% reads, ~10% writes.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Optimize reads first:
  → Add Redis cache (most impactful)
  → Add read replicas
  → Add CDN for static data

Only then optimize writes:
  → Sharding
  → Async writes (queue + batch)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🗃️ Connection Pooling
&lt;/h2&gt;

&lt;p&gt;Never create a new DB connection per request. Use a pool.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// pg (Node.js PostgreSQL)&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;pool&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Pool&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
  &lt;span class="na"&gt;max&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="c1"&gt;// max 20 connections&lt;/span&gt;
  &lt;span class="na"&gt;idleTimeoutMillis&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;30000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;connectionTimeoutMillis&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;query&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;SELECT * FROM users WHERE id = $1&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;]);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Without pooling: 1000 req/sec = 1000 new connections = DB crashes.&lt;/p&gt;

</description>
      <category>database</category>
      <category>systemdesign</category>
      <category>sql</category>
      <category>redis</category>
    </item>
    <item>
      <title>Consistent Hashing</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Sat, 06 Jun 2026 02:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/consistent-hashing-3fn7</link>
      <guid>https://dev.to/gouranga-das-khulna/consistent-hashing-3fn7</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;One-liner:&lt;/strong&gt; A hashing technique that minimizes the number of keys that need to be remapped when servers are added or removed.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  📌 The Problem It Solves
&lt;/h2&gt;

&lt;p&gt;Imagine you have 3 cache servers and you distribute keys using:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;server = hash(key) % N  (N = number of servers)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Works fine until N changes:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;N=3: hash("user123") % 3 = 1 → Server 1
N=4: hash("user123") % 4 = 3 → Server 3  ← DIFFERENT!
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When you add or remove a server, &lt;strong&gt;almost ALL keys remap&lt;/strong&gt; → massive cache miss storm → database gets hammered.&lt;/p&gt;




&lt;h2&gt;
  
  
  💡 The Solution: Consistent Hashing
&lt;/h2&gt;

&lt;h3&gt;
  
  
  How It Works
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;Map both &lt;strong&gt;servers&lt;/strong&gt; and &lt;strong&gt;keys&lt;/strong&gt; onto the same circular hash ring (0 to 2³²)&lt;/li&gt;
&lt;li&gt;Each key is assigned to the &lt;strong&gt;first server clockwise&lt;/strong&gt; from its position&lt;/li&gt;
&lt;li&gt;Adding/removing a server only affects &lt;strong&gt;adjacent keys&lt;/strong&gt;
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;         0
      /     \
  S3         S1
    \       /
         S2
        360°

Key K1 → clockwise → lands on S1
Key K2 → clockwise → lands on S2
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Adding a Server
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Before:  K1→S1, K2→S1, K3→S2, K4→S3
Add S4 between S1 and S2:
After:   K1→S1, K2→S4, K3→S2, K4→S3  ← Only K2 moved!
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Removing a Server
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Before: K1→S1, K2→S2, K3→S2, K4→S3
Remove S2:
After:  K1→S1, K2→S3, K3→S3, K4→S3  ← K2 and K3 moved to next server
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  ⚖️ Problem: Uneven Distribution
&lt;/h2&gt;

&lt;p&gt;With few servers, the ring can be unbalanced:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;S1 handles 60% of keyspace
S2 handles 30% of keyspace
S3 handles 10% of keyspace
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Solution: Virtual Nodes (VNodes)
&lt;/h3&gt;

&lt;p&gt;Instead of 1 point per server, use &lt;strong&gt;many virtual points&lt;/strong&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;S1 → S1_1, S1_2, S1_3 ... S1_100  (spread around ring)
S2 → S2_1, S2_2, S2_3 ... S2_100
S3 → S3_1, S3_2, S3_3 ... S3_100
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Result: Much more even distribution. Each physical server maps to ~N/total keys.&lt;/p&gt;




&lt;h2&gt;
  
  
  🏗️ Implementation Sketch
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;ConsistentHash&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nf"&gt;constructor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;replicas&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;replicas&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;replicas&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// virtual nodes per server&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ring&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Map&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt; &lt;span class="c1"&gt;// hash → serverName&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;sortedKeys&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="nf"&gt;addServer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;server&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;replicas&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;hash&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;md5&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;server&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;:&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;i&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
      &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ring&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;hash&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;server&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
      &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;sortedKeys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;hash&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;sortedKeys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sort&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="nf"&gt;getServer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;hash&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;md5&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;key&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="c1"&gt;// Find first server clockwise&lt;/span&gt;
    &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;k&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;sortedKeys&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;k&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="nx"&gt;hash&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ring&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;k&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ring&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;sortedKeys&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]);&lt;/span&gt; &lt;span class="c1"&gt;// wrap around&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  📊 Consistent Hashing vs Modulo Hashing
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Dimension&lt;/th&gt;
&lt;th&gt;Modulo Hashing&lt;/th&gt;
&lt;th&gt;Consistent Hashing&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Keys remapped on resize&lt;/td&gt;
&lt;td&gt;~100%&lt;/td&gt;
&lt;td&gt;~1/N&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Uneven distribution&lt;/td&gt;
&lt;td&gt;Even&lt;/td&gt;
&lt;td&gt;Possible (fix with VNodes)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Complexity&lt;/td&gt;
&lt;td&gt;O(1)&lt;/td&gt;
&lt;td&gt;O(log N) per lookup&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Use case&lt;/td&gt;
&lt;td&gt;Fixed set of servers&lt;/td&gt;
&lt;td&gt;Dynamic server pool&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  🌍 Real-World Use Cases
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;System&lt;/th&gt;
&lt;th&gt;How it's used&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Cassandra&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Partitions data across nodes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;DynamoDB&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Key-based partitioning&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Memcached/Redis Cluster&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Distributing cache keys&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Nginx&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Consistent upstream selection&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;CDN&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Routing requests to edge nodes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Kafka&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Partition assignment&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  🎨 Excalidraw Diagram
&lt;/h2&gt;

&lt;p&gt;The diagram shows:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The hash ring as a circle&lt;/li&gt;
&lt;li&gt;Server nodes placed on the ring&lt;/li&gt;
&lt;li&gt;Keys mapped to nearest server clockwise&lt;/li&gt;
&lt;li&gt;Virtual nodes distributed around the ring&lt;/li&gt;
&lt;li&gt;Before/after adding a server (minimal key movement)&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  🔑 Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Modulo hashing causes thundering herd on any server change&lt;/li&gt;
&lt;li&gt;Consistent hashing limits disruption to ~1/N keys per change&lt;/li&gt;
&lt;li&gt;Virtual nodes (VNodes) solve the uneven distribution problem&lt;/li&gt;
&lt;li&gt;This is used &lt;strong&gt;everywhere&lt;/strong&gt; in distributed databases and caches&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>distributedsystems</category>
      <category>systemdesign</category>
      <category>kafka</category>
    </item>
    <item>
      <title>Back of Envelope Calculations</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Fri, 05 Jun 2026 02:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/back-of-envelope-calculations-22d7</link>
      <guid>https://dev.to/gouranga-das-khulna/back-of-envelope-calculations-22d7</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;One-liner:&lt;/strong&gt; Rough estimates done quickly to understand the scale of a system before designing it. They tell you what tier of infrastructure you need.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  📌 Why This Matters
&lt;/h2&gt;

&lt;p&gt;Before designing ANY system, you need to know:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;How many &lt;strong&gt;users&lt;/strong&gt; will use it?&lt;/li&gt;
&lt;li&gt;How many &lt;strong&gt;requests per second&lt;/strong&gt;?&lt;/li&gt;
&lt;li&gt;How much &lt;strong&gt;storage&lt;/strong&gt; do we need?&lt;/li&gt;
&lt;li&gt;How much &lt;strong&gt;bandwidth&lt;/strong&gt; is required?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These estimates shape every architectural decision.&lt;/p&gt;




&lt;h2&gt;
  
  
  🔢 Numbers Every Engineer Should Know
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Time
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Unit&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1 millisecond&lt;/td&gt;
&lt;td&gt;10⁻³ seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1 microsecond&lt;/td&gt;
&lt;td&gt;10⁻⁶ seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1 nanosecond&lt;/td&gt;
&lt;td&gt;10⁻⁹ seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Seconds in a day&lt;/td&gt;
&lt;td&gt;~86,400 ≈ 10⁵&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Seconds in a month&lt;/td&gt;
&lt;td&gt;~2.5M ≈ 2.5 × 10⁶&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Seconds in a year&lt;/td&gt;
&lt;td&gt;~31.5M ≈ 3 × 10⁷&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Latency Numbers (Approximate)
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Operation&lt;/th&gt;
&lt;th&gt;Latency&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;L1 cache reference&lt;/td&gt;
&lt;td&gt;1 ns&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;L2 cache reference&lt;/td&gt;
&lt;td&gt;4 ns&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;RAM reference&lt;/td&gt;
&lt;td&gt;100 ns&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SSD random read&lt;/td&gt;
&lt;td&gt;150 µs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;HDD seek&lt;/td&gt;
&lt;td&gt;10 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Network: same datacenter&lt;/td&gt;
&lt;td&gt;500 µs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Network: cross-region (US→EU)&lt;/td&gt;
&lt;td&gt;150 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Network: cross-continent&lt;/td&gt;
&lt;td&gt;200–300 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Storage Units
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Unit&lt;/th&gt;
&lt;th&gt;Size&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1 KB&lt;/td&gt;
&lt;td&gt;10³ bytes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1 MB&lt;/td&gt;
&lt;td&gt;10⁶ bytes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1 GB&lt;/td&gt;
&lt;td&gt;10⁹ bytes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1 TB&lt;/td&gt;
&lt;td&gt;10¹² bytes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1 PB&lt;/td&gt;
&lt;td&gt;10¹⁵ bytes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  📐 The Estimation Framework
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Step 1: DAU → QPS
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Daily Active Users (DAU)    = 10 million
Avg requests per user/day   = 10
Total daily requests        = 100 million

QPS = 100M / 86,400 ≈ 1,200 req/sec
Peak QPS (2-3x average)     ≈ 3,000 req/sec
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Step 2: Storage
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;New posts per day    = 1M
Avg post size        = 1 KB (text) + 500 KB (image)
Daily storage        = 1M × 501 KB ≈ 500 GB/day
Yearly storage       = 500 GB × 365 ≈ 180 TB/year
5-year storage       ≈ 1 PB
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Step 3: Bandwidth
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Reads per second     = 100K req/sec
Avg response size    = 10 KB
Outbound bandwidth   = 100K × 10 KB = 1 GB/sec
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🔬 Real Example: Design Twitter
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Assumptions
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;300M DAU&lt;/li&gt;
&lt;li&gt;Each user reads 10 tweets/day&lt;/li&gt;
&lt;li&gt;Each user writes 1 tweet/week ≈ 0.14 tweets/day&lt;/li&gt;
&lt;li&gt;Avg tweet: 280 chars = 280 bytes ≈ 300 bytes&lt;/li&gt;
&lt;li&gt;10% of tweets have an image (~200 KB)&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Read QPS
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;300M users × 10 reads/day = 3B reads/day
QPS = 3B / 86,400 ≈ 35,000 reads/sec
Peak QPS ≈ 100,000 reads/sec
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Write QPS
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;300M × 0.14 = 42M tweets/day
QPS = 42M / 86,400 ≈ 500 writes/sec
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Storage per day
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Text: 42M × 300 bytes = 12.6 GB
Images: 42M × 10% × 200 KB = 840 GB
Total: ~850 GB/day ≈ 310 TB/year
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Bandwidth
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Read traffic: 35K req/sec × 1 KB/response ≈ 35 MB/sec = ~300 Gbps
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🔬 Real Example: Design WhatsApp
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Assumptions
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;2B DAU&lt;/li&gt;
&lt;li&gt;Each user sends 20 messages/day&lt;/li&gt;
&lt;li&gt;Avg message: 100 bytes&lt;/li&gt;
&lt;li&gt;30% are media messages (500 KB avg)&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  QPS
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;2B × 20 = 40B messages/day
QPS = 40B / 86,400 ≈ 460,000 msg/sec
Peak QPS ≈ 1M msg/sec
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Storage per day
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Text: 40B × 100B = 4 TB
Media: 40B × 30% × 500 KB = 6 PB/day (too high → add retention/compression policy)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  💡 Tips for Interviews
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Always ask&lt;/strong&gt; before estimating — "Should I assume 10M or 100M users?"&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Round aggressively&lt;/strong&gt; — 86,400 → 10⁵, 3.14 → 3&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Peak = 2-3× average&lt;/strong&gt; (or 5-10× for viral/event-based systems)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Show your math&lt;/strong&gt; — interviewers care about the process, not the exact number&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Derive storage needs from estimates&lt;/strong&gt; — don't pull numbers from thin air&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;1 char = 1 byte (ASCII)&lt;/strong&gt;, 1 char = 2-4 bytes (Unicode)&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  📋 Estimation Cheat Sheet
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;1K users  → Single server fine
10K users → Need to think about DB separation
100K users → Load balancer, read replicas
1M users  → Caching layer, CDN, sharding
10M users → Distributed systems, microservices
100M+     → You work at a FAANG (or interview there)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  🔑 Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Back-of-envelope is about &lt;strong&gt;order of magnitude&lt;/strong&gt;, not precision&lt;/li&gt;
&lt;li&gt;Know the key numbers cold: bytes, seconds in a day, latency tiers&lt;/li&gt;
&lt;li&gt;Always separate &lt;strong&gt;read QPS&lt;/strong&gt; from &lt;strong&gt;write QPS&lt;/strong&gt; — they're usually very different&lt;/li&gt;
&lt;li&gt;Storage estimations reveal whether you need sharding/archival early&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>systemdesign</category>
      <category>backend</category>
    </item>
    <item>
      <title>TypeScript 7.0 Beta is Here — and It's Rewritten in Go. Here's What Actually Changed.</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Thu, 04 Jun 2026 04:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/typescript-70-beta-is-here-and-its-rewritten-in-go-heres-what-actually-changed-4l78</link>
      <guid>https://dev.to/gouranga-das-khulna/typescript-70-beta-is-here-and-its-rewritten-in-go-heres-what-actually-changed-4l78</guid>
      <description>&lt;p&gt;I've been watching the TypeScript Go port news since it first leaked, and I'll be honest — when they said "10x faster," I was ready to call marketing spin. Then I ran it on a real codebase.&lt;/p&gt;

&lt;p&gt;It's fast. Like, uncomfortably fast. The kind of fast where you assume something must be broken.&lt;/p&gt;

&lt;p&gt;TypeScript 7.0 Beta dropped on April 21st, and it's not your usual TypeScript release. This isn't "we added a few new types and fixed some edge cases." The entire compiler was ported from TypeScript (the language) to &lt;strong&gt;Go&lt;/strong&gt; — and then they're calling the result TypeScript 7.0.&lt;/p&gt;

&lt;p&gt;Let me break down what this actually means for you.&lt;/p&gt;




&lt;h2&gt;
  
  
  Wait, Why Go?
&lt;/h2&gt;

&lt;p&gt;First question everyone asks. Why not Rust? Why not WASM?&lt;/p&gt;

&lt;p&gt;The honest answer: speed of porting, not peak performance. Go's concurrency model and memory layout happened to align well with how the existing TypeScript compiler was structured. A Rust rewrite would've taken years and needed a ground-up redesign. The Go port took roughly a year, and it's architecturally identical to TypeScript 6.0's type-checking logic — same semantics, same results.&lt;/p&gt;

&lt;p&gt;Pragmatic call. I think it was the right one.&lt;/p&gt;




&lt;h2&gt;
  
  
  How to Try It Right Now
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npm &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-D&lt;/span&gt; @typescript/native-preview@beta
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then run &lt;code&gt;tsgo&lt;/code&gt; instead of &lt;code&gt;tsc&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx tsgo &lt;span class="nt"&gt;--version&lt;/span&gt;
&lt;span class="c"&gt;# Version 7.0.0-beta&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Note: the package is &lt;code&gt;@typescript/native-preview&lt;/code&gt; for now. The stable release will eventually ship as the regular &lt;code&gt;typescript&lt;/code&gt; package with the normal &lt;code&gt;tsc&lt;/code&gt; command.&lt;/p&gt;

&lt;p&gt;For VS Code, grab the &lt;strong&gt;TypeScript Native Preview&lt;/strong&gt; extension. It's not a rough prototype — teams at Bloomberg, Figma, Slack, Google, and others have been running pre-release builds for over a year. It's been solid for months.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Performance Is the Main Event
&lt;/h2&gt;

&lt;p&gt;The Go rewrite isn't just a compiler curiosity. It unlocks parallelization that the old JS runtime couldn't do:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;code&gt;--checkers&lt;/code&gt;&lt;/strong&gt; controls how many type-checking workers run in parallel (default: 4). Bigger codebases on beefy machines can push this higher. CI runners with limited cores should drop it down.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;code&gt;--builders&lt;/code&gt;&lt;/strong&gt; controls parallel project reference builds — a big deal for monorepos. Combined with &lt;code&gt;--checkers&lt;/code&gt;, it multiplies. &lt;code&gt;--checkers 4 --builders 4&lt;/code&gt; = up to 16 type-checkers running simultaneously.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;code&gt;--singleThreaded&lt;/code&gt;&lt;/strong&gt; if you need to debug or benchmark against TypeScript 6 without parallelism noise.&lt;/p&gt;

&lt;p&gt;The team reports ~10x speedups on large codebases. Based on what early adopters are saying publicly, that tracks.&lt;/p&gt;




&lt;h2&gt;
  
  
  Running 7.0 Alongside 6.0 (Without Chaos)
&lt;/h2&gt;

&lt;p&gt;This part tripped me up at first. Here's the clean way to handle it.&lt;/p&gt;

&lt;p&gt;There's a new compatibility package: &lt;code&gt;@typescript/typescript6&lt;/code&gt;. It exposes a &lt;code&gt;tsc6&lt;/code&gt; entry point, so you can run both without naming collisions.&lt;/p&gt;

&lt;p&gt;If you use tools like &lt;code&gt;typescript-eslint&lt;/code&gt; that peer-depend on &lt;code&gt;typescript&lt;/code&gt; directly, do this in your &lt;code&gt;package.json&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"devDependencies"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"typescript"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"npm:@typescript/typescript6@^6.0.0"&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This aliases the &lt;code&gt;typescript&lt;/code&gt; package to 6.0, so your existing tooling stays happy while you experiment with &lt;code&gt;tsgo&lt;/code&gt; on the side.&lt;/p&gt;




&lt;h2&gt;
  
  
  Breaking Changes — The Ones That Will Actually Bite You
&lt;/h2&gt;

&lt;p&gt;TypeScript 7.0 adopts 6.0's defaults as hard requirements. A lot of these were opt-in before. Now they're just on, with no escape hatch.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Defaults that changed:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;strict&lt;/code&gt; is &lt;code&gt;true&lt;/code&gt; by default&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;module&lt;/code&gt; defaults to &lt;code&gt;esnext&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;noUncheckedSideEffectImports&lt;/code&gt; is &lt;code&gt;true&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;types&lt;/code&gt; defaults to &lt;code&gt;[]&lt;/code&gt; — your &lt;code&gt;@types/node&lt;/code&gt; won't be picked up automatically anymore&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;The &lt;code&gt;rootDir&lt;/code&gt; change is subtle but will catch you:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If your &lt;code&gt;tsconfig.json&lt;/code&gt; sits outside your &lt;code&gt;src/&lt;/code&gt; folder (pretty common), you need to add this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"compilerOptions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"rootDir"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"./src"&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"include"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"./src"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;The &lt;code&gt;types&lt;/code&gt; change will also catch you:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"compilerOptions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"types"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"node"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"jest"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;List every &lt;code&gt;@types&lt;/code&gt; package your project needs explicitly. No more implicit globals from everything in your &lt;code&gt;node_modules/@types&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Things that are just gone:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;target: es5&lt;/code&gt; — not supported&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;moduleResolution: node&lt;/code&gt; / &lt;code&gt;node10&lt;/code&gt; — use &lt;code&gt;nodenext&lt;/code&gt; or &lt;code&gt;bundler&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;module: amd&lt;/code&gt;, &lt;code&gt;umd&lt;/code&gt;, &lt;code&gt;systemjs&lt;/code&gt;, &lt;code&gt;none&lt;/code&gt; — gone&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;baseUrl&lt;/code&gt; — use &lt;code&gt;paths&lt;/code&gt; relative to the project root instead&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;downlevelIteration&lt;/code&gt; — gone&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you're still on any of these, migrating to TypeScript 6.0 first will make the 7.0 transition cleaner.&lt;/p&gt;




&lt;h2&gt;
  
  
  JavaScript Support Got a Full Rethink
&lt;/h2&gt;

&lt;p&gt;This one flew under my radar until I read the details. TypeScript 7.0 rewrites how &lt;code&gt;.js&lt;/code&gt; files are handled — moving away from Closure-style JSDoc support toward patterns consistent with &lt;code&gt;.ts&lt;/code&gt; files.&lt;/p&gt;

&lt;p&gt;Key changes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;@enum&lt;/code&gt; is no longer special — you need &lt;code&gt;@typedef&lt;/code&gt; instead&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;@class&lt;/code&gt; on a function doesn't make it a constructor anymore — use a class declaration&lt;/li&gt;
&lt;li&gt;Closure-style function syntax like &lt;code&gt;function(string): void&lt;/code&gt; is dropped — use &lt;code&gt;(s: string) =&amp;gt; void&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;values&lt;/code&gt; can't be used where types are expected — use &lt;code&gt;typeof someValue&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you have a pure-JS codebase that relied on TypeScript's JSDoc inference, check the &lt;a href="https://github.com/microsoft/TypeScript-Go" rel="noopener noreferrer"&gt;CHANGES.md&lt;/a&gt; before upgrading.&lt;/p&gt;




&lt;h2&gt;
  
  
  What's Still Missing
&lt;/h2&gt;

&lt;p&gt;The beta label is real, but it's not hiding a disaster — it's flagging a few known gaps:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;No stable programmatic API yet (that's TypeScript 7.1 at earliest)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;--watch&lt;/code&gt; mode is less efficient than it'll eventually be&lt;/li&gt;
&lt;li&gt;Some VS Code features are still coming: semantics-enhanced highlighting, more granular import commands&lt;/li&gt;
&lt;li&gt;Declaration file emit from &lt;code&gt;.js&lt;/code&gt; files is still being finished&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you depend on the TypeScript compiler API for custom tooling, wait for 7.1. For everything else — type-checking in CI, editor experience, build times — 7.0 Beta is ready.&lt;/p&gt;




&lt;h2&gt;
  
  
  My Take
&lt;/h2&gt;

&lt;p&gt;The Go port is a genuinely impressive engineering decision. Not the most glamorous language choice, not the theoretical peak, but it shipped in about a year and already works on multi-million line codebases.&lt;/p&gt;

&lt;p&gt;The breaking changes are real, but most of them were coming anyway. TypeScript 6.0 gave everyone a year to see them coming. If you've been keeping up with releases, 7.0 should feel like a minor migration with a huge performance payoff.&lt;/p&gt;

&lt;p&gt;If you haven't touched your &lt;code&gt;tsconfig.json&lt;/code&gt; in two years and still have &lt;code&gt;moduleResolution: node&lt;/code&gt; in there... you have some cleanup to do. But that cleanup was overdue regardless of whether TypeScript 7 existed.&lt;/p&gt;

&lt;p&gt;Try it today:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npm &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-D&lt;/span&gt; @typescript/native-preview@beta
npx tsgo &lt;span class="nt"&gt;--version&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Run it on something real and let the speed be someone else's excuse to finally upgrade the old config.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Found issues? The team wants feedback at the &lt;a href="https://github.com/microsoft/typescript-go" rel="noopener noreferrer"&gt;microsoft/typescript-go&lt;/a&gt; issue tracker — not the main TypeScript repo. That distinction matters during the beta.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>typescript</category>
      <category>javascript</category>
      <category>go</category>
      <category>programming</category>
    </item>
    <item>
      <title>Single Server Setup</title>
      <dc:creator>Gouranga Das Samrat</dc:creator>
      <pubDate>Wed, 03 Jun 2026 02:00:00 +0000</pubDate>
      <link>https://dev.to/gouranga-das-khulna/single-server-setup-20il</link>
      <guid>https://dev.to/gouranga-das-khulna/single-server-setup-20il</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;One-liner:&lt;/strong&gt; The simplest possible architecture — one machine running everything. Great for starting out, but a single point of failure.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  📌 What Is It?
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Browser] ─── HTTP ──► [Single Server]
                          ├── Web Server (Nginx/Apache)
                          ├── App Code (Node/Python/Java)
                          └── Database (MySQL/PostgreSQL)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One machine handles everything: web serving, business logic, database.&lt;/p&gt;




&lt;h2&gt;
  
  
  ✅ When It's Fine
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Prototype / MVP stage&lt;/li&gt;
&lt;li&gt;&amp;lt;1,000 daily users&lt;/li&gt;
&lt;li&gt;Internal tool / side project&lt;/li&gt;
&lt;li&gt;Learning / development&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  ❌ Problems
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Problem&lt;/th&gt;
&lt;th&gt;Impact&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;SPOF&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Server dies → everything down&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;No scaling&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Can't add more machines&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Resource contention&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;App and DB fight for CPU/RAM&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Hard to update&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Any deploy = full downtime&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  🚀 First Step: Separate App &amp;amp; DB
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Browser] → [App Server] → [DB Server]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now you can scale each independently.&lt;/p&gt;

</description>
      <category>systemdesign</category>
      <category>networking</category>
      <category>backend</category>
    </item>
  </channel>
</rss>
