DEV Community: Gouranga Das Samrat

Distributed Locks

Gouranga Das Samrat — Sun, 13 Sep 2026 04:00:00 +0000

One-liner: A distributed lock ensures that only one node in a cluster can perform a critical operation at a time — preventing race conditions across services.

❓ Why Do You Need Distributed Locks?

In a single-server world, a mutex or semaphore handles concurrency. But in distributed systems:

Multiple service replicas run simultaneously
They all share the same database or resource
Without coordination, two instances might process the same job, double-charge a user, or corrupt shared state

Classic example: Flash sale — 100 units in stock, 10,000 concurrent requests. Without a lock, you oversell.

🔑 Redis Distributed Lock (Redlock)

The most common approach: use Redis SET NX EX.

# Acquire lock
SET lock:resource_id <unique_token> NX EX 10
# NX = only set if key does NOT exist
# EX 10 = auto-expire in 10 seconds (TTL safety net)

# Returns OK → you have the lock
# Returns nil → someone else has it

# Release lock (Lua script — atomic!)
if redis.call("GET", KEYS[1]) == ARGV[1] then
  return redis.call("DEL", KEYS[1])
else
  return 0
end

Why a unique token? Prevents a slow process from releasing a lock that was already acquired by someone else after TTL expiry.

⚙️ How It Works (Step by Step)

Service A                    Redis                   Service B
   |                           |                        |
   |── SET lock:job1 tokenA ──>|                        |
   |   NX EX 10                |                        |
   |<── OK (lock acquired) ────|                        |
   |                           |                        |
   |   (doing work...)         |── SET lock:job1 ──────>|
   |                           |   tokenB NX EX 10      |
   |                           |<── nil (locked) ───────|
   |                           |   (retry or fail)      |
   |                           |                        |
   |── DEL lock:job1 ─────────>|                        |
   |   (if token matches)      |                        |
   |<── lock released ─────────|                        |

⚠️ Edge Cases & Gotchas

Problem	What Happens	Solution
Process crashes	Lock stuck forever	TTL auto-expire
Network partition	Two leaders think they have lock	Redlock (multi-node)
GC pause longer than TTL	Process resumes, lock already expired	Fencing tokens
Redis crashes	Lock lost	Use Redis Cluster / persistence

Fencing Token Pattern

Each lock acquisition returns a monotonically increasing token number. The resource (e.g., DB) only accepts writes with tokens higher than the last seen:

Lock acquired → token 42
Lock expired, re-acquired → token 43
Stale process tries to write with token 42 → REJECTED

🏛️ Alternatives to Redis Locks

Tool	Mechanism	Use When
Redis (SET NX)	Atomic key-value	Most common, low latency
ZooKeeper	Ephemeral nodes + watchers	Need strong consistency
etcd	Leases + CAS	Kubernetes-style infra
DB row lock	`SELECT FOR UPDATE`	Already using SQL, low scale

✅ Pros

Sub-millisecond lock acquisition with Redis
TTL prevents deadlocks from crashed processes
Simple to implement with SET NX EX
Scales with Redis Cluster

❌ Cons

Redis is single point of failure (use Redlock for HA)
Clock drift can cause issues with TTL-based expiry
Redlock is controversial (Martin Kleppmann vs. Antirez debate)
Not suitable for long-held locks (use queue instead)

⚖️ When to Use / When NOT to Use

✅ Use when:

Preventing double processing (payment, job scheduling)
Flash sales / inventory deduction
Leader election among service instances
Rate limiting per resource

❌ Avoid when:

Operations take longer than a reasonable TTL (use idempotency + queue instead)
You need guaranteed strong consistency (use ZooKeeper + fencing)
The resource is already serialized (single-threaded queue consumer)

Circuit Breaker Pattern

Gouranga Das Samrat — Sat, 12 Sep 2026 04:00:00 +0000

One-liner: A circuit breaker stops calling a failing service to give it time to recover — instead of hammering it with requests that are guaranteed to fail.

❓ The Problem: Cascading Failures

User Request
    ↓
Service A  ──► Service B  ──► Service C (DOWN 💥)
    ↑              ↑
Threads hang   Threads hang
(timeout 30s)  (timeout 30s)

→ Service A's thread pool exhausts
→ Service A goes down
→ Everything upstream dies
→ Full cascade failure 🔥

Without circuit breakers, one slow service kills your entire system.

🔌 The Three States

┌─────────────────────────────────────────────────────────┐
│                      CLOSED                             │
│              (Normal operation)                         │
│   Requests flow through. Track failure rate.            │
│   Failure threshold exceeded → trip to OPEN             │
└──────────────────────────┬──────────────────────────────┘
                           │ failures > threshold
                           ▼
┌─────────────────────────────────────────────────────────┐
│                       OPEN                              │
│              (Service is DOWN)                          │
│   All requests IMMEDIATELY fail (no network call)       │
│   Return cached/default response                        │
│   Wait for reset timeout (e.g., 60s) → HALF-OPEN       │
└──────────────────────────┬──────────────────────────────┘
                           │ timeout elapsed
                           ▼
┌─────────────────────────────────────────────────────────┐
│                    HALF-OPEN                            │
│              (Testing recovery)                         │
│   Let a few probe requests through                      │
│   Success → CLOSED (recovered! ✅)                      │
│   Failure → back to OPEN (still down ❌)                │
└─────────────────────────────────────────────────────────┘

💻 Implementation Example (Conceptual)

class CircuitBreaker {
  constructor(fn, { failureThreshold = 5, resetTimeout = 60000 }) {
    this.fn = fn;
    this.state = "CLOSED";
    this.failureCount = 0;
    this.failureThreshold = failureThreshold;
    this.resetTimeout = resetTimeout;
  }

  async call(...args) {
    if (this.state === "OPEN") {
      // Fail fast — don't even try
      return this.fallback();
    }

    try {
      const result = await this.fn(...args);
      this.onSuccess();
      return result;
    } catch (err) {
      this.onFailure();
      throw err;
    }
  }

  onSuccess() {
    this.failureCount = 0;
    this.state = "CLOSED";
  }

  onFailure() {
    this.failureCount++;
    if (this.failureCount >= this.failureThreshold) {
      this.state = "OPEN";
      setTimeout(() => {
        this.state = "HALF_OPEN";
      }, this.resetTimeout);
    }
  }

  fallback() {
    return { error: "Service temporarily unavailable", cached: true };
  }
}

// Usage:
const paymentBreaker = new CircuitBreaker(callPaymentService, {
  failureThreshold: 5,
  resetTimeout: 30000,
});

🛠️ Production Libraries

Language	Library
Java	Resilience4j, Hystrix (deprecated)
Node.js	opossum
Go	sony/gobreaker
Python	pybreaker
.NET	Polly
Service mesh	Istio (no code changes needed)

🔗 Circuit Breaker + Fallback Strategies

Fallback Type	Example
Cached response	Return last known good data
Default value	Show "0 recommendations" instead of error
Static response	Return empty array, blank page
Redirect	Send to static maintenance page
Queue	Accept request, process later

✅ Pros

Prevents cascading failures — protects the whole system
Fail-fast gives users a quick response instead of 30s timeout
Gives the failing service breathing room to recover
Enables graceful degradation with fallbacks

❌ Cons

Adds complexity to every service call
Threshold tuning is tricky (too sensitive = flapping, too lax = slow to trip)
Half-open probes can still let some failures through
Stale cached data shown as fallback may mislead users

⚖️ When to Use / When NOT to Use

✅ Use when:

Calling external APIs or microservices (anything that can fail)
Third-party integrations (payment gateways, SMS providers)
Inter-service communication in microservices architecture
Any synchronous call that has a timeout risk

❌ Avoid when:

Calling a local in-process function (no network, no need)
Using async message queues (they decouple naturally)
Already using a service mesh like Istio (it handles this for you)

DDoS Protection & Rate Limiting Abuse Cases

Gouranga Das Samrat — Sun, 06 Sep 2026 04:00:00 +0000

One-liner: A DDoS (Distributed Denial of Service) attack overwhelms your system with fake traffic until real users can't get through. Defense is layered — no single solution works alone.

❓ Types of Attacks

Type	How	Target
Volumetric	Flood bandwidth (Gbps UDP)	Network layer
Protocol	Exhaust TCP connections (SYN flood)	Transport layer
Application (L7)	HTTP flood, slowloris	Your app
Credential stuffing	Try billions of stolen passwords	Auth endpoints
Scraping	Extract all your data	Business logic
Account enumeration	Guess valid emails	User existence

🧱 Defense in Depth (Layers)

Internet
   │
   ▼
┌──────────────────────────────────┐
│  CDN / DDoS Scrubbing Center     │  ← Cloudflare, AWS Shield
│  Filters volumetric attacks      │    Absorbs Tbps-scale traffic
│  IP reputation, anycast routing  │
└──────────────────┬───────────────┘
                   │ (clean traffic only)
                   ▼
┌──────────────────────────────────┐
│  WAF (Web Application Firewall)  │  ← Cloudflare WAF, AWS WAF
│  Blocks SQLi, XSS, bad patterns  │    Rate limits by IP/user-agent
│  OWASP Top 10 rules              │    Bot fingerprinting
└──────────────────┬───────────────┘
                   │
                   ▼
┌──────────────────────────────────┐
│  API Gateway / Load Balancer     │  ← Rate limiting per API key
│  Request throttling              │    IP allowlist/blocklist
│  Auth enforcement                │    Quota per user tier
└──────────────────┬───────────────┘
                   │
                   ▼
┌──────────────────────────────────┐
│  Application Rate Limiting       │  ← Redis-backed counters
│  Per-user, per-endpoint limits   │    Sliding window algorithm
│  CAPTCHA triggers                │    Business logic rules
└──────────────────┬───────────────┘
                   │
                   ▼
            Your Services

🔑 Rate Limiting Abuse Cases

1. Credential Stuffing Defense

// Stricter limits on auth endpoints
rateLimiter({
  "/api/login": { max: 5, window: "15min", by: "ip" },
  "/api/forgot-password": { max: 3, window: "1hr", by: "ip" },
  "/api/register": { max: 10, window: "1hr", by: "ip" },
});

// After 5 failures: require CAPTCHA
// After 10 failures: temp block IP for 1 hour
// Alert security team if 1000+ failures from IP range

2. API Abuse Tiers

Free tier:     100 req/day, 10 req/min
Pro tier:      10,000 req/day, 100 req/min
Enterprise:    Unlimited, custom limits

// Headers to return (industry standard):
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1699999999  (unix timestamp)
Retry-After: 60               (when limit hit, return 429)

3. Slow Loris Defense

Attacker sends HTTP headers very slowly to hold connections open.

# Nginx settings
client_body_timeout 10s;
client_header_timeout 10s;
keepalive_timeout 5s 5s;
send_timeout 10s;

🤖 Bot Detection Signals

Signal	Legitimate User	Bot
Request rate	~1-2 req/sec	100s req/sec
User-agent	Chrome/Firefox	Empty or spoofed
TLS fingerprint	Real browser	curl, Python requests
Behavioral pattern	Random, varied	Uniform, sequential
JavaScript execution	Yes	No (headless bots)
IP reputation	Clean	Known datacenter/VPN

Cloudflare Bot Management and reCAPTCHA v3 automate most of this.

⚙️ AWS Shield Tiers

Tier	Protection	Cost
Shield Standard	Automatic L3/L4 protection	Free
Shield Advanced	L7 protection, DDoS cost protection, 24/7 DRT	$3,000/month

Most startups: Cloudflare Free or Pro tier is sufficient.

✅ Pros

CDN/scrubbing centers absorb traffic before it reaches you
WAF handles OWASP Top 10 without code changes
Rate limiting protects business logic and database
Layered approach means no single point of failure

❌ Cons

Cloudflare/Shield costs money at scale
Sophisticated L7 attacks (mimic real users) are hard to distinguish
Blocking legitimate users with aggressive rate limits hurts UX
Misconfigured WAF rules cause false positives

⚖️ When to Use / When NOT to Use

✅ Use when:

Any public-facing API or website (DDoS is a real threat)
Financial or health apps (credential stuffing target)
APIs with expensive backend operations (LLM calls, DB writes)
Any endpoint that is unauthenticated

❌ Don't over-engineer when:

Internal-only API (behind VPN/private network)
Early prototype with 10 users — Cloudflare Free is enough
Traffic is already behind authentication (harder to abuse)

Authentication & Authorization — JWT & OAuth 2.0

Gouranga Das Samrat — Sat, 05 Sep 2026 04:00:00 +0000

One-liner: Authentication proves who you are; Authorization proves what you're allowed to do. JWT and OAuth 2.0 are the industry standards for doing both at scale.

🎫 JWT — JSON Web Token

A self-contained, signed token that carries claims about the user. No database lookup needed to verify.

Structure: Header.Payload.Signature

eyJhbGciOiJIUzI1NiJ9.eyJ1c2VySWQiOjQyLCJyb2xlIjoiYWRtaW4iLCJleHAiOjE3MDAwMDB9.abc123sig

  HEADER              PAYLOAD                                     SIGNATURE
  {                   {                                           HMACSHA256(
    "alg": "HS256",     "userId": 42,                              base64(header) + "." +
    "typ": "JWT"        "role": "admin",                           base64(payload),
  }                     "iat": 1699999999,                         secretKey
                        "exp": 1700003599  ← expires in 1hr        )
                      }

JWT Flow

1. Login:    Client ──► POST /login {email, password}
                        Server validates, creates JWT
             Client ◄── { token: "eyJ..." }

2. Use API:  Client ──► GET /profile
                        Headers: Authorization: Bearer eyJ...
                        Server decodes JWT, checks exp, checks role
             Client ◄── { user data }

Stateless = Scalable

Server doesn't store sessions. Any server instance can verify the token using the secret key. No Redis/DB lookup needed per request.

JWT Risks & Mitigations

Risk	Mitigation
Token stolen	Short expiry (15min) + Refresh tokens
Can't invalidate before expiry	Refresh token blacklist in Redis
Payload visible (base64 ≠ encrypted)	Never store sensitive data in payload
Weak secret	Use RS256 (asymmetric) for multi-service

🔑 OAuth 2.0 — Delegated Authorization

OAuth lets a third party app access resources on your behalf, without sharing your password.

"Sign in with Google" is OAuth 2.0 in action.

Authorization Code Flow (Recommended)

User         Client App        Authorization Server       Resource Server
  |               |             (Google/GitHub/etc)          (Your API)
  |──"Login"─────►|
  |               |──Redirect──►|
  |               |  client_id  |
  |◄──────────────|             |
  |────Login to──►|             |
  |   Google      |             |
  |               |◄─auth code──|
  |               |             |
  |               |──POST /token|
  |               |  code +     |
  |               |  client_secret
  |               |◄─access_token + refresh_token
  |               |                                         |
  |               |──GET /userinfo (Bearer access_token)───►|
  |               |◄── { id, email, name }──────────────────|
  |               |
  |◄──Logged in!──|

Key OAuth Concepts

Term	What It Is
Client	Your app
Resource Owner	The user
Authorization Server	Google, GitHub, Auth0 (issues tokens)
Resource Server	API that accepts access tokens
Access Token	Short-lived (1hr) — use to call APIs
Refresh Token	Long-lived (30 days) — exchange for new access token
Scope	Permissions granted: `read:email`, `write:profile`

🔐 Session vs JWT Comparison

	Session	JWT
Storage	Server-side (Redis/DB)	Client-side (localStorage / cookie)
Revocation	Instant (delete session)	Must wait for expiry (or blacklist)
Scale	Needs shared session store	Stateless — any server works
Payload	Just session ID	Full claims (userId, role, etc.)
Best for	Monolith, small-medium scale	Microservices, APIs, mobile

✅ Pros

JWT: Stateless, scalable, no DB lookup per request, cross-service

OAuth: No password sharing, granular scopes, industry standard for SSO

❌ Cons

JWT: Revocation is hard, payload visible, long tokens add overhead

OAuth: Complex flow, misconfig is a security disaster, token leakage risks

⚖️ When to Use / When NOT to Use

✅ JWT — use when:

REST APIs serving mobile or SPA clients
Microservices that need to pass identity between services
Stateless, horizontally scaled backends

✅ OAuth — use when:

Third-party "Sign in with X" integrations
Allowing external apps to access user data on your platform
Enterprise SSO (Single Sign-On)

❌ Avoid JWT when:

You need instant logout / token revocation (use sessions with Redis)
Highly sensitive systems where payload visibility is a concern

HLD: Twitter / X

Gouranga Das Samrat — Sun, 30 Aug 2026 02:00:00 +0000

Design a social media platform where users post tweets, follow each other, and see a timeline of tweets from accounts they follow.

1️⃣ Clarify Requirements

Functional Requirements

Post a tweet (text, images, videos)
Follow / unfollow users
View home timeline (tweets from followed users, newest first)
Like, retweet, reply
User profile with tweet history
Search tweets

Non-Functional Requirements

High availability — Twitter is a utility
Eventual consistency — timeline can be slightly stale (seconds)
Low latency — timeline load < 200ms
Read-heavy — 100:1 read-to-write ratio
Scale: 300M DAU, 500M tweets/day

2️⃣ Estimate Scale

Tweets/day: 500M
Write QPS:  500M / 86,400 ≈ 5,800 writes/sec
Read QPS:   5,800 × 100 = 580,000 reads/sec
Peak read:  ~1-2M reads/sec

Storage:
  Tweet: 280 chars = ~300 bytes + metadata ≈ 1 KB
  500M tweets/day × 1 KB = 500 GB/day
  Media: ~100 TB/day (images, videos)

Timeline cache:
  300M users × 100 tweet IDs × 8 bytes = 240 GB

3️⃣ API Design

POST /tweets
Body: { text, mediaIds[] }
Response: { tweetId, createdAt }

GET /timeline/home?cursor=&limit=20
Response: { tweets: [...], nextCursor }

GET /users/:userId/tweets?cursor=
Response: { tweets: [...], nextCursor }

POST /tweets/:tweetId/likes
DELETE /tweets/:tweetId/likes

POST /users/:userId/follow
DELETE /users/:userId/follow

4️⃣ The Core Problem: Home Timeline

The hardest part. User A follows 500 people. How do you fetch their timeline?

Approach 1: Pull (Fan-out on Read)

Timeline request:
1. Get all 500 followed user IDs
2. Query tweets from each user (last 24h)
3. Merge and sort by time
4. Return top 20

❌ Slow: 500 DB queries per timeline request

❌ Gets worse as you follow more people

❌ Doesn't scale at 580K reads/sec

Approach 2: Push (Fan-out on Write) — Twitter's Original Approach

User A tweets:
1. Find all A's followers (e.g., 1,000 followers)
2. Push tweet ID to each follower's timeline cache
3. Timeline request → just read from your cached timeline

Tweet created → [Fan-out Service]
                    → Redis: user1_timeline.lpush(tweetId)
                    → Redis: user2_timeline.lpush(tweetId)
                    → Redis: user3_timeline.lpush(tweetId)
                    ... (1,000 followers)

✅ Timeline reads are O(1) — just read from Redis

❌ Celebrity problem: Lady Gaga has 100M followers → 100M Redis writes per tweet

❌ Wasted if followers don't check timeline

Approach 3: Hybrid (Twitter's Current Approach)

Regular users (< 1M followers): Fan-out on Write
Celebrity users (> threshold):  Fan-out on Read

Timeline request:
1. Fetch pre-computed timeline from Redis (fan-out writes)
2. Fetch recent tweets from celebrity accounts you follow (fan-out reads)
3. Merge both, sort, return

5️⃣ High-Level Architecture

[Client]
   │
   ▼
[CDN] ← static assets, media
   │
[API Gateway + LB]
   │
   ├──────────────────────────────────────┐
   │                                      │
[Tweet Service]                   [Timeline Service]
   │                                      │
   ├─► [Media Service] → [S3/CDN]         ├─► [Timeline Cache (Redis)]
   │                                      │         (500 tweet IDs per user)
   └─► [Tweets DB]                        └─► [Fan-out Service]
       (Cassandra: write-heavy,                   │
        time-series)                        [Message Queue (Kafka)]
                                                  │
                                        [Worker Pool: fan-out writers]
                                                  │
                                          [Timeline Cache Redis]

6️⃣ Database Design

Tweets Table (Cassandra)

CREATE TABLE tweets (
  tweet_id     BIGINT,          -- Snowflake ID (time-sortable)
  user_id      BIGINT,
  text         TEXT,
  media_ids    LIST<UUID>,
  created_at   TIMESTAMP,
  likes_count  COUNTER,
  retweet_count COUNTER,
  PRIMARY KEY (user_id, tweet_id)  -- partition by user, sort by tweet
) WITH CLUSTERING ORDER BY (tweet_id DESC);

Why Cassandra?

Write-heavy (500M tweets/day)
Time-series access pattern (recent tweets)
Naturally partitioned by user_id

Followers Table (Graph in Cassandra)

-- Who does user X follow?
CREATE TABLE following (
  follower_id  BIGINT,
  followed_id  BIGINT,
  created_at   TIMESTAMP,
  PRIMARY KEY (follower_id, followed_id)
);

-- Who follows user X?
CREATE TABLE followers (
  followed_id  BIGINT,
  follower_id  BIGINT,
  created_at   TIMESTAMP,
  PRIMARY KEY (followed_id, follower_id)
);

Denormalized for O(1) lookup in both directions.

Timeline Cache (Redis)

Key: timeline:{userId}
Value: Sorted Set of tweet IDs (score = timestamp)
Size: Keep last 800 tweet IDs
TTL: 7 days (trim inactive users)

7️⃣ Media Storage

Tweet with image:
1. Client uploads image → [Media Service]
2. Media Service → S3 (original)
3. Media Service → async resize: thumbnail, medium, large
4. All sizes stored in S3
5. CloudFront CDN in front of S3
6. Tweet stores mediaId → resolved to CDN URL on read

8️⃣ Search

Tweets → Elasticsearch
Index: tweet_id, text, user_id, created_at, hashtags

Search query → Elasticsearch → ranked results
Trending hashtags → pre-computed every 5 min, stored in Redis

9️⃣ The Celebrity (Hotspot) Problem

Elon Musk tweets → 100M followers
Fan-out on write: 100M Redis writes in seconds → IMPOSSIBLE

Solution:
- Maintain a "high-follower" list
- Skip fan-out for celebrities
- At read time: fetch celebrity tweets separately, merge with pre-computed timeline
- Cache celebrity recent tweets (they're read by millions)

🎨 Diagram

The diagram shows:

Full architecture: Client → CDN → Gateway → Services
Fan-out on write flow (Kafka → workers → Redis)
Hybrid timeline assembly (pre-computed + celebrity merge)
Cassandra tweet storage partition scheme
Media upload and CDN serving path

✅ Trade-offs Summary

Decision	Choice	Rationale
Timeline generation	Hybrid push/pull	Pure push fails for celebrities
Tweet storage	Cassandra	Write-heavy, time-series
Timeline cache	Redis Sorted Set	O(log N) insert, O(1) range read
Fan-out	Async via Kafka	Don't block tweet creation
Media	S3 + CDN	Scalable object storage
Consistency	Eventual	Timeline can be 1-2s stale — fine

HLD: Instagram Feed

Gouranga Das Samrat — Sat, 29 Aug 2026 02:00:00 +0000

Design a photo-sharing platform where users post photos and see a feed of photos from people they follow.

1️⃣ Clarify Requirements

Functional Requirements

Upload photos/videos
Follow/unfollow users
View home feed (photos from followed users, newest first)
Like, comment on photos
User profile with photo grid

Non-Functional Requirements

High availability
Eventual consistency for feed (seconds lag is fine)
Low latency feed load < 200ms
Image-heavy — media storage is the core challenge
Scale: 500M DAU, 100M photos uploaded/day

2️⃣ Estimate Scale

Photo uploads/day: 100M
Write QPS: 100M / 86,400 ≈ 1,160 uploads/sec
Read QPS: 500M × 10 feed loads/day / 86,400 ≈ 58,000 reads/sec

Storage:
  Avg photo: 3 MB (original) + thumbnails (100KB, 500KB) ≈ 4 MB total
  100M photos/day × 4 MB = 400 TB/day
  5-year storage: ~730 PB (need tiered storage!)

CDN bandwidth:
  58K reads/sec × 10 photos/feed × 500KB avg = ~290 GB/sec outbound

3️⃣ Architecture Overview

[Client]
   │
[CDN] ← photos served here (not from origin)
   │
[API Gateway + LB]
   │
   ├── [Photo Service] → upload → [S3] → trigger → [Resize Worker]
   │                                                      → S3 (thumbnail, medium, large)
   │                                                      → Update metadata DB
   │
   ├── [Feed Service] → Redis (pre-computed feed) → return photoIds → CDN URLs
   │
   ├── [User Service] → PostgreSQL
   │
   └── [Social Graph] → Cassandra (followers/following)

4️⃣ Photo Upload Flow

1. Client → POST /upload → Photo Service
2. Photo Service → generate uploadId → return S3 pre-signed URL
3. Client → upload directly to S3 (bypasses your servers!)
4. S3 → triggers Lambda / sends event to queue
5. Resize Worker → create 3 sizes → store in S3
6. Update metadata DB: photoId, userId, S3 keys, created_at
7. Fan-out → push photoId to followers' feed caches (async)

Pre-signed URL pattern = your servers never touch the image bytes.

5️⃣ Feed Generation (Hybrid Fan-out)

Same pattern as Twitter:

Regular users → fan-out on write
Celebrities → fan-out on read, merge at read time

Redis feed cache:
  Key: feed:{userId}
  Value: Sorted Set of {photoId, timestamp}
  Size: 500 entries max
  TTL: 7 days

6️⃣ Database Schema

-- Photos
CREATE TABLE photos (
  photo_id    BIGINT PRIMARY KEY,  -- Snowflake ID
  user_id     BIGINT NOT NULL,
  s3_key_orig VARCHAR(500),
  s3_key_med  VARCHAR(500),
  s3_key_thumb VARCHAR(500),
  caption     TEXT,
  created_at  TIMESTAMP DEFAULT NOW(),
  likes_count BIGINT DEFAULT 0
);

-- Social graph (Cassandra)
CREATE TABLE following (
  follower_id BIGINT,
  followed_id BIGINT,
  PRIMARY KEY (follower_id, followed_id)
);

7️⃣ Key Challenges

Storage Tiering

Hot (0-7 days):   S3 Standard (fast, expensive)
Warm (7-90 days): S3 Infrequent Access (cheaper)
Cold (90+ days):  S3 Glacier (very cheap, slow retrieval)

Serving Photos

NEVER serve from origin → always from CDN
URL format: https://cdn.instagram.com/photos/{photoId}/{size}.jpg
Immutable URLs (content never changes for same ID) → cache forever

✅ Trade-offs Summary

Decision	Choice	Rationale
Photo storage	S3 + CDN	Scalable object store, edge serving
Upload pattern	Pre-signed URL	Don't bottleneck servers with bytes
Feed	Hybrid fan-out	Celebrities break pure push
Social graph	Cassandra	Write-heavy, large-scale relationships
Image sizes	3 variants	Different contexts (thumbnail, feed, full)

From One Merged PR to a Repology Page: A Termux Maintainer Roundup

Gouranga Das Samrat — Wed, 26 Aug 2026 05:44:45 +0000

A few weeks ago, running pkg show on my own phone and seeing my name next to Maintainer: felt like a fluke. It wasn't a plan — it was one feature request for a password manager CLI that turned into weeks of build-system archaeology. Since then it hasn't really stopped.

I've written four posts about individual pieces of this so far. This one is the zoomed-out version — everything currently sitting across termux/termux-packages (the main repo) and termux/termux-user-repository (TUR), plus what's shipped since the last post that I haven't written up on its own yet. If you want the receipts instead of my word for it, my Repology maintainer page tracks every package live under my name in real time.

The story so far

In case you're coming in fresh, here's the series in order:

From a High School Termux User to a Package Maintainer — how proton-pass-cli became my first merge on the main repo, and the sqlcipher symlink bug it dragged along with it.
Rejected on Main, Accepted on TUR — nine Nerd Fonts packages that didn't fit the main repo's size policy, and my first merge on TUR instead.
From a 4-Year-Old Feature Request to pkg install bun — closing a four-year-old issue by moving bun from a rejected main-repo PR to a TUR merge.
The Day I Merged 6 Go Packages Into Termux Before Dinner — goimports, golangci-lint, air, gotests, goreleaser, and govulncheck, all merged in one sitting.

Everything below picks up right after that last post.

Main repo: cleanup, then a second wave

The six Go packages didn't land perfectly clean. Right after merge, each one still carried a build dependency it didn't actually need at runtime, so I went back through all six with individual fix PRs: goimports, golangci-lint, air, goreleaser, gotests, and govulncheck. Small, boring PRs — the kind that don't make for a good story on their own, but matter for anyone who installs these packages later. govulncheck also got bumped to 1.7.0 the same day.

With the toolkit's foundation cleaned up, I kept adding to it:

git-cliff — changelog generator driven by conventional commits.
git-absorb — automatically absorbs staged changes into the right earlier commits.
gtrash — a trash can for the CLI, so rm stops being permanent by accident.
sqlc — generates type-safe code straight from SQL.
golang-migrate — database migrations, CLI and library both.
gotestsum — a nicer go test output wrapper, pairs naturally with gotests from the earlier batch.

And proton-pass-cli, the package that started all of this, got bumped to 2.3.3 — still getting routine maintenance almost two weeks after the original merge.

Not everything from this stretch made it into the main repo, and that's expected at this point: PR #31157 (the original 9-font attempt) and PR #31115 (the prebuilt-binary bun attempt) both closed on policy grounds, and PR #31013 closed as a duplicate of a fix that landed first. All three have their full story in the earlier posts — I'm only listing them here so the trail is complete.

TUR: the fonts grow up

TUR has been quieter in terms of headline moments, but not idle. The eight Nerd Fonts packages from PR #2743 started out as eight independent ttf-*-nerd packages. In PR #2755 I refactored all eight into a single unified nerd-fonts parent/subpackage structure and bumped everything to v3.5.1 — one source package, eight subpackages, instead of eight nearly-identical build.sh files to keep in sync by hand. It hit the usual round of CI friction along the way (a missing cache directory variable, an overlength description, a license auto-detection miss, an array-vs-scalar source URL, a subpackage dependency value that needed correcting) — nothing dramatic, just the kind of thing that shows up once you actually run a consolidation through CI instead of just planning it.

Right after that, PR #2762 added a dependency on the main repo's pre-existing ttf-nerd-fonts-symbols package, to match how Arch handles the same font set — closing a small naming/parity gap that had been sitting there since the original PR.

bun also has a bump sitting as a draft — 1.4.0, not merged yet. And there's an early-stage docker-qemu package in draft too, at PR #2708.

Where things stand

Two repos, two different sets of rules, and by now a pattern I'm used to: try the main repo first, and if the policy genuinely doesn't fit — package count discipline, source-build requirements — TUR is where it goes instead, not a consolation prize.

The Repology maintainer page is the easiest way to see the live total, since PR merges don't always mean immediate publish. As of today it lists 15 distinct projects across both repos — 13 on Termux's main repo, 2 on TUR — every single one at 100% newest, with nothing outdated, problematic, or flagged as potentially vulnerable:

air-live-reload, bun, fonts:nerd-fonts, git-absorb, git-cliff, go:migrate, goimports, golangci-lint, goreleaser, gotests, gotestsum, govulncheck, gtrash, proton-pass-cli, sqlc

Two entries are worth a second look because the names don't match the PR titles: Repology tracks air under its upstream project name air-live-reload, and golang-migrate as go:migrate. The other one is fonts:nerd-fonts showing up as a single TUR project instead of eight — which is exactly what PR #2755 was for. The consolidation into one parent/subpackage structure didn't just clean up the build.sh files, it collapsed the whole set into one tracked project on Repology too.

The Day I Merged 6 Go Packages Into Termux Before Dinner 🐹

Gouranga Das Samrat — Mon, 24 Aug 2026 04:27:22 +0000

I didn't wake up planning to get six pull requests merged. I just wanted to add one missing Go tool to termux/termux-packages. Then I figured, while I'm in here, why not bring the rest of my Go toolkit along too? A few hours and six PRs later, all of them were sitting in main.

Here's how it went down — and what actually shipped.

What Landed

Termux users can now pkg install a proper Go developer toolkit, straight from the official repo:

goimports 0.49.0 — the tool that quietly fixes your import block, adding what's missing and trimming what isn't used anymore. Learn more →
golangci-lint 2.13.1 — dozens of Go linters bundled into one fast binary. At 45 MB installed, it's the heavyweight of the group, but worth every megabyte. Learn more →
air 1.67.4 — live reload for Go apps, so your binary rebuilds and restarts the second you save a file. Learn more →
gotests 1.9.0 — generates table-driven test scaffolding straight from your source, so you stop typing the same boilerplate. Learn more →
goreleaser 2.17.1 — packages and ships Go binaries in one command, cross-compilation included. Learn more →
govulncheck 1.1.4 — scans your code and dependencies for known vulnerabilities. Learn more →

Six packages, one theme: making Termux a real place to write Go.

The Curveball: A Last-Minute Rebase

Right as review was wrapping up, maintainer @robertkirkman asked for something I hadn't planned for: rebase and force-push all six PRs so they'd recompile against the just-released Go 1.27.0. You can see the ask in the goimports thread.

No arguing with that logic — better to ship on the latest compiler than patch it later. I rebased all six branches, force-pushed, and got a quick "Thank you!" in response. Not long after, the merges started coming in one by one.

Proof It's Real

Once everything landed, I ran the most satisfying command in the Termux workflow — pkg show — against all six:

Package: goimports        | 0.49.0  | https://pkg.go.dev/golang.org/x/tools/cmd/goimports
Package: golangci-lint    | 2.13.1  | https://golangci-lint.run/
Package: air              | 1.67.4  | https://github.com/air-verse/air
Package: gotests           | 1.9.0   | https://github.com/cweill/gotests
Package: goreleaser        | 2.17.1  | https://goreleaser.com
Package: govulncheck       | 1.1.4   | https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck

Six for six. All installable right now with:

pkg install goimports golangci-lint air gotests goreleaser govulncheck

What I'd Tell Someone Trying This Themselves

Batch related packages together. Reviewers can apply one piece of feedback — like a compiler bump — across the whole set instead of repeating themselves six times.

Move fast on review feedback. The gap between "please rebase" and "rebased and pushed" is often the difference between a PR that merges today and one that sits for a week.

Don't underestimate the boring packages. None of these tools are glamorous. Together, though, they turn a phone into a legitimate Go dev environment.

Thanks to @robertkirkman and the rest of the termux/termux-packages team for the fast, thorough reviews. More packages coming soon.

Working on a Go tool you think Termux is missing? Tell me about it in the comments — I'm always looking for the next PR.

HLD: Notification System

Gouranga Das Samrat — Sun, 23 Aug 2026 02:00:00 +0000

Design a notification system that sends push notifications, emails, and SMS to users based on events in the platform.

1️⃣ Clarify Requirements

Functional Requirements

Send notifications via: Push (iOS/Android), Email, SMS
Trigger types: real-time (someone liked your post) and scheduled (weekly digest)
Users can set preferences: opt-in/out per channel and notification type
Support for templated notifications (not hardcoded)
Delivery guarantee — important notifications must not be dropped

Non-Functional Requirements

High throughput — 10M notifications/day
Low latency for real-time notifications (< 5 seconds)
Reliability — retry on failure
Scalability — handle spikes (breaking news, product launches)
Observability — track delivered/failed/opened

2️⃣ Estimate Scale

Notifications/day: 10M
Notifications/sec: 10M / 86,400 ≈ 116/sec (avg)
Peak: 1,000/sec (event spikes)

Breakdown:
  Push: 60% → 6M/day
  Email: 30% → 3M/day
  SMS: 10% → 1M/day

Storage (notification log):
  1 record ≈ 500 bytes
  10M/day × 500B = 5 GB/day
  Retention: 90 days → 450 GB

3️⃣ API Design

# Send a notification (internal service call)
POST /notifications/send
Body: {
  "userId": "user_123",
  "type": "like_received",
  "channels": ["push", "email"],  // or leave empty → use user preferences
  "data": {
    "actorName": "Rahul",
    "postTitle": "My Design Notes"
  }
}

# User preference management
GET  /users/:userId/notification-preferences
PUT  /users/:userId/notification-preferences
Body: {
  "push": { "like_received": true, "new_follower": true, "marketing": false },
  "email": { "weekly_digest": true, "marketing": false }
}

# Scheduled notifications
POST /notifications/schedule
Body: {
  "templateId": "weekly_digest",
  "scheduledAt": "2024-01-15T09:00:00Z",
  "userSegment": "active_users"
}

4️⃣ High-Level Architecture

[Event Sources]
  - User Service ("user A liked post")
  - Order Service ("your order shipped")
  - Scheduler ("send weekly digest")
        │
        ▼
[Notification Service API]
        │
        ├─► Check User Preferences → skip if opted out
        ├─► Build notification from template
        ├─► Route to appropriate channels
        │
        ▼
[Message Queue (Kafka / SQS)]
  ├──► [Push Queue]
  ├──► [Email Queue]
  └──► [SMS Queue]
        │
        ▼
[Channel Workers]
  ├──► [Push Worker] → FCM (Android) / APNs (iOS)
  ├──► [Email Worker] → SendGrid / SES / Mailgun
  └──► [SMS Worker] → Twilio / SNS
        │
        ▼
[Delivery Log DB] ← track status per notification

5️⃣ Component Deep Dives

Preference Service

# User preferences (stored in DB, cached in Redis)
preferences = {
  "push": {
    "like_received": True,
    "new_follower": True,
    "marketing": False
  },
  "email": {
    "weekly_digest": True,
    "security_alert": True,
    "marketing": False
  },
  "sms": {
    "security_alert": True,
    "marketing": False
  }
}

def should_notify(userId, notifType, channel):
    prefs = redis.get(f"prefs:{userId}") or db.get(userId)
    return prefs.get(channel, {}).get(notifType, True)  # default: on

Template Service

// Template: "like_received"
{
  "templateId": "like_received",
  "push": {
    "title": "{{actorName}} liked your post",
    "body": "\"{{postTitle}}\" is getting attention!"
  },
  "email": {
    "subject": "{{actorName}} liked your post",
    "templateFile": "like-received.html"
  }
}

Render: inject actual values, produce final notification content.

Push Notification Worker

def send_push(notification):
    device_tokens = db.get_device_tokens(notification.userId)

    for token in device_tokens:
        if token.platform == "android":
            fcm.send({
                "token": token.value,
                "title": notification.title,
                "body": notification.body,
                "data": notification.data
            })
        elif token.platform == "ios":
            apns.send({
                "deviceToken": token.value,
                "aps": { "alert": { "title": notification.title, "body": notification.body } }
            })

Retry & Dead Letter

Worker fails to send → retry with exponential backoff
  Attempt 1: immediate
  Attempt 2: 30 seconds
  Attempt 3: 5 minutes
  Attempt 4: 30 minutes
  Attempt 5: DLQ (manual inspection or discard)

6️⃣ Database Schema

-- Notification log (Cassandra — write-heavy, time-series)
CREATE TABLE notifications (
  notification_id  UUID,
  user_id          BIGINT,
  type             VARCHAR(50),
  channel          VARCHAR(20),  -- push/email/sms
  status           VARCHAR(20),  -- pending/sent/failed/delivered/opened
  payload          JSONB,
  created_at       TIMESTAMP,
  sent_at          TIMESTAMP,
  PRIMARY KEY (user_id, created_at, notification_id)
) WITH CLUSTERING ORDER BY (created_at DESC);

-- User device tokens
CREATE TABLE device_tokens (
  user_id    BIGINT,
  token      TEXT,
  platform   VARCHAR(10),  -- ios/android/web
  created_at TIMESTAMP,
  PRIMARY KEY (user_id, token)
);

-- User preferences (PostgreSQL, read-heavy)
CREATE TABLE notification_preferences (
  user_id     BIGINT PRIMARY KEY,
  preferences JSONB,
  updated_at  TIMESTAMP
);

7️⃣ Handling Scale Spikes

Event: Product Launch → 10M users notified

Problem: Sudden flood of 10M notifications in seconds.

Solution: Rate limiting the outbound queue

Email: SendGrid allows 1000 emails/sec
→ Queue backs up → workers drain at 1000/sec → all 10M sent in ~3 hours
→ Users get it "within a few hours" → acceptable for marketing

Priority Queues

HIGH priority queue:   security alerts, OTPs → process immediately
MEDIUM priority queue: social notifications → within seconds
LOW priority queue:    marketing, digests → within hours

Workers check HIGH first, then MEDIUM, then LOW.

8️⃣ Observability

Track every notification:

Events to log:
  - notification_queued     (timestamp, userId, type, channel)
  - notification_sent       (timestamp, provider response)
  - notification_delivered  (timestamp, from FCM/APNs delivery receipt)
  - notification_opened     (timestamp, from SDK tracking)
  - notification_failed     (timestamp, error, retryCount)

Metrics:
  - Delivery rate per channel
  - Average send latency (queued → sent)
  - Failure rate per provider
  - Open rate per notification type

🎨 Diagram

The diagram shows:

Event sources → Notification Service → Kafka queues → channel workers → external providers
Preference check before queuing
Template rendering step
Retry with DLQ
Priority queue lanes (high/medium/low)
Delivery status tracking

✅ Trade-offs Summary

Decision	Choice	Rationale
Queue type	Kafka/SQS	Decouple events from delivery, absorb spikes
Priority	Multiple queues	Don't let marketing delay OTPs
Retry	Exponential backoff	Avoid hammering failed providers
Preference storage	Redis cache + DB	Fast reads on every notification
Delivery tracking	Cassandra	High-volume, time-series write pattern

HLD: URL Shortener (like bit.ly)

Gouranga Das Samrat — Sat, 22 Aug 2026 02:00:00 +0000

Design a system that takes a long URL and returns a short URL, and redirects short URLs to their originals.

1️⃣ Clarify Requirements

Functional Requirements

Given a long URL, generate a short URL
Given a short URL, redirect to the original long URL
(Optional) Custom aliases: short.ly/my-brand
(Optional) URL expiration
(Optional) Analytics: click count, referrer, location

Non-Functional Requirements

High availability — downtime = broken links everywhere
Low latency redirects — < 10ms (cached)
Durability — shortened URLs should work for years
Reads >> Writes (100:1 read-to-write ratio typical)

2️⃣ Estimate Scale

Write QPS:
  10M URLs shortened per day
  = 10M / 86,400 ≈ 115 writes/sec

Read QPS (100:1 ratio):
  100 × 115 = 11,500 reads/sec
  Peak: ~50,000 reads/sec

Storage:
  Each record: shortURL(7B) + longURL(200B) + metadata(100B) ≈ 307 bytes
  10M URLs/day × 10 years = 36.5B URLs
  36.5B × 307 bytes ≈ 11 TB (very manageable)

Cache:
  80% reads on 20% of URLs (Pareto principle)
  Cache 20% of daily URLs: 10M × 0.2 × 307B ≈ 600 MB/day

3️⃣ API Design

# Create short URL
POST /api/shorten
Body: { "longUrl": "https://...", "customAlias": "optional", "expiresAt": "optional" }
Response: { "shortUrl": "https://short.ly/abc1234" }

# Redirect
GET /:shortCode
Response: 301/302 Redirect to longUrl

# Analytics (optional)
GET /api/stats/:shortCode
Response: { clicks: 1000, topCountries: [...] }

301 vs 302:

301 Permanent → browser caches redirect → fewer server hits → can't update
302 Temporary → browser always checks server → can update/expire → use this for analytics

4️⃣ Short Code Generation

Option A: Hash + Truncate

MD5(longURL) = "1a79a4d60de6718e8e5b326e338ae533"
Take first 7 chars: "1a79a4d"
shortURL: short.ly/1a79a4d

Problem: Collisions — different URLs can produce same 7-char prefix.

Option B: Base62 Encoding (Recommended)

Character set: [0-9a-zA-Z] = 62 characters

7 characters × 62^7 = 3.5 trillion unique codes ✅

ID: 123456789
Base62(123456789) = "8M0kX"
shortURL: short.ly/8M0kX

Where does the ID come from?

Auto-increment DB ID — simple, but reveals volume/sequence
UUID — random, no collision, but long → truncate carefully
Snowflake ID — distributed, time-sortable, unique (Twitter's approach)

Option C: Counter Service

A dedicated service generates unique IDs:

Counter Service: ID = 1, 2, 3, ... (globally unique)
URL Service: Base62(ID) → short code

5️⃣ High-Level Design

                                    [Cache (Redis)]
                                          │
Client ──► [API Gateway + LB] ──► [URL Service] ──► [DB (PostgreSQL)]
                                          │
                                   [Analytics Queue]
                                          │
                                   [Analytics DB]

Redirect flow:
Client ──GET /abc1234──► [URL Service]
                              │
                    ┌─── Redis HIT → 302 Redirect ──► Client
                    │
                    └─── Redis MISS → DB query → cache → 302 Redirect

6️⃣ Database Schema

CREATE TABLE urls (
  id           BIGSERIAL PRIMARY KEY,
  short_code   VARCHAR(10) UNIQUE NOT NULL,
  long_url     TEXT NOT NULL,
  user_id      BIGINT,
  created_at   TIMESTAMP DEFAULT NOW(),
  expires_at   TIMESTAMP,
  click_count  BIGINT DEFAULT 0
);

CREATE INDEX idx_short_code ON urls(short_code);

-- Analytics table
CREATE TABLE clicks (
  id          BIGSERIAL PRIMARY KEY,
  short_code  VARCHAR(10),
  clicked_at  TIMESTAMP DEFAULT NOW(),
  country     VARCHAR(2),
  referrer    TEXT,
  user_agent  TEXT
);

7️⃣ Caching Strategy

Key:   short_code ("abc1234")
Value: long_url
TTL:   24 hours (refresh on hit if needed)

Cache-aside:
1. Check Redis for short_code
2. MISS → query PostgreSQL → store in Redis with TTL
3. HIT → return long_url immediately

Cache hit rate target: >95% (80/20 rule — most traffic on top URLs)

8️⃣ Scalability Deep Dive

Read Scaling

Redis cache handles ~95% of redirect traffic
Read replicas for the remaining DB reads
CDN for the redirect response itself (if 301)

Write Scaling

Writes (new URLs) are much rarer — primary DB + async replication is fine
If needed: hash short_code → route to shard

High Availability

Redis Sentinel or Cluster for cache HA
DB: Primary + read replicas + automated failover
Multiple URL service instances behind LB

9️⃣ Custom Aliases & Expiry

def shorten(long_url, custom_alias=None, expires_at=None):
    if custom_alias:
        if db.exists(custom_alias):
            raise ConflictError("Alias taken")
        short_code = custom_alias
    else:
        id = db.insert(long_url)
        short_code = base62_encode(id)

    db.store(short_code, long_url, expires_at)
    return f"https://short.ly/{short_code}"

def redirect(short_code):
    url_data = cache.get(short_code) or db.get(short_code)
    if not url_data:
        raise NotFoundError()
    if url_data.expires_at and url_data.expires_at < now():
        raise GoneError()  # 410 Gone
    return url_data.long_url

🎨 Diagram

The diagram shows:

Full system: Client → LB → URL Service → Redis → PostgreSQL
Shorten flow vs redirect flow (separate paths)
Cache hit vs miss paths
Analytics async flow via queue
DB schema with indexes highlighted

✅ Trade-offs Summary

Decision	Choice	Trade-off
301 vs 302	302	More server hits, but analytics & expiry work
ID generation	DB auto-increment + Base62	Simple but sequential — use Snowflake for privacy
Cache TTL	24hr	Stale entries for expired/deleted URLs
SQL vs NoSQL	PostgreSQL	Simple queries, ACID for uniqueness

From a 4-Year-Old Feature Request to `pkg install bun`: My Second Merge on the Termux User Repository

Gouranga Das Samrat — Fri, 21 Aug 2026 09:35:47 +0000

Last post i wrote was about getting turned away from the main repo over a policy disagreement and landing my first TUR merge instead with 8 Nerd Fonts packages.

This is the same shape of story again, except the "no" from the main repo wasn't about repo size this time — it was about something much harder to argue around: how Termux insists packages get built.

An issue older than some of the tools people used to work around it

Issue #11188 asking for bun in Termux goes back to July 2022, opened by @leap0x7b. Reading the whole thread back to front is basically a history of people improvising around a missing package for four straight years:

Early on, bun's Linux aarch64 build just didn't run on Android — npm install -g bun failed outright with an EBADPLATFORM error, because upstream hadn't shipped an Android target.
The workaround that kept surfacing was grun, a glibc compatibility shim, letting people run the Linux binary against a glibc environment layered on top of Termux's normal Bionic libc. It mostly worked, with recurring reports of install failures (AccessDenied errors from bun install trying to hardlink files) and one-off fixes like bun install --backend=copyfile.
A few people tried building bun from source directly. That hit its own wall: Bun's own docs state that Bun itself must be installed to compile Bun — you need an existing bun binary just to build a new one — plus a strict Zig version dependency that broke against whatever Termux had packaged at the time.
@TomJo2000 summed up the stall bluntly in mid-2025: nobody had written a build script, no other distro built it from source either, and it depended on a kernel version most Android devices didn't meet.

The turn came in April 2026, when @licy183 flagged that upstream had finally added an Android build target. @TomJo2000 noted the obvious next step — someone still had to actually write the build script. By mid-May, @Jobians confirmed Bun was running natively using the official Android ARM64 build. The pieces were finally there. I decided to be the one to write the script.

Round one: prebuilt binaries, meet main-repo policy

I opened termux-packages PR #31115 — addpkg(main/bun): 1.3.14 — built around the official prebuilt Android binaries that shipped in Bun v1.3.14. The binaries are Position-Independent Executables linked straight against standard Bionic libraries (libc.so, libm.so, libdl.so), so no glibc shim needed at all — a real fix for the grun workaround people had been leaning on for years. The only real limitation was architecture: upstream only builds Android targets for aarch64 and x86_64, so I set those two as excluded-safe and expected arm/i686 to just skip cleanly in CI.

@robertkirkman stopped that approach at the door: termux-packages requires compiling from source, so downloading upstream's prebuilt binaries wasn't going to fly. He floated a specific alternative — if Bun were published as an installable crate on crates.io, cargo-binstall could fetch upstream binaries in a way the main repo's tooling already sanctions. Worth checking, but it didn't pan out: Bun isn't a simple Rust crate at all. It's a monorepo built through its own CMake pipeline, linking against JavaScriptCore/WebKit and LLVM — nothing cargo build --release or cargo-binstall can touch.

There was a fun tangent buried in the same exchange — @robertkirkman noting Node.js is already built from source in Termux and reasoning Bun should in theory be possible too, then wondering out loud whether Bun is meaningfully faster than both Node and Deno (Deno also being a from-source build in Termux) — the kind of "is this actually a different tool or just a faster clone" curiosity that comes up naturally whenever a runtime like this gets proposed.

I went and actually tried the from-source route properly, then came back and closed the PR myself on August 20 with a detailed writeup of exactly where it broke down:

Bun's build system does have official Android cross-compile support (--abi=android, --android-ndk=<path>), confirmed against the pattern in its .buildkite/Dockerfile (NDK r27c, host-clang + sysroot).
WebKit doesn't need a from-source build for Android — a prebuilt tarball gets fetched automatically, so that wasn't the blocker I expected.
The actual wall was a statically, cross-compiled ICU for Android ($BUN_ANDROID_ICU_ROOT) with no public build recipe I could find anywhere — not in oven-sh/bun, not in oven-sh/bun-development-docker-image, not in the actual .buildkite/Dockerfile, despite a comment in webkit.ts pointing at a "Dockerfile.android" that doesn't seem to exist publicly.

Add a strictly pinned LLVM 21.1.8 and a pinned Rust nightly on top of that, and it was more toolchain archaeology than I could commit to finishing in one PR. So I closed it, thanked @robertkirkman for the guidance, and left the door open to revisit if upstream's Android/ICU story ever gets easier to build from source.

The TUR side already had its own long-running Bun request

There was already a parallel thread for this on TUR: tur issue #571, opened back in 2023 by @earningpoints. @licy183 had kept it open rather than closing it outright, citing two blockers at the time — Bun's kernel 5.1 requirement, which most Android devices didn't meet, and the fact that it didn't compile against non-glibc libcs like musl. @jothi-prasath tried getting it running via grun and shared a working build script, but @licy183 was clear about where TUR draws its own line: binaries in TUR packages need to link against Bionic libc, not GNU libc, with no plan to support a custom-loader GNU libc setup.

That distinction turned out to matter a lot for round two — because the whole reason the official Bun v1.3.14 Android binaries worked for Termux at all is that they're linked directly against Bionic. No grun, no glibc shim, no custom loader path. They fit TUR's actual requirement perfectly; they just didn't fit the main repo's from-source rule.

Round two: TUR PR #2746

I opened tur PR #2746 — same package, same version, same prebuilt-binary approach — explicitly closing both threads at once: Fixes termux/termux-packages#11188 #571, with a note that it was migrated from PR #31115 specifically because of the main repo's build-from-source policy. No repeat of the source-vs-prebuilt argument — TUR's policy is built for exactly this kind of package.

Review was quick and purely technical. @licy183 caught an inefficiency in build.sh: rather than my approach, the more common pattern is to download binaries for both aarch64 and x86_64 up front and only install the one matching the build architecture — pointing me at how pypy3's build script already does it. I made the change same day. A second review comment flagged that my binary-path handling was split oddly across build steps — it only needed to run in termux_step_make_install, while termux_step_post_get_source should just be fetching the LICENSE file. I moved the logic accordingly.

Then: a simple "Thanks!" from @licy183, and that was that.

What changed, twice over

The version string says 1.3.14, same as the PR I closed on the main repo, but what actually shipped is bigger than the number:

A four-year-old feature request that survived grun hacks, glibc shims, Zig version breakage, and a genuine "you need Bun to build Bun" chicken-and-egg problem, finally resolved with pkg install bun.
A real technical answer, on the record, for why termux-packages couldn't take it as-is: a from-source Android build is blocked on a static cross-compiled ICU with no public recipe — useful for whoever eventually revisits it.
Confirmation from TUR's side of exactly which binaries are acceptable there: Bionic-linked, no glibc shim required — which is exactly what upstream Bun now ships.

Two PRs in a row now where the front door said no for a real, defensible reason, and TUR turned out to be exactly the right back door. Different reasons each time — repo-size policy for the fonts, source-build policy for Bun — but the same shape of resolution. I'm starting to think that's just what TUR is for.

Get it

If you're on Termux and TUR isn't already added as a repo, three commands and bun is installed with no grun, no glibc shim, none of the workarounds from that four-year thread:

pkg install tur-repo
pkg update
pkg install bun

Proof it's live:

~ $ pkg show bun
Package: bun
Version: 1.3.14-1
Maintainer: Gouranga Das Samrat <gouranga.das.khulna@gmail.com>
Installed-Size: 89.8 MB
Homepage: https://bun.com
Download-Size: 22.8 MB
APT-Sources: https://tur.kcubeterm.com tur-packages/tur aarch64 Packages
Description: Incredibly fast JavaScript runtime, bundler, test runner, and package manager

The full trail:

Rejected on Main, Accepted on TUR: How 8 Nerd Fonts Became My First Merge on the Termux User Repository

Gouranga Das Samrat — Thu, 20 Aug 2026 16:57:28 +0000

I wrote before about the day proton-pass-cli merged into termux-packages — my first merge into the main repo, full stop. This post is about what happened right after: I tried to do it again with nine fonts, got talked out of it on the main repo, and ended up with my first merge on TUR instead — 8 packages, all in one shot.

Round one: nine fonts, one PR, the main repo

I opened termux-packages PR #31157 — addpkg(nerd-fonts): add 9 Nerd Fonts packages (v3.5.0). The plan: pull nine fonts from the v3.5.0 release of ryanoasis/nerd-fonts, package each as its own opt-in .deb following the same pattern as the existing ttf-jetbrains-mono, and ship only the plain Regular/Bold/Italic/BoldItalic styles to keep things lean.

@robertkirkman was first to comment, and it wasn't about the packages themselves — it was naming. Termux already had ttf-nerd-fonts-symbols, which mirrors Arch Linux's naming convention, so he asked me to rename all nine of mine to match it too. Fair catch — I'd shipped them as ttf-<font>-nerd-font, and I renamed everything to ttf-<font>-nerd the same day.

That turned out to be the easy part.

Round one, continued: the argument that actually mattered

Somewhere in review, @TomJo2000 raised a concern about repo size — enough that my next comment opened with thanking him for it directly, even though he hadn't posted publicly in the thread yet. I made the case for keeping all nine anyway: Homebrew and Arch both ship every Nerd Font as its own opt-in package, nothing else in Termux would depend on mine so there's zero cost to anyone who skips them, and ttf-jetbrains-mono already set the precedent. I even offered to trim styles further if size was the real issue.

@robertkirkman didn't disagree with the utility — he laid out why Termux draws the line differently than Homebrew or Arch in the first place. Termux sits at a little over 3,000 packages; those other ecosystems are past 10,000. The project's focus, as he put it, is packages that are otherwise inconvenient to get running on Android — not "niche, static cosmetic assets that never require patching to work directly copied and pasted into Termux, like wallpapers, fonts, themes and icon packs." He did concede the fonts had a genuinely high Repology score, and offered a condition: add them once at least one other person actually asks.

I pushed back on that condition specifically, because it's a trap by design — once a PR like mine is sitting open and visible, anyone who wants the package just watches the PR instead of filing a separate request, so the bar never gets cleared. I tried a subset compromise (trim the heaviest fonts, keep the rest), pointed to PR #29412 (OBS Studio) as a case where real demand never shows up as comments on an open PR, and floated moving everything to TUR if it stayed stuck.

@robertkirkman held the line on the condition, and also corrected my OBS comparison — that PR wasn't stalled from lack of interest, it was blocked on an unresolved Android 7 compatibility bug being discussed mostly on Discord, not GitHub. @TomJo2000 weighed in too, rejecting the "ship a subset" middle ground outright and pointing out that termux-styling has its own version of this exact problem already.

I tried one more angle — comparing font packages to -static libraries that already sit in the main repo with zero reverse dependencies — which pulled the thread into a genuinely interesting tangent about why Termux ships -static packages at all. @robertkirkman pointed to portaudio-static easing a real packaging headache for CDP8 and to PR #29787 as an example of the maintenance cost involved; @TomJo2000 admitted he wasn't even sure why they're split out in the first place, guessing it predates his time on the project. Somewhere in there I also brought up that sqlcipher itself had zero reverse dependencies in the main repo right up until I shipped proton-pass-cli #30987 — which @robertkirkman settled with a technical clarification: the no-reverse-dependency rule mainly targets packages that only drop files into lib, libexec, share, or opt. sqlcipher always shipped an executable in bin too, so it was never actually in violation.

None of that moved the font policy itself. So I closed the PR myself and did what I'd already floated: took it to TUR.

Round two: TUR PR #2743

TUR exists precisely for software that's useful and wanted but doesn't fit the main repo's policy — so I opened tur PR #2743, explicitly noting in the description that it was migrated from termux/termux-packages#31157 "to respect main repo's policies." Iosevka got dropped along the way, so this round shipped eight fonts, not nine — though the PR title, copy-pasted from the original, still says "9 Nerd Fonts packages" even with a table listing exactly eight underneath it. Nobody caught it, and honestly, neither did I until just now.

This time there was no policy fight — just infrastructure. @robertkirkman flagged that a GitHub race condition was currently blocking the creation of new package repos in TUR, so merging would have to wait. A couple hours later, @licy183 stepped in, offering to manually set up a nerd-fonts repository in tur-dists in the meantime, with a longer-term fix — retrying the upload script up to five times with a one-second sleep between attempts — planned for the following weekend.

It worked.

Proof it's real

~ $ pkg search '^ttf-.*nerd'
ttf-cascadia-code-nerd/tur-packages 3.5.0 all
  Cascadia Code patched with Nerd Fonts icons/glyphs (Font Awesome, Devicons, Octicons, Powerline, etc)

ttf-hack-nerd/tur-packages 3.5.0 all
  Hack patched with Nerd Fonts icons/glyphs (Font Awesome, Devicons, Octicons, Powerline, etc)

ttf-inconsolata-nerd/tur-packages 3.5.0 all
  Inconsolata patched with Nerd Fonts icons/glyphs (Font Awesome, Devicons, Octicons, Powerline, etc)

ttf-jetbrains-mono-nerd/tur-packages 3.5.0 all
  JetBrains Mono patched with Nerd Fonts icons/glyphs (Font Awesome, Devicons, Octicons, Powerline, etc)

ttf-meslo-nerd/tur-packages 3.5.0 all
  Meslo (MesloLGS) patched with Nerd Fonts icons/glyphs (Font Awesome, Devicons, Octicons, Powerline, etc), commonly used with Powerlevel10k

ttf-roboto-mono-nerd/tur-packages 3.5.0 all
  Roboto Mono patched with Nerd Fonts icons/glyphs (Font Awesome, Devicons, Octicons, Powerline, etc)

ttf-sourcecodepro-nerd/tur-packages 3.5.0 all
  Source Code Pro patched with Nerd Fonts icons/glyphs (Font Awesome, Devicons, Octicons, Powerline, etc)

ttf-victor-mono-nerd/tur-packages 3.5.0 all
  Victor Mono patched with Nerd Fonts icons/glyphs (Font Awesome, Devicons, Octicons, Powerline, etc)

Eight of those nine lines are mine — every one tagged tur-packages with my name on Maintainer:. The odd one out is ttf-nerd-fonts-symbols/stable, which just happens to match the same search pattern: it's the pre-existing package from the main repo, maintained by @termux, not by me. Easy to mix up in a grep, so worth saying plainly here.

~ $ pkg show ttf-cascadia-code-nerd
Package: ttf-cascadia-code-nerd
Version: 3.5.0
Maintainer: Gouranga Das Samrat <gouranga.das.khulna@gmail.com>
Installed-Size: 11.4 MB
Homepage: https://www.nerdfonts.com/
Download-Size: 1550 kB
APT-Sources: https://tur.kcubeterm.com tur-packages/tur aarch64 Packages
Description: Cascadia Code patched with Nerd Fonts icons/glyphs (Font Awesome, Devicons, Octicons, Powerline, etc)

Same story across all eight — ttf-hack-nerd, ttf-inconsolata-nerd, ttf-jetbrains-mono-nerd, ttf-meslo-nerd, ttf-roboto-mono-nerd, ttf-sourcecodepro-nerd, and ttf-victor-mono-nerd all show up with my Maintainer: line and tur.kcubeterm.com as the source. One PR, one merge, eight packages live at once — the biggest single contribution I've shipped to Termux so far, and my first time landing anything on TUR specifically.

What actually stuck with me

Getting held to a stricter bar on the main repo didn't feel great mid-thread — three-thousand-package discipline running headfirst into nine fonts I was sure people wanted. But @robertkirkman and @TomJo2000 weren't wrong to hold it, and the rejection wasn't a dead end so much as a redirect to the tool built for exactly this situation. proton-pass-cli taught me how to get something into termux-packages. This one taught me that knowing where a package doesn't belong — and having TUR as somewhere to actually put it — is just as much a part of contributing as the packaging itself.

If you're on Termux and tired of manually curling .ttf files into ~/.termux/font.ttf:

pkg install tur-repo
pkg update
pkg install ttf-jetbrains-mono-nerd

(swap in hack, cascadia-code, meslo, sourcecodepro, inconsolata, roboto-mono, or victor-mono for whichever prompt you're patching.)

The full trail:

DEV Community: Gouranga Das Samrat

Distributed Locks

❓ Why Do You Need Distributed Locks?

🔑 Redis Distributed Lock (Redlock)

⚙️ How It Works (Step by Step)

⚠️ Edge Cases & Gotchas

Fencing Token Pattern

🏛️ Alternatives to Redis Locks

✅ Pros

❌ Cons

⚖️ When to Use / When NOT to Use

Circuit Breaker Pattern

❓ The Problem: Cascading Failures

🔌 The Three States

💻 Implementation Example (Conceptual)

🛠️ Production Libraries

🔗 Circuit Breaker + Fallback Strategies

✅ Pros

❌ Cons

⚖️ When to Use / When NOT to Use

DDoS Protection & Rate Limiting Abuse Cases

One-liner: A DDoS (Distributed Denial of Service) attack overwhelms your system with fake traffic until real users can't get through. Defense is layered — no single solution works alone.

❓ Types of Attacks

🧱 Defense in Depth (Layers)

🔑 Rate Limiting Abuse Cases

1. Credential Stuffing Defense

2. API Abuse Tiers

3. Slow Loris Defense

🤖 Bot Detection Signals

⚙️ AWS Shield Tiers

✅ Pros

❌ Cons

⚖️ When to Use / When NOT to Use

Authentication & Authorization — JWT & OAuth 2.0

🎫 JWT — JSON Web Token

Structure: Header.Payload.Signature

JWT Flow

Stateless = Scalable

JWT Risks & Mitigations

🔑 OAuth 2.0 — Delegated Authorization

Authorization Code Flow (Recommended)

Key OAuth Concepts

🔐 Session vs JWT Comparison

✅ Pros

❌ Cons

⚖️ When to Use / When NOT to Use

HLD: Twitter / X

1️⃣ Clarify Requirements

Functional Requirements

Non-Functional Requirements

2️⃣ Estimate Scale

3️⃣ API Design

4️⃣ The Core Problem: Home Timeline

Approach 1: Pull (Fan-out on Read)

Approach 2: Push (Fan-out on Write) — Twitter's Original Approach

Approach 3: Hybrid (Twitter's Current Approach)

5️⃣ High-Level Architecture

6️⃣ Database Design

Tweets Table (Cassandra)

Followers Table (Graph in Cassandra)

Timeline Cache (Redis)

7️⃣ Media Storage

8️⃣ Search

9️⃣ The Celebrity (Hotspot) Problem

🎨 Diagram

✅ Trade-offs Summary

HLD: Instagram Feed

1️⃣ Clarify Requirements

Functional Requirements

Non-Functional Requirements

2️⃣ Estimate Scale

3️⃣ Architecture Overview

4️⃣ Photo Upload Flow

5️⃣ Feed Generation (Hybrid Fan-out)

6️⃣ Database Schema

7️⃣ Key Challenges

Storage Tiering

Serving Photos

✅ Trade-offs Summary

From One Merged PR to a Repology Page: A Termux Maintainer Roundup