DEV Community: Joud Awad

Day 36/60 System Design Questions

Joud Awad — Thu, 11 Jun 2026 17:46:09 +0000

Your API response went from 320ms to 95ms after a CDN switch.

Nothing changed on the server. Same origin. Same payload.

The only difference: the CDN started speaking HTTP/3 to your clients.

Here's the setup:
Mobile app → Load Balancer → Origin (NestJS) → DB

Every request goes through the CDN first. Latency is acceptable, until high packet-loss environments (mobile 4G to 5G transitions, flaky WiFi, Asia-Pacific routes). You're seeing 800ms+ tail latency for the p99. The CDN supports HTTP/2 and HTTP/3. What do you enable?

A) HTTP/2 only — multiplexing over a single TCP connection eliminates the old HTTP/1.1 head-of-line problem. Proven, widely supported.

B) HTTP/3 only. Built on QUIC (UDP), eliminates TCP head-of-line blocking entirely, 0-RTT connection resumption. Modern clients handle it fine.

C) HTTP/2 to origin, HTTP/2 to clients. Maximize multiplexing end-to-end, avoid QUIC instability in enterprise firewalls.

D) HTTP/3 to clients, HTTP/2 to origin. QUIC handles the lossy last mile, HTTP/2 handles the stable datacenter leg.

One of these is how Netflix, Cloudflare, and most major CDNs actually handle this in production.

Pick one (A, B, C, or D) and tell me why. Full breakdown in the comments.

If your team is debating CDN config or protocol upgrades, share this. The tradeoff is sharper than most people think.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #HTTP3 #WebPerformance

Day 35/60 System Design Questions

Joud Awad — Wed, 10 Jun 2026 16:23:56 +0000

This article was written with the assistance of AI tooling for structure and syntax. The concepts, tradeoffs, and production context are based on my own engineering experience and research. #ABotWroteThis

You're building the "find nearby drivers" feature for a ride-hailing app.

At peak, you have 500,000 active drivers updating their GPS location every 5 seconds. Riders query for drivers within 2km. At scale, you're doing ~100,000 proximity queries per second.

Your naive implementation does this:

SELECT * FROM drivers
WHERE lat BETWEEN ? AND ?
AND lng BETWEEN ? AND ?

It works fine at 1,000 drivers. At 500,000, it's a full table scan on every query. Latency hits 800ms. Riders see a spinner. Drivers miss trips.

Your team proposes 4 approaches to fix this:

A) Geohash partitioning — Encode each driver's location into a geohash string. Index by geohash prefix. Proximity queries become a string lookup on the index.

B) PostGIS with spatial indexes — Add a PostGIS extension to Postgres. Use a proper R-tree/GiST spatial index for bounding-box and radius queries.

C) Quadtree in memory — Keep all active driver positions in a quadtree data structure in a Redis-backed in-memory service. Decompose space recursively until each cell has ≤ N drivers.

D) H3 hexagonal grid (Uber's system) — Divide the earth into hexagonal cells at multiple resolutions. Assign each driver to a cell. Queries check the target cell + 6 neighbors at the right resolution.

You need sub-50ms p99 latency, real-time updates, and it has to stay accurate at cell boundaries.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

If your team has argued about spatial indexing before, share this. Worth the debate.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #SoftwareArchitecture #DistributedSystems

34/60 Days System Design Questions

Joud Awad — Tue, 09 Jun 2026 16:30:01 +0000

Your AI feature works in the demo.

It fails in production 3 weeks later. Nobody touched the model. Nobody changed the code.

The only thing that changed: the inputs got messier.

Here's the setup:

You're at a SaaS company. 50,000 support tickets a week. Your team builds an AI triage system — GPT-4o classifies each ticket into 6 categories (billing, bug, feature request, account access, security, other) so the right team gets it instantly.

In dev, it nails 71% accuracy. You need 90%+ to cut manual review.

The model is locked. The budget for inference isn't unlimited. You need to close the 19-point gap.

Here are your four options:

A) Zero-shot with a better system prompt — rewrite the instructions, add explicit category definitions, specify edge case rules. No examples.

B) Few-shot examples — add 3–5 real classified tickets directly in the prompt. One example per category edge case.

C) Chain-of-Thought — add "think step-by-step before answering" to the prompt. Force the model to reason through the ticket before outputting the category.

D) Self-Consistency — run each ticket through the model 5 times with temperature=0.7, take the majority vote across outputs.

Same model. Same ticket. Four different accuracy + cost profiles.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

30DaysOfSystemDesign #AI #SystemDesign #BackendEngineering

33/60 Days System Design Questions

Joud Awad — Mon, 08 Jun 2026 15:55:29 +0000

Your order service takes 200 writes/sec at peak.
You audit 6 months of data. Something's off — two orders show the same ID, different totals.

You have the current state. You don't have how it got there.

Your DB is a graveyard of overwritten rows.

Here's the system:

• OrderService → Postgres (current state only)
• Events: placed, updated, cancelled, refunded
• Every UPDATE overwrites the previous row
• No audit log. No event history. No replay.

A billing dispute just landed. You need to reconstruct exactly what happened to Order #8471. You can't.

That's the problem Event Sourcing solves.

Instead of storing the current state, you store the sequence of events that produced it.

What's your approach when redesigning this service?

A) Event Sourcing — append-only event log as the source of truth, current state derived from replaying events.
B) Change Data Capture (CDC) — keep Postgres as-is, but stream all row changes to Kafka for an audit trail.
C) Add an audit_log table — trigger-based shadow writes on every INSERT/UPDATE/DELETE.
D) Dual-write — write to both the current-state table and a separate events table on every operation.

One of these gives you full replay, projection flexibility, and a real source of truth. The others are patches.

Pick one — A, B, C, or D — and tell me why. I'll drop the full breakdown in the comments.

If your team is arguing about audit trails or event-driven redesigns, tag someone who needs to see this.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #EventSourcing #SoftwareArchitecture

32/60 Days System Design Questions!

Joud Awad — Sun, 07 Jun 2026 18:22:35 +0000

Your startup just got its first SOC 2 audit.

The auditor asks: "Where are your database passwords, API keys, and service tokens stored?"

Your senior engineer goes quiet.

Turns out half of them are in .env files committed to git 18 months ago. Three are hardcoded in Lambda environment variables. One is in a Slack message from 2023.

You have 6 services in production, 4 environments, and zero rotation policy.

Here's the setup:

• NestJS API → Postgres (password in env var)
• NestJS API → Stripe (API key in env var)
• Background workers → SQS, S3 (AWS credentials in env var)
• 3rd-party webhooks → HMAC secrets in env var
• Zero rotation. Zero audit trail. Zero centralized access control.

You need to fix this. And you can't take downtime.

A) Move everything to AWS Secrets Manager — SDK calls at runtime, IAM controls access, auto-rotation built in.

B) Use HashiCorp Vault — dynamic secrets, fine-grained policies, works across any cloud or on-prem.

C) Use environment variables injected at deploy time via CI/CD — secrets stored in GitHub Actions / GitLab CI secrets vault, never touch disk.

D) Encrypt secrets with KMS and store ciphertext in your own database — decrypt at runtime, full control.

All four are used in production at real companies.

Pick one — A, B, C, or D — and tell me why. I'll drop the full breakdown in the comments.

If your team is having this argument right now, share this post. Someone needs to see it.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #BackendEngineering #CloudArchitecture

31/60 Days System Design Questions!

Joud Awad — Sat, 06 Jun 2026 17:40:39 +0000

Your e-commerce platform just crossed 2M daily active users. 70% are in the US, 30% in Europe.

Latency complaints are piling up from European users — 380ms average round-trip to your US-East region. Support tickets are up 40%. Black Friday is in 6 weeks.

Your infrastructure: single AWS us-east-1 region, RDS PostgreSQL (primary), Redis cache, 12 microservices behind an API Gateway.

You need to get European latency under 80ms. The engineering team is debating four approaches.

Here's your constraint: you cannot afford a full database rewrite, and you need this shipped before Black Friday.

A) Active-Active multi-region — deploy the full stack in eu-west-1, use a distributed database (CockroachDB or Aurora Global), route users to the nearest region. Writes go to both regions simultaneously.

B) Active-Passive with read replicas — keep us-east-1 as primary, spin up eu-west-1 as a hot standby with read replicas. European reads go local, writes still go to US. Failover in minutes if US goes down.

C) CDN + Edge caching — keep the single region, push static assets and cacheable API responses to CloudFront edge nodes in Europe. No database changes.

D) Active-Active with eventual consistency — deploy full stack in both regions, allow each region to own its writes, sync asynchronously. Accept that a European user might see a US write 200ms late.

Three of these are real patterns production teams use. Only one actually solves the problem you have — under your constraints, before Black Friday.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

If your team has been in this exact debate, share this with them. The tradeoffs are what matter.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #DistributedSystems #SoftwareArchitecture

30/60 Days System Design Questions!

Joud Awad — Fri, 05 Jun 2026 16:01:09 +0000

You're building a file upload service. 10TB of user files today. 100TB in 12 months.

Your team is having a fight.

The backend lead says: "Just use S3. Done."
The DevOps engineer says: "Mount an EBS volume. Simpler, faster."
The platform architect says: "We need EFS — multiple services need to read the same files."
The startup CTO says: "We can't afford cloud storage at scale. Self-host with MinIO."

All four have shipped this in production. All four have opinions backed by scars.

Here's the setup:
— Upload service (NestJS) receives files from mobile + web clients
— ML pipeline needs to read uploaded images for processing
— Audit service needs read access to the same files
— Files range from 5KB profile pics to 2GB video exports
— You're on AWS

What do you pick?

A) S3 — object storage, infinite scale, pay per GB, no servers to manage.
B) EBS — block storage, SSD-backed, attach to your EC2, low latency reads.
C) EFS — managed NFS, shared across multiple EC2 instances simultaneously.
D) MinIO on EC2 — S3-compatible self-hosted object storage, you own the infra.

One of these is the obvious right answer for this scenario.
One of them will destroy your architecture quietly over 6 months.
One is great — but not for this problem.
One is a trap that looks like cost savings.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

If your team has had this exact fight, share this. Someone needs to win it with data, not opinions.

Drop your answer 👇
You're building a file upload service. 10TB of user files today. 100TB in 12 months.

Your team is having a fight.

All four have shipped this in production. All four have opinions backed by scars.

What do you pick?

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

If your team has had this exact fight, share this. Someone needs to win it with data, not opinions.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #AWS #CloudArchitecture

29/60 Days System Design Questions

Joud Awad — Thu, 04 Jun 2026 17:48:30 +0000

You have an AI product with 4 specialized agents: a Planner, a Researcher, a Coder, and a Reviewer.

The Planner breaks down the task. The Researcher pulls context. The Coder implements. The Reviewer catches bugs.

Simple on paper. In production, it's falling apart.

Here's what's happening:

• The Researcher sometimes returns before the Planner finishes → Coder gets incomplete context
• The Reviewer flags issues → but there's no retry loop, so bugs ship anyway
• One agent timeout hangs the entire pipeline for 40 seconds
• You have no visibility into which agent failed or why

You need to redesign the orchestration layer. What do you do?

A) Centralized orchestrator — one controller calls each agent in sequence, owns retry logic, tracks state in a DB, times out per step individually.
B) Choreography via event bus — agents publish/subscribe to events, no central controller, each agent triggers the next autonomously.
C) DAG-based execution — model the pipeline as a directed acyclic graph, parallelize independent steps, block only on real dependencies.
D) Supervisor pattern — a meta-agent monitors all others, detects failures, decides whether to retry, reroute, or escalate to a human.

All four exist in production AI systems. Only one handles your specific failure modes without introducing new ones.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

If you're building agentic systems, share this. Most teams hit these exact problems at month 2.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #AgenticAI #AIEngineering

28/30 Days System Design Questions!

Joud Awad — Wed, 03 Jun 2026 16:52:11 +0000

You're building a semantic search feature for a B2B SaaS product.

The corpus: 4 million support articles, docs, and user-generated tickets. Users type natural language queries. They expect Google-quality results — not keyword matching.

Your current stack: PostgreSQL 15, Redis, and a Node.js backend. The search team says ILIKE and pg_trgm aren't cutting it. Embeddings are the answer. Now you need a place to store and query 1536-dimensional vectors (OpenAI ada-002) at <100ms p99.

4 million rows. ~24GB of raw embeddings. Query volume: 300 req/s with weekend spikes to 900 req/s.

Where do you store and query those vectors?

A) pgvector extension on your existing PostgreSQL — store embeddings in a new column, query with <-> cosine similarity.
B) Pinecone — fully managed vector database, serverless tier, no infra to run.
C) Weaviate — open-source vector DB, self-hosted on Kubernetes, full control over indexing.
D) Qdrant — open-source vector DB, Rust-based, self-hosted or cloud, optimized for high-throughput filtering.

All four are used in production at scale. But only one fits this scenario without hidden costs that bite you at 300 req/s.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

If this is the debate your team is about to have, share it. These decisions are hard to reverse.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #VectorDatabases #MachineLearning

27/30 Days System Design Questions!

Joud Awad — Tue, 02 Jun 2026 17:16:57 +0000

Your LLM answers are wrong. Not hallucination-wrong — outdated-wrong.

You shipped a customer support bot on GPT-4. It's trained through early 2024. Your product changed 14 times since then. Every week, users get answers that were accurate 8 months ago and are flat-out wrong today.

The team is debating the fix.

Here's the setup:
NestJS API → OpenAI GPT-4 + PostgreSQL (product knowledge base)
~2,000 support queries/day, 15% return wrong answers tied to stale knowledge
Knowledge base updates weekly — new pricing, new features, deprecated flows
Budget: mid-size startup, not training custom models from scratch
You need accurate, up-to-date answers without re-training on every product update

What do you do?

A) RAG — embed your knowledge base, retrieve relevant chunks at query time, inject into context. Model stays the same, knowledge is always fresh.
B) Fine-tune the base model — train GPT-4 (or open-source equivalent) on your product docs. The model internalizes your domain.
C) Fine-tune + RAG hybrid — fine-tune for style/tone/domain fluency, RAG for factual grounding. Best of both worlds.
D) Prompt engineering only — detailed system prompt + few-shot examples. No infra, no training, just better instructions.

All four are in production somewhere. Only one actually solves the problem in front of you.

Pick one — A, B, C, or D — and tell me why. I'll drop the full breakdown in the comments (including the one that feels like the obvious upgrade but makes your freshness problem worse, not better).

If your team is arguing RAG vs fine-tuning right now, share this. The tradeoff is worth mapping before you commit.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #RAG #MachineLearning

26/30 Days System Design Questions!

Joud Awad — Mon, 01 Jun 2026 17:28:07 +0000

Your cache and DB are out of sync. Again.

A user updates their profile. The cache still serves the old name for the next 10 minutes. Support gets a ticket. You patch it with a cache flush. It happens again next week.

You're asked to fix write consistency before it becomes a customer-facing incident.

Here's the setup:
NestJS API → PostgreSQL (source of truth) + Redis (cache)
~600 req/s reads, ~80 req/s writes at peak
Current pattern: write to DB, manually invalidate cache key on success
3 incidents this month — all traced back to stale cache after writes
You need a strategy that survives race conditions, retries, and partial failures

What do you change?

A) Write-through — write to cache and DB together, synchronously. Cache is always warm, always consistent.
B) Write-behind — write to cache first, async flush to DB. Fast writes, eventual persistence.
C) Write-around — skip the cache on writes entirely. Write to DB only. Cache fills on next read miss.
D) Dual-write with an outbox — write to DB + publish an event. A consumer updates the cache from the event log.

All four are used in production. Only one actually survives the failure modes in this setup.

Pick one — A, B, C, or D — and tell me why. I'll drop the full breakdown in the comments (including the one that looks safest but will burn you at scale).

If your team argues about this at design review, share it with them. The debate is worth having before an incident forces it.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #Caching #SoftwareArchitecture

25/30 Days System Design Questions!

Joud Awad — Sun, 31 May 2026 17:50:02 +0000

Your order processing service runs on SQS.

Normal load: 200 orders/min. Consumers keep up fine.

Then Black Friday hits. Producers start pushing 4,000 orders/min. Queue depth climbs to 80,000 messages in 20 minutes. Your downstream DB is at 95% CPU. Consumers are falling behind and you're watching the queue grow in real time.

You need to handle this backpressure. What do you do?

A) Scale consumers horizontally — add more Lambda functions / EC2 workers to chew through the backlog faster.

B) Set a visibility timeout and route failures to a dead-letter queue to protect against poison pills.

C) Rate-limit producers at the source — use a token bucket or sliding window to cap how fast messages enter the queue.

D) Switch to SQS delay queues — defer message visibility to spread out delivery and reduce consumer pressure.

Three of these are real patterns engineers reach for. Only one actually solves backpressure.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

If this made you second-guess your instinct, share it — someone on your team is designing this right now.

30DaysOfSystemDesign #SystemDesign #AWS #SoftwareArchitecture