DEV Community: Joud Awad

56/60 Days System Design Questions

Joud Awad — Wed, 01 Jul 2026 16:29:39 +0000

Your background job ran for 4 minutes and nobody knows if it finished.

That's not a job queue problem. That's a missing design problem.

Long-running jobs break every assumption you built for synchronous APIs. Your load balancer times out after 30s. Your mobile client doesn't know whether to retry. Your retry logic re-runs a job that already half-completed.

Here's the real scenario:

You're processing a video upload. The job takes 2–8 minutes. Millions of users.

What do you expose to the client?

A) Polling endpoint — client hits /jobs/:id/status every 5s until done
B) Webhook — job fires a POST to client's callback URL on completion
C) SSE / WebSocket — server pushes progress updates in real time
D) Synchronous wait — keep the HTTP connection open until the job finishes

One scales to millions without coupling your infrastructure to client uptime.

The others have hard production failure modes most teams don't discover until 3 AM.

The deeper problem isn't transport — it's these 4 things nobody gets right the first time:

→ Idempotency. Every job must be safe to re-run. If your retry logic can double-charge, double-send, or double-process — you don't have retries, you have bugs waiting.

→ Progress granularity. "0% → 100%" is useless for a 6-minute job. You need intermediate states: queued, processing, transcoding, uploading, complete. Clients need something to show users.

→ Timeout vs failure. A job that stops responding isn't the same as a job that failed. Dead workers, OOM kills, spot instance evictions — your queue needs a heartbeat or a visibility timeout, not just a try/catch.

→ Deduplication. The client will retry. Your queue will redeliver. You need a dedup key scoped to the original request — not the job run.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

30DaysOfSystemDesign #SystemDesign #BackendEngineering #DistributedSystems

55/60 Days System Design Questions

Joud Awad — Tue, 30 Jun 2026 16:46:23 +0000

You built an agent. It works.

Now you need 5 of them running in parallel, sharing state, and handing off work to each other.

Your pipeline breaks on the first real workload.

Here's the setup:
You're building a research agent system. A user asks a complex question. You need to:

• Fan out to 3 specialized sub-agents simultaneously
• One agent might spawn 2 more based on what it finds
• They all write back to shared context
• A final agent synthesizes everything

Classic multi-agent orchestration. You have 4 options for how agents coordinate.

A) Centralized Orchestrator — one controller agent dispatches tasks, collects results, manages shared state. Agents are dumb workers.

B) Decentralized Peer Handoff — each agent decides who gets the task next. No central controller. Agents communicate directly.

C) Shared Message Queue + Blackboard — all agents read/write to a shared blackboard. Coordination happens through state, not calls.

D) Hierarchical Nesting — orchestrator spawns sub-orchestrators. Each sub-tree is self-coordinating. Recursive decomposition of the problem.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

Drop your answer 👇

54/60 Days System Design Questions

Joud Awad — Mon, 29 Jun 2026 16:20:56 +0000

You built a RAG pipeline. Works great in dev.

6 months later, your users complain: "The search results are garbage."

You haven't changed a line of code.

Here's what happened:

Your product evolved. New features, new docs, new support tickets. The data drifted — but your embedding index didn't.

Now you're serving a 400GB FAISS index that was last rebuilt in January. Your chunks are stale. Your nearest-neighbor results point to deprecated docs. Your LLM is confidently hallucinating from outdated context.

You need to fix this. 4 engineers each propose a solution:

A) Scheduled full rebuild
Every Sunday, re-embed the entire corpus from scratch. Replace the index atomically. Slow (4h+ at scale), expensive, but always fresh.

B) Incremental upserts + soft delete
On every document change, re-embed only the affected chunks. Mark deleted chunks as tombstoned. Keep a version field on each vector. Index size grows over time; compact quarterly.

C) Embedding version registry + hot swap
Track which embedding model version produced each vector. When the model drifts (fine-tuned or upgraded), invalidate the mismatched vectors and rebuild only those. Two indexes run in parallel during migration. Route traffic by model version.

D) Approximate staleness detection
Run a nightly job that samples 1% of your corpus, re-embeds it, and measures cosine distance against the stored vector. If drift exceeds a threshold, trigger a full rebuild. Otherwise, skip it. Cheap monitoring, reactive rebuilds.

Real constraint: your corpus is 50M chunks. Full rebuild = 4 hours + ~$800 in embedding API cost. You deploy model updates every 6 weeks.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

30DaysOfSystemDesign #SystemDesign #MachineLearning #MLEngineering

53/60 Days System Design Questions

Joud Awad — Sun, 28 Jun 2026 16:20:06 +0000

Your migration ran fine in staging.

Then you ran it in production.

The app went down.

Not because the SQL was wrong. Because you ran it on a live table with 40 million rows while 8 services were actively writing to it.

Your setup:
→ PostgreSQL. users table. 40M rows. Active writes from 8 services.
→ Product request: split full_name into first_name + last_name.
→ You have a 2-hour maintenance window tonight.

The engineering question: how do you ship this without downtime?

A) Run ALTER TABLE to drop full_name and add first_name + last_name in a single migration during the maintenance window.

B) Add first_name + last_name as nullable columns first → backfill → update all services to write to both → drop full_name only after everything is migrated.

C) Create a new users_v2 table with the target schema → dual-write to both tables → flip the read pointer → drain the old table.

D) Add a DB view that aliases full_name as first_name || ' ' || last_name → let each service migrate off it at its own pace.

Drop your answer 👇

52/60 Days System Design Questions

Joud Awad — Sat, 27 Jun 2026 16:12:50 +0000

Your API just shipped a breaking change.

/users now returns fullName instead of first_name + last_name. 3 mobile clients broke. 1 partner integration went down. Your on-call is not happy.

You had a versioning strategy. It just wasn't the right one.

There are 4 ways to version an API. Here's what actually happens when you pick each one in production:

A — URL path versioning (/v1/users, /v2/users)
Simple. Explicit. Every request makes the version visible in logs and caches. But now you're maintaining 2 full route trees. A bugfix in the business logic layer has to be patched in both. Teams quietly let v1 rot.

B — Header versioning (API-Version: 2)
Clean URLs. Version negotiation in the transport layer, not the path. Harder to test in a browser, invisible in logs unless you instrument for it, and clients forget to send the header — defaulting to whatever your server decides "latest" means.

C — Query param versioning (/users?version=2)
Fast to implement. Zero client SDK changes. Cache-unfriendly — every CDN layer treats ?version=1 and ?version=2 as separate cache keys. Works until you have 40 endpoints and version drift becomes untrackable.

D — Content negotiation (Accept: application/vnd.api.v2+json)
The REST-purist approach. Semantically correct — you're asking for a representation, not a route. Almost nobody implements it right. Client library support is inconsistent. One wrong Accept header and you get a 406.

Which strategy does your team use?

51/60 Days System Design Questions

Joud Awad — Fri, 26 Jun 2026 16:37:54 +0000

You're building a B2B SaaS product. 50 enterprise customers. Each one wants their data isolated. Some are on free plans. A few are paying $50k/year and demanding SLA guarantees.

Your current setup:
→ One database. One schema. A tenant_id column on every table.
→ One app server handling all traffic.
→ A free-tier customer running a badly-written bulk export just hammered your DB for 40 seconds. A paying enterprise customer's checkout flow timed out.

Your investors are not happy. Neither is that enterprise customer.

The engineering question: how do you isolate tenants without rebuilding the whole product?

A) Keep one shared DB — add row-level security + query budgets per tenant to enforce limits.

B) Schema-per-tenant — every customer gets their own schema in the same Postgres instance, migrations run per-schema.

C) Database-per-tenant (silo model) — each enterprise customer gets a dedicated DB. Free tier stays pooled.

D) Middleware bridge — route requests to tenant-specific DB clusters based on a tenant registry, free tier stays on shared pool.

One of these is a band-aid. One will collapse under 500 tenants. One is how Notion, Salesforce, and every serious B2B at scale actually operates.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments (including the noisy neighbor failure mode nobody warns you about).

If your team is building multi-tenant systems, share this. The wrong isolation model is a rewrite waiting to happen.

Drop your answer 👇

30DaysOfSystemDesign #SystemDesign #BackendEngineering #SoftwareArchitecture

50/60 Days System Design Questions!

Joud Awad — Thu, 25 Jun 2026 15:49:24 +0000

Your ML team trained a model that hits 94% accuracy in the notebook.

Then you deploy it.

Peak traffic: 3,000 inference requests/second.
P99 latency shoots to 4.2 seconds.
GPU utilization: 23%.

The model works. The serving layer is the bottleneck.

Here's the setup:

• PyTorch model, ~6B params
• Single A100 GPU, 80GB VRAM
• FastAPI wrapper calling model.predict() one request at a time
• No batching, FP32 weights, no quantization

Users are hitting timeouts. GPUs are sitting mostly idle. Your infra bill is climbing.

What do you fix first?

A) Enable dynamic batching — buffer incoming requests, group into batches, process in one forward pass.

B) Quantize the model to INT8 — reduce weight precision from 32-bit to 8-bit, shrink memory footprint and speed up inference.

C) Switch to tensor parallelism — split the model across multiple GPUs, distribute the compute.

D) Add a request queue + async workers — decouple HTTP receiving from model inference, process jobs in background.

All four are real production patterns. Only one directly fixes the combination of low GPU utilization + high latency at 3K RPS.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

49/60 Days System Design Questions

Joud Awad — Wed, 24 Jun 2026 19:43:26 +0000

Your data team just opened a $4,200 BigQuery bill.

For a single month. One analyst. 12 queries.

The queries weren't wrong. They weren't inefficient SQL. They were reasonable analytics queries — "give me last 30 days of events for customer X." The problem was that every single one scanned the full 3.2 TB table. No partition pruning. No cost control. Just full scans, every time.

This is the most expensive silent bug in data engineering. You write a query. It looks fast. It returns results. And every run quietly eats through terabytes you're paying per-byte to scan.

The fix is partition strategy — but picking the wrong one doesn't just fail to help, it actively makes things worse.

Here's the setup:

You're running a 3.2 TB events table on BigQuery. 18 months of data. Ingested daily. Analyst queries almost always filter on two things: a date range ("last 30 days") and a customer_id ("for customer X").

Which partition + clustering strategy do you pick?

A) Partition by ingestion date — date range queries only scan the relevant day-partitions.

B) Partition by customer_id ranges — customer queries only scan the relevant ID bucket.

C) No partitioning — use Redshift DISTKEY on customer_id + SORTKEY on event_time.

D) Partition by ingestion date + cluster on customer_id — date pruning cuts the partition, clustering prunes within it.

Three of these reduce your bill. Only one fully prunes on both filters your analysts actually use.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments

Drop your answer

30DaysOfSystemDesign #SystemDesign #DataEngineering #BigQuery

48/60 Days System Design Questions

Joud Awad — Tue, 23 Jun 2026 17:14:44 +0000

Your AI agent just got a user message: “Book me a flight to Dubai next Friday.”

The LLM has access to 12 tools: search_flights, get_user_preferences, check_calendar, book_flight, send_confirmation, get_weather…

How does the agent decide which tools to call, in what order, and when to stop?

A) ReAct loop — model reasons step-by-step, emits a “thought” then picks one tool at a time, observes output, repeats until it self-decides it’s done

B) Parallel tool calling — model emits ALL required tool calls in a single response, executes them concurrently, feeds all results back in one context update

C) Forced function schema — you lock the model into a strict JSON schema per turn; it can’t produce free text, only structured tool calls you defined

D) Planner-executor split — a lightweight planner LLM creates a tool call DAG upfront, a separate executor runs the graph, results flow back to planner only at checkpoints

Pick one — A, B, C, or D — and tell me why you’d use it in production.

Full breakdown in the comments. 👇

47/60 Days System Design Questions

Joud Awad — Mon, 22 Jun 2026 16:44:53 +0000

Your monolith is 6 years old.

500k lines of code. One deploy every 3 weeks. One bad migration takes down the whole thing.

Leadership says: "rewrite it." You've seen what happens when teams do that. 18 months. Budget overruns. Half the features never make it back. The new system launches and nobody trusts it.

There's a better way. You don't rewrite — you strangle.

Here's your system:

• OrderService (monolith) handles: order creation, fulfillment, invoicing, returns
• 40 engineers. 3 teams. All deploying to the same codebase.
• Returns processing is a bottleneck. The business wants it extracted first.
• You can't freeze feature work during migration.

You need to extract Returns into a standalone service without a big-bang rewrite.

What's your migration strategy?

A) Strangler Fig — route /returns traffic to a new service via a proxy/facade, keep the monolith intact until the new service is proven. Deprecate monolith code after.

B) Branch by Abstraction — introduce an interface inside the monolith, build the new implementation behind it, flip the switch when ready, then move it out.

C) Big Bang Rewrite — freeze the monolith, rebuild everything in microservices in parallel, cut over when done.

D) Database-First Migration — extract the returns database tables into a separate schema first, build the service around it, then reroute traffic.

One of these lets you migrate live, with zero freeze periods, and with a clear rollback at every step.

Pick one — A, B, C, or D — and tell me why. Full breakdown in the comments.

Drop your answer 👇

Queues vs Streams vs Event Bus: The Mental Model That Makes System Design Click

Joud Awad — Mon, 22 Jun 2026 10:51:27 +0000

I struggled with system design and event-driven architecture for years.

Then I learned that queues, streams, and event buses are three completely different things, even though they look identical on a whiteboard.

You draw the same boxes every time:

Producers
a broker
And something subscribing on the other end.

That picture is what kept tripping me up, because it hides the only part that actually matters: what's going on inside the broker, and the way each one pushes your system to scale in a different direction.

Here's the mental model that finally got it to stick.

A queue is a to-do list. A message lands in line, one worker picks it up, finishes it, and it's gone. Consumed once, then deleted. Want to go faster? Add more workers pulling from the same line, and that's pretty much the whole trick. SQS, RabbitMQ, Celery all work this way: one job, one consumer, done a single time. You reach for it when the work genuinely can't run twice, like charging a card or firing off a welcome email.
A stream is a logbook you can rewind. Reading a message doesn't delete it. Each consumer just keeps a little bookmark, an offset, marking where it left off, so two teams can read the same data at once at completely different speeds. One of them can jump back to last Tuesday while the other stays on the live edge.
An event bus is a switchboard. An event shows up, and the bus works out who should hear about it from rules you write, instead of you wiring every connection by hand. "order.paid" fires once, and shipping, analytics, and the fraud check all react to it without any of them knowing the others exist.

So same picture, three problems that barely overlap.

I went through all three properly in my latest video, including how each one works under the hood and how it changes the way your system scales.

46/60 Days system Design Questions

Joud Awad — Sun, 21 Jun 2026 17:06:55 +0000

our production LLM agent just returned this JSON to your order processing service:

{ "action": "refund", "amount": "fifty dollars", "order_id": null, "confidence": "pretty high" }

Your downstream service crashes. The retry hits the same model. Same broken output. The refund never fires — but the user got a confirmation email.

You need your agent to return valid, typed, structured output — every time. What do you do?

A) Prompt-engineer harder — add "Always return valid JSON with these exact fields" to your system prompt and document the schema inline.

B) Use structured outputs / function calling (OpenAI, Bedrock tool use, Gemini response schema) — constrain the model at the API level to return a typed schema.

C) Post-process with a validation layer — parse the output, run JSON Schema or Pydantic validation, retry with corrective context if it fails (max 2 retries).

D) Add a second LLM as a judge — pass the first model's output to a smaller, faster model that scores and flags invalid responses before they reach your service.

Three of these are patterns used in production AI systems. One of them is wishful thinking.

Pick one — A, B, C, or D — and tell me why. I'll drop the full breakdown in the comments, including the pattern that looks defensive but actually makes hallucinations worse under load.

Drop your answer

30DaysOfSystemDesign #SystemDesign #AIEngineering #AgenticsAI