DEV Community: Joud Awad

REST API Design Like a Senior Engineer (REST, GraphQL, Networking, Pagination, Auth, Security)

Joud Awad — Mon, 02 Nov 2026 15:32:42 +0000

Most first APIs are a list of URLs, one per button. A senior engineer's API follows a few rules, so you can guess every address.

That list breaks once a second developer joins.

Here are the 5 rules that fix it:

Plural noun plus ID: GET /posts/42, not GET /getPost?id=42. The HTTP method already carries the verb.
IDs, not names: /users/7, not /users/jane_old. A link built on the ID survives a rename.
One level of nesting: /posts/42/comments lists a post's comments. Comment 99 keeps its own short home at /comments/99.
Actions become resources: a like is POST /posts/42/likes, not POST /likeIt. An unlike deletes it.
A verb only when no noun fits: POST /posts/42:archive, for actions that leave nothing to store.

You can always add servers when traffic grows. An API is harder to change, because installed apps keep calling the old shape for years.

I walk through all of this in a full REST API course on my YouTube channel, designing one photo-sharing app from zero: https://youtu.be/BvtIqaiabJA

Over to you: which URL in your current API would a new teammate never guess?

How Servers Handle Requests: Threads vs Event Loop Explained

Joud Awad — Wed, 28 Oct 2026 15:40:04 +0000

One CPU-heavy line, like hashing a password, can freeze every request on a Node server. In Go, it slows down just one. Here's why:

Both serve thousands of connections. They differ in who runs your code.

Node (event loop):

• All your JavaScript runs on one thread.

• The loop picks up a ready request, runs its code, then moves to the next one.

• A synchronous hash holds that thread. Every other request waits until it finishes.

Go (goroutines):

• Each request gets its own goroutine.

• The runtime spreads goroutines across OS threads, one per CPU core by default.

• The scheduler can preempt a long-running goroutine, so the others still get a turn.

Neither one is broken. Node works well when requests mostly wait on the network or the database. Go is more forgiving when requests also compute.

In Node, keep heavy work off the loop: use the async hash (it runs in the libuv thread pool) or a worker thread.

The video follows one login request from the moment it reaches the machine to the response, and shows how Python and Java handle the same wait.

https://youtu.be/buPqtt6Lm8Y

Over to you: What's the slowest line of code you've found blocking a whole service?

Data Lake vs Data Warehouse vs Data Lakehouse: Data Engineering Simplified

Joud Awad — Mon, 26 Oct 2026 15:39:02 +0000

I have seen a lot of engineers think about data one way: how do I store it, and how do I query it?

Data engineers ask something else: where does all this data land, and what can I build from it?

That gap gets wider as your system grows.

5 services means 5 data shapes, 5 formats, 5 update patterns. Reading and writing rows is the least interesting thing you can do with them.

Clean them, join them, and you get product features, stakeholder dashboards, and an honest view of what’s actually happening inside your system.

There are 3 places that data can live:

Data Lake
Where everything lands first. Raw, in any format: JSON events, logs, CSVs, Parquet, images.
No schema forced on write. You describe it later (schema-on-read).
On AWS that’s S3, plus the Glue Data Catalog to define tables, plus Athena to query them.
Weakness: no transactions, and updates or deletes are painful. Without governance it quietly turns into a data swamp nobody trusts.
Data Warehouse
Clean, modeled, schema-on-write. Built for heavy joins and aggregations, so it’s what your BI dashboards sit on (Redshift, Snowflake, BigQuery).
Weakness: you pay to ETL and store a second copy. It wants structured data only, and ML teams usually end up exporting data back out of it.

So most teams ran both.

A lake for raw data and ML. A warehouse for BI. Pipelines copying between them. Two copies, two bills, and one of them is always stale.

Data Lakehouse Keep the data in cheap object storage, in open formats, and put a table layer on top: Apache Iceberg, Delta Lake, or Hudi. That layer gives you ACID transactions, schema enforcement, time travel, and row-level updates. One copy of the data. BI and ML read the same tables.

It’s not a new storage system. It’s metadata on top of files you already have.

Weakness: you own more of the operations, like compaction, the small-files problem, and picking a catalog.

Rule of thumb:
→ Raw data, any shape, cheapest storage: lake
→ Fast, trusted SQL for the business: warehouse
→ Both, without duplicating data: lakehouse

I walk through all three in the full video:
https://youtu.be/dRDaQ7wZHTg

Over to you: is your team still running a lake and a warehouse side by side, or have you moved to Iceberg or Delta?

Batch vs Stream Processing: Full Course for System Design

Joud Awad — Mon, 19 Oct 2026 15:48:50 +0000

Batch or stream?

That's the wrong first question. The better one is: does your data ever end?

If you've worked anywhere near big data, you've sat through this debate. Someone wants Kafka and Flink. Someone else says a nightly Spark job is fine. But what it is all about?

Stream processing reacts to each event the moment it happens.
Batch waits for a time window or a volume threshold, then processes everything in one go.

At first glance it looks like a per-feature choice. Pick one, move on.

but it isn't.

Bounded data has an end. Yesterday's song plays, last month's orders. Because it ends, a batch job can give you a complete answer.

Unbounded data never ends. Plays are still coming in while you read this. A stream can't wait for "all the data", so the best it can give you is a fresh answer.

Fresh, complete, cheap. You rarely get all three.

So the real work is understanding your data and how events move through your system. Then for each feature you ask one thing: how fast does this event lose its value?

Take a music app:

A bot spamming plays to fake streams? Worth catching in seconds. Stream.
Paying artists for last month's plays? Nobody needs that in real time, they need it correct. Batch.
The trending chart? Honestly, both. A stream for what's hot right now, and a batch job that fixes the counts once late data lands.

Most real systems end up looking like that. A few features on each side, and a couple that need both.

I turned this into a small course: Spark and the shuffle, Kafka offsets, Flink watermarks, exactly-once, backpressure, replaying a week of bad results, and the four questions I use to route any feature to batch, stream, or both.

https://youtu.be/8FXSOrk3Bhs

Event-Driven Architecture Full Course: Beginner to Advanced

Joud Awad — Mon, 12 Oct 2026 15:01:31 +0000

Everyone draws event-driven architecture the same way. Three boxes: producer, broker, consumer.

That drawing hides almost every decision that comes back to bite you.

Start with the producer. Is it publishing an event or a command? Both end up in the same broker, so a lot of teams treat them the same.

An event is a fact that already happened. "VideoUploaded." The producer has no idea who's listening, and it shouldn't.
A command is a request. "TranscodeVideo." It has one intended handler, and that handler is allowed to say no.

Blur the two and you get a system that looks decoupled on the diagram while every service quietly depends on the others.

Then the broker. "We'll use Kafka" isn't a design decision. That middle box can be three different things:

A queue hands each message to one worker, and once it's processed it's gone. Great for spreading work out.
A log keeps events in order and keeps them around, so each consumer reads at its own pace and can replay the past.
A router checks each event against rules and pushes it to whoever subscribed.

Pick the wrong one and you end up bolting replay onto a queue, or fighting a log to do basic job distribution.

The consumer side has its own trap. A producer renames one field, and a consumer nobody remembered breaks in production.

And the one people mix up most: event-driven and async are different things.

You can throw work onto a background job queue and still have a tightly coupled system. Async is about when the work runs. Event-driven is about who needs to know about whom.

So EDA won't fix your scaling problem on its own. It hands you trade-offs, and you want to pick them on purpose.

I just published a free course that opens all three boxes, whether you are new to EDA or you have worked with it before. This video is going to change how you think and use EDA in your projects.

https://youtu.be/W25_BbZCZD0?si=VJXt5d1RZ6qjGptH

Load Balancer Explained

Joud Awad — Mon, 05 Oct 2026 15:15:27 +0000

A load balancer is more than a box you draw in your system design diagram. It's one of the most critical pieces in your whole system, and if it goes down, it takes everything behind it down with it.

The general idea is simple. It's the entry point to your servers. It sits in front of them and spreads the traffic between them. For most people that's where the understanding stops, and honestly it's enough to draw it in an interview.

But this one component carries a lot of complexity. Changing a single option can completely change how your servers behave. Switch the hashing method and adding one server can move most of your users to a different machine. Put retries in the wrong place and one struggling database can get hit with hundreds of times its normal load. Pick layer 4 when you needed layer 7 and you lose the ability to route by URL at all.

That's why understanding the different algorithms, what each network layer gives you, and how the balancer itself stays up matters so much. It's the difference between drawing a load balancer and actually knowing what it does and how to scale it with your system.

In this video I break down every concept of load balancing, starting with the simplest thing it does, distributing load, and building up to the bigger pieces like high availability, global load balancing across regions, and load balancing between services.

https://youtu.be/WmzB4MJcCiU?si=5i4Oyh7cme-BEgke

Object Storage for System Design

Joud Awad — Mon, 28 Sep 2026 15:45:39 +0000

Every growing app eventually runs into a file storage problem.

Videos, images, backups, datasets. At some point every growing app hits the same wall: files get too big and too many for a database or a single server's disk.

That's where object storage comes in.

S3, Google Cloud Storage, Azure Blob Storage: they all solve the same problem. One central place that can hold any type of file, at almost any scale.

But most engineers only use them at the surface level: upload, download, done.

In my latest video, I go deeper:

→ How object storage works internally
→ Why it scales in ways a file system can't
→ The design patterns for using it correctly in real systems

If you're working in backend development or preparing for system design interviews, this is one of those fundamentals worth understanding properly.

CDN For System Design

Joud Awad — Mon, 14 Sep 2026 16:15:23 +0000

Say "add a CDN" in a system design interview and the next four questions decide whether you pass.

One photo answers all four. Uploaded once to a server in Virginia. Opened by someone in Sydney.

Before a single line of your code runs, geography has already charged that viewer about 930 milliseconds. A bigger server fixes none of it.

The four questions, usually within thirty seconds:

What is your cache key? What does a TTL expiry cost the origin? Which edge answers, and why? How do you serve a private file from a public machine?

I followed that one photo for 44 minutes through all four. A few things I wish someone had told me earlier.

Most engineers think the cache key is the URL. It isn't. It's the method plus the URI plus whatever your config or a Vary header bolts on. CloudFront's default CachingOptimized policy ignores query strings entirely. Cloudflare's default includes the full query string, verbatim, order included. Same photo URL, opposite behavior, two vendors.

Add Vary: User-Agent and you get one cache entry per browser build. One extra header can take a hit ratio from the high nineties into single digits.

A long TTL is permission, not a guarantee.

Eviction happens whenever the edge wants the space back. And when a viral object does expire, it expires on every tier at once, so edges everywhere turn around and hit your origin in the same window. That's the thundering herd. Request collapsing and Origin Shield exist so that as few as one request per object reaches you.

Purges are priced by the path you submit, not the files it clears. CloudFront gives you 1,000 free paths a month, then $0.005 each. Which is the actual argument for versioned filenames: photo.a3f9.jpg never needs purging.

Which edge answers? GeoDNS guesses from your resolver's address, not yours. EDNS Client Subnet patches the guess. Anycast removes it: every edge announces the same IP, and failover becomes a routing event with no DNS TTL to wait out.

And the number interviewers ask for is the wrong one. Going from 90% to 95% hit ratio sounds like five points. It halves your origin load. Origin offload is the metric. Hit ratio is the proxy.

The private file? Signed URLs for a single object. Signed cookies when one viewer needs a hundred of them.

Full walkthrough, one photo end to end, built from the vendor docs and the RFCs rather than folklore:

https://www.youtube.com/watch?v=gV_z0uG8_uo

Circuit Breaker Pattern Explained

Joud Awad — Mon, 07 Sep 2026 15:25:36 +0000

Payments went fully down, and checkout was fine.

The next week payments stayed up and answered in 30 seconds instead of 40 milliseconds. The entire site went dark.

A dead dependency is a small problem. A slow one takes everything with it.

Every call still waiting is holding a thread, a database connection and a chunk of memory that the product page needed too. Nothing throws an error. The pool is just gone.

That's the cascading failure a circuit breaker exists to stop — and it's why the breaker is never installed on its own. A request meets a timeout, then a retry policy, then the breaker, then a fallback. In that order.

Most of us built that chain from a tutorial that said "open after 5 consecutive failures."

Go read your actual config.

Resilience4j, Hystrix, Polly and gobreaker don't count consecutive failures. They trip on a failure rate across a sliding window, gated by a minimum call volume. Which is better — until you notice that on a low-traffic endpoint, two of those defaults combine to leave the breaker unable to open at all. Not "slow to open." Unable.

Three more things I had wrong for years:

Retries are selfish. Exponential backoff without jitter doesn't spread the load, it just reschedules the same spike for later.

You can have full throughput and zero goodput at the same time, and every dashboard will look busy while nothing succeeds.

And AWS's own Builders' Library warns that circuit breakers add modal behavior and recovery time — its SDK ships a retry token bucket instead. Amazon has a written case against fallback code, too. The pattern everyone teaches as the answer is the one Amazon argues you should think twice about.

So I built the version that includes the numbers and the counterarguments: 36 minutes, one call from checkout to payments, all the way through.

If you take one thing: an open breaker with no alarm on its state changes is a silent partial outage.

https://youtu.be/rvKKj7JU92g

Database Internals For System Design

Joud Awad — Mon, 31 Aug 2026 15:56:17 +0000

They added an index and the query got slower.

Then they doubled the instance size and bought back 400 milliseconds.

So they put Redis in front of it. The hit rate came back at 4 percent. Then they added a read replica, and it fell 45 seconds behind.

Four fixes. All reasonable. All wrong.

None of it was bad luck. Every one of those numbers was predictable from what the engine was doing underneath the query.

The index lost because the planner's row estimate was off — and an index scan is two hops, find the entry, then go fetch the row. Past enough rows, reading the pages straight through just wins.

The 4 percent wasn't a Redis problem. It was the diagnosis. A cache is a second buffer pool. If the working set doesn't fit in the first one, it isn't going to fit in the second one either.

And the replica lag came from the same place. Replication is the write-ahead log, shipped. An UPDATE in Postgres never overwrites a row — it writes a new version. More versions, more WAL, more lag.

Same layer, three times. Storage.

Most of us learned databases top-down: write the query, add an index when it's slow, add a replica when the index stops helping. That's backwards. Indexes make no sense without pages. Joins make no sense without indexes. And nobody explains the WAL until the replica is already 45 seconds behind.

So I built the bottom-up version I wish I'd had.

2 hours 22 minutes. Seven layers — storage, indexes, pagination, execution, transactions, read scaling, schema. One query, one 200 million row table, all the way up. Seven checkpoint quizzes with real countdowns, so you answer before you hear the answer.

If you only watch part of it, watch the bottom four.

https://youtu.be/uy2yom0AlhA

DynamoDB Client API Tutorial: Query, Pagination, Cost Explained

Joud Awad — Mon, 24 Aug 2026 09:24:02 +0000

There is no SQL in DynamoDB.

No query language, no syntax to memorize, no planner deciding how your request gets run. You import a client and you call a method.

In Postgres, "give me one row" and "give me a thousand rows" are the same SELECT with different clauses, and the engine works out the rest. In DynamoDB those are different methods and you're the one choosing. GetItem for a single item by its key. BatchGetItem for up to 100 of them. Query for a run of items under one partition key. Scan for when you've given up and want the whole table.

The method name is the operation. That's the shift people miss coming from SQL: you're not composing a statement, you're picking an action. One item or many, atomic or not, read or write. Three decisions, and the entire client API falls out of them.

What you do have to learn is the cost, because it isn't hidden behind a planner any more. 4 KB read blocks, 1 KB write blocks, and a round-up rule that makes a 200-byte item cost a full block. A strongly consistent read costs double an eventual one. A read inside a transaction doubles again. Once you know which method you called, you can price the call before you run it.

So I ran every one of them on camera, from real NestJS code, with the consumed capacity on screen next to each call.

https://youtu.be/JeyZ-kN0H7Y?si=ZkvqVIC1TCjCX06k

DynamoDB Table Design: Complete Data Modeling Guide

Joud Awad — Thu, 20 Aug 2026 16:16:51 +0000

DynamoDB Data modeling is one of the hardest things that appears on paper. Still, once you understand the different access patterns and how DynamoDB queries your data, it starts being an easy task.

DynamoDB uses a unique combination of Partition Key (PK) and sort keys (SK) in order to store and query your data, and understanding how those two works under the hood is the first part of working with DynamoDB, the hard part and the most challenging is to understand how to model your data, how to model your access pattern, and how you can extract questions from your own system and convert them into a working DynamoDB Tables.

In this video, we walk you through a full DynamoDB end-to-end table design, covering all the access patterns DynamoDB uses to build three real-world case examples.

At the end of this video, you will be able to:

Understand how DynamoDB Data Modeling works.
Build your own DynamoDB Tables.
Convert your own use case into a question DynamoDB is able to answer for you.
Model any real-world use case that you may face in your own application.
Master the Single-Table Design in DynamoDB
Work and interact with all the differnet access patterns DynamoDB provide to you.

Video Link: https://www.youtube.com/watch?v=rbP1jGNzG2A