DEV Community: interviewgpt

20 Tricky SQL Interview Questions That Separate Good Candidates from Great Ones (2026)

interviewgpt — Sun, 17 May 2026 16:29:48 +0000

20 Tricky SQL Interview Questions That Separate Good Candidates from Great Ones (2026)

SQL interviews at top tech companies have evolved. Forget basic SELECT queries — modern data engineering and backend interviews test your ability to handle window functions, complex aggregations, recursive CTEs, and real-world analytical scenarios.

This guide covers the most challenging SQL interview questions you'll face, each linked to a full solution.

What SQL Interviewers Are Really Testing

Most candidates can write a GROUP BY query. What separates top performers is their ability to:

Use window functions (ROW_NUMBER, RANK, LAG, LEAD, SUM OVER) fluently
Write self-joins for hierarchical or sequential data problems
Handle NULLs carefully and intentionally
Think about performance — indexes, query plans, and avoiding full scans
Pivot and reshape data dynamically
Apply set theory to solve complex membership and exclusion problems

The SQL Questions (With Solutions)

📊 Window Functions & Rankings

Nth Highest Distinct Salary

Given an 'Employee' table with 'id' (Primary Key) and 'salary' (Integer), write a SQL query to retrieve the Nth highest distinct salary. If there are fewer than N distinct salaries in the table, the query should return NULL. The solution should be flexible enough to handle any integer input N.
Monthly Department vs Company Salary Benchmarking

You are given two tables: 'Salary' (containing 'id', 'employee_id', 'amount', and 'pay_date') and 'Employee' (containing 'employee_id' and 'department_id'). For every month present in the data, determine if the average salary within each department was higher, lower, or equal to the average salary of the entire company for that same month. Return a result table with the month (formatted as 'YYYY-MM'), the 'department_id', and a 'comparison' column indicating 'higher', 'lower', or 'same'.

🔁 Sequential & Time-Based Problems

Consecutive Product Usage Streak

Given a table filed_taxes with columns filing_id, user_id, filing_date, and product, find all users who have filed their taxes using any 'TurboTax' product for at least three consecutive years. Note that 'TurboTax' products may have different version names (e.g., 'TurboTax Basic', 'TurboTax Deluxe'). Assume each user files at most once per calendar year. Return the results sorted by user_id.
Consecutive SaaS Subscription Retention

Given a table filed_taxes with columns filing_id, user_id, filing_date, and product, identify users who have demonstrated loyalty by using any version of the 'TurboTax' product suite for at least three consecutive calendar years. Note that 'TurboTax' may appear in the product column under various names (e.g., 'TurboTax Deluxe 2023', 'TurboTax Free Edition'). A user may have multiple entries, but only one filing per year should be considered for the continuity count. Output the list of unique user IDs sorted numerically/alphabetically.
Consecutive High Attendance Records

You are given a table Stadium containing stadium visit data with columns id (integer), visit_date (date), and people (integer). The id is an auto-incrementing value that corresponds with the chronological order of the visit_date. Write a SQL query to identify all sequences of three or more consecutive IDs where each visit in that sequence had a population of 100 or more people. The result should include the original row details and be ordered by the visit date.

🏅 Customer Retention & Loyalty

Identifying Multi-Year Customer Loyalty

Find customers who made purchases in every year of a multi-year window. Tests intersection and set-based thinking.
Identifying Multi-Year Customer Retention

A variation focusing on customer cohort retention rates across annual periods.

🔄 Data Transformation & Pivoting

Finding Median Rows per Partition Given a table Employee with columns id (INT, PK), company (VARCHAR), and salary (INT), write a SQL query to identify the specific rows that represent the median salary for each company.

Rules for median calculation:

Sort salaries in ascending order for each company.
In case of identical salaries, use id as a secondary ascending sort key to break ties.
If a company has an odd number of employees, return the single middle row.
If a company has an even number of employees, return the two middle rows.

The result should include the id, company, and salary columns for these specific median records.

Title Case Transformation with Hyphen Sensitivity You are given a table user_content with a column content_text. Your task is to generate a report that transforms the text into Title Case based on specific business rules.

The first letter of every word must be capitalized, and all subsequent letters in that word must be lowercase.
The transformation must recognize a hyphen (-) as a word boundary, meaning segments on both sides of a hyphen (e.g., 'high-speed') must be capitalized ('High-Speed').
All original spacing, including multiple consecutive spaces, must be preserved exactly as they appear in the source data.

Return the original content_id, content_text, and the newly transformed_text column.

Correcting Swapped Sequences in Delivery Logs A delivery platform experienced a technical glitch where every pair of consecutive items in their order logs was swapped (i.e., the item intended for ID 1 was recorded for ID 2, and vice versa). You are given a table orders with columns order_id (unique, sequential integers starting from 1) and item (the name of the food).

Write a query to restore the correct mapping. If the total number of orders is odd, the final order ID should remain paired with its original item. The output should be sorted by order_id in ascending order.

Alphabetical Continent Pivot

Given a table Student containing name and continent columns (with potential duplicates), pivot the table so that student names are grouped under their respective continent headers: 'America', 'Asia', and 'Europe'. Within each column, names must be sorted alphabetically. If one continent has more students than another, the extra rows for the smaller continents should be filled with NULL. The result should show the first alphabetical student for each continent in the first row, the second in the second row, and so on.
Identifying Consistent Median Performers

Given a Student table containing names and IDs, and an Exam table containing scores for various exams, define a 'Quiet Student' as one who has participated in at least one exam but has never achieved the highest or lowest score in any exam they sat for. Write a SQL query to find the student_id and student_name of all quiet students, ensuring that students who never took an exam are excluded. Note: If an exam has multiple students with the same top score, all are considered 'loud' for that exam. Sort the output by student_id.

🏢 Organizational Hierarchy

Identifying Senior Management Tiers Given a table employees with columns emp_id, manager_id, and manager_name, define a 'Senior Manager' as an employee who manages at least one individual who is himself/herself a manager, but does not manage anyone who qualifies as a Senior Manager.

Write a SQL query to return the name of every Senior Manager and the total count of their direct reports. If an employee reports to multiple managers, they should be counted as a report for each. Sort the results by the report count in descending order.

Note: A manager is any employee who has at least one direct report.

🔧 Data Quality & Deduplication

Identifying Incomplete Assembly Parts Find assembly records where required component parts are missing. A manufacturing/logistics favorite.

How to Study SQL for Interviews

The three patterns that cover 80% of hard SQL questions:

1. Window Functions

SELECT 
  employee_id,
  salary,
  RANK() OVER (PARTITION BY department_id ORDER BY salary DESC) AS salary_rank
FROM employees;

Master RANK, DENSE_RANK, ROW_NUMBER, NTILE, LAG, LEAD, SUM OVER, AVG OVER.

2. CTEs for Readability and Recursion

WITH ranked AS (
  SELECT *, ROW_NUMBER() OVER (PARTITION BY category ORDER BY revenue DESC) AS rn
  FROM sales
)
SELECT * FROM ranked WHERE rn = 1;

3. Self-Joins for Sequential Logic

SELECT a.user_id
FROM activity a
JOIN activity b 
  ON a.user_id = b.user_id 
  AND DATEDIFF(b.activity_date, a.activity_date) = 1;

CRM Marketing Touch Streak Analysis A marketing team wants to reward highly engaged potential customers. Given a marketing_touches table (event_id, contact_id, event_type, event_date) and a crm_contacts table (contact_id, email), find the email addresses of all contacts who meet two conditions:
They had at least one marketing touch per week for at least three consecutive weeks (weeks are defined as starting on Monday).
They have performed at least one touch of type 'trial_request' at any point in their history.

Ensure the solution handles contacts with multiple touches in a single week correctly and accounts for streaks that might cross year boundaries.

Best-Selling Products per Category with Tie-Breakers

You are given two tables: 'products' (containing product_id, product_name, and category_name) and 'product_sales' (containing product_id, sales_quantity, and rating). Write a PostgreSQL query to identify the top-performing product in every category. The primary metric for performance is 'sales_quantity'. In the event of a tie in sales, the product with the higher 'rating' should be prioritized. If multiple products remain tied after checking both metrics, include all of them. The final result should display the category name and product name, sorted alphabetically by the category name.
Server Fleet Utilization Analysis

A cloud provider tracks server uptime through a series of status logs. You are given a table server_utilization with columns server_id (int), status_time (timestamp), and session_status (string, either 'start' or 'stop'). Each server can start and stop multiple times throughout the period.

Calculate the total uptime for the entire fleet of servers. The result should be returned as the total number of full days (24-hour periods) of cumulative uptime across all servers, rounded down to the nearest integer.

Ensure your solution accounts for the chronological order of events per server and correctly pairs start events with their subsequent stop events.

Identify Retried Payment Transactions

Given a transactions table with columns transaction_id, merchant_id, credit_card_id, amount, and transaction_timestamp, write a query to identify the total number of 'accidental' repeat payments. A repeat payment is defined as a transaction that occurs at the same merchant, with the same credit card, for the exact same amount, within a 10-minute window of a previous transaction. Note: In a sequence of three identical transactions each within 10 minutes of the last, the second and third should be counted as repeats, but the first should not.
Daily Odd and Even Sensor Measurement Totals

Given a table measurements containing measurement_id, measurement_value, and measurement_time, calculate the daily sum of values based on their chronological order within each day. A measurement is 'odd-numbered' if it is the 1st, 3rd, or 5th recorded event of that day, and 'even-numbered' if it is the 2nd, 4th, or 6th. Your output should contain the date, the total sum for odd-numbered measurements, and the total sum for even-numbered measurements for each day represented in the dataset.
Top Grossing Products by Category

Given a product_spend table documenting customer transactions, write a query to retrieve the top two highest-grossing products within each category for the calendar year 2022. Gross spend is defined as the sum of the spend column for a given product. Your output should include the category name, the product name, and the total calculated spend, ordered by category and then by the highest spend.
Top-Selling Products by Category

Given a table product_spend representing Amazon transactions, write a query to identify the top 2 products in terms of total revenue for each category during the year 2022. Your output should display the category name, the product name, and the total spend associated with that product. In cases of identical spend within a category, ensure the ranking logic is consistent with standard business reporting (e.g., handling ties).
Manufacturing Pipeline Bottlenecks

You are analyzing production data for a high-volume manufacturing facility. The table parts_assembly tracks individual steps in the assembly process for various components.

A part is considered 'In Progress' if it has at least one entry in the table. A part is considered 'Unfinished' if there is at least one step in its sequence where the finish_date is missing (NULL).

Write a query to return a unique list of all parts that are currently in the assembly process but have not yet been completed.

Practice Tips

Always think about NULL handling first — COUNT(*) vs COUNT(col) matters
Use EXPLAIN to validate your solution isn't doing full table scans
For "consecutive" problems, the LAG/LEAD approach is almost always cleaner than self-joins
For "per group" questions, think PARTITION BY before GROUP BY
Time-window problems almost always need BETWEEN or DATEDIFF with careful boundary thinking

50 Must-Know System Design Interview Questions (With Solutions) for 2026

interviewgpt — Mon, 11 May 2026 06:57:58 +0000

System design interviews are the make-or-break round at top tech companies like Google, Meta, Amazon, Apple, and Netflix. Unlike coding questions, there's no single "correct" answer — interviewers are evaluating your ability to reason about trade-offs, scale, and architecture under pressure.

This article compiles the most popular and frequently-asked system design questions, each with a full expert-written solution you can study interactively on InterviewGPT.

Why System Design Interviews Are Different

Most candidates spend 90% of their prep on LeetCode and neglect system design. The truth? At senior and staff levels, system design often weighs more than coding in the final hiring decision.

Strong answers demonstrate:

The ability to clarify requirements before diving in
Back-of-the-envelope estimation (scale, storage, throughput)
Knowledge of distributed systems fundamentals (consistency, availability, CAP theorem)
Real-world technology choices (Kafka, Redis, PostgreSQL, etc.)
An understanding of trade-offs — not just "what" but "why"

High-Throughput GPU Inference Batching System Design

interviewgpt — Tue, 07 Apr 2026 22:19:43 +0000

High-Throughput GPU Inference Batching System

Abstract: How do you build a system that supports high-concurrency requests against an API you cannot change? This article walks through a complete infrastructure design for a GPU inference batching system — one that optimizes GPU utilization via a server-side batching mechanism that intelligently balances latency and throughput. From clarifying questions to deep trade-off analysis, this is a FAANG-level deep dive into one of the hardest infrastructure problems in applied ML.

Clarifying Questions
Crash Strategy & Key Points
Elite Bonus Points (FAANG Rubrics)
Functional Requirements
Non-Functional Requirements
Back-of-Envelope Estimation
High-Level Design
Low-Level Design
Trade-offs, Alternatives & Optimizations

1. Clarifying Questions

Before designing anything, you need to nail down assumptions. Here are the key questions — and the assumptions we'll carry forward:

Question	Assumption
What is the peak QPS and the target latency SLO?	10,000 QPS with a p99 latency requirement of < 500ms
What is the maximum batch size supported by the fixed inference API?	Max batch size is 64 requests
What is the payload size for input and output?	Text-based, ~2KB per request
Is client communication synchronous or asynchronous?	Clients expect a synchronous-like experience; we use an async-polling or long-polling pattern internally
Do we need to handle request priorities (e.g., premium vs. free users)?	No — FIFO for the MVP

Clarifying questions are not just a formality. Each assumption here directly shapes an architectural decision downstream. The batch size cap (64) determines our batcher's flush trigger. The 500ms SLO sets our tolerable wait window. Always clarify before drawing boxes.

2. Crash Strategy & Key Points

The Core Bottleneck

When requests arrive individually at a GPU worker, two problems emerge: under-utilization (GPUs thrive on parallelism) and memory overhead (per-request context switching). The naive design — one request in, one inference out — kills throughput.

The Key Strategy: Dynamic Batching

The solution is a Dynamic Batching Service that acts as a buffer between your high-concurrency HTTP API and the fixed GPU workers. It's a traffic shaper: absorb the spikes, group requests intelligently, dispatch in bulk.

Progressive Problem Decomposition

Think through this layer by layer:

How do we ingest 10k+ requests without blocking?
Use a distributed message queue. Each incoming request is a lightweight enqueue operation.
How do we group them efficiently?
The Batcher implements "Wait-or-Full" logic — flush when batch size hits 64, or when 50ms elapses, whichever comes first.
How do we deliver results back to the user?
A Result Store (Redis) holds completed inference outputs. Clients poll with a task_id.
How do we scale the Batcher itself?
Partition-based batching — each Batcher instance consumes from a dedicated partition of the queue, eliminating global locks and contention.

3. Elite Bonus Points (FAANG Rubrics)

These are the insights that separate a good design from a great one:

Adaptive Batching

Dynamically adjust the wait_time based on current traffic volume. During low-traffic periods, flush quickly to minimize latency. During spikes, extend the wait window to fill batches and maximize GPU throughput. A simple EWMA (Exponentially Weighted Moving Average) on the arrival rate drives this.

Zero-Copy Serialization

Use Protobuf or Apache Arrow for internal data transfer. This reduces CPU overhead during batch construction and deconstruction — critical when you're processing millions of requests per hour.

GPU Backpressure Propagation

Implement a feedback loop: if the GPU Worker's internal queue or memory utilization exceeds 90%, the Batcher slows ingestion. This prevents the queue from becoming a buffer for an already-saturated backend. Your system should degrade gracefully, not catastrophically.

Locality-Aware Batching

At global scale, ensure batching happens at the edge or within the same Availability Zone. Cross-region data transfer costs are real, and cross-AZ latency adds up quickly under a 500ms SLO.

4. Functional Requirements

Core Use Cases

Users submit inference requests via a REST API.
Requests are batched and processed by the GPU model.
Users retrieve the inference result via polling.

Scope Control

In-Scope	Out-of-Scope
API Gateway	Model training
Request Queue	Model optimization (TensorRT/ONNX)
Batching Logic	User authentication service
Result Storage

Scope control is not laziness — it's clarity. Defining the boundary prevents scope creep and lets you go deep on what matters.

5. Non-Functional Requirements

Dimension	Requirement
Scale	Handle 10k QPS, scale horizontally
Latency	Batching overhead < 50ms; total E2E latency < 500ms
Availability	99.9% uptime; requests must not be lost on worker failure (at-least-once delivery)
Consistency	Eventual consistency for results; strict ordering within a batch is not required
Fault Tolerance	Dead-letter queues (DLQ) for failed inference attempts

6. Back-of-Envelope Estimation

Let's do the math to validate our architecture can actually work.

Traffic

10,000 requests/sec

Storage

10,000 req/s × 2KB/req = 20 MB/s
For 1 hour of retention: ~72 GB

Bandwidth

Ingress: 10,000 × 2KB = 20 MB/s
Egress (Results): ~20 MB/s

GPU Worker Count

One batch of 64 requests takes ~200ms to process
One GPU worker handles: 64 / 0.2s = 320 QPS
Workers needed: 10,000 / 320 ≈ 32 GPU workers

This gives us a concrete target — 32 GPU workers — and validates that the batching approach is load-bearing, not cosmetic.

7. High-Level Design

Design Summary

A high-throughput pipeline using a distributed queue to decouple request ingestion from GPU execution, featuring a dedicated Batcher Service for optimal GPU utilization.

Major Components

Component	Purpose
API Gateway	Entry point — SSL termination, request validation, rate limiting
Request Queue (Redis Streams)	Fast, in-memory buffer for incoming inference tasks
Batcher Service	Core logic — aggregates N messages or waits T milliseconds before calling the GPU API
Result Store (Redis)	Short-lived storage for finished inference results

System Architecture Diagram

┌────────┐      ┌─────────────┐      ┌───────────────┐      ┌──────────────────┐      ┌───────────────┐
│ Client │ ───► │ API Gateway │ ───► │ Request Queue │ ───► │ Batcher Service  │ ───► │ GPU Worker API│
└────────┘      └─────────────┘      │ (Redis Stream)│      │  (Wait-or-Full)  │      └───────┬───────┘
     ▲                ▲              └───────────────┘      └──────────────────┘              │
     │                │                                                                        ▼
     └────────────────┴──────────────────────────────────── Result Store (Redis) ◄────────────┘

Simplicity Audit

This architecture intentionally avoids complex stream-processing frameworks like Apache Flink. A lightweight consumer-group-based batcher is easier to deploy, debug, and scale for an MVP — and can be upgraded later if needed.

8. Low-Level Design

8.1 Edge Layer

Traffic Routing: A Global Server Load Balancer (GSLB) routes traffic to the nearest regional API Gateway.
Security: The API Gateway handles JWT validation and rate limiting — 1,000 requests per user per minute — to protect the GPU cluster from abuse.

8.2 Service Layer

Topology: Stateless API instances deployed in Kubernetes, auto-scaling on CPU (70% threshold).

API Schema:

POST /v1/inference
Body: { "input": "...", "client_id": "..." }
Response: { "task_id": "abc-123" }

GET /v1/result/{task_id}
Response: { "status": "PENDING" | "SUCCESS", "output": "..." }

Resilience: 3 retries with exponential backoff for the Batcher when calling the GPU API.

8.3 Storage Layer

Access Pattern: High write/read (1:1 ratio). Data is transient — results expire after 10 minutes.

Result Table (Redis Hash):

Key: task_id
Fields: status, output, timestamp

Distribution: Partitioning by task_id using Redis Cluster to handle 20k+ operations per second.

8.4 Cache Layer

Deduplicate identical inference requests (e.g., the same prompt submitted multiple times) to save GPU cycles.

Key: SHA256(input_payload)
Value: task_id or cached_result
TTL: 5 minutes
Failure Handling: If Redis fails, bypass the cache and go straight to the queue.

8.5 Messaging Layer

Topic Schema — inference-requests:

{
  "task_id": "abc-123",
  "payload": "...",
  "ts": "2026-03-29T19:50:55Z"
}

Throughput: Redis Streams with 16 shards to allow parallel Batcher consumers.

Why Redis Streams? Low latency, built-in consumer groups for at-least-once delivery, and operational simplicity compared to Kafka for this scale.

8.6 Data Processing Layer — The Batcher (Core Design)

This is the heart of the system. The Batcher Service uses a hybrid trigger model:

Trigger	Condition
Size Trigger	64 messages accumulated
Time Trigger	50ms elapsed since the first message in the current window

Whichever fires first wins. This guarantees that no request waits longer than 50ms regardless of traffic volume.

Processing DAG:

Read from Stream
      │
      ▼
Accumulate in Memory (per-partition buffer)
      │
  Size=64 OR Time=50ms
      │
      ▼
Call Fixed GPU Worker API (one HTTP/gRPC call with full batch)
      │
      ▼
Disperse Results → Write each result to Redis by task_id
      │
      ▼
ACK Stream (mark messages as processed)

Scalability: Multiple Batcher instances consume from different partitions of the Redis Stream — no shared state, no global locks.

8.7 Infrastructure & Observability

Key Metrics to Track:

batch_size_distribution — Are we consistently filling batches? Or flushing early on timeouts?
gpu_worker_latency — Are workers saturating?
queue_depth — Leading indicator of system stress.

Distributed Tracing: Jaeger for end-to-end tracing from the API Gateway through the Batcher to the GPU API response. Essential for diagnosing latency regressions.

Technology Stack: Redis Streams, Redis Cluster, Kubernetes, Envoy, gRPC, Prometheus, Jaeger, JWT, mTLS.

9. Trade-offs, Alternatives & Optimizations

The Fundamental Trade-off: Latency vs. Throughput

We accept up to 50ms of additional latency per request in exchange for dramatically higher GPU utilization. A single request arriving at an idle batcher window waits up to 50ms. In return, the system can sustain 10x the load of a naïve pass-through design.

This is the right trade-off for batch inference workloads, where throughput matters more than tail latency for individual requests.

Reliability: At-Least-Once Delivery

Redis Streams' Consumer Groups ensure durability. If a Batcher instance crashes mid-processing, unacknowledged messages are automatically re-delivered to another healthy instance (NACK mechanism). No request is silently dropped.

Bottleneck Analysis

The Fixed GPU API is the ultimate bottleneck. If it slows down, the Request Queue depth grows. Two mitigations:

TTL on the Queue: Drop stale requests before they age out of user tolerance.
Backpressure: Signal upstream components to slow ingestion when GPU memory > 90%.

Security

All internal communication between the Batcher and GPU API uses mTLS.
Input sanitization is performed at the API Gateway to prevent prompt injection attacks.

The "Hot Key" Problem: Request Collapsing

When many users submit the same inference request (e.g., a trending query), the Batcher can implement request collapsing: identify duplicate payloads within the same batch using a hash, send only one to the GPU, then replicate the result to all matching task_id entries. This is a powerful optimization at scale.

Summary

Layer	Technology	Key Design Decision
API Gateway	Envoy / K8s Ingress	Rate limiting, JWT validation
Request Queue	Redis Streams (16 shards)	Partitioned for parallel consumers
Batcher	Custom service, per-partition	Wait-or-Full hybrid trigger (N=64, T=50ms)
GPU Worker	Fixed external API	Called with full batch payload
Result Store	Redis Cluster	TTL=10min, keyed by task_id
Observability	Prometheus + Jaeger	Batch size distribution, queue depth, GPU latency

The architecture centers on one insight: GPUs are expensive, and idle GPUs are waste. Every design decision — the queue, the batcher, the partitioning strategy, the cache — serves the goal of keeping GPU workers saturated while keeping the user experience fast and reliable.

Designed to handle 10,000 QPS with a p99 latency under 500ms. 32 GPU workers. Zero request loss. One core idea: batch everything.

For more system-design articles, check the InterviewGPT blog.

DEV Community: interviewgpt

20 Tricky SQL Interview Questions That Separate Good Candidates from Great Ones (2026)

20 Tricky SQL Interview Questions That Separate Good Candidates from Great Ones (2026)

What SQL Interviewers Are Really Testing

The SQL Questions (With Solutions)

📊 Window Functions & Rankings

🔁 Sequential & Time-Based Problems

🏅 Customer Retention & Loyalty

🔄 Data Transformation & Pivoting

🏢 Organizational Hierarchy

🔧 Data Quality & Deduplication

How to Study SQL for Interviews

The three patterns that cover 80% of hard SQL questions:

Practice Tips

50 Must-Know System Design Interview Questions (With Solutions) for 2026

Why System Design Interviews Are Different

Top System Design Interview Questions & Solutions

🏦 Finance & Payments

💬 Messaging & Real-time Communication

📹 Video & Streaming

🌐 Social Networks & Feeds

🔗 Infrastructure & Storage

🔍 Search & Discovery

🔗 URL & Crawling

🚗 Ride-Sharing & Location

🏨 Booking & Reservations

🤖 AI & ML Infrastructure

High-Throughput GPU Inference Batching System Design

Table of Contents

1. Clarifying Questions

2. Crash Strategy & Key Points

The Core Bottleneck

The Key Strategy: Dynamic Batching

Progressive Problem Decomposition

3. Elite Bonus Points (FAANG Rubrics)

Adaptive Batching

Zero-Copy Serialization

GPU Backpressure Propagation

Locality-Aware Batching

4. Functional Requirements

Core Use Cases

Scope Control

5. Non-Functional Requirements

6. Back-of-Envelope Estimation

Traffic

Storage

Bandwidth

GPU Worker Count

7. High-Level Design

Design Summary

Major Components

System Architecture Diagram

Simplicity Audit

8. Low-Level Design

8.1 Edge Layer

8.2 Service Layer

8.3 Storage Layer

8.4 Cache Layer

8.5 Messaging Layer

8.6 Data Processing Layer — The Batcher (Core Design)

8.7 Infrastructure & Observability

9. Trade-offs, Alternatives & Optimizations

The Fundamental Trade-off: Latency vs. Throughput

Reliability: At-Least-Once Delivery

Bottleneck Analysis

Security

The "Hot Key" Problem: Request Collapsing

Summary