Backend Interview Question: Your API Takes 900 ms. SQL Takes 40.

Before adding a cache, build a request timeline and find the work your user is actually waiting for.

Someone says the API is slow, and suddenly the room has six solutions. Add Redis, add an index, add more servers, rewrite the endpoint, parallelize everything, or blame the network and hope nobody asks a follow-up.

Meanwhile, we still have not explained where the time went.

I like a much smaller starting point. Give me one slow request, the boundary we are timing, and enough evidence to account for its elapsed time. Then we can have a useful conversation about what to change.

For this original backend interview exercise, imagine a product-details API that takes 900 milliseconds from gateway arrival to completion of the response. Its SQL query uses 40 milliseconds. Those numbers are a synthetic fixture, not a production incident or a claim about a particular company.

Our job is to improve the experience without accidentally making the answer less correct. A faster response containing the wrong price is not the little victory we are looking for.

Choose the stopwatch before choosing the fix

First, clarify where the measurement starts and stops. Browser click to usable screen, gateway arrival to completed response, and database query execution are different intervals.

For our example, we are timing the server path from the gateway receiving the request until the response finishes there. We are not yet measuring DNS, connection setup before arrival, the network trip back to the browser, or rendering in the client.

That boundary matters because a beautiful server trace cannot explain time it never measured.

Next, make the complaint specific. Is this one route, one region, one tenant, or every request? Did the slowdown begin after a deployment, appear only during busy periods, or affect unusually large responses? Compare successful and failed requests separately so fast failures do not make the service look reassuringly quick.

Google’s SRE guidance on monitoring distributed systems emphasizes latency alongside traffic, errors, and saturation. Those signals help us describe the symptom before choosing a remedy.

For this exercise, start with one successful slow request. Treat it as a clue to investigate, not proof that every user has the same problem.

Give every part of the 900 ms a place on the page

Imagine our instrumentation gives us the following non-overlapping stages. In this deliberately simple fixture, they happen sequentially.

• The request waits 120 ms before an application worker starts.
• Authentication takes 40 ms.
• The SQL query takes 40 ms.
• An outbound dependency call takes 600 ms as observed by our service.
• Serialization and response completion take the remaining 100 ms.

Together, those stages account for all 900 ms. We are assuming that the measurement includes the queue and that no relevant interval is missing.

If your real trace does not add up, resist the urge to name the missing time network overhead and move on. It could be uninstrumented work, a gap between spans, or a boundary you misunderstood.

OpenTelemetry’s explanation of traces and spans is useful here. A trace connects operations through a request, while spans describe individual operations with timing and context. That structure helps you ask where the request went, but it cannot reveal work you never instrumented.

Also check what the 600 ms actually measures. An HTTP client span can include time spent acquiring a connection, connecting, waiting for the remote service, and receiving its response. It is evidence that this interval is expensive, not proof that the remote application spent 600 ms computing.

Honestly, this is where the investigation starts becoming interesting. We have narrowed the search without pretending we already know the answer.

Put an upper bound on your favorite optimization

Suppose the first suggestion is to cache the SQL result. Even if that eliminated the entire 40 ms query at zero added cost, our fixed trace would still take 860 ms.

That is a maximum saving of about 4.4% for this example, with every other stage held constant. It is not a prediction of a real cache’s performance, and it says nothing yet about cache misses, invalidation, or freshness.

By comparison, reducing the dependency interval from 600 to 200 ms would bring this same simplified timeline to 500 ms. That is a hypothesis worth investigating because the potential impact is much larger.

This does not mean caching is always the wrong answer. Under load, removing database work might reduce contention and shorten other waits too. Our one-request calculation deliberately holds those effects constant, so a load experiment could reveal a different outcome.

What the calculation does give us is a disciplined question. What mechanism makes this change reduce the delay we actually observed, and how much improvement is plausible if that mechanism works?

I prefer answering that before introducing another service we will need to operate.

Follow the waiting time instead of blaming the biggest box

Take the 600 ms dependency interval and split the next investigation into possibilities. Maybe the local client waits for a connection. Maybe connection setup is slow. Maybe the remote service is slow. Maybe a large response takes time to arrive.

Each explanation needs different evidence and suggests a different change.

For example, if connection acquisition dominates, inspect pool occupancy, pending requests, and connection lifetime. Increasing the pool size is only a proposal; it might overload the downstream service or move the queue somewhere else.

If remote processing dominates, correlate the outbound request with the downstream trace when available. If payload transfer dominates, inspect response size and whether the caller needs every field. Keep missing evidence visible rather than promoting a plausible explanation into a fact.

Do the same for the 120 ms before the application worker starts. Low CPU does not rule out waiting on a constrained resource. On the other hand, a single queued request does not prove the service needs more workers.

Your next measurement should distinguish between competing explanations. Adding ten dashboard screenshots that all say slow is not the same as narrowing the cause.

Parallel work changes the arithmetic

Now change the example. Suppose two independent downstream calls start together, one taking 400 ms and the other 300 ms. The response needs both results, followed by 100 ms of response work.

Ignoring other overhead, elapsed time is 500 ms because the response waits for the slower branch and then finishes its own work. Adding 400, 300, and 100 to report 800 ms would double-count overlap.

That also changes which optimization matters. Making the 300 ms branch take 100 ms would not shorten this particular response while the 400 ms branch still determines the join. It could reduce resource use, but that is a different claim.

Before parallelizing real calls, establish that neither depends on the other’s result and that concurrency will not violate ordering or overload a shared dependency. Parallel is a scheduling decision, not a synonym for faster in every situation.

Read a request timeline as a set of dependencies. Find the chain that determines when the response can finish, and avoid adding parent spans to the child spans already contained within them.

Write the experiment before celebrating the improvement

For our synthetic request, a useful investigation note might say that the dependency interval is 600 ms, connection-pool waiting is one possible explanation, and acquisition timing is the next evidence needed. That is a plan somebody else can question and reproduce.

Once the evidence supports a change, compare equivalent requests under equivalent conditions. Keep the route, payload, authorization context, data shape, and load profile clear. Separate warm and cold behavior when they differ.

Check end-to-end latency along with errors, correctness, completed throughput, and the resource you changed. Include slow requests in the comparison rather than relying only on the average, and record the sample size and measurement window. An average improvement can hide a worse experience for a smaller group.

A patch that returns early by skipping required work is a behavior change, not a free optimization. If the product accepts a partial response or stale data, state that contract explicitly and test it.

For another practice round, use this shopping cart latency problem. Draw the current request path before proposing improvements, then identify which work must finish before the user can see a correct cart. Treat stock freshness as part of the result you must preserve.

Before moving on, leave yourself a short investigation note with the measurement boundary, the observed timeline, one hypothesis, the next piece of evidence, and the result that would count as success.

That is the interview answer I want to practice. We can absolutely talk about caches and indexes, but first let’s make every millisecond in the complaint earn its place on the page.

~ Emily