Before we begin, ask yourself a simple question:
What does service performance mean to you? What undeniable proof do you have that your service is not performing well—or vice versa?
You see, this question solves the first problem in scaling any service. What does scale mean to you and your service? For some, handling 10k requests per second is an optimised system because at peak they are receiving 5–7k requests. For some, 1 million requests per second is not optimised enough. They are still facing bottlenecks.
I have seen many times that people give very little importance to observability / monitoring, but it is the very first step that one can take and ideally should take on their journey to scaling services.
Why does Observability / Monitoring matter?
Without these, you will not be able to figure out what the low and high ceilings of your service are. What is the average load on your servers? On a random Tuesday, how many requests per second reach your service?
If you don't have answers to the above questions and you are trying to scale your service without actually understanding where the bottleneck is, you might be increasing your CPU cores thinking that this will make your service faster, but CPU was never the bottleneck. If you had measured CPU usage, you would have known that your CPU cores were idle most of the time. It was actually your database that was the real culprit. Or maybe Redis was the actual issue? Or was it?
This is what Observability solves for you.
After setting it up, you have answers to questions like: why does our API take 5s to return a response? You can see, from the list of services, where the actual slowness is and target that specific component.
How do you know that your service now requires Redis? I know most of the LLMs these days add Redis even for prototypes. But as an engineer, you know that the fewer services and components you have, the less operational overhead you introduce. Does this mean adding Redis, OpenSearch, or any other service is a bad thing? No, not if you have a measured reason. If you see that the DB is fast, replying in milliseconds, but the responses are getting delayed due to calculations that we are doing, then the bottleneck is compute. That's what we should solve.
What are some of the mainstream stacks for Observability and Monitoring?
Datadog → full-stack observability: metrics, logs, traces, infrastructure, APM
SigNoz → open-source observability: metrics, logs, traces, APM, built around OpenTelemetry
Grafana → observability/visualisation ecosystem: dashboards, metrics, logs, traces
New Relic → application observability / APM
Dynatrace → enterprise observability / APM
Prometheus → metrics monitoring, often part of an observability stack
Sentry → error tracking + application performance monitoring
Which of the above is best?
I don't know your use case, figure that out yourself. Are you happy to pay for a managed solution? Maybe Datadog then. Want an open source option? SigNoz or the Grafana ecosystem might be a better fit.
The important thing is not which tool you choose. The important thing is that you have enough visibility into your system to understand what is actually happening. I hope this post gave you an understanding of how to think about scaling your services. Before actually scaling them, Measure first, find the bottleneck and thhen scale.
Top comments (0)