Percentile for measuring performance.

To understand the difference between both of the factors, let us take an example where we calculate marks percentage and percentile both.

How is the percentile different from than percentage?

A percentile is different from a percentage. A percentage is calculated based on this formula

percentage = marks obtain / total marks * 100 

This equation will give the percentage of marks.

Percentile is calculated?

Percentile is a collective unit whereas percentage is individual unit.

The percentile is calculated based on this formula

percentile = (No of people behind you) / (Total number of people) * 100

So, let's say you got 90/100(i.e 90%) marks in a class of 100 students. And no one has got more than you so the percentile is 99th percentile

Another example:
Let's say you got a mark of 50% and only 20 students are behind you so the percentile will be
20th percentile

Okay, We now know the difference between percentage and percentile and we are starting to get the picture of why we measure performance based on percentile and not a percentage.
Because a percentile is a collective unit and we need to measure our response time for our users collectively and not individually

This means we don't want to say our API responded with 0.25ms for one user. We would love to say that our API response for 90% of users was less than 0.25ms.

Got it so why in percentile for measuring performance?

By far average is the easiest way to calculate and can be used to measure performance but one should not rely on average for calculating the performance and only focus on a percentile (90th percentile) to improve and measure performance.

Let us look at why?
To answer this question look at the simple bell curve.

For a bell curve average and median(50th percentile) will always be the same.

But for a real-world scenario, the graph will be something like this.

based on this graph average will be 0.97ms but that's not what most of our users are having response time.

Let us sort this graph.

We can see that 90%(P90) of our user is facing 2sec of response time but only 10%(P10) of users are having 0.97ms of response time. So we know that we need to improve our API's response time.

Conclusion

We tried to learn about different techniques and why percentile should be preferred over other techniques. And one should care about P90(i.e 90th percentile) for measuring the performance in real-world applications.