0

it's been many moons since I've been in a math class, so I'm not sure how to formulate the problem precisely. I'll describe it informally instead.

This is related to response times in a computer system. Let's say I have some system, foo, that I can make requests to and I can measure its response time. I've collected enough data to be able to describe the TP50 and TP90 latencies. Now, let's say I'm building a new system, bar, and for every request that bar needs to service, I need to make two requests to foo. The calls to foo are done in parallel and let's say they are independent of each other. bar needs to wait for them both to complete. How can I calculate TP50 of bar? What if instead of 2 calls, I need to make n calls?

Through a little toy simulation, I get the TP50 of bar is ~TP70 of foo, assuming bar makes two calls. I'd like to understand how to think about this mathematically though.

Thanks for any help!

Olaf
  • 103

1 Answers1

1

Unfortunately, knowing only the $50^{\rm th}$ and $90^{\rm th}$ percentiles of the latency distribution for the first system is inadequate information to estimate the $50^{\rm th}$ percentile of the second system.

For instance, suppose the first system's latency is exponentially distributed with mean $1$ time units (say, seconds); i.e., the random response time $T$ has cumulative distribution function $$\Pr[T \le t] = 1 - e^{-t}, \quad t > 0.$$ Then the "TP50" is the $50^{\rm th}$ percentile $m_{0.5}$ of this distribution, namely $$\Pr[T \le m_{0.5}] = 1/2$$ which implies $$m_{0.5} = \log 2 \approx 0.693147.$$ This means that $50\%$ of all requests to the first system will be served in at most $693$ milliseconds. Similarly, $m_{0.9} = \log 10 \approx 2.30259$ seconds.

Now under this distributional assumption, we can establish that the maximum response time of $n$ requests to this first system has the cumulative distribution function $$\Pr[T_{max} \le t] = (1 - e^{-t})^n, \quad t > 0.$$ The $50^{\rm th}$ and $90^{\rm th}$ percentiles are $$m_{0.5} = - \log (1 - (1/2)^{1/n}), \\ m_{0.9} = - \log (1 - (9/10)^{1/n})$$ and for $n = 2$, these become $m_{0.5} \approx 1.22795$ and $m_{0.9} \approx 2.96974$.

But this applies only to exponentially distributed latencies. Suppose I instead choose to model the latency as normally distributed, in such a way that matches the median and $90^{\rm th}$ percentiles above; i.e., $$\mu = m_{0.5} = \log 2,$$ and we need to numerically solve for the variance $\sigma^2$ such that the $90^{\rm th}$ percentile equals $\log 10$. This gives us $$\sigma^2 \approx 1.57716.$$ Then the maximum of $n = 2$ requests does not have a nice closed form, but again with a computer, we can solve for the percentiles. These are $m_{0.5} \approx 1.37753$ and $m_{0.9} \approx 2.74297$, different from the exponential model.

If the use of a normal distribution is objectionable, then we could use a Gamma distribution with suitable shape parameter. But the point that I have shown mathematically is that even when the $50^{\rm th}$ and $90^{\rm th}$ percentiles are fixed, I could choose two different distributional assumptions and get different results, illustrating that knowing these values is not in itself sufficient to determine the TP50 of the second system. Consequently, you must provide additional information or assumptions about the latency of the first system with which to properly model the behavior of the second.

heropup
  • 135,869