The Normal Distribution

Question

I'd like to correct some of my misconceptions about The Normal Distribution.

Firstly, I don't know how to interpret the following formula: $ \displaystyle Z = \frac{X-\mu}{\sigma}$

I was first introduced to this formula in a earlier chapter and it was introduced to me as the Z-Score. Apparently this was useful since it could find the number of standard deviations a value in a data set $X$ is from the mean. However, upon starting The Normal Distriubution it seems $Z$ is interpreted differently. From what I can understand, it seems $Z$ is treated as another data set with $\mu = 0$ and $\sigma=1$ which can be transformed onto by taking all data values in $X$ and applying $ \displaystyle \frac{X-\mu}{\sigma}$

I'm struggling to accept this concept and don't understand how the formula $ \frac{X-\mu}{\sigma} $ maps any normal distribution X to the Standard Normal distribution $Z$.

Secondly, I understand the Standard Normal Distribution has the equation $ \displaystyle f(x) = \frac{e^{-\frac{x^2}{2}}}{\sqrt{2\pi}} $.

Suppose we wanted to find the $P(Z=1)$, I understand this can be found by just calculating $f(1)$, since the vertical axis outputs probabilities for $x\in X$. However, when looking for probabilities such as $P(Z\leq a)$, I don't understand how this equals the area under the curve over the interval $(-\infty,a]$. Wouldn't it be some type of sum of all the f(x) values between $(-\infty,a]$, so $f(a) + f(a-1)+f(a-2)+f(a-3) + ... $

Answer 1

Your interpretation of $Z = \frac{X - \mu}{\sigma}$ is correct. For example, IQ scores are sometimes considered as being distributed $\mathsf{Norm}(\mu=100,\sigma=15).$ Then if someone gets a 'raw' test score of $x = 115,$ one can say that their 'standard' score is $z = 1,$ meaning that they scored one standard deviation above the mean.

Then if we want to know the probability of a raw score less than $x \le 115,$ one can use the standard score to get that probability:

$$P(X \le 115) = P\left(\frac{X = \mu}{\sigma} \le \frac{115 - 100}{15}\right) = P(Z \le 1) = 0.8413,$$

where you can get the probability 0.8413 from a printed table of the normal distribution. If your table is a pure CDF table then you can read 0.8413 directly from the table. If your table shows the probability between 0 and z, you will see 0.3413 in the table, to which you'd need to add 0.5 to get 0.8413. In symbols,

$$P(Z \le 1) = P(Z \le 0) + P(0 < Z \le 1) = 0.5000 + 0.3413 = 0.8413.$$

We have $P(Z \le 0) = 0.5000$ because the standard normal density curve is symmetric about $0$ so half of the area under the curve (representing probability) lies on either side of $0.$

In the figure below, the density function of $\mathsf{Norm}(100,15)$ is shown on the left and the standard normal density curve is shown on the right. In both curves the area between the vertical red lines represents the probability 0.3413.

Sometimes one says that the raw score 115 lies at about the 84th percentile because about 84% of the population gets raw scores below 115 on an IQ test.

It is possible to use a table for the standard normal random variable $Z$ to get probabilities relating to the random variable $X \sim \mathsf{Norm}(100,15)$ because all normal distributions have the same "fundamental shape." When you speak in terms of standard deviations above and below the mean, that applies to any normal distribution. Just as $P(Z \le 1) = 0.8413$ and $P(X \le 115) = 0.8413,$ it is also true that $Y \sim \mathsf(\mu = 50, \sigma = 2)$ has $P(Y \le 52) = 0.8413$ because the raw score $y = 52$ also corresponds to standard score $z = 1.$

Using a printed table, it is necessary to use standard scores because it would be impossible to print separate tables, one for each conceivable value of $\mu$ and each conceivable value of $\sigma.$

Finally, it is not possible to use the standard normal density function directly to get probabilities. You need a standard normal cumulative distribution (CDF) table for that. For some distributions, you can use the density function to get probabilities by integration, but that does not work for the normal distribution. (Informally, you might say that the expression $\varphi(z) = \frac{1}{\sqrt{2\pi}}e^{-0.5x^2}$ is "too messy" to integrate, using the usual methods of calculus.)

This means that it is tedious to make a normal table and you should be glad someone has done it for you. Roughly speaking, one has to break the desired region into rectangles and sum the areas of the rectangles. One could get 'pretty close" to 0.3413 by summing the areas of the five rectangles between the vertical red bars in the figure below. The height of each rectangle (at its center) can be found from the standard normal density function $\varphi(z)$ and the widths are all 0.02.