What is the T-distribution, and what is it used for?

Question

(I'll post my own answer to this, but don't hesitate to post your own!)

Student's t-distribution, or T-distribution, was introduced in 1908 by William Sealey Gossett writing under the pseudonym "Student".

What is it, and what is it for?

Answer 1

Let $X_1,\ldots,X_n$ be independent identically distributed random variables that are normally distributed with expected value $\mu$ and variance $\sigma^2$. Think of them as a random sample from a normally distributed population whose expected value and variance are unknown and must be estimated based on this sample.

Then the random variable $$ \frac{\bar X-\mu}{\sigma/\sqrt{n}},\qquad\text{where } \bar X = \frac{X_1+\cdots+X_n}{n} $$ is normally distributed with expected value $0$ and variance $1$. Thus we have $$ \Pr\left(-1.96<\frac{\bar X-\mu}{\sigma/\sqrt{n}}<1.96\right) = 0.95, $$ so we have a $95\%$ confidence interval for $\mu$, with endpoints $\bar X\pm1.96\dfrac{\sigma}{\sqrt{n}}$, provided we know the value of $\sigma$. But of course in practice we don't know $\sigma$. We can estimate $\sigma$ by using $$ S = \sqrt{\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar X)^2}, $$ and it is improbable that that is very different from $\sigma$ if $n$ is large.

But what if $n$ is small? Then think about $$ T=\frac{\bar{X}-\mu}{S/\sqrt{n}}. $$ The distribution of this random variable is approximately normal if $n$ is large. The exact distribution, regardless of the value of $n\ge2$, is Student's t-distribution with $n-1$ degrees of freedom. Given $n$, one can find by standard algorithms a number $A$ such that $\Pr(-A<T<A)=0.95$ (many software packages give you such numbers, and they're found in tables in backs of books, and in the old days, there were volumes containing extensive tables). This gives us a $95\%$ confidence interval for $\mu$ with endpoints $\bar X\pm A\dfrac{S}{\sqrt{n}}$. This number $A$ is bigger than $1.96$ because of the additional uncertainty from not knowing $\sigma$. The smaller $n$ is, the bigger $A$ is. And $A\downarrow1.96$ as $n\to\infty$.

In order that all of this work, it is of course necessary that the distribution of $T$ not depend on $\mu$ or $\sigma$. In effect $\mu$ cancels out with the subtraction in the numerator, and $\sigma$ cancels out with the division.

Answer 2

Oh, that's much more than I was going to say: $T$ is the distribution of $${\bar x - \mu \over s / \sqrt{n}}$$ where $x$ is a normally distributed variable with mean $\mu$, and $\bar x$ and $s$ are the sample mean and standard deviation (resp.) of a random sample of size $n$. (I'm assuming everyone understands that a random sample meets the same conditions as Michael Hardy's $X_1,\dots,X_n$.)

I can add two things: Gossett worked for Guiness, which did not allow its researchers to publish; hence he resorted to the pseudonym. The t-distribution is developed in The Probable Error of the Mean, Biometrika, 6(1), (Mar. 1908) 1-25.