What exactly is meant by "the distribution of a random variable"?

Question

So I've started statistical distributions this week, and there's one piece of terminology that throws me, and this is the idea of a distribution. I understand for most people (on this site, at least) this term is common knowledge, but as someone who is in the very beginner stages of coming to grips with some of the terminology in statistics, bear with me.

If we have a random variable, call it $X$, where $X=$ the number facing up when a dice is rolled, then from this information, we know that the sample space of this $X$ is therefore $$\lbrace 1,2,3,4,5,6\rbrace$$ and we know that the probability distribution of rolling any element, $x$, in $X$ is given by $$~~~~~~~~x~~~~~~~~~1~~~~2~~~3~~~4~~~5~~~6$$ $$P(X=x)~~~\frac{1}{6}~~~\frac{1}{6}~~~\frac{1}{6}~~~\frac{1}{6}~~~\frac{1}{6}$$ or $$ P(X=x) = \left\{ \begin{array}{ll} \frac{1}{6} & \quad x=1,2,3,4,5,6 \\ 0 & \quad otherwise \end{array} \right. $$

from this, we infer that the probability distribution of any random variable $X$ completely describes the probability of each possible outcome, but what, then, can we infer from just the distribution of $X$?

The reason why this throws me only becomes apparent in binomial distribution, since here, we talk a random variable being binomially distributed. Until this point in statistical distributions, we can talk about finding $P(X=x)$, by simply stating that the variable is random, but once a variable is distributed binomially, we need to describe it as such, and despite describing a variable as being binomially distributed, it doesn't make much sense to me. Any help is appreciated.

Thank you.

Answer 1

I think I understand your problem. It seems to be you are uncomfortable about defining a random variable $X$ just by describing its distribution. To answer your question satisfactorily it is necessary to talk about probability spaces (see wikipedia).

First let me do your dice example in some more detail. Let $\Omega := \{1,...,6\}$, and define $X: \Omega \rightarrow \mathbb{R}$ by $X(\omega) = \omega$. Define a probability measure on $\Omega$ by specifying that $P(\{ \omega \}) = 1/6$ for all $\omega \in \Omega$. Let the set $\mathcal{F}$ be the set of all possible outcomes; this means that $\mathcal{F}$ is the set of all subsets of $\Omega$. For instance $\{1,2\} \in \mathcal{F}$ and the 'event' $\{1,2\}$, for instance, means 'the dice shows a $1$ or a $2$'. Technically, I have constructed an underlying probability space $(\Omega, \mathcal{F}, P)$ and a random variable $X$ (that is, a map $X: \Omega \rightarrow \mathbb{R}$) with the property that $$ P(X = \omega) := P(\omega \in \Omega: X(\omega) = \omega) = P(\{\omega\}) = 1/6.$$

Now, to answer your concern: Let us take a sequence of numbers in $[0,1]$ that add up to $1$: $(p_1, p_2,...)$. What does it mean to say "Let $X$ be a random variable with the property $P(X=n)=p_n$"? It means: We can construct a space $(\Omega, \mathcal{F}, P)$ and a map $X: \Omega \rightarrow \mathbb{R}$ with the property that $$P(\omega \in \Omega: X(\omega) = n) = p_n.$$ And here is the general fact: you can always do this. Note: there are lots of different ways of doing this (i.e. we could choose our underlying space to be lots of different things). So random variables with a given distribution aren't 'unique' in this sense. However: all we care about are probabilities, and the underlying space isn't important to talk about these (in elementary contexts).

If you want to understand this fully, you need to take a course in measure-theoretic probability theory. But the take home message is: if you just care about probabilities, you can assume there exists a 'formal set-up' that enables you to do so.

Answer 2

The probability distribution tells you, as you have mentioned, the probability you get a certain value. When you talk about binomial, I guess you mean the continuous binomial distribution. For continuous distribution you have the gotcha that you cannot talk anymore whether what the probability of a single value is (this would be zero), but instead you have to ask, what the probability is, that you are between a lower value $a$ and an upper value $b$. This probability is then: $P(a < X < b) = \int_a^b f(x)dx$, where $f$ is the probability density function.

I hope this eases your confusion a bit.

Answer 3

It is actually not true that you can find $P(X=x)$ by simply stating that the variable is random, or even by stating that the variable is random and giving its sample space.

What may be true is that the binomial distribution is the first one you have encountered that is not a uniform discrete distribution. After all, fair coins, fair dice, bags from which you draw a single colored ball, and well-shuffled decks from which you draw a single card all are typically assigned uniform distributions (distributions in which every point of the sample space has an equal probability to occur).

It seems many people have the experience of learning their first few examples of probability this way and drawing the conclusion that "random" means that every point in the sample space has equal probability. This can lead to erroneous results when someone with this notion constructs a sample space over which the desired variable actually should not have a uniform distribution.

So a sample space and the word "random" are not enough information to say anything about probability. On the other hand, if you know a distribution, it practically implies what the sample space is. (If the distribution says that $P(X=2) = \frac13,$ for example, you know that $2$ is in the sample space.) So the distribution usually has the information you need.