11

Consider X as a continuous random variable which can assume any value in [0, 1]. It is known that P(X=x)=0 where P is the probability density function. I want to understand this intuitively.

The math insight article helps me somewhat:

In other words, the probability that the random number X is any particular number x∈[0,1] (confused?) should be some constant value; let's use c to denote this probability of any single number. But, now we run into trouble due to the fact that there are an infinite number of possibilities. If each possibility has the same probability c and the probabilities must add up to 1 and there are an infinite number of possibilities, what could the individual probability c possibly be? If c were any finite number greater than zero, once we add up an infinite number of the c's, we must get to infinity, which is definitely larger than the required sum of 1. In order to prevent the sum from blowing up to infinity, we must have c be infinitesimally small, i.e., we must insist that c=0.

But, what if I choose, c=1/N (where N is that large number) for the uniform case? The sum will still be 1 as far as I can understand. For the non-uniform case, I can pick some 0's and others non-zeros and still be theoretically able to get a sum of 1 for all the possible values.

Anything that helps me understand this clearly will be of immense help. The answer here helps but I still don't get it.

Community
  • 1
Amit
  • 213

3 Answers3

10

A continuous random variable can realise an infinite count of real number values within its support -- as there are an infinitude of points in a line segment.

So we have an infinitude of values whose sum of probabilities must equal one. Thus these probabilities must each be infinitesimal. That is the next best thing to actually being zero. We say they are almost surely equal to zero.

$$\Pr(X=x) = 0 \text{ a.s.}$$

( To have a sensible measure of the magnitude of these infinitesimal quantities, we use the concept of probability density, which yields a probability mass when integrated over an interval. This is, of course, analogous to the concepts of mass and density of materials. )

$$f_X(x) = \frac{\mathrm d}{\mathrm d x}\Pr(X\leq x)$$

For the non-uniform case, I can pick some 0's and others non-zeros and still be theoretically able to get a sum of 1 for all the possible values.

You are describing a random variable whose probability distribution is a mix of discrete (massive) points and continuous intervals. This has step discontinuities in the cumulative distribution function.

$\Pr(X\leq x) = \begin{cases} 0 & x < 0 \\ 0.25 & x=0 \\ 0.25+x/4 & 0< x< 1/2 \\ 0.75 & x=1/2 \\ 0.5+x/4 & 1/2< x< 1 \\ 1 & x\geq 1\end{cases}\\[2ex] \Pr(X=x) = \begin{cases} 0 & x<0 \cup x> 1 \\ 0.25 & x=0 \cup x=1/2 \cup x=1 \\ 0\text{ a.s.} & 0<x<1/2 \cup 1/2<x< 1 \end{cases}$

Graham Kemp
  • 129,094
  • Thank you for the explanation. I like your explanation. However, for the non-uniform case, what if I reframe my question to ask that I can have different infinitesimal values instead of being exactly 0? Will the same reason as uniform probability apply? That, moving to uncountably infinite space, the values are infinitesimal, and for practical purposes, close to 0? – Amit May 01 '15 at 12:35
  • 2
    Please see the comments under this question concerning the use of "almost surely" in this answer. – joriki Jul 07 '18 at 13:21
  • what a.s. means? – ESCM Jun 26 '19 at 21:26
  • "almost surely" @EduardoS. – Graham Kemp Jun 26 '19 at 22:40
5

The situation is easier to formalize when we say that there is no uniform distribution on a countably infinite set $S$. This is because if there were, then $P(X=x)=c>0$ for every $x$, and now

$$P(X \in S)=\sum_{x \in S} c = +\infty.$$

This follows from the property of countable additivity of probability, which is usually treated as an axiom. I think that at least finite additivity of probability should be intuitively obvious, and countable additivity is a straightforward extension.

In the uncountable case, we need one more result, which can be proven from the countable additivity axiom: if $A \subset B$ then $P(A) \leq P(B)$; so now if $S$ is uncountable, and we are to assign a uniform distribution to it, then we can extract a countably infinite subset $C$. Then

$$P(X \in S) \geq P(X \in C) = \sum_{x \in C} c = +\infty$$

as before.

Ian
  • 101,645
  • Thanks Ian. I will think more about this, I feel I have got a new way of thinking about it. – Amit May 01 '15 at 00:36
  • what about the non-uniform case? – Amit May 01 '15 at 00:37
  • @Amit In the non-uniform case, if you take for granted that the CDF of a continuous variable is continuous, then the situation is simple: $P(X=x)=\lim_{y \to x^-} F(x)-F(y)=0$. The first equality is general, while the second equality is from continuity. – Ian May 01 '15 at 00:42
1

Use the fact that if $A \subset B$, then $p(A) \le p(B)$.

If $X$ is a continuous random variable, this means that the cdf is continuous, that is $f(\alpha) = p(\{\omega | X(\omega) \le \alpha \})$ is continuous.

It follows that $p((a,b]) = f(b)-f(a)$.

Suppose $a < x \le b$. Then we have $p((a,b]) = f(b)-f(a)$.

Since $\{x\} \subset [a,b]$ for all such $a,b$, we have $p(\{x\}) \le f(b)-f(a)$ for all such $a,b$. Hence $p(\{x\})=0$, since $f$ is continuous.

Nikolas
  • 58
copper.hat
  • 172,524
  • Technically, because you are allowing $x=b$, this is essentially begging the question, since $[x,x]={ x }$. But everything works out provided you assume $x<b$ (and separately manage the case $x=1$). – Ian May 01 '15 at 00:06
  • Everything works out the way it is above. I don't understand why you want $x<b$? Containment is all that is needed here. – copper.hat May 01 '15 at 00:08
  • By saying that $P([a,b])=b-a$ for $a \leq b$, I can just take $a=b=x$ to get $P([x,x])=P({ x })=0$. So you are actually just asserting the claim without justifying it. – Ian May 01 '15 at 00:09
  • @cooper.hat sorry, I am looking for an answer with less math, and more intuition. – Amit May 01 '15 at 00:09
  • The point was not to compute the probability explicitly, but to show that it must be zero because it is contained in increasingly unlikely events. This was meant to help intuition, not computation. – copper.hat May 01 '15 at 00:10
  • Oops, somewhere I thought is said uniform distribution. @Ian: I understand what you were saying now. – copper.hat May 01 '15 at 00:12