9

Quote from the book I'm reading:

Any collection of possible outcomes, including the sample space $\Omega$ and its complement, the empty set $\emptyset$, may qualify as an event. Strictly speaking, however, some sets have to be excluded. In particular, when dealing with probabilistic models involving an uncountably infinite sample space; there are certain unusual subsets for which one cannot associate meaningful probabilities.

Question 1

  • What is meant by "meaningful" probabilities?

Question 2

  • Can you provide an example in which we cannot assign meaningful probabilities to the events of the sample space?
Stokolos Ilya
  • 3,371
  • 5
    I suppose, that in addition to the notions mentioned in the other comments, this also refers to the fact that when working for example with the reals as a probability space, you commonly have to exclude some sets from the sigma-algebra (i.e. The collection of all sets you can assign probabilities to) These sets will however be not very intuitive and may not have a good example that you can put in colloquial terms. You can however read up on this: https://en.m.wikipedia.org/wiki/Non-measurable_set – a_student Apr 27 '20 at 12:00

4 Answers4

14

Say your sample space is $\Omega=[0,1]$, and you want want to define probabilities on the possible events $X$ according to a distribution function $F(x)$. You are thus looking for a function $\mathbb{P}$ that assigns a probability to every subset $X \subset \Omega$ such that

  1. $\mathbb{P}(X) = F(b) - F(a)$ if $X=[a,b]$

and such that the probabilities assigned by $\mathbb{P}$ are meaningful in the sense that they obey the usual laws for probabilities,

  1. $\mathbb{P}(\emptyset)=0$, $\mathbb{P}(\Omega) = 1$, $0 \leq \mathbb{P}(X) \leq 1$ for all $X \subset \Omega$,

  2. $\mathbb{P}(X_1 \cup X_2 \ldots) = \mathbb{P}(X_1) + \mathbb{P}(X_1) + \ldots$ for all disjoint sequences $X_1,X_2,\ldots \subset \Omega$ (disjoint means $X_i \cap X_j = \emptyset$ if $i \neq j$, and note that countably infinite sequences are allowed!).

Even for some very well-behaved $F$ (e.g. for $F(x)=x$, it's hard to imagine a more well-behaved function than this), this turns out to be not possible. There simply isn't a function $\mathbb{P}$ that assign each subset of $[0,1]$ a probability (i.e. a real number between 0 and 1) such that the requirements above are fulfilled.

But it is possible if we exclude certain very weird and hard to imagine sets. All of these sets require the axiom of choice to even construct them, so you may imagine them to be artifacts of mathematical set theory, and not sets that you ever want to actually compute a probability for (unless you're a set theorist, maybe). Such sets are called non measurable.

Vitali sets are examples of non measurable sets, but unfortunately their construction requires a bit of number theory.

fgp
  • 21,050
  • Can you explain what is the problem if our measurable space is $(\mathbb R, \mathcal P(\mathbb R))$ and we take the Dirac measure at $0$? Why is this assignment meaningless? –  Apr 27 '20 at 16:06
  • 4
    @Shahab is not meaningless. For any set $S$ it's always possible to define de Dirac measure on $\mathcal{P}(S)$. But this is not always useful. What Durrett says is that, given a certain probability measure on certain subsets $\Omega$, it's not always possible to extend this measure to all subsets of $\Omega$. fgp gives an example of certain measures in $[0,1]$ that CANNOT be extended to all subsets of $[0,1]$. That does NOT mean that there aren't measures defined on all subsets of $[0,1]$. – Célio Augusto Apr 27 '20 at 21:22
  • @Shahab It's true that you can always pick (countably many) points $\omega_1,\omega_2,\ldots \in \Omega$ and probabilities $p_1,p_2,\ldots \geq 0$ with $\sum_i p_i = 1$, and define $\mathbb{P}(X)$ for all $X  \subset \Omega$ as the sum of all $p_i$ where the corresponding $x_i$ lie in $X$. However, you don't really have an uncountable sample space in that case, you have the countable space ${\omega_1,\omega_2,\ldots}$ plus a lot of sets in between these $\omega_i$ that have probability 0. For truly uncountable sample space, you always have to exclude some sets to define probabilities. – fgp Apr 28 '20 at 17:40
1

  1. I think, in this context a meaningful assignment of probabilities is an assignment of probabilities to events which make our model a good model.

  2. Suppose, we mix certain chemicals in the lab and get a solution. Assume that due to some constraints we cannot ascertain the exact composition of the solution. Fortunately however, let us presume that we were only interested in whether the solution turns red or blue. On the basis of empirical evidence we may conclude that the solution turns red with probability $p$ and blue with probability $1-p$. Clearly only four events have been assigned probabilities here: the empty set has probability $0$, the set of all outcomes has probability $1$, the event that the solution turns red has probability $p$ and the event that the solution turns blue has probability $1-p$. Other events (for example a subset of the red solutions) do not have any probability assigned to them because we cannot gauge with how much chance that happens. If we forcibly assign them probabilities as per our whim, then even if the resultant structure is a probability space, our model may not be a good model.

0

This example is fairly contrived but I hope it conveys the point. Let's suppose we have a continuous random variable $X$ distributed over its sample space according to some probability density function $f$. We'll take the sample space of $X$ in this example to simply be $\mathbb{R}$ for simplicity. Using our knowledge of probability theory, the event space of $X$ is then $\mathcal{P}(\mathbb{R})$, the powerset of the sample space. Technically speaking, any set $S \in \mathcal{P}(\mathbb{R})$ should have some associated probability, for instance $P(X \in (-1,1))=\int ^{1}_{-1} f( x) dx$ where the interval $(-1,1)$ is clearly an element of $\mathcal{P}(\mathbb{R})$. However, other elements of $\mathcal{P}(\mathbb{R})$ don't have meaningful probabilities. Take for instance the subset $\{1,2,3\}$. $P(X \in \{1,2,3\})=\int ^{1}_{1} f( x) dx \ +\int ^{2}_{2} f( x) dx\ +\int ^{3}_{3} f( x)dx=0.$ If you want to take a measure-theoretic approach to the matter, any subset $S$ with measure $0$ in $\Omega$ will have probability $0$ and therefore may be discarded.

K.defaoite
  • 12,536
0

I'm going to continue fgp's answer and describe a set without a meaningful probability. Our set is a maximal subset of $[0,1]$ so that no two elements differ by a rational number (this requires the axiom of choice to exist). Call this set $V$.

For any small number $\epsilon$, we could have chosen all of our elements to be in the subinterval $[0,\epsilon]$. This shows that our set has probability at most $\epsilon$, so that the only possible probability our set could have is zero.

Now, for any number $x$ in $[0,1]$, it must differ from some $v$ in $V$ by a rational number $r$ (otherwise, we should have started with $V\cup\{x\}$). This shows that $[0,1]\subseteq\cup_{r\in\mathbb{Q}}(r+V)$: if we shift $V$ by all possible rational numbers, then we must cover all of $[0,1]$. Furthermore, this $v$ and $r$ must be unique (otherwise, the other $v'$ would have differed from our $v$ by a rational amount, which was disallowed by the construction of $V$). This shows that the $r+V$ are all disjoint.

But this means that $$1 =P([0,1])\le P\bigl(\bigcup_{r\in\mathbb{Q}}(r+V)\bigr) =\sum_{r\in\mathbb{Q}}P(r+V) =\sum_{r\in\mathbb{Q}}P(V) =\sum_{r\in\mathbb{Q}}0= 0,$$ which is a contradiction.

A more abstract example would be a maximal subset of $[0,1]$ which has probability zero (which again requires the axiom of choice to exist). Call this set $U$. We can't have $U=[0,1]$ (that would make its probability 1), so there is some point $x$ in $[0,1]\setminus U$. But then $P(U\cup\{x\})=P(U)+P(\{x\})=0$ is a larger set that still has probability zero, again a contradiction.

Teepeemm
  • 812