4

Suppose we have a probability space $(\Omega, \mathcal{F}, P)$. Thus, when we write $P(A \vert B)$, either $A \vert B$ is an element of the event space $\mathcal{F}$ (i.e. the domain of $P$), or this an abuse of notation. I suspect that this is simply an abuse of notation, but I'm looking for two things here:

  1. Am I interpreting (that is "unabusing") this notation correctly?
  2. Is it possible to view $A \vert B$ as an event, even if this isn't the standard way of thinking about it?

Notational Abuse?

Entertaining the second possibility (notational "abuse") I can easily imagine how to "desugar" this notation into something more formal. For instance, we could read $P(A \vert B)$ as $P_B(A)$, where $P_B$ is a probability measure on a new/derived probability space $(B, \mathcal{F}_B, P_B)$. That is, this new space has $B$ as its sample space and a different event space derived from our original event space. We can then define $P_B(A)$ in terms of our original probability measure $P$ as: $P_B(A) = \frac{P(A \cap B)}{P(B)}$. I haven't yet worked out the details (e.g. What is $\mathcal{F}_B$?, Is $P_B$ a probability measure?, etc.), but I suspect they're not too difficult.

Conditional Events?

But, is it possible to view $A \mid B$ as an event (that is, some element of $\mathcal{F}$)? This section of the Wikipedia article on Conditional Probability states that

Conditional probability can be defined as the probability of a conditional event $A_B$.

it then proceeds to define the "Goodman–Nguyen–Van Fraassen conditional event", but I haven't been able to decipher this yet.

Possible Counterexample

However, there appear to be simple examples in which $A \mid B$ just can't be an event. For instance, suppose we have a sample space $\Omega = \{ TT, TH, HT, HH \}$ and we define a probability measure using the mass function $p : \Omega \to [0, 1]$ like so:

$$ \begin{cases} p(TT) &= \frac{1}{6} \\ p(TH) &= \frac{1}{6} \\ p(HT) &= \frac{1}{6} \\ p(HH) &= \frac{1}{2} \end{cases} $$

We can think of this as some experiment involving flipping two coins that are biased in some strange way that makes double-heads more likely than other outcomes. Then the probability that ($A$) both coins land heads up given ($B$) the first coin lands heads up is

$$ \begin{align*} P(A \mid B) &= \frac{P(A \cap B)}{P(B)} \\ &= \frac{P(\{ HH \} \cap \{ HT, HH \})}{P(\{ HT, HH \})} \\ &= \frac{P(\{ HH \})}{P(\{ HT, HH \})} \\ &= \frac{p(HH)}{p(HT) + p(HH)} \\ &= \frac{\frac{1}{2}}{\frac{1}{6} + \frac{1}{2}} \\ &= \frac{3}{4} \end{align*} $$

However, no subset of the sample space has probability $\frac{3}{4}$, so $A \mid B$ can't be an event in the original probability space.

Have I made a mistake in this example, or am I misunderstanding the basic idea of "conditional events"?

Pedro
  • 122,002
William Lewis
  • 163
  • 7
    This is a lot of text that I have not read fully yet, but answering the question in the title and first few lines... $A\mid B$ is not an event. You could if you wanted rewrite $\Pr(A\mid B)$ as $\Pr_B(A)$ where $\Pr_B$ is the probability measure over the probability space corresponding to $B$ as the sure event and everything scaled accordingly. – JMoravitz Dec 12 '23 at 15:06
  • 1
    "Is this an abuse of notation" Not really, no. It is well understood (by most) that this notation merely adds context to what it is that it describes and everyone familiar with the notation understands that $A\mid B$ is not itself an event, but rather $A$ is the event and $B$ is the event it is being conditioned on. Suggesting that it is an abuse of notation seems to me like suggesting the question mark in "He threw the ball." vs "He threw the ball?" is an abuse of notation since the meaning of the two sentences changes based on the presence or absence of the question mark. – JMoravitz Dec 12 '23 at 15:09
  • Thanks @JMoravitz! By abuse of notation I don't mean that it isn't well-understood, but rather that it isn't "formally valid". For instance, if someone told me that $f : \mathbb{N} \to \mathbb{N}$, and then proceeded to define $f(\infty)$, I'd call that an abuse of notation because $\infty \not\in \mathbb{N}$. "Abuse" has such a negative connotation, but I'm not implying there's anything harmful about it. – William Lewis Dec 12 '23 at 15:15
  • 2
    I suppose... but again, it is well understood and well defined notation that if written as $\Pr(~\cdot~ \mid ~\cdot~)$ that this is to be treated as a function that takes multiple inputs... the first of which being the event we are interested in finding the probability of, and the second of which following the bar being the event we are conditioning things on. Is this written differently than if we had used a comma like usual for multi-input functions? Sure, but it helps to keep it separate from $\Pr(A,B)$ notation which does appear in textbooks. – JMoravitz Dec 12 '23 at 15:19
  • 1
    It just so happens that it is traditional now to use a bar rather than a comma to represent that this is a multiple-input function. If you are interested further in the history of the notation... you'll want to look at Kolmogorov (1933), Upensky (1937), Feller (1950), and similar. In particular, the vertical bar notation was popularized from Feller. – JMoravitz Dec 12 '23 at 15:23
  • 1
    I would say it's not abusing the existing $P(A)$ notation, merely extending it. Once you realize that $A \mid B$ is not an event, then there is no ambiguity between the two. – Nate Eldredge Dec 12 '23 at 16:40
  • I can also tell you that while CEA may be a valid formalism to interpret $P(A \mid B)$ as a function acting on a well-defined mathematical object (which is not an event in the usual sense of the word), I don't know of any working probability theorists who actually work within in that formalism. – Nate Eldredge Dec 12 '23 at 16:43
  • 1
    Simply because $P(X\vert Y)$ also uses $P$ doesn't mean it is the same function as $P(X)$ where $X$ is in your $\sigma$-algebra, in the same way that $(x_1,x_2,x_3)$ is not an abuse of notation if you are also using $(x_1,x_2)$ because "$(x_2,x_3)$ is not a scalar". The conditional probability $P(X\vert Y)$ is a function of two events, not one. Granted the duplication of $P$ might be confusing, but this happens. For example, $\int_A f(x,y)dxdy$ is a perfectly valid notation, even though $\int_I f(t)dt$ also is, but $A, f$ in the first are completely different from $I$ and $f$ in the second. – Pedro Dec 12 '23 at 16:50
  • Thanks, everyone! I think I understand now (the tuple example did it for me, @Pedro). I was being too narrow-minded about how I interpreted $P(\ldots)$: I was thinking that this referred exclusively to applying $P$ to an event; I don't see any issue with reading $P(\cdot)$ as application and $P(\cdot \mid \cdot)$ as shorthand for the conditional probability. Please forgive my obstinance; I'll be more diplomatic about these kinds of questions in the future :P – William Lewis Dec 12 '23 at 22:17

1 Answers1

4

$A|B$ is not an event.

It is also not considered an abuse of notation to write $P(A|B)$. Rather, $P(A)$ for the unconditional probability of an event and $P(A|B)$ for the conditional probability of one event given another event are considered different, related notations. (As one comment puts it, this is a storm in a teacup, but if you double down on calling it notational abuse, you'll be in the minority.)

There are several different ways of defining the meaning of the notation $P(A|B)$ on a probability space $(\Omega,\mathcal{F},P)$, and they are all rigorous, correct, and "mostly" equivalent. The simplest way of defining it is the same way it's defined in basic probability courses. If $A,B\in\mathcal{F}$ are events, then: $$P(A|B) := \frac{P(A\cap B)}{P(B)}\qquad (1)$$ Note that $A\cap B\in \mathcal{F}$, since $\mathcal{F}$ is closed under intersections, so the numerator always makes sense. If $P(B)=0$, then it's pretty much dealer's choice -- we can leave $P(A|B)$ undefined or take it to be zero. It doesn't much matter.

(Update: As an important aside, you sometimes see things like the following. Let $X$ and $Y$ be iid continuous random variables (e.g., standard Gaussian) and define $S=X+Y$. Now, what do we mean by $P(S>0|X=1)$? You'd have a reasonable argument here that this is abuse of notation. This conditional probability is not given by definition (1) because $X=1$ is not supposed to be interpreted as the zero-probability event $P(X=1)=0$ here!)

There is no need to define a subspace $(B,\mathcal{F}_B,P_B)$ first, but you can. If you do, then you'll want to ensure that $P(B)>0$ and take $$\mathcal{F}_B := \{A \cap B : A \in\mathcal{F}\}$$ and $$P_B(A) = \frac{P(A)}{P(B)}, A\in\mathcal{F}_B$$ Note that, by the definition of $\mathcal{F}_B$, we have $P(A)=P(A\cap B)$ for all $A\in\mathcal{F}_B$, so the numerator can be written either way.

While it's not completely obvious, $\mathcal{F}_B$ is a $\sigma$-field on $B$, just like $\mathcal{F}$ was a $\sigma$-field on $\Omega$, and $P_B$ is a probability measure on the measure space $(B,\mathcal{F}_B)$. (Proving this would not be out of place as a quick exercise in a graduate-level probability text.) So, $(B,\mathcal{F}_B,P_B)$ is a probability space equipped with a probability measure $P_B(\cdot)$ that acts a lot like $P(\cdot|B)$.

We could now proceed to define $P(A|B)=P_B(A)$, but there's a problem with this definition. For all $A\in \mathcal{F}_B$, the quantity $P_B(A)$ is defined and equal to $P_B(A)=P(A\cap B)/P(B)$, so it matches the usual definition of $P(A|B)$. But, for $A\in\mathcal{F}\setminus\mathcal{F}_B$, the probability measure $P_B(A)$ isn't defined, even though $P(A|B)$ should be. That is, we want $P(\cdot|B)$ to be a probability measure on the entire space $(\Omega,\mathcal{F})$, not just $(B,\mathcal{F}_B)$. We could define: $$P(A|B) := P_B(A\cap B), A \in\mathcal{F}$$ but since this is exactly the same as the definition (1) above, it's not clear why we bothered defining a subspace first.

The concept of a "conditional event" involves transforming the entire probability space $(\Omega,\mathcal{F},P)$ into a new space $(\Omega^\star,\mathcal{F}^\star,P^\star)$ equipped with some function $C\colon \mathcal{F}\times\mathcal{F}\to\mathcal{F}^\star$ satisfying $P(A|B) = P^\star(C(A,B))$ for all $A,B\in\mathcal{F}$. In this context, $C(A,B)$ is a true event in the new probability space that "acts" like $A|B$ in the original probability space. It seems like an interesting construction, but I've never seen it used anywhere, and it doesn't make $A|B$ an event in the original space.

Your counterexample shows a situation where $A|B$ can't be an event in the original space, but applying this construction would involve defining $\Omega^\star$ to be infinite sequences of elements from $\Omega=\{TT,TH,HT,HH\}$ equipped with a probability measure equivalent to generating iid realizations from $\Omega$ (i.e., an infinite sequence of independent experiments that involve flipping two coins).

The conditional event $C(A,B)$ would then be defined as the subset of all sequences where the first occurrence of event $A\cap B$ comes before the first occurrence of $A\setminus B$ (for your example, where the first $HH$ comes before the first $HT$), and if you calculated the probability of this "event" (this subset of sequences), you'd discover that it has probability 3/4. (Specifically, if we search a random sequence for the first occurrence of either $HH$ or $HT$, by your assumed original probabilities, it's 3 times more likely to be $HH$ than $HT$, so the first of these will be $HH$ 3/4 of the time.)

K. A. Buhr
  • 1,672