Intuitive Understanding of Independence of 3 Events

Question

I understand what it means to say that an even E is independent of an event F, it means that knowing that F has occurred does not change the probability that E will occur. Algebraically, it implies:

$$P(E) = P\left(E \mid F\right) \stackrel{\text{def}}{=} \frac{P\left(E \cap F\right)}{P\left(F\right)} \Leftrightarrow P\left(E \cap F \right) = P\left(E\right) P\left(F\right)$$

Now I'm trying to tackle what it means for events E, F, and G to be independent, the definition stated in the book says:

$$\begin{gathered} P\left(E \cap F \cap G\right) = P\left(E\right) P\left(F\right) P\left(G\right)\\ P\left(E \cap F\right) = P\left(E\right) P\left(F\right) \\ P\left(E \cap G\right) = P\left(E\right) P\left(G\right) \\ P\left(F \cap G\right) = P\left(F\right) P\left(G\right)\end{gathered}$$

Based on my previous understanding the bottom three lines would be implying that E is independent of F, E is independent of G, and that F is independent of G. But I'm not sure what the first line is saying. The way I've made sense of it so far is that:

$$ P\left(E \cap F \cap G\right)= P\left( E \cap \left( F \cap G \right) \right) \stackrel{?}{=} P\left(E \right) P\left(F \cap G\right) \stackrel{\alpha}{=} P\left(E\right) P\left(F\right)P\left(G\right)$$

(Where $\alpha$ comes from the fact that E and F were independent.)

In order for the equality with the question mark to hold, I think we would need $E$ to be independent from $F \cap G$.

I am a little confused by their definition and tried to figure it out algebraically, but that's what I've gotten up to.

I am hoping someone can:

• Give me an intuitive understanding of what it means for 3 events to be independent
• Help me connect this understanding to the definition they have provided
• Bonus: Help me take that understanding to the independence of $n$ events

Answer 1

For three events to be independent, knowledge of the others should not affect the probability of each event. For pairwise independence, knowing one other event tells you nothing. But knowing both other events might shift the odds. For example: draw 3 overlapping circles. Put a 1 in the 3 way overlap. Put 7 in each of the 2-way overlaps that are not in the center. Put a 1 in each of the regions belonging to exactly one set. Put a 7 in the complement of the union. The total is 32; each set contains 16, each 2-way overlap contains 8, so for example $A$ and $B$ are independent since $P(A) = \frac{1}{2}, P(B) = \frac{1}{2},$ and $P(A\cap B) = \frac{1}{4}$. Likewise for the other pairs, so we have full two way independence. But 3-way independence fails: $\frac{1}{8} = P(A)P(B)P(C) ≠ P(A\cap B \cap C) = \frac{1}{32}$.

Now what does this mean? Suppose I know $A$ is true. $P(C)$ is unaffected, since half of the elements of $A$ are in $C$.

Now uppose I know $A$ is true and $B$ is true. This makes $P(C)$ drop from $\frac{1}{2}$ to $\frac{1}{32}$. Knowing one of the other truth values told you nothing, but knowing both of them does give you information that changes the probability.

Answer 2

It can help to compare the definition of independent events with the concept of disjoint events.
Let's say I have some collection of $k$ events, $E_1, E_2, ..., E_k$, in a sample space of outcomes $\Omega$.

The formal definition of what it means for these events to be disjoint is that, for every subset $\{ j_1, j_2, ..., j_r \} \subseteq \{ 1, 2, ..., k \}$, we have $$ E_{j_1} \cap ... \cap E_{j_r} = \emptyset.$$ Disjointness means these events are mutually exclusive: one of these events happening excludes any of the others from happening.
In order for the whole group of $k$ events to be disjoint, it suffices for them to be pairwise disjoint: if I know $E_m \cap E_n = \emptyset$ for all $1 \leq m < n \leq k$, then I have that the whole collection of events $E_1, E_2, ..., E_k$ is disjoint as well, since any $3$-way, $4$-way, $5$-way, etc. intersection of events is contained in some $2$-way intersection by definition. For this reason, many sources will define disjointness using the condition for pairwise disjointness, since it is equivalent but easier to check.

The formal definition of what it means for these events to be independent is that, for every subset $\{ j_1, j_2, ..., j_r \} \subseteq \{ 1, 2, ..., k \}$, we have $$\Bbb{P}(E_{j_1} \cap ... \cap E_{j_r}) = \Bbb{P}(E_{j_1}) ... \Bbb{P}(E_{j_r}).$$ This does actually imply that every event is independent of any intersection of the other events, as per your question, and we want our definition of independence to have this property. That question-marked equality is a consequence of the definition, and not a thing that needs to be proven.

The reason is that, even if a group of events is pairwise independent, dependencies can come up in groups of more than $2$ events that would not be apparent from looking at pairs of events individually. If I want to be able to generalize my way of calculating probabilities of intersections of independent events to groups of more than $2$ events, I have to build it in as part of the definition, because it will not be implied by the pairwise property, as it was for disjointness.

Simple example: I flip a fair coin 3x. Every individual flip is independent--there's no possible effect they can have on each other. Let event $A$ = "first and second flips are the same," event $B$ = "second and third flips are the same", event $C$ = "first and third flips are the same". For these events, we have $$\Bbb{P}(A) = \Bbb{P}(B) = \Bbb{P}(C) = \frac{1}{2},$$ and also $$\Bbb{P}(A \cap B) = \Bbb{P}(A \cap C) = \Bbb{P}(B \cap C) = \frac{1}{4},$$ so these events are pairwise independent. However, they are not fully independent, because knowing whether both $A$ and $B$ happened tells me whether $C$ happened, since $$C = (A \cap B) \cup (A^c \cap B^c).$$ And in fact, $\Bbb{P}(A \cap B \cap C) = \frac{1}{4}$, not $\frac{1}{8}$ as it would be if we multiplied $\Bbb{P}(A)$, $\Bbb{P}(B)$, $\Bbb{P}(C)$. That's why our definition has to include the $k$-way intersections for $k > 2$ explicitly, and can't just use the two-way intersections. When one event is completely controlled or determined by a group of other events in this way, we don't want to consider the group as a whole "independent", even if the events in the group are independent in pairs.