4

I frequently see Bayes Theorem phrased in two different ways.

Simple Bayes Theorem: $$P(X|Y) = \frac{P(X)P(Y|X)}{P(Y)}$$

Complex Bayes Theorem: Let $X_1, \dots, X_k$ be a partition of the sample space. $$P(X_i|Y) = \frac{P(X_i)P(Y|X_i)}{\sum_{j = 0}^k P(X_j)P(Y|X_j)}$$

The second version follows pretty quickly from the first by noticing $$\sum_{j = 1}^k P(X_j | Y) = 1.$$

Thus $$P(Y) = \sum_{j = 1}^k P(Y)P(X_j | Y) = \sum_{j = 1}^k P(X_j)P(Y | X_j)$$

My question is, why is the second version even mentioned? Is it used in that form frequently? How often can I easily find $P(X_j)P(Y|X_j)$ for each $j$, but not know $P(Y)$ off hand?

zrbecker
  • 4,048
  • 1
    That's an interesting way to derive the denominator. I would have just done $P(Y)=\sum P(Y\cap X_j)=\sum P(X_j)P(Y|X_j)$. –  Apr 21 '17 at 14:44

1 Answers1

2

It is the application, the scenario that is more often encountered, that makes the second version important.

We are talking of conditional probabilities here. Bayes theorem attempts to say of condition reversal.

A factory has many machines manufacturing a product with a small probability of defect that is a characteristic of individual machines. This is $P(D|M_i)$. (Conditional probability that a piece is defective given it is from machine $M_i$)

Once the product is sold in the market, and we got a defective product we want to know if it is from a specific machine. (We want to fix the blame!) It is the conditional probability: given it is defective, the probability that $M_j$ manufactured it.

Remember that in this type of scenario $P(D|M_i)$ is more natural (that is machine-wise defective rate) than the unconditional probability $P(D)$, the probability that a piece is defective.

There is a categorisation in the population and the probability is known (or easier to ascertain) within each category. The conditional probability, naturally fits this situation. And this type of categorisation is what we might have access to in real life situations. Many times data might be owned by each category only some summary of the data might be available to others: probability that a grade 8 student knows how to write a computer program: this might depend on individual schools.

P Vanchinathan
  • 19,504