0

I have a vector $x$ of random variables and a finite set of classes $C$. Also $x$ has to belong to one class. In the context of classification problems I have the transformation

$P(x) = P(x|c_1)P(c_1) + P(x|c_2)P(c_2) \ \cdots P(x|c_n)P(c_n)= \sum_{c \in C}P(x|c)P(c)$

Why is this equality true?

Thank you very much!

elfeck
  • 427
  • What is the exact meaning of $P(x)$? It will stand for a probability, but of wich event? If $x$ is a vector of random variables then it is not an event. – drhab Jun 03 '14 at 10:23
  • x is a vector of discrete random variables (which in my case have binary state). P(x) is the probability of the vector having its exact configuration (out of all possible finite configurations). – elfeck Jun 03 '14 at 10:27

2 Answers2

0

In this answer $X$ stands for a random vector and $x$ is one of the values that it can take. I am not sure whether this meets your question (as my comment makes clear).

Maybe it will be of help anyhow. If not then let me know. In that case I will delete it.

Formally $P\left(X=x\mid c_{i}\right)P\left(c_{i}\right)=P\left(X=x\wedge c_{i}\right)$ where $c_{i}$ stands for the event that $X$ belongs to class $i$.

Then $P\left(X=x\mid c_{1}\right)P\left(c_{1}\right)+\cdots+P\left(X=x\mid c_{n}\right)P\left(c_{n}\right)=P\left(X=x\wedge c_{1}\right)+\cdots+P\left(X=x\wedge c_{n}\right)$

Also $P\left(X=x\right)=P\left(\left[X=x\wedge c_{1}\right]\vee\cdots\vee\left[X=x\wedge c_{n}\right]\right)=P\left(X=x\wedge c_{1}\right)+\cdots+P\left(X=x\wedge c_{n}\right)$

This because $X$ must belong to exactly one class.

So LHS of first equality equals LHS of second equality.

drhab
  • 151,093
0

It is due to the Law of Total Probability.

If the members of a set, $\{B_i\}$, are mutually exclusive and exhaustive events, then for any event $A$:

$$P(A) = \sum\limits_{B_i \in \{B_i\}} P(B_i)P(A\mid B_i)$$

Since each instance of $\vec X$ belongs to one and only one of the classes in $C$, we do have a set of mutually exclusive and exhaustive events.

$$\begin{align}\mathbb{\large P}(\vec X=\vec x) & = \mathbb{\large P}(\mathop{\large\cup}_{c\in C} (\vec X \in c\cap \vec X=\vec x)) \\ & = \sum\limits_{c \in C} \mathbb{\large P}(X\in c)\cdot \mathbb{\large P}( \vec X=\vec x \mid \vec X\in c) \\ & \color{grey}{= \mathbb{\large P}(\vec X\in c_1)\mathbb{\large P}(\vec X=\vec x\mid \vec X\in c_1)+\cdots+\mathbb{\large P}(\vec X\in c_n)\mathbb{\large P}(\vec X=\vec x\mid \vec X\in c_n) }\end{align}$$

Graham Kemp
  • 129,094