1

Say we have two random variables $X\sim B(p_1),\ Y\sim B(p_2)$ where $B(p)=$ Bernoulli with probability $0\le p\le 1$. I am interested in the case when the correlation $\rho$ of $X,Y$ tends to $1$.

If we set events $A=\{X=1\}$, $B=\{Y=1\}$, the conditional probability property gives us $$\begin{align}\tag{1} P(A\cap B) = P(A\mid B)\cdot P(B)\\ = P(B\mid A)\cdot P(A) \end{align}$$ When $\rho=1$ we must have $P(A\mid B)=P(B\mid A)=1$ (since one event implies the other). Equation $(1)$ then yields $$P(A) = P(B)$$ Furthermore, if correlation is high (= tending to 1), then whenever event $A$ occurs then $B$ must also occur (and vice versa). So the probabilities of $A,B$ should be $$P(A)=P(B)=\max(p_1,p_2)\tag{2}$$ However, for any $\rho=1-\epsilon$ with $\epsilon>0$, we still have $P(A)=p_1$ and $P(B)=p_2$ as per definition.

So can $(2)$ even be correct? What am I missing?

Unfortunately, Joint distribution of dependent Bernoulli Random variables only discusses non-deterministic sequences, so it doesn't quite apply.

Phil-ZXX
  • 3,194
  • "we still have $P(A)=p_1$ and $P(B)=p_2$". Try $p_1=0.2$ and $p_2=0.8$ and try to get a value of $\rho$ being 0.9. Can't do it. Given $p_1$ and $p_2$, there are certain values of $\rho$ that can't be obtained. So you can't fix $p_1$ and $p_2$ and take the limit of $\rho$ -> $1$. Hint: start with the definition of $\rho$. – JimB Jan 25 '19 at 14:36

1 Answers1

3

The apparent inconsistency is due to the fact that $\rho$ cannot take on all values between $-1$ and $1$ given specific values for $p_1$ and $p_2$.

Using the definition of a correlation coefficient

$$\rho={{Cov(x_1,x_2)} \over {\sqrt{Var(x_1)Var(x_2)}}}={{\text{Pr}(x_1=1,x_2=1)-p_1 p_2}\over{\sqrt{p_1(1-p_1)p_2(1-p_2)}}}$$

and noting that $\text{Pr}(x_1=1,x_2=1)\leq \text{Min}(p_1,p_2)$ we can find the maximum value of $\rho$ for all possible values of $p_1$ and $p_2$ (here using Mathematica):

sol = Maximize[{(P11 - p1 p2)/Sqrt[p1 (1 - p1) p2 (1 - p2)], 
    0 < p1 < 1 && 0 < p2 < 1 && 0 < P11 <= Min[p1, p2]}, P11][[1]]

$$\begin{array}{cc} \{ & \begin{array}{cc} \sqrt{\frac{\text{p1} (\text{p2}-1)}{(\text{p1}-1) \text{p2}}} & \left(0<\text{p2}<\frac{1}{2}\land 0<\text{p1}\leq \text{p2}\right)\lor \left(\frac{1}{2}\leq \text{p2}<1\land \text{p1}=\text{p2}\right)\lor \left(\frac{1}{2}\leq \text{p2}<1\land 0<\text{p1}<\text{p2}\right) \\ \sqrt{\frac{(\text{p1}-1) \text{p2}}{\text{p1} (\text{p2}-1)}} & \left(0<\text{p2}<\frac{1}{2}\land \text{p2}<\text{p1}<1\right)\lor \left(\frac{1}{2}\leq \text{p2}<1\land \text{p2}<\text{p1}<1\right) \\ -\infty & \text{True} \\ \end{array} \\ \end{array}$$

ContourPlot[sol, {p1, 0, 1}, {p2, 0, 1}, ContourLabels -> True]

Contour plot of maximum rho

In short, $\rho=1$ can only occur when $p_1=p_2$.

JimB
  • 1,861