4

If we suppose that the two independent random variables $X \sim \mathcal{N}(0,\sigma^2_x)$ and $N \sim \mathcal{N}(0,\sigma^2_n)$ and that $S = X + N$, how would I work out the conditional expectation $E[X\mid S=s]$?

Michael Hardy
  • 1
John Lee
  • 97
  • 1
  • 6
  • 1
    how is the dependence structure defined? – Math-fun Jul 21 '15 at 17:01
  • Are $X$ and $N$ independent? – zoli Jul 21 '15 at 17:01
  • $X$ and $N$ are independent! Fixed my post. – John Lee Jul 21 '15 at 17:02
  • Hint: if $N$ and $X$ are jointly normal, then so are $X$ and $S$ jointly normal. Can you work out the parameters of $(X,S)$? Do you know the standard result for the conditional mean of $X$ given $S$ when $X$ and $S$ enjoy joint normality? – Dilip Sarwate Jul 21 '15 at 17:04
  • @DilipSarwate, I guess the parameters must be $\mu = (0,0)$ and $\Sigma = (\sigma_x^2,\sigma_x^2;\sigma_x^2;\sigma_x^2+\sigma_n^2)$? I do not know any of the standard results.. – John Lee Jul 21 '15 at 17:17

2 Answers2

5

Since $X$ and $N$ are independent and normal, any combinations of $X$ and $N$ are normal. As any combination of $X+N$ and $\frac{1}{\sigma_x^2}X - \frac{1}{\sigma_n^2}N$ is also a combination of $X$ and $N$, that is, any combinations of $X+N$ and $\frac{1}{\sigma_x^2}X - \frac{1}{\sigma_n^2}N$ are normal, then $X+N$ and $\frac{1}{\sigma_x^2}X - \frac{1}{\sigma_n^2}N$ are jointly normal. Moreover, note that \begin{align*} E\bigg((X+N)\Big( \frac{1}{\sigma_x^2}X - \frac{1}{\sigma_n^2}N\Big) \bigg) &= 0. \end{align*} Then $X+N$ and $\frac{1}{\sigma_x^2}X - \frac{1}{\sigma_n^2}N$ are independent. Consequently, \begin{align*} E\big( (X+N) \mid S\big) &= S,\\ E\bigg( \Big( \frac{1}{\sigma_x^2}X - \frac{1}{\sigma_n^2}N\Big) \mid S\bigg) &= 0. \end{align*} Solving this system, we obtain that \begin{align} E(X\mid S) = \frac{\sigma_x^2}{\sigma_x^2+\sigma_n^2}S. \end{align} In other words, \begin{align} E(X\mid S=s) = \frac{\sigma_x^2}{\sigma_x^2+\sigma_n^2}s. \end{align}

Gordon
  • 4,461
  • how do you conclude that they are independent? For uncorrelation implying independence you need afaik that they are also jointly normal?! – user190080 Jul 21 '15 at 18:12
  • @user190080: As $X$ and $N$ are normal, then any combinations of them are normal, and consequently, any combinations of $X+N$ and $\frac{1}{\sigma_x^2}X - \frac{1}{\sigma_n^2}N$ are normal. This implies the joint normality. Then uncorrelated implies independence. – Gordon Jul 21 '15 at 18:16
  • @Gordon : If $X$ and $N$ are normal and independent, the certainly every linear combination of them is normal and every tuple of linear combnations of them is jointly normal. But this hypothesis of independence, although it can be greatly weakened, cannot be simply dropped. I can show you two random variables $X$ and $N$ that are normally distributed but for which some linear combinations are not normally distributed. For example, let $X\sim N(0,1)$ and let $N=\pm X$, each with probability $1/2$, independently of $X$. Then $\Pr(X+N=0)>0$, so $X+N$ is not normally distributed. ${}\qquad{}$ – Michael Hardy Jul 21 '15 at 18:24
  • $X,N$ are jointly normal since they are independent, agreed. Therefore all linear transformations are normal, agreed. But now you want to conclude that for $S=X+N$ and $T=\frac{1}{\sigma_x^2}X - \frac{1}{\sigma_n^2}N$ and $\mathbf{E}(ST)=0$ it follows that $S$ and $T$ are independent, but this is only true if $S,T$ are also jointly normal, which is nowhere shown - do we agree here? – user190080 Jul 21 '15 at 18:26
  • @user190080: any combination of $S$ and $T$ is also a combination of $X$ and $N$. That is, $S$ and $T$ are also jointly normal. – Gordon Jul 21 '15 at 18:33
  • 1
    @MichaelHardy: See also my above comments -- any combination of $S$ and $T$ is also a combination of $X$ and $N$. That is, $S$ and $T$ are also jointly normal. – Gordon Jul 21 '15 at 18:42
  • So your answer to the first comment was not correct as it stood. – Michael Hardy Jul 21 '15 at 18:44
  • 1
    I would answer that first comment as follows: Linear combinations of independent normals are normal, and linear combinations of $X+N$ and $aX+bN$ are linear combinations of $X$ and $N$; hence linear combinations of $X+N$ and $aX+bN$ are lnormal; hence $X+N$ and $aX+bN$ are jointly normal. ${}\qquad{}$ – Michael Hardy Jul 21 '15 at 18:47
  • 1
    @MichaelHardy: can you please be more specific which part in my answer is wrong? I have never said that uncorrelated implied independence. What I have said is that uncorrelated plus joint normality implies independence. Your above comment was what I said. Now I made it more explicit in my answer. By the way, as this is too basic, I think people understand anyways. – Gordon Jul 21 '15 at 18:49
  • @Gordon : Quoting your comment above: "As X and N are normal, then any combinations of them are normal," But that doesn't follow. – Michael Hardy Jul 21 '15 at 18:54
  • @MichaelHardy: the independence is already assumed. – Gordon Jul 21 '15 at 18:55
  • As it concerns me, I am convinced. I forgot that this is an "if and only if" relation between the linear transformations and jointly normal distributions, so your argumentation works, thanks to you two for pointing it out. . – user190080 Jul 21 '15 at 18:55
  • @Gordon How does the product imply the below equations? E((X+N)*1) = 0 but E(X+N|S) = S and E(1|S) = 1? What am I missing here? Thanks – gst1502 Jun 12 '23 at 06:35
0

Here's another method:

https://stats.stackexchange.com/questions/30588/deriving-the-conditional-distributions-of-a-multivariate-normal-distribution

Not as "clever" as Gordon's method above however.

Community
  • 1
John Lee
  • 97
  • 1
  • 6