0

Show E[E(X|Y)]=E[X]

Now if X and Y are independent then it is very straightforward as E(X|Y)]=E[X]. However is there a better explanation to this ? Can we prove this mathematically ?

What if X and Y are not independent ? Will it hold ?

Aviral Gupta
  • 65
  • You can try to show it for discrete variables X and Y first. Bayes rule may come handy. – megas Sep 29 '16 at 05:29
  • E(X|Y) is a fixed value, what you mean the expected value of that? – cxz Sep 29 '16 at 05:31
  • E(X|Y) is not always a fixed value. It can be a function of y too! I mean using the properties of conditional expectation, how can we prove it ? – Aviral Gupta Sep 29 '16 at 05:35
  • @Aviral: So, in the case that $Y$ is a discrete random variable, here's the start: $$E[E[X|Y]] = \sum_y E[X|Y=y]p(Y=y) = \sum_y \big(xp_{X|Y}(x|y)\big) p_Y(y).$$ – Ted Shifrin Sep 29 '16 at 05:40
  • "Show E[E(X|Y)]=E[X]" Isn't this direct from the definition of E(X|Y)? What is this definition, already? – Did Sep 29 '16 at 05:49
  • @cxz "E(X|Y) is a fixed value, what you mean the expected value of that?" Actually E(X|Y) is a random variable, by definition. – Did Sep 29 '16 at 05:50
  • @Ted The identity in your comment might lack an integral sign. – Did Sep 29 '16 at 06:00

1 Answers1

1

This is called the Law of total expectation. First, note that

$$E[X|Y]=\int_{-\infty}^{\infty}p_{X|Y}(x|y)xdx\tag{1}$$

where $p_{X|Y}(x|y)$ is the conditional probability density function. Then note that

$$E[E[X|Y]]=\int_{-\infty}^{\infty}E[X|Y=y]\;p_Y(y)dy\tag{2}$$

Combining $(1)$ and $(2)$ gives

$$\begin{align}E[E[X|Y]]&=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}p_{X|Y}(x|y)p_Y(y)x\,dxdy\\&=\int_{-\infty}^{\infty}x\int_{-\infty}^{\infty}p_{XY}(x,y)\,dy\,dx\\&=\int_{-\infty}^{\infty}xp_X(x)dx\\&=E[X]\tag{3}\end{align}$$

where I've used $p_{X|Y}(x|y)p_Y(y)=p_{XY}(x,y)$, and $p_X(x)=\int_{-\infty}^{\infty}p_{XY}(x,y)dy$.

Matt L.
  • 10,636
  • It is odd to place the notion of conditional PDF at the center of everything related to conditional expectations. It is not. To begin with, it may not exist, just like ordinary PDFs may not exist and are not the rigorous way to define ordinary expectations. – Did Sep 29 '16 at 05:49
  • The proof for the general case is in the Wikipedia article linked above. Direct link: https://en.wikipedia.org/wiki/Law_of_total_expectation#Proof_in_the_general_case – Wood Sep 29 '16 at 05:54
  • @Did: Thanks for your feedback; I guess that most engineers are taught expectations in that way (based on PDFs), so that's the way I'm used to thinking about them. It would be great if someone else posted a more rigorous answer. – Matt L. Sep 29 '16 at 06:47
  • 1
    @MattL. In the present case there is no real need since the thing is (textbook stuff and) explained in tons of places, including on this very site. – Did Sep 29 '16 at 06:52