0

To solve $\sum_{i=1}^N E[X_i^2]$ , where $N$ is a random variable, and $X_i$'s are IID, we could apply Wald's equality, which is:

$$ E[Z_N] = E[X_1]E[N] = \cdots = E[X_N]E[N] $$ where $Z_N = X_1 + \cdots + X_N$

So applying this, we can get $$ \sum_{i=1}^N E[X_i^2] = E[X_1^2]E[N] $$

Now my question is since each term of the summation is identical since $X_i$ is IID, why couldn't we instead write the following? $$ \sum_{i=1}^N E[X_i^2] = NE[X_1^2] $$ Obviously, this result is a random variable whereas using Wald's equality did not result in a random variable. What is wrong with this approach? My guess is it's something related to summing from $i=1$ to $N$, where the latter is a random variable.

What I did doesn't make sense, but I don't really have a formal reason why it doesn't work. But I was just started thinking about a different example: $$ \sum_{i=1}^N 1 $$ What is the expected value of this expression? $$ E[\sum_{i=1}^N 1 ] = \sum_{i=1}^N E[1] = \sum_{i=1}^N 1 = E[N] $$ ?

user5965026
  • 677
  • You can get properly sized brackets (and other paired delimiters) that adjust to the size of their content by preceding them with \left and \right. – joriki May 11 '20 at 05:02

1 Answers1

1

You misapplied the identity.

$\sum_{i=1}^N E[X_i]^2$ is a random variable, since the upper limit of the summation is random and you’re not taking the expectation over it.

Applying Wald’s identity, $E\left[\sum_{i=1}^NY_i\right]=E[Y_1]E[N]$, to this sum, with $Y_i=E[X_i]^2$, yields

$$ E\left[\sum_{i=1}^NE[X_i]^2\right]=E[E[X_1]^2]E[N]=E[X_1]^2E[N]\;. $$

So there are two things wrong with what you wrote as the result of applying the identity: On the left you’re missing the expectation operator, and on the right the square is inside the expectation when it should have stayed outside. You can tell that your equation can’t be right because the left-hand side is a (non-constant) random variable and the right-hand side is a constant.

On the other hand, your second result (the one about which you asked what’s wrong with it) is correct. Here you sum the same constant $N$ times and thus get $N$ times that constant; both sides of the equation are random variables, so everything is OK.

Your last equation is also correct, but proving it using the linearity of expectation as you did is a bit of a detour; you could go directly from the left to the very right by noting that $\sum_{i=1}^N1=N$, which is a basic arithmetic fact that doesn’t require any probabilistic concepts like linearity of expectation.

joriki
  • 238,052
  • Oh I see. So just to confirm, if I want to take the expectation of $\sum_{i=1}^N X_i$ where $N$ is a random variable, $E \bigg[ \sum_{i=1}^N X_i \bigg] \neq \sum_{i=1}^N E[X_i]$? The LHS is a numerical value, whereas the RHS is a random variable? – user5965026 May 11 '20 at 13:02
  • This question was actually inspired by David's answer from https://math.stackexchange.com/questions/2242553/simple-question-stopped-martingale-expected-value. Under his "Method 2", he stated that the first summation term is equal to $E[N]$, which seems to be wrong if I understand wrong answer correctly? – user5965026 May 11 '20 at 13:05
  • One more question I have is it seems that $E\left[ \sum_{i=1}^N X_i \right] = E\left [ \sum_{i=1}^N E[X_i] \right]$? Is this always true? – user5965026 May 11 '20 at 14:57