3

Apparently linearity of expectation holds under certain circumstances for a collection of random variables even when that collection is countably infinite. One of these circumstances, which I encountered reading this book, is as follows (excerpted from exercise 2.29).

If $X_1, X_2, X_3...$ is a sequence of random variables such that $\sum_{i=1}^{\infty} E[|X_i|]$ converges, then linearity of expectation holds:

$$E[\sum_{i=1}^{\infty} X_i] = \sum_{i=1}^\infty E[X_i]$$

What is the proof of this statement? Although this statement is mentioned in another question on this site, the accepted answer omits a proof. The answer to this question goes a bit further and claims that the Fubini theorem is enough to show the sufficiency of the condition above, but I still fail to see how it can be used to produce a proof of the statement.

Johnny
  • 280
  • 1
    The desired equality involves changing the order of two "integrals"; one "integral" is the expectation, and the other "integral" is the sum over $i$. This is precisely the conclusion of Fubini's theorem, and the last link writes out the condition needed for the theorem to hold. – angryavian Aug 03 '21 at 15:11

1 Answers1

6

Let $Y = \sum_{i=1}^\infty |X_i|$. Apply the monotone convergence theorem to the partial sums $\sum_{i=1}^n |X_i|$ to show that $E[Y] = \sum_{i=1}^\infty E|X_i| < \infty$. In other words, let $Y_n = \sum_{i=1}^n |X_i|$. The random variables $Y_n$ are nonnegative and monotone increasing, and their pointwise limit is $Y$. So the monotone convergence theorem lets us conclude that $\lim_{n \to \infty} E[Y_n] = E[Y]$. But by linearity of expectation over finite sums, we have $E[Y_n] = \sum_{i=1}^n E|X_i|$. So we have shown $$\lim_{n \to \infty} \sum_{i=1}^n E|X_i| = E[Y]$$ which is exactly what it means to say that $$\sum_{i=1}^\infty E|X_i| = E[Y].$$

Now apply the dominated convergence theorem to the partial sums $\sum_{i=1}^n X_i$, noting that they are all dominated by $Y$. That is, similarly to what was done above, let $S_n = \sum_{i=1}^n X_n$ be the partial sum without the absolute values. The fact that $E[Y] < \infty$ means in particular that $Y < \infty$ almost surely, which is to say that almost surely, the series $\sum_{i=1}^\infty X_i$ converges absolutely. This means that the sequence $S_n$ converges almost surely to some $S$, which we might as well call $S = \sum_{i=1}^\infty X_i$. Now by the triangle inequality, for each $n$ we have $|S_n| \le \sum_{i=1}^n |X_i| \le Y$, so the sequence $S_n$ is dominated by the integrable random variable $Y$. By dominated convergence we conclude $E[S_n] \to E[S]$.

On the other hand, by linearity of expectation for finite sums, we have $E[S_n] = \sum_{i=1}^n E[X_i]$. So we have shown $$\lim_{n \to \infty} \sum_{i=1}^n E[X_i] = E[S] = E\left[\sum_{i=1}^\infty X_i\right]$$ which is precisely the desired statement.

Nate Eldredge
  • 97,710
  • How do we know that the sequence of partial sums $\sum_{i=1}^{n}|X_i|$ is bounded? – Johnny Aug 04 '21 at 17:19
  • @kotu: You don't, but for the monotone convergence theorem, no such assumption is needed. – Nate Eldredge Aug 04 '21 at 19:48
  • I apologize, I don't know much about analysis but are we referring to the same monotone convergence theorem? Doesn't this say that the sequence needs to be bounded? – Johnny Aug 06 '21 at 16:27
  • Oh, no, not that one. I mean the monotone convergence theorem from measure theory. It's stated further down on that Wikipedia page, where it's called "Beppo Levi's lemma". FWIW, my experience is that in measure theory and probability, "monotone convergence theorem" always refers to this result, not the one about bounded sequences of real numbers. – Nate Eldredge Aug 06 '21 at 16:34
  • Could you explained where you said "apply the monotone convergence theorem to the partial sums...to show that......." because where do you use the partial sum there? – Frish Vanmol May 17 '23 at 16:35
  • 1
    @FrishMagmol: I filled in more details, hopefully it helps. – Nate Eldredge May 17 '23 at 17:31