$E [Z(u) \overline{Z(v)}]=\lim _{N \rightarrow \infty} E [Z_N(u) \overline{Z_N(v)}]=\int\sum_n f_n(t) \overline{f_n(s)} u(t) \overline{v(s)} d u d s$?

Question

A stochastic process $\{Z(t), t \in \mathbb{R}\}$ is called a generalized stationary Gaussian process with mean $0$ and covariance $\{\gamma(s), s \in \mathbb{R}\}$ if for every test function $u(t)$, the random variable $Z(u)$ is normal with mean $0$, and if, for two test functions $u(t)$ and $v(t)$, the covariance of the random variables $Z(u)$ and $Z(v)$ equals $$ \Gamma(u, v)=E \left[ Z(u) \overline{Z(v)} \right] =\int_{-\infty}^{\infty} u(t) \overline{v(s)} \gamma(t-s) d t d s . $$ Here $\gamma(t)$ may be a distribution (or generalized function). One writes informally $E Z(t) \overline{Z(s)}=$ $\gamma(t-s)$. For example, if $Z$ is white noise $(H=1 / 2)$, then $\gamma$ is the Dirac $\delta$ generalized function $(<\delta, u>=u(0))$ and $\Gamma(u, v)=\int_{-\infty}^{\infty} u(t) \bar{v}(t) d t$.

I am reading the paper titled Wavelets, generalized white noise and fractional integration: The synthesis of fractional Brownian motion, where I found in the proof of Proposition 1, the following argument:

If $\epsilon_n, n \geq 0$ are i.i.d. with mean zero and unit variance and if $f_0(t), \ldots, f_n(t), \ldots$ is an orthonormal basis of $L^2(\mathbb{R})$ such that $$ Z(t)=\sum_{n=0}^{\infty} f_n(t) \epsilon_n $$ converges almost surely as a generalized function, then $E \left[ Z(t) \overline{Z(s)} \right] =\delta(t-s)$, that is, the limit is a generalized white noise. Indeed, if $Z_N(t)=\sum_{n=0}^N f_n(t) \epsilon_n$, then for any two test functions $u$ and $v$, $$ \begin{aligned} E \left[ Z(u) \overline{Z(v)} \right] & =\lim _{N \rightarrow \infty} E \left[ Z_N(u) \overline{Z_N(v)} \right] \\ & =\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \sum_{n=0}^{\infty} f_n(t) \overline{f_n(s)} u(t) \overline{v(s)} \ d u d s \\ & =\int_{-\infty}^{\infty} u(t)\left(\sum_{n=0}^{\infty} f_n(t) \int_{-\infty}^{\infty} \overline{f_n(s) v(s)} d s\right) d t \\ & =\int_{-\infty}^{\infty} u(t)\left(\sum_{n=0}^{\infty} f_n(t) \overline{\langle f_n, v \rangle}\right) d t \\ & =\int_{-\infty}^{\infty} u(t) \overline{v(t)} d t . \end{aligned} $$ In the last steps there are few things that I can't understand.

I can't explain the first two equalities, i.e. I can't explain why $$E \left[ Z(u) \overline{Z(v)} \right] =\lim _{N \rightarrow \infty} E \left[ Z_N(u) \overline{Z_N(v)} \right]=\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \sum_{n=0}^{\infty} f_n(t) \overline{f_n(s)} u(t) \overline{v(s)} \ d u d s.$$

Edit

Since $Z(u)=\langle Z, u\rangle=\langle \sum_n f_n \epsilon_n, u\rangle=\sum_n \epsilon_n\langle f_n , u\rangle$, we have that $$\langle Z(u), Z(v) \rangle= \langle \sum_n \epsilon_n \langle f_n , u\rangle, \sum_m \epsilon_m \langle f_m , v\rangle \rangle$$ (in the last formula the brackets are related to the expectation and are different from those that define $Z(u)$ as a generalized process). Hence $$\langle Z(u), Z(v) \rangle= \sum_n \langle f_n , u\rangle \sum_m \overline{\langle f_m , v\rangle} \langle \epsilon_n , \epsilon_m \rangle$$ Since $\{\epsilon_n\}$ are i.i.d. with mean zero and unit variance, we have that $$\langle Z(u), Z(v) \rangle= \sum_n \langle f_n , u\rangle \overline{\langle f_n , v\rangle}$$

Answer 1

The authors are using that the limit

$$ Z(u) = \lim_{N\to\infty}Z_N(u) $$

converges not just almost-surely, but also in $L^2(\mathbb P)$. To see this, note that for $M \ge N,$

$$ E\lvert Z_M(u) - Z_N(u) \rvert^2 = E\left\lvert\sum_{n=N+1}^M \langle f_n, u \rangle\epsilon_n\right\rvert^2=\sum_{n=N+1}^M \lvert\langle f_n, u \rangle\rvert^2\le\sum_{n=N+1}^\infty \lvert\langle f_n, u \rangle\rvert^2\to 0 $$ as $N \to \infty$, since $u \in L^2(\mathbb R)$, and the $f_n$ are an orthonormal basis. Thus the sequence $Z_N(u)$ is Cauchy in $L^2(\mathbb P)$, so has an $L^2(\mathbb P)$-limit $\tilde Z(u)$, which must coincide with the almost-sure limit $Z(u)$.

Writing $\langle X, Y \rangle = E[X\overline{Y}]$ also for the inner product on $L^2(\mathbb P)$, we then have that $$ \langle Z(u), Z(v) \rangle = \lim_{N \to \infty} \langle Z_N(u), Z_N(v) \rangle = \sum_{n=0}^\infty \langle f_n, u \rangle \overline{\langle f_n, v \rangle} = \langle u, v \rangle. $$ using the continuity of the inner product, and that the $f_n$ are an orthonormal basis of $L^2(\mathbb R)$.

Edit: The first equality in the final display holds because $Z_N(u) \to Z(u)$ in $L^2(\mathbb P)$, $Z_N(v) \to Z(v)$, and the inner product of an $L^2$ space is continuous. The second equality holds for sums up to any finite $N$ by linearity of the inner product, so it must hold also in the limit $N \to \infty$. The third equality holds because $f_n$ is an orthonormal basis, so $u=\sum_{n=0}^\infty \langle f_n, u \rangle f_n$, $v=\sum_{n=0}^\infty \langle f_n, v \rangle f_n$, and we can argue as in the first two equalities.