I'm considering the Ornstein - Uhlenbeck process $ X(t)=x_{\infty}+e^{-at}(x_{0}-x_{\infty})+b \int_{0}^{t} e^{-a(t-s)} dW(s)$ where $a, b > 0 $ are given constants. I used the Itô Isometry to compute the variance but I did not figure out how to use it to compute the covariance. What is the most general context in which I can use Itô Isometry to compute the covariance?
Thank you for your help.
Complementing the answer above to make explicit use of Ito's isometry as you requested. The appropriate version of ito's isometry to use in this case is the following: $$ \mathbb{E} \int_0^T f(r)dW(r) \int_0^T g(r) dW(r) = \mathbb{E} \int_0^T f(s)g(s)ds,$$ where in your case $f(r) = e^{ar} \mathbf{1}_{(0,s)}(r)$, and $f(r) = e^{ar} \mathbf{1}_{(0,t)}(r)$. I followed all the steps and got that the covariance in your case is $$ \frac{ b^2 }{a} e^{-a ( t \vee s ) } { \rm cosh } ( s \wedge t ),$$ where for any pair of numbers, $ x,y \in \mathbb{R} $, $x \vee y = \max \{ x,y \}$, and $x \wedge y = \min \{ x,y \}$.
Let $Y_t=\displaystyle\int_0^t\mathrm e^{au}\mathrm dW_u$ then, for every $t$, $X_t=b\mathrm e^{-at}Y_t+z(t)$ where $z(\ )$ is deterministic hence $$\mathrm{cov}(X_t,X_s)=(b\mathrm e^{-at})(b\mathrm e^{-as})\mathrm{cov}(Y_t,Y_s),$$ and, for every $t$, $Y_t=\displaystyle\int_0^\infty g_t(u)dW_u$ where $g_t$ is deterministic since $g_t(u)=\mathrm e^{au}\mathbf 1_{u\lt t}$ hence $$\mathrm{cov}(Y_t,Y_s)=\int_0^\infty g_t(u)g_s(u)du=\int_0^\infty \mathrm e^{2au}\mathbf 1_{u\lt\min\{t,s\}}du=\int_0^{\min\{t,s\}}\mathrm e^{2au}du=\ldots$$
