0

Is there an analogous version of $\operatorname{var(X+cY)} = \operatorname{var}(X) + c^2\operatorname{var}(Y)$ for vectors of uncorrelated random variables ($X$ and $Y$ are random variables here)?

For example, consider $\operatorname{var(\boldsymbol{X}+A\boldsymbol{var}(Y)$, where the bold indicators a vector random variables. Would the variance look something like

$$ \operatorname{var(\boldsymbol{X})+A^TA\operatorname{var}(\boldsymbol{Y})} $$ ?

user5965026
  • 677
  • Did you mean $c^2Var(Y)$? – Mdoc Jul 10 '20 at 03:07
  • @Mdoc Yes, sorry. I just fixed it. – user5965026 Jul 10 '20 at 03:08
  • As far as I remember,we have $Var(AY)=A^TVar(Y)A$ and it should be easy to derive this straight from the definition of the variance. – Mdoc Jul 10 '20 at 03:11
  • @Mdoc I think one thing I'm confused about is what is the form of the variance of a random vector? Is it a matrix, or should it be a vector the same size as the random vector? If it is a matrix, is it simply just the variance-covariance matrix, or is it a diagonal matrix (disregarding the covariance terms)? – user5965026 Jul 10 '20 at 03:19
  • @Mdoc When you write $A^TVar(Y)A$, that indicates to me that $Var(Y)$ has to be a matrix otherwise the dimensions wouldn't work out. But I don't get why it's a matrix when we're only talking about variance. – user5965026 Jul 10 '20 at 03:20
  • If Y is a vector then Var(Y) is the covariance matrix. – Mdoc Jul 10 '20 at 03:23
  • @Mdoc Just came to correct something in case someone else sees this. $var(AY) = AVar(Y)A^T$ and not $A^TVar(Y)A$. – user5965026 Jul 11 '20 at 00:28

1 Answers1

3

For vector-valued random variables $\overrightarrow{X}$ and $\overrightarrow{Y}$, the appropriate concept is a covariance matrix: if $\overrightarrow{X} = [x_1, ..., x_n]$, then

$$\operatorname{cov}(\overrightarrow{X}) = [a_{ij}],$$

where $a_{ij} = \operatorname{cov}(x_i, x_j)$. It is then true that for any matrix $A$ of appropriate dimension,

$$\operatorname{cov}(A\overrightarrow{X}) = A\operatorname{cov}(\overrightarrow{X})A^t,$$

and direct computation should show that $\operatorname{cov}(\overrightarrow{X} + \overrightarrow{Y}) = \operatorname{cov}(\overrightarrow{X}) + \operatorname{cov}(\overrightarrow{Y})$ when the components of $\overrightarrow{X}, \overrightarrow{Y}$ are uncorrelated. So yes, you do get a version of your original statement:

$$\operatorname{cov}(\overrightarrow{X} + A\overrightarrow{Y}) = \operatorname{cov}(\overrightarrow{X}) + A\operatorname{cov}(\overrightarrow{Y})A^t,$$

and if $A = cI$ for some scalar $c$, then it is even true that

$$\operatorname{cov}(\overrightarrow{X} + c\overrightarrow{Y}) = \operatorname{cov}(\overrightarrow{X}) + c^2\operatorname{cov}(\overrightarrow{Y}).$$

Rivers McForge
  • 5,787
  • Is there a concept of just variance matrix? – user5965026 Jul 10 '20 at 03:23
  • No, there's no "just variance matrix", or rather, the covariance matrix is the unique generalization of "variance" to the vector case.

    However, it is true that if the components of $\overrightarrow{X} = [x_i]$ are uncorrelated, then $\operatorname{cov}(\overrightarrow{X})$ is a diagonal matrix ($a_{ij} = 0$ if $i \neq j$) and the diagonal elements are just the variances of the $x_i$: $a_{ii} = \operatorname{Var}(x_i)$.

     – Rivers McForge Jul 10 '20 at 03:32