7

Suppose I have two independent random vectors $X,Y\in \mathbb R^n$ such that each entry $x_i\sim\mathcal{N}(0,1)$ i.i.d., $y_i\sim\mathcal{N}(0,1)$ i.i.d. Define $$ S = X+Y, \qquad D = X-Y .$$ I seem to remember learning that $S$ and $D$ are independent RVs but can't find a reference. Is this even true, and how could I go about showing this?

Does the same apply for any other distributions for $X$ and $Y$ (in particular with non-independent entries)?

mass
  • 214

3 Answers3

3

Since $X$ and $Y$ have Gaussian distributions, $S$ and $D$ do as well. So, all you have to show is that $S$ and $D$ are uncorrelated. To do this, we calculate some expectations.

$E[S^TD] = E[(X+Y)^T(X-Y)] = E[X^TX]-E[Y^TY]=\mathbf 1-\mathbf 1=0$

and

$E[S]^TE[D] = (E[X]+E[Y])^T(E[X]-E[Y]) = (0+0)^T(0-0) = 0$.

Since $E[S^TD] = E[S]^TE[D]$ then $S$ and $D$ are uncorrelated, hence independent.

Off the top of my head, it doesn't seem like this is going to work for arbitrary distributions, since if $E[X^TX]\neq E[Y^TY]$ but $X$ and $Y$ still have $0$ mean then the whole thing falls apart. For non-Gaussian distributions, uncorrelated does not even imply independent, but since it's still a necessary condition, it seems like the same counterexample holds.

Y. S.
  • 1,816
  • You need to put in various transpose symbols in your answer. If $S$ and $D$ are vectors, $SD$ is undefined (the matrix dimensions don't match up), and so you need to write either $S^TD$ or $SD^T$ depending on whether you like rows or columns as vectors. – Dilip Sarwate Jan 04 '15 at 20:43
  • 1
    Forgive me if this is obvious, but how do we know $S$ and $D$ are jointly Gaussian? To my knowledge Gaussian+uncorrelated does not in general imply independence but Jointly Gaussian+uncorrelated does. – mass Jan 04 '15 at 20:45
  • Ack you are right! Sorry about that, should be fixed now! – Y. S. Jan 04 '15 at 20:45
  • 1
    @SCarm That's a good point, I didn't consider it and I should have. But for this case, I think you're ok if $X$ and $Y$ are independent. I'm not solid on how to show this though. – Y. S. Jan 04 '15 at 20:57
  • 1
    Adding more detail - uncorrelated implies independent is NOT true for general Normal variables. It is true only if they are components of a Multivariate Normal.

    In this case, S and D put together in a vector [S D] can be expressed in the form $$[S D] = AZ + u$$ where $Z is standard multivariate gaussian$, so [S D] form a Multivariate Normal. Since they are components of a multivariate gaussian, you can use the fact that uncorrelated implies independence.

     – spandan madan Nov 01 '19 at 06:33
  • 1
    You probably forgot the transposition in the second line, here: $E[S]^TE[D]$ – blahblah Apr 21 '20 at 07:28
3

$(X,Y)$ is a vector of dimension $2N$ whose entries are independent $N(0,1)$ random variables and so its covariance matrix is the $(2N)\times (2N)$ identity matrix $I_{2N}$. The $2N$ variables enjoy a multivariate normal distribution, that is, they are jointly Gaussian random variables, and $(X,Y)$ is called a Gaussian vector.

Any linear transformation of a Gaussian random vector results in a Gaussian vector (this is sometimes taken as the definition of a Gaussian vector), and so $$(X+Y,X-Y) = (X,Y)\left[\begin{matrix}1&1\\1&-1\end{matrix}\right] = (X_1,\ldots,X_n,Y_1,\ldots Y_n)\left[\begin{matrix}I_N&I_N\\I_N&-I_N\end{matrix}\right]$$ is also a Gaussian vector with covariance matrix $$\left[\begin{matrix}I_N&I_N\\I_N&-I_N\end{matrix}\right]^TI_{2N}\left[\begin{matrix}I_N&I_N\\I_N&-I_N\end{matrix}\right] = \left[\begin{matrix}2I_N&0\\0&2I_N\end{matrix}\right]$$ which shows independence of $X+Y$ and $X-Y$ and also reveals that the $X_i\pm Y_i$ are $N(0,2)$ random variables.

Now try and see if all the above will work when the zero-mean Gaussian vectors $X$ and $Y$ are independent of each other but $(X_1,X_2,\ldots, X_n)$ has the same covariance matrix $\Sigma$ as $(Y_1,Y_2,\ldots, Y_n)$ with $\Sigma$ not necessarily restricted to being the identity matrix or even a diagonal matrix, that is, the $X_i$ (and similarly the $Y_i$) need not be independent of each other.

Dilip Sarwate
  • 25,197
  • From this answer it is clear that Gaussian is the only distribution for which vectors with i.i.d. components satisfy the stated property, but could we say this for distributions without i.i.d. components e.g. uniform on the $(n-1)$-sphere? (Might just have to normalize the Gaussians) – mass Jan 04 '15 at 21:14
  • I think that if the covariance matrix of $(X,Y)$ is $\left[\begin{matrix}\Sigma&\bf 0\\bf 0&\Sigma\end{matrix}\right]$ with $\Sigma$ not necessarily being diagonal or $I_N$, the proof will still work; the components $X_i$ need not be independent of each other (nor the $Y_i$ be independent of each other) as long as the vectors $X$ and $Y$ are independent and have the same covariance matrix. – Dilip Sarwate Jan 05 '15 at 03:41
2

The pdf of the joint distribution of two iid normal variables is a function of the distance from the origin alone. (You may have seen this as part of a derivation of the constant factor in the pdf of $\mathcal N(0,1).$)

This means you can rotate the pdf as much as you want around the origin and it will still be the joint pdf of two iid normal variables. You can also reflect the pdf around an axis without changing it, for example by substituting $-X$ for $X$.

Now consider what transformation is required to map $(X,Y)$ to $(S,D)$. (That may be a bit of overkill for the problem as asked, but it's a useful mental image to acquire.)

This particular trick does not work for variables that are not independent. Nor does it necessarily work for iid variables that are not normally distributed.

It is not hard to come up with an example of two non-independent random variables $X$ and $Y$ such that $X+Y$ and $X-Y$ also are not independent.

David K
  • 98,388