1

Consider the 2 way ANOVA model:

$y_{i j} = \mu + l_j + \alpha_i + e_{i j}$, $e_{i j} \sim \mathcal{N} (0, \sigma^2)$

where $i=1\dots m, j=1\dots n$, how do you prove that the residual degrees of freedom is $(n - 1)(m - 1)$? I can sort of see this intuitively, as the model can still be fitted exactly when $l_1 = 0, \alpha_1 = 0$, but how would I prove this rigorously (I think it's defined using the dimension of the vector space spanned by the coefficients on RHS)?

simonzack
  • 1,701
  • 13
  • 28

1 Answers1

1

To show this rigorously, we should view the $mn$ observations $y_{ij}, i = 1, \ldots m,\; j = 1, \ldots, n$ as an $mn$-vector in the vector space $\mathbb{R}^{mn}$. By the RHS specification of the model, the linear part is spanned by vectors: $$e, r_1, \ldots, r_m, c_1, \ldots, c_n$$ where $e$ is the $mn$-vector of all ones. $r_i$ is the $mn$-vector with its $(i - 1)m + 1, \ldots, (i - 1)m + n$ components $1$ and all else zeros (known as "row vectors"), and $c_j$ is the $mn$-vector with its $j, n + j, \ldots, (m - 1)n + j$ components $1$ and all else zeros (known as "column vectors").

We may denote the linear subspace spanned by these vectors by $\mathcal{L}$. It is easily seen $$\dim(\mathcal{L}) = 1 + (m - 1) + (n - 1) = m + n - 1.$$

Since the residual degrees of freedom is understood as the dimensionality as the orthogonal complement subspace of $\mathcal{L}$ in $\mathbb{R}^{mn}$, we have: $$df_{\text{residual}} = \dim(\mathbb{R}^{mn}) - \dim(\mathcal{L}) = mn - (m + n - 1) = (m - 1)(n - 1).$$

Remark: why $R \cap C = \text{span}(e)$? You may interpret the intersection of two subspaces just in the sense of intersection of two ordinary sets. Rigorously, I should write $R \cap C = \text{span}(e)$ instead of $R \cap C = e$ (sorry for any confusion).

To show this, first, by definition, $e \in R \cap C$. On the other hand, any vector belongs to $R$ has the form $\alpha_1 r_1 + \cdots + \alpha_m r_m$ (this is because, as I argued in the comment, for example, $R$ is in fact spanned by $\{r_1, \ldots, r_m\}$, which is known as a "basis" of $R$) whereas any vector belongs to $C$ has the form $\beta_1 c_1 + \cdots + \beta_n c_n$, so if $x \in R \cap C$, then $x = \alpha_1 r_1 + \cdots + \alpha_m r_m = \beta_1 c_1 + \cdots + \beta_n c_n$. By carefully writing down these two representations and equalizing them, you can deduce that $$\beta_1 = \alpha_1 = \cdots = \alpha_m, \beta_2 = \alpha_1 = \cdots = \alpha_m, \ldots, \beta_m = \alpha_1 = \cdots = \alpha_m$$ Thus all the $\alpha, \beta$ coefficients must be identical, let the common value be $c$, then $$x = c(\alpha_1 + \cdots + \alpha_m) = c(\beta_1 + \cdots + \beta_n) = ce \in \text{span}(e).$$

In summary, we have shown that $$\text{span}(e) \subset R \cap C \subset \text{span}(e).$$

Therefore, $R \cap C = \text{span}(e)$.

Zhanxiong
  • 14,040
  • How is this easily seen? I'm not sure how you arrive at $m + n - 1$. – simonzack Jun 19 '15 at 03:12
  • @simonzack There are many ways to see this. One rigorous proof: are you familiar with the ``dimension equation" in linear algebra? If not, see the link: http://math.stackexchange.com/questions/102579/dimension-of-the-sum-of-subspaces. Now let's denote the subspace spanned by ${e, r_1, \ldots, r_m}$ by $R$, while the subspace spanned by ${e, c_1, \ldots, c_n}$ by $C$. Clearly $\dim(R) = m$, $\dim(C) = n$. Since $R\cap C = e$, whose dimension is obviously 1. Therefore, $\dim(\mathcal{L}) = \dim(R) + \dim(C) - \dim(R \cap C) = m + n - 1$. – Zhanxiong Jun 19 '15 at 04:11
  • In case you are wondering why $\dim(R) = m$ or $\dim(C) = n$, note that, for example, ${r_1, \ldots, r_m}$ are linearly independent, hence $\dim(R) \geq m$, but the all $m + 1$ span vectors are linearly dependent as $e = r_1 + r_2 + \cdots + r_m$, hence $\dim(R) < m + 1$. Therefore $\dim(R) = m$. – Zhanxiong Jun 19 '15 at 04:15
  • Also, $\mathcal{L}$ is the sum of the subspaces $R$ and $C$, namely, $\mathcal{L} = R + C$. – Zhanxiong Jun 19 '15 at 04:19
  • Ah thank you for your explanation and time, I'm a bit rusty on linear algebra, I still don't see how $R\cap C = e$? – simonzack Jun 19 '15 at 08:22
  • @simonzack The explanation is too long to be a comment so I will edit my answer. – Zhanxiong Jun 19 '15 at 12:22
  • I get it now, many thanks! – simonzack Jun 19 '15 at 12:44