1

In a famous 1938 paper ("The large-sample distribution of the likelihood ratio for testing composite hypotheses", Annals of Mathematical Statistics, 9:60-62), Samuel Wilks derived the asymptotic distribution of $2 \times LLR$ (log likelihood ratio) for nested hypotheses, under the assumption that the larger hypothesis is correctly specified. The limiting distribution is $\chi^2$ (chi-squared) with $h-m$ degrees of freedom, where $h$ is the number of parameters in the larger hypothesis and $m$ is the number of free parameters in the nested hypothesis. However, it is supposedly well-known that this result does not hold when the hypotheses are misspecified (i.e., when the larger hypothesis is not the true distribution for the sampled data).

Can anyone explain why? It seems to me that the proof should still work with minor modifications. It relies on the asymptotic normality of the maximum likelihood estimate (MLE), which still holds with misspecified models (assuming an invertible covariance matrix). The only difference is the nature of the covariance matrix: for correctly specified models, we can approximate it with the inverse Fisher information matrix $J^{-1}$, with misspecification, we can use the sandwich estimate of the covariance matrix ($J^{-1} K J^{-1}$). The latter reduces to the inverse of the Fisher information matrix when the model is correctly specified (since $J = K$). AFAICT, Wilks proof doesn't care where the estimate of the covariance matrix comes from, as long as we have an invertible asymptotic covariance matrix of the multivariate normal for the MLEs.

ratsalad
  • 133
  • 5

2 Answers2

0

Vuong actually showed that if you don't assume the information matrix equality holds then the wilks 1938 statistics has a weighted chi square distribution. In the special case of correct specification (Which implies the information matrix equality holds) then the weights are all equal to one and this reduces to the 1938 result. You might find my paper on this to be helpful "Discrepancy risk model selection test theory for comparing possibly misspecified or non nested models" psychometrika 2003 vol 68 issue 2 229-249. Richard Golden

RMG
  • 121
0

Misspecification via redundant variables will produce proportional rows (or columns, if you perfer) in the covariance matrix. This causes the defect you observe -- the inverse covariance does not exist. (Other defects may do this, depending on details not available to a general analysis.)

Update 20140619: See also [Vuong 1989], cor. 7.3 et seq. and cor. 7.5 et seq. where Wilks's result holds if the larger model is correctly specified. (If it is not, its limiting distribution is not necessarily $\chi^2$ with the correct number of degrees of freedom.) The easiest failure is getting the wrong number of d.o.f. by having redundancy. Other failure modes are possible. This result doesn't seem to notice whether the smaller model is or is not misspecified. (It seems to be acceptable for the larger model to incorporate any misspecification of the smaller model.)

Eric Towers
  • 67,037
  • Redundancy is an interesting form of misspecification, but Wilks' proof assumes an invertible covariance matrix (and of course there are infinitely many misspecified models with an invertible cov matrix). Given the invertibility assumption, I still don't see why the proof fails with misspecified models. – ratsalad Jun 19 '14 at 12:13
  • @ratsalad: Updated, with reference. – Eric Towers Jun 19 '14 at 15:27
  • Vuong's paper is actually one of the motivations for my question. Has Vuong actually shown that the limiting distribution is not $\chi^2$ under misspecification? It is not clear to me. But my question is about Wilks' proof, which appears to still work under misspecification. (Aside: AIUI, the smaller model inherits specification/misspecification from the larger model). – ratsalad Jun 19 '14 at 16:28
  • No. Vuong has shown that under misspecification the limiting distribution may not be $\chi^2$, or may be $\chi^2$ with the wrong number of d.o.f. It is still possible for a happy coincidence to give the right distribution, so no such proof can exist (without additional hypotheses capturing that such a coincidence does not occur). Wilks's theorem has no additional hypotheses capturing that such a coincidence has occurred, so Wilks's proof cannot work under uncontrolled misspecification. (I may have the inheritance sentence backwards; it's been a while since I thought about this in detail.) – Eric Towers Jun 19 '14 at 17:03
  • OK, though I still don't understand how Vuong has shown what you claim. But assuming he does: why then does Wilks proof not work? His proof does not seem to need any "happy coincidence" assumption; it appears to work even for misspecification, since the key assumption in the proof is simply that the asymptotic distribution of the MLE is multivariate normal (with positive definite covariance matrix, which a misspecified model can have). – ratsalad Jun 19 '14 at 17:35
  • A misspecified model can have those things, it can also fail to have those things. Redundant variables break definiteness and can cause the asymptotic MLE to be multivariate normal in fewer dimensions than it lives in, so the distribution may well be invariant with respect to changes in some combination of the variables. For instance, it may be normal in the combination $x+y$ and constant in the combination $x-y$, so has zero covariance in the $x-y$ direction, so is no longer positive definite. – Eric Towers Jun 20 '14 at 02:57