3

I am currently researching on providing some bounds on estimation using some information theoretic tools (I won't expend on that here for now, I may make a post about it later) and turns out that given a phenomenon $X$, an observation $Y$, then $\hat{x}(Y)$, the maximum likelihood estimator of $X$ based on $Y$, may apparently not be a sufficient statistic and this is a something I would like to study, the answer to this post states that such an example exists when $Y$ consists of samples that are not i.i.d but fails to provide such an example and I haven't found anything about it. Have anyone seen something of the sort ?

P. Quinton
  • 6,031

1 Answers1

6

If $T$ is a sufficient statistic for $\theta$ and a unique MLE of $\theta$ exists, then the MLE must be a function of $T$.

So if you can find a situation where there can be several maximum likelihood estimators, there remains a possibility that you can choose one MLE that might not be a function of a sufficient statistic alone.

A simple example to consider is the $U(\theta,\theta+1)$ distribution.

Consider i.i.d random variables $X_1,X_2,\ldots,X_n$ having the above distribution.

Then the likelihood function given the sample $(x_1,\ldots,x_n)$ is

$$L(\theta)=\prod_{i=1}^n \mathbf1_{\theta<x_i<\theta+1}=\mathbf1_{\theta<x_{(1)},x_{(n)}<\theta+1}\quad,\,\theta\in\mathbb R$$

A sufficient statistic for $\theta$ is $$T(X_1,\ldots,X_n)=(X_{(1)},X_{(n)})$$

And an MLE of $\theta$ is any $\hat\theta$ satisfying $$\hat\theta<X_{(1)},X_{(n)}<\hat\theta+1$$

or equivalently, $$X_{(n)}-1<\hat\theta<X_{(1)} \tag{1}$$

Choose $$\hat\theta'= (\sin^2 X_1)(X_{(n)}-1) + (\cos^2 X_1)(X_{(1)})$$

Then $\hat\theta'$ satisfies $(1)$ but it does not depend on the sample only through $T$.

StubbornAtom
  • 17,052
  • I am a bit curious, do you think there is always a element in the set of MLE that is a sufficient statistic ? (in this case yes, if we take $(X_{(1)}+X_{(n)})/2$) and is it unique ? Also I am interested in other measures of error, do you know if there are some examples of minimum MSE estimator that is not a sufficient statistic too ? – P. Quinton Dec 20 '18 at 16:40
  • @P.Quinton There is of course an MLE that is a function of $T$. Any MLE of the form $\alpha(X_{(n)}-1)+(1-\alpha)X_{(1)}$ is a function of the sufficient statistic $T$ where $\alpha\in(0,1)$ is a constant. – StubbornAtom Dec 20 '18 at 18:10
  • Yes but that's not really what I asked, in my case I was wondering if $\theta - \hat\theta - (X_1, \dots, X_n)$ is a Markov chain (supposing that $\theta$ is a random variable), I think that in your formula, if we know alpha then we can find $X_{(1)}$ and $X_{(n)}$ back, so it's a sufficient statistic, is that correct ? – P. Quinton Dec 20 '18 at 18:16