6

Here's an example of what I am asking :

$X_1,\ldots,X_n $ i.i.d $N(\phi , 1)$ where $\phi \in \mathbb{R}$. Let $\gamma = P(X_1\leq 1)$.

Give a sufficient statistic of $\gamma$.

This question is part of a midterm exam I took on april.

I see that $\gamma_{m\ell} = \Phi(1 - \phi_{m\ell}))$, where $\Phi$ is the standard normal distribution function. I know that in this case, $\phi_{m\ell} = n^{-1}\sum(X_i^2)$ which is a sufficient statistic. Does that make $\gamma_{m\ell}$ one also ?

Thank you for your answers.

Community
  • 1
Donno
  • 153
  • 1
  • 5
  • I just answered the highlighted question here: https://math.stackexchange.com/questions/2334885/sufficient-statistic-for-a-strange-parameter-on-normal-distribution/2334896#2334896 – Michael Hardy Jun 24 '17 at 16:48
  • The question in the subject line, however, is different. I think there are cases in which the MLE is not sufficient. – Michael Hardy Jun 24 '17 at 16:49
  • 2
    In answer to your headline, no a Mle is not always sufficient (consider a case like the Cauchy distribution with a location parameter that has no one dimensional sufficient statistic). But a mle is always a function of the sufficient statistic. But in regard to your detailed question an invertable function of a sufficient statistic is sufficient. – spaceisdarkgreen Jun 24 '17 at 16:59
  • Following up on the comment by "spaceisdarkgreen": This answer deals with a case in which the MLE is not a sufficient statistic: https://stats.stackexchange.com/questions/174117/maximum-likelihood-estimator-of-location-parameter-of-cauchy-distribution – Michael Hardy Jun 24 '17 at 18:16
  • 2
    There is an error in the question: For an i.i.d. sample from the family $N(\varphi,1)$, the sum of squares is not a sufficient statistic, but the sum of first powers is. – Michael Hardy Jun 24 '17 at 18:28
  • As spaceisdarkgreen already mendtioned, MLE is not always sufficient, because there are situations where $\theta$ is one-dimensional and there are no one-dimensional sufficient statistic. Simple example: $U[\theta, \theta+1]$ - uniform distribution on $[\theta, \theta+1]$. Here $(X_{(1)}, X_{(n)})$ is minimal sufficient statistics. – Botnakov N. Dec 15 '23 at 13:52

1 Answers1

5

Several points of confusion are in this question:

  • The subject line says "Is the maximum likelihood estimator always a sufficient statistic?". That would leave the impression that that is what this question is about. The short answer is "no". More on that below . . .

  • Then it says "Here's an example of what I am asking :" and then presents the following problem:
          $X_1,\ldots,X_n $ i.i.d $N(\varphi , 1)$ where $\varphi \in \mathbb{R}$. Let $\gamma = P(X_1\leq 1)$.
          Give a sufficient statistic of $\gamma$.
    To approach that question by thinking about MLEs might make sense if the MLE were always a sufficient statistic. However, it is easy to show that $X_1+\cdots+X_n$ is sufficient for $\gamma$ without going into that. I answered that question today, here. I briefly considered closing this present question as a duplicate of that.

Following up on the comment by "spaceisdarkgreen": The following answer deals with a case in which the MLE is not a sufficient statistic:

https://stats.stackexchange.com/questions/174117/maximum-likelihood-estimator-of-location-parameter-of-cauchy-distribution

That answer gives a numerical method for finding the MLE. How do we know it is not a sufficient statistic? That is dealt with in Bernard Lindgren's Statistical Theory, 4th edition, and right now I can't find it with the help of the index. On one occasion I told the author of that book that I couldn't find it, and he pointed out where it is, and I've forgotten, so definitely this is a deficiency in the index. He died shortly after that, so I won't be able to ask him again. However, it is commonplace to find it asserted, even if unusual to find it proved, that the complete set of all $n$ order statistics from a sample of size $n$ is the coarsest you can get with the Cauchy family. For any i.i.d. sample, the complete set of order statistics is sufficient, but for many it is not minimal. For this one it is minimal. So this is a case in which the MLE is not a sufficient statistic. I think there are simpler examples, but I don't have one at the tip of my tongue. Romano & Siegel's book Counterexamples in Probability and Statistics might have one.

Michael Hardy
  • 1
  • Simple example: $U[\theta, \theta+1]$ - uniform distr. on $[\theta, \theta+1]$. It's known that $(X_{(1)}, X_{(n)})$ is minimal sufficient statistics. Hence there are no one-dimensional sufficient statistics. Thus MLE isn't sufficient. Remark: as Romano, Siegel (see 7.2) show, there is a real-valued sufficient statistic for a sample from a Gaussian distr. with unknown mean and unknown variance, and this example shows that we should be very careful while speaking about dimensions, because their argument also works for $(X_{(1)}, X_{(n)})$ and for Cauchy distr. with a location parameter etc. – Botnakov N. Dec 15 '23 at 14:14