2

Given the density function $f(x;\theta)=\theta x^{\theta -1}$ with $\theta \gt 0$ and $x \in (0,1)$, once found the MLE estimator $\hat{\theta}=\frac{-n}{\sum{\log{X_i}}}$ i want to show the consistency of such an estimator. I was suggested to apply the strong law of large numbers, although i am not sure how. Should I first obtain the expected value of the random variable $\log(X_i)$ and then apply the SLLN on them or is there another way?

Mattiatore
  • 643

2 Answers2

1

You can show that $Y = -\ln X \sim Exp(\theta)$, hence $\mathbb{E}[Y]=1/\theta$, thus by the SLLN and the continuous mapping theorem $$ \hat{\theta}_n \xrightarrow{a.s.} (1/\theta)^{-1} = \theta, $$ as $n \to \infty$.

V. Vancak
  • 16,444
1

If you are not familiar with SLLN, there are several other ways to approach the problem,

  1. The simplest: a property of ML Estimators is that they are consistent.

  2. Consistency you have to prove is $\hat{\theta}\xrightarrow{\mathcal{P}}\theta$

So first let's calculate the density of the estimator.

Observe that (it is very easy to prove this with the fundamental transformation theorem)

$$Y=-logX\sim Exp(\theta)$$

Thus $W=\Sigma_iY_i\sim Gamma(n;\theta)$ and $\frac{1}{W}\sim \text{Inverse Gamma}$ and consequently

$$\mathbb{E}[\hat{\theta}]=\frac{n}{n-1}\theta$$

$$\mathbb{V}[\hat{\theta}]=\frac{n^2}{(n-1)^2(n-2)}\theta$$

This grant the following are both true

$$\lim\limits_{n\to \infty}\mathbb{E}[\hat{\theta}]=\theta$$

$$\lim\limits_{n\to \infty}\mathbb{V}[\hat{\theta}]=0$$

This is an IFF condition for the convergence in $L^2$ which implies also the convergence in probability (the consistency of your estimator)

  1. Calculate the convergence in probability by the definition

Using 2 is a better way because you proved also the convergence in quadratic mean, that it is stronger

Trying all the method we showed is an useful exercise (IMHO)

tommik
  • 32,733
  • 4
  • 15
  • 34
  • Thanks for your answer! Regarding 1, can it be proved that every MLE is consistent? – Mattiatore Nov 25 '20 at 12:05
  • 1
    @MattiaMariantoni : Yes, of course! Actually in the basic courses this and other interesting properties of ML estimators are only given without proof. You can find the theorem (Th 10.1.6) and a sketch of the proof in Casella Berger. Details of the proof can be found, for example, in Stuart, Ord and Arnold - 1999, - Advanced Theory of Statistics, Chapter 18 – tommik Nov 25 '20 at 12:44