0

I want to calculate the maximum likelihood estimation of $P(x|\theta) = 1/(1-\theta)$ for $\theta<= x <=1$.

I end up with $log1 - nlog(1-\theta)$ and when I want to take the derivitie I end up with $-n*-1/(1-\theta)$ how should I proceed? Because I cannot set this equal to zero and get a value for $\theta$

I know this has some relations possibly with order statistics but I am not sure how to derive MLE for it.

Mona Jalal
  • 331
  • 4
  • 16
  • Hello @Mona, are you sure of the PDF $f(x \vert \theta)$ that you've shared ? It's not a PDF – Ahmad Bazzi Sep 16 '18 at 19:53
  • that's a good question tbh! Instructor mentioned if there is a typo tell it and get bonus points! Last semester I had a similar one 1/theta and for that I used order statistics. I am not sure how order statistics would make sense here exactly. can you tell why it is wrong? – Mona Jalal Sep 16 '18 at 19:58
  • @AhmadBazzi also why does it need to be a PDF to begin with? thanks for explanation – Mona Jalal Sep 16 '18 at 19:59
  • 1
    Maximum likelihood is the maximum of the likelihood function, which is a pdf.. – Ahmad Bazzi Sep 16 '18 at 20:00
  • 1
    oh the lower bound of $x$ is $\theta$.. so it is now a PDF – Ahmad Bazzi Sep 16 '18 at 20:02

2 Answers2

2

So lets write down the PDF of $n$ independent samples generated from $x \vert \theta$, i.e \begin{equation} f(x_1 \ldots x_n \vert \theta) = f(x_1 \vert \theta) \ldots f(x_n \vert \theta) = \frac{1}{(1 - \theta)^n}, \qquad \theta < x_k < 1, \quad k = 1 \ldots n \end{equation} The log likelihood is the log of the above function, i.e. \begin{equation} l(\theta) = \log \frac{1}{(1 - \theta)^n} = - n \log (1 - \theta) \end{equation} Maximizing the $l(\theta)$ is equivalent to minimizing $-l(\theta)$ \begin{equation} \hat{\theta} = \operatorname{argmin}_{\theta} \log (1 - \theta) \end{equation} Now, if you do the derivative, as you say, you will not get anywhere. However, the minimum is clear, it is at $\theta = 1$, you'd get $-\infty$ as the optimum value. BUT, you have constraints here, which is that \begin{equation} \theta < x_k < 1 \qquad k = 1 \ldots n \end{equation} So to minimize $\log (1 - \theta)$ subject to the $n$ constraints, you must estimate $\theta$ as \begin{equation} \hat{\theta} = \min \lbrace x_i \rbrace_{i=1}^n \end{equation}

PS: A more rigorous mathematical arguement is by deriving the Lagrangian function taking into account the above inequalities, but here the problem is easy to solve. You need $\theta$ as close as possible to $1$ without violating the constraints. The only way to do so is by picking the minimum of the sample values.

Ahmad Bazzi
  • 12,076
1

When you cannot set derivative to zero, it means the maximum occurs at the boundary. Indeed, it is clear the maximum occurs at $\theta=\min x_i$ if you actually kept the indicator in your likelihood function: $$P(x\mid\theta)=(1-\theta)^{-1}1_{\theta\leq x\leq 1}$$ So with independent $x_1,\dots,x_n$, we get $$L(\theta)=L(\theta\mid x_1,\dots,x_n)=(1-\theta)^{-n}1_{\theta\leq\min x_i}1_{\max x_i\leq 1}$$ and hence the derivative $$ (\log L)'(\theta\mid x_1,\dots,x_n)= \begin{cases} \dfrac{n}{1-\theta}>0 & \theta<\min x_i\leq\max x_i\leq 1\\ 0 & \theta>\min x_i\text{ or }\max x_i>1 \end{cases} $$ Hence the maximum occurs at $\theta=\min x_i$ by left-continuity of our $L(\theta)$

user10354138
  • 33,239