1

I'm given $f(x;\theta) = \frac12 e^{-|x - \theta|}$, $-\infty < x < \infty$ and $0 < \theta < \infty$. I want to find the maximum likelihood estimator of $\theta$. I found:

$$\ln L(\theta; x_1,..., x_n) = -n \ln 2 - \sum |x_i - \theta|$$

Usually I would differentiate and find the maximum. Here differentiation does not work. But by inspection, $\sum |x_i - \theta|$ is always positive so $L$ has a maximum when $\sum |x_i - \theta| = 0$. But how can I express $\theta$ in terms of the $x_i$'s?

George
  • 543
  • Why is $-n\ln 2$ positive? (but, of course, that term doesn't matter.) Observe that for given $x_1,\cdots, x_n$, it might not be possible for the sum to be zero. Why doesn't differentiation work in this problem? – Michael Burr Aug 16 '15 at 11:49
  • 2
    The value should be maximized when $n$ is odd and $\theta$ is the middle $x_i$ and when $n$ is even, (anywhere) between the middle two $x_i$'s. – Michael Burr Aug 16 '15 at 11:52
  • right, I don't know why I said it was positive. How can I differentiate? Each $x_i$ is possibly less than $\theta$ and possibly greater than it. How can I remove the absolute value? – George Aug 16 '15 at 11:53
  • 1
    Think about differentiating between the $x_i$'s. You'll get a bunch of $\pm 1$. If you think about whether $\theta$ is to the left or the right of $x_i$, you can figure out if the function is increasing or decreasing (you can figure out how many $1$ vs $-1$'s you have). – Michael Burr Aug 16 '15 at 11:55
  • Hint: Let $\theta$ be smaller than the smallest $x_i$. Then for all $i$, $|x_i-\theta|=x_i-\theta$. Then, by differentiating with respect to $\theta$, you get $-1$. Since this is true for all $i$, you find that the derivative is $n$. Therefore, the value increases if you go to the right. – Michael Burr Aug 16 '15 at 11:58
  • got it! if $\theta$ is the least of the $x_i$'s then the derivative is $n$.. to minimize the derivative then we should make $\theta$ greater than some $x_i$'s, but making it greater than all of them makes the derivative $-n$, so we should make a balance so that $\theta$ is the middle value of the $x_i$'s when $n$ is odd and we take it to be somewhere between the middle two $x_i$'s when $n$ is even. So $\hat{\theta}$ is the median. – George Aug 16 '15 at 12:06
  • Hint: $$\frac{d}{d\theta}|x-\theta|=\mathrm{sign}(\theta-x)$$ $$\frac{d}{d\theta}L(\theta)=-\sum_x\mathrm{sign}(\theta-x)$$ – Did Aug 16 '15 at 12:06

1 Answers1

3

Hint: Suppose that the $x_i$'s have been reindexed so that they are increasing order: $x_1\leq x_2\leq\cdots\leq x_n$. Suppose that $x_k<\theta<x_{k+1}$. Then, $$ \ln(L;x_1,\cdots,x_n)=-n\ln 2-\sum_{i=1}^k(\theta-x_k)-\sum_{i=k+1}^n(x_k-\theta). $$ Taking the derivative with respect to $\theta$ gives $$ -\sum_{i=1}^k(1)-\sum_{i=k+1}^n(-1)=-k+(n-k)=n-2k. $$ Therefore, if $k<\frac{n}{2}$, then as $\theta$ grows, the derivative is positive, so $L$ increases. If $k>\frac{n}{2}$, then as $\theta$ grows, the derivative is negative, so $L$ decreases. This mimics the first derivative test for a maximum.

Michael Burr
  • 32,867