2

Suppose $\{X\}_{i=1}^n\overset{i.i.d}{\sim}X$, and $X\in\mathbb{R}^d$ has density,$$f_{\theta}(x)=c\exp\left\{-||x-\theta||\right\},\theta\in\mathbb{R}^d,$$ where $||\cdot||$ denotes the Euclidean norm.

Show that the MLE $\hat{\theta}$ exists but is not unique when $n$ is even.

I know how to prove that the MLE $\hat{\theta}$ exists, by noticing $\underset{\theta\rightarrow\partial\mathbb{R}^d}{\lim} \log f(\theta)=-\infty$, however I don't know how to prove it is not unique. I understand that the log-likelihood function, $l(\theta)=-\sum_{i=1}^n||x_i-\theta||$ is a convex, but not strictly convex, function, but this does not guarantee the nonuniqueness. I also tried taking the first and second gradient of $l(\theta)$. But even I have a non-negative definite $l(\theta)$, I still don't have the non-uniqueness...

---Update---

The original question is here. This is from Mathematical Statistics (Bickel). enter image description here

Tan
  • 621
  • 2
    It might help to take a visual example. Take n to be 2 and visualise your two data points in R^d (and take d = 2 or 3 if that helps). Now what estimator can you pick that minimises the sum of the two distances (from your estimate to the two points)? The answer is that you can pick any point on the straight line between the two measurements. – Jeroen van der Meer Jan 30 '21 at 18:42
  • Think about the median for even set of different numbers. – Royi Jan 30 '21 at 18:55
  • @Royi It is Euclidean norm, but not $l-$1 norm. I think "median" is not the minimizer for our case. – Tan Jan 30 '21 at 18:59
  • @JeroenvanderMeer Thanks! Let's say $d=2$. I think when $n=2$, I got it. However, for $n=4$, I feel that the $\theta$ that is closest to the four points in $\mathbb{R}^2$ is really one single point? – Tan Jan 30 '21 at 19:05
  • I'm inclined to agree, which is why I feel conflicted about the premise of the question. But there is a very rare (and stupid} situation in which you still have nonuniqueness: namely, when all four points are on a single line. For instance if $x_1 = x_2$ and $x_3 = x_4$ then the situation is the same as when n = 2. I wonder if that's what the author of the question had in mind... – Jeroen van der Meer Jan 30 '21 at 19:13
  • 1
    Also regarding your comment to @Royi: I do think I agree with him that what you're looking for is called the geometric median. Wikipedia tells me it's also called the $L_1$ estimator but I don't think it has anything to do with the $L^1$-norm. – Jeroen van der Meer Jan 30 '21 at 19:14

1 Answers1

1

The minimizer of such set of points is given by the Geometric Median.

For example, for $ d = 1 $ you get the Median which is not unique for an odd set of different numbers.

For higher dimensions you need to take care of the case the points are collinear, which basically means that the problem, is again, equivalent to 1D.

Royi
  • 8,711
  • Thank you. I've made the notation consistent. Also glad to know $\hat{\theta}$ is the geometric median. The question is for general case (not only collinear points). I am still not sure how to prove the non-uniqueness of the MLE. – Tan Jan 30 '21 at 19:40
  • @Tan did you see the special case for 4 coplanar points on Wikipedia? – LinAlg Jan 30 '21 at 20:29