Maximum Likelihood Estimate of a a discrete r.d - I spent more than 4 hours on this questions, help!!

Question

Suppose $X$ is a discrete r.d with the following p.d.f:

$$ \begin{array}{c|lc} X & \text{0} & \text{1} & \text{2} & \text{3} \\ \hline p(x) & 2\theta/2 & \theta/3 & 2(1-\theta)/3 & (1-\theta)/3 \end{array} $$

where $0\leq\theta\leq1$ was taken from such a distribution $(3,0,2,1,3,2,1,0,2,1)$. What is the MLE of $\theta$?

Likelihood function: $$(2\theta/2)^2.(\theta/3)^3.(2(1-\theta)/3)^3.((1-\theta)/3)^2$$ Log-Likelihood function: $$2\log(2\theta/2)+3\log(\theta/3)+3\log(2(1-\theta)/3)+2\log((1-\theta)/3)$$ Set derivative to zero: $$2/\theta+3/\theta-6/(2-2\theta)-2/(1-\theta)=0$$ $$\frac{5-5\theta}{\theta}-\frac{6+6\theta}{2-2\theta}=2$$ $$5-5\theta=2\theta+\frac{6\theta+6\theta^2}{2-2\theta}$$ $$2-2\theta(5-7\theta)=6\theta+6\theta^2$$ Finally I got something like this: $$10+8\theta^2=30\theta$$ Don't know how to find $\theta$. But I guess I have made mistake because the answer is $\theta=0.5$.

I spent more than 4 hours on this, tried different ways, like simplify the likelihood function before $"\log"$ them and still cannot work out, anyone help please?

heropup · Accepted Answer · 2016-07-19T17:51:04.457

Your calculation could be greatly simplified if you first algebraically simplify the likelihood while ignoring all constants of proportionality (i.e. factors of the likelihood that are constant with respect to the parameter). That is to say, we can write $$\mathcal L(\theta \mid \boldsymbol x) \propto \theta^5 (1-\theta)^5,$$ from which we obtain $$\ell(\theta \mid \boldsymbol x) = 5 \left( \log \theta + \log (1-\theta) \right).$$ Now it is trivial to compute $$\frac{\partial \ell}{\partial \theta} = 5 \left( \frac{1}{\theta} - \frac{1}{1-\theta} \right) = 0,$$ hence $\hat \theta = 1/2$ as claimed.

Note also that, although it makes no difference to the likelihood whether $\Pr[X = 0 \mid \theta] = 2\theta/2$ or $2\theta/3$, it is important to understand in a qualitative sense why it makes no difference. For had it been $\theta + c$ for some constant $c$, then it does affect your MLE estimation. Why? The reason is that when $\theta$ is regarded as a variable quantity (as it is in the likelihood, given the observed sample), we see that the relative likelihood is unaffected by any scaling factors.

This of course does not excuse the error of the sum of probabilities not adding to unity.

score 1 · Answer 2 · answered Jul 19 '16 at 15:23

1

Your $p(x)$ is wrong: its sum is $4/3$, not $1$. And who would write $2\theta/2$ instead of $\theta$? I suspect that $2\theta/2$ should be $2\theta/3$ and $2(1-\theta)/2$ should be $2(1-\theta)/3$.

answered Jul 19 '16 at 15:23

Robert Israel

448,999

Sorry for that. $2(1−θ)/2$ was a typo, it should be $2(1−θ)/3$. But $2θ/2$ is right. So the question is wrong? – BigData Jul 19 '16 at 15:29
@BigData sum up and check, does it equal $1$? – user190080 Jul 19 '16 at 15:33
The total probability always has to be $1$. – Robert Israel Jul 19 '16 at 15:34
sum up $= θ+(\frac{-3θ+3}{3})$ – BigData Jul 19 '16 at 15:42
May I ask are my steps correct? If yes then I will try to change $2θ/2$ as $2θ/3$ and do it again – BigData Jul 19 '16 at 15:43
@BigData you should get something like $θ+(\frac{-2θ+3}{3})\ne1$...and yes, try to set $2\theta/2 \to 2\theta /3$ as Robert already suggested. Perform your calculations and check whether it fits your desired result – user190080 Jul 19 '16 at 15:48
$2θ/2$ is same as $2θ/3$? because if we take the derivative of $2log(2θ/2) = \frac{2}{2θ/2} = 2/θ$ and $2log(2θ/3) = \frac{4/3}{2θ/3} also = 2/θ$ – BigData Jul 19 '16 at 17:09

Maximum Likelihood Estimate of a a discrete r.d - I spent more than 4 hours on this questions, help!!

2 Answers2