0

To maximize the log likelihood of my parameters, I need to find the argument that maximizes the following function: $$\sum_{\substack{i,\,j\\ i \neq j}} n_i \log(x_i) + n_j \log(x_j) - (n_i + n_j) \log(x_i+x_j) $$

The $n_i$ have a known value, the unknowns are the $x_i$

I don't think there will be a closed formula, I think it needs an iterative approach.

This is a practical problem, so the use of optimizer is fine. I tried using the optimizer of scipy in Python, but it did not converge.

Any help much appreciated, thanks.

DevShark
  • 113
  • it seems you have a lot of choice variables ... namely $n^2-n$ ... – Math-fun Aug 04 '15 at 10:34
  • Should it be $(n_i+n_j)$ rather than $x_i+n_j$ in the third term? – Empy2 Aug 04 '15 at 10:35
  • You are right, sorry, I will fix it. – DevShark Aug 04 '15 at 10:37
  • Yes. Typically, I have about 1,000 observations, so i and j are between 0 and 1,000. I am trying to find the $x_i$ that maximize the likelihood, so I can choose them to be anything I want. It makes sense for them to be positive though. – DevShark Aug 04 '15 at 10:51
  • It has a symmetry if you replace all the $x_i$ by $kx_i$, the function is unchanged. So you might set $\sum x_i=1$. But that might make it harder. – Empy2 Aug 04 '15 at 11:33

2 Answers2

1

So you are actually maximizing $\sum_{i \neq j} \log{\left(\frac{x_{i}^{n_{i}}x_{j}^{n_{j}}}{(x_{i}+x_{j})^{n_{i}+n_{j}}}\right)}$.

This can also be written as $\log{\left( \prod_{i \neq j}\frac{x_{i}^{n_{i}}x_{j}^{n_{j}}}{(x_{i}+x_{j})^{n_{i}+n_{j}}}\right)}$. Since $\log$ is an increasing function, you need to maximize $\prod_{i \neq j}\frac{x_{i}^{n_{i}}x_{j}^{n_{j}}}{(x_{i}+x_{j})^{n_{i}+n_{j}}}$. I don't know of any software that would help to optimize for this kind of objective function (usually quadratic objective functions are hard enough), but you have enough symmetry in the variables that perhaps something could be done with more information.

[edit] Strike the comment about symmetry, you lose the symmetry if the $n_{i}$ are not all equal. Still, the way to optimize is to take the partial derivatives with respect to each $x_{i}$ (!!!) and try to get them all $=0$; then you can try to figure out where the max/mins are.

xxxxxxxxx
  • 13,302
0

Differentiate the log-likelihood, we want to solve $$(N-1)\frac{n_i}{x_i}=\sum_{j\neq i}\frac{n_i+n_j}{x_i+x_j}\\ \sum_{j\neq i}\left(\frac{n_i+n_j}{x_i+x_j}-\frac{n_i}{x_i}\right)=0$$
This has an obvious solution $x_i=n_i$.

Empy2
  • 50,853