1

I have

$$ J_{\theta}(X) = - \frac 1 m \cdot \left[ y \cdot ln( h_{\theta} (X ) ) + ( 1 - y) \cdot ln ( 1 - h_{\theta}(X) ) \right] $$

I need $\frac d {d\theta} J_{\theta}(X)$. I tried many time, and here's my result

$$ \frac d {d\theta} J_{\theta}(X) = - \frac 1 m \cdot \left[ \frac y {h_{\theta}(X)} - \frac { (1 - y) } { 1 - h_{\theta}(X)} \right] \cdot \frac d {d\theta} h_{\theta}(X) $$

But when I plug in the result of $\frac d {d\theta}h_{\theta}(X)$, I cannot get the expected result, which is

$$ \frac 1 m \cdot (h_{\theta}(X) - y) \cdot X $$

$h_{\theta}(X)$ and its derivative are defined here.

David S.
  • 269

1 Answers1

0

It looks like you are trying to arrive at the maximum likelihood equations for a logistic regression model, where $h_\theta(X)$ is the parametrization of the probabilities of binomial random variables $Y$'s w.r.t. data $X$'s and parameters $\theta$.

I suggest to have a look at the first answer to this question and its calculations. Some care has to be taken w.r.t. the indices "hidden" in your equation; they play an important role. Remember, for example, that the $i$-th observation $y_i$ of the binomial r.v. $Y_i$ depends only on the $i$-th observation $x_i$ of the whole dataset $X$ (consequently, the probability for $Y_i$ is parametrized as $h_i:=h(x_i,\theta)$.)

Avitus
  • 14,018
  • 1
    Thanks a lot. It looks like that guy is doing the same thing as I am doing :) – David S. Jul 29 '14 at 03:09
  • Very good! Please, check my answer in the link if you need computations with indices (I am lazy and I did not want to write it twice, sorry) – Avitus Jul 29 '14 at 15:09