0

I have the following function:

$-logL(\beta|t,X)= -\sum_{n=1}^{N} \{t_n \cdot log(y_n) +(1-t_n)\cdot log(1-y_n)\}$

where $y_n=(1+exp(-x_n'\beta))^{-1}$ and where $\beta$ and $x_n$ are both kx1 vectors and $t$ is a vector of zeros (corresponding to $y_n=0$) and ones (corresponding to $y_n=1$) and $X$ is a NxK matrix if stacked $x_n$ vectors. Basically it is an application of the maximum likelihood method where $x$ represents data and $y_n$ represents the probability of succcess of a binary variable.

I need to show that the given function (that I need to minimize with repsect to beta) is convex such that the extremum is a minimum. I know the ruls about the hessian matrix but I have troubles in applying them. The hessian is given by

$H= \sum_{n=1}^{N} \{y_n \cdot (1-y_n)x_nx_n'=X'RX$

where $R$ is a diagonal matrix with entries $R_{nn}=y_n \cdot (1-y_n)$

giorgio
  • 5
  • 1
    this follows trivially from convexity of the log-sum-exp function – LinAlg Jan 26 '19 at 14:02
  • could you please be more explicit? I am confused because the exponential function has a negative power, and of that I have the reciprocal. is that still convex? Besides, log is concave, exp is convex, what's the composition? – giorgio Jan 26 '19 at 14:05
  • Are the entries on the diagonal of R nonnegative? Can you factor H to show that it's positive semidefinite? – Brian Borchers Jan 26 '19 at 16:03
  • Yes the entries are all nonnegative. There was a typo and I corrected it. – giorgio Jan 26 '19 at 16:06
  • @giorgio have you done a quick search on log-sum-exp? How does a power interact with a logarithm? When $f(x)$ is convex, is $f(Ax)$ also convex? Are sums of convex functions convex? Combine those answers and you got your answer. – LinAlg Jan 26 '19 at 17:32

0 Answers0