Convexity of log-determinant

Question

I'm having a difficult time determining if the following function is convex

$$f(X) = \log {\rm det}(X^T A X)$$

where $A \in \mathbb{R}^{r \times r}$ is a symmetric positive definite matrix and $X \in \mathbb{R}^{r \times u}$ with $u < r$ and $X'X = I$. I've worked on find the second derivative with respect to $X$, but am not very confident in my solution. Any suggests on steps to proceed would be appreciated.

EDIT: Because the domain of $X$ is nonconvex, the function cannot be convex, as pointed out by @RobertIsrael.

Use $\det(XY) = \det X \det Y$, $\det X^T = \det X$ and $\det X > 0$ if $X$ is positive definite to reduce the problem to $g(\alpha) = \log \alpha$ where $\alpha \in R^{+}$. — Biswajit Banerjee, Dec 04 '15 at 03:59
Is this true for all $X$ and $Y$? My impression was that this only held for square $X$ and $Y$. — user23658, Dec 04 '15 at 04:09
If its any help, $\frac{\partial}{\partial X} \log{\rm det}(X^TAX) = 2(X^TAX)^{-1}AX$. I was stuck on taking the second derivative. — user23658, Dec 04 '15 at 04:12
I assume you've used $\log \det(X) = \text{tr} \log (X)$ in coming up with the first derivative and that hasn't helped? — Biswajit Banerjee, Dec 04 '15 at 04:47
No I didn't. For d1, use $\frac{\partial \log {\rm \det}(X^TAX)}{\partial (X^TAX)} \frac{\partial X^TAX }{\partial X}$. And I am not sure that what you've written makes sense; how are you defining $\log(X)$? Elementwise? — user23658, Dec 04 '15 at 04:50
$X^T X = I$? The set of such $X$ is not convex, so how can you even ask if $f(X)$ is convex? — Robert Israel, Dec 04 '15 at 04:55
One way to examine Robert's point is to take $A = I$ to get $f(X) = 0$ and think about the convexity of that function. — Biswajit Banerjee, Dec 04 '15 at 05:31
@RobertIsrael's point ends the discussion. The domain of the function is not a convex set, and that is a necessary criterion for convexity. Even $f(X)=|X|_F^2$, which is trivially convex, becomes non-convex when restricted to a domain like $X^TX=I$. — Michael Grant, Dec 05 '15 at 22:28

charlestoncrabb · Answer 1 · 2015-12-04T04:10:53.723

2

(EDIT: If $A,X$ are both square, then ...) As the comment suggest, we know that since the determinant is multiplicative and $\det X^T=\det X$, we have that $$\det(X^TAX)=\alpha\det(X)^2$$ where $\alpha=\det(A)>0$.

However, the function $g(x)=x^2$ is not logarithmically convex, therefore the function $f(X)$ is not either.

edited Dec 04 '15 at 04:10

answered Dec 04 '15 at 04:05

charlestoncrabb

2,628

Does anything change when you consider that $X$ and $A$ are not of the same dimension and $X$ is not square? – user23658 Dec 04 '15 at 04:07
@user23658, that is an interesting question. – charlestoncrabb Dec 04 '15 at 04:10
The multiplicity of the determinant only holds under square $X$ and $A$, I believe. Not sure tho! That's why I ask. – user23658 Dec 04 '15 at 04:10
My instinct is telling me the square case suggests the general $f(X)$ is not concave, let me think on it... – charlestoncrabb Dec 04 '15 at 04:12

Biswajit Banerjee · Answer 2 · 2015-12-04T05:19:34.867

The Hessian can be used to find a sufficient (but not necessary) condition for convexity. According to the author, the first derivative of the objective function is $$ D^1 := \frac{\partial}{\partial X} (\log\det(X^T A X)) = 2(X^T A X)^{-1} AX = 2 B^{-1} A X $$ The second derivative can be computed by going into index notation (summation over repeated indices). Then $$ (D^1)_{in} = 2B^{-1}_{il} A_{l m} X_{mn} $$ Then $$ (D^2)_{inpq} := \frac{\partial (D^1)_{in}}{\partial X_{pq}} = 2\left[\frac{\partial B^{-1}_{il}}{\partial X_{pq}} X_{mn} + B^{-1}_{il} \frac{\partial X_{mn}}{\partial X_{pq}}\right] A_{lm} $$ Now, (without symmetrizing) $$ \frac{\partial X_{mn}}{\partial X_{pq}} = \delta_{mp}\delta_{nq} $$ Therefore, $$ B_{il}^{-1} \frac{\partial X_{mn}}{\partial X_{pq}} A_{lm} = B_{il}^{-1} \delta_{mp}\delta_{nq} A_{lm} = \delta_{nq} B_{il}^{-1} A_{lp} $$ Consider the case where $A = I$. Then $B = X^T A X = X^T X = I$ and $D^1 = 2X$. So $$ (D^2)_{inpq} = 2\delta_{ip}\delta_{nq}\,. $$ That is the simplest result you can get because $X$ is not square. Not sure if this helps in determining convexity.

Not sure if you saw the edit above but the derivative you use should have an inverse and be scaled by 2. — user23658, Dec 04 '15 at 04:45

Convexity of log-determinant

2 Answers2