Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3828 questions
3
votes
1 answer

General Chain Rule for element wise function and Matrix vector product composition

I am trying to apply the general chain rule for a composition of two functions. The first outer function is a element wise function $\phi(x)$ and the second a matrix vector multiplication $W^Tx$: $$D_x[\phi(W^Tx)]$$ As far as i understand in…
Fiorentino
  • 31
  • 2
3
votes
2 answers

Derivative of transpose of inverse of matrix with respect to matrix

I want to calculate $$ \frac{\mathrm{d}\left(\mathbf{C}^{-1}\right)^T}{\mathrm{d}\mathbf{C}} = \quad? $$ From The Matrix Cookbook I know…
Amit Singh
  • 43
2
votes
1 answer

Derivative of a matrix function with respect to a matrix

I am trying to differentiate the following function, with respect to a matrix $X$: $$ \operatorname{tr}(AX(X^TX)^{-1}B) $$ where tr corresponds to the trace. Is there an easy way to see what the derivative will be? I've come across rules for…
NSR
  • 864
2
votes
1 answer

lu decomposition of submatrix

We have a matrix $A$ and $A'$ is a submatrix of $A$. first we make LU decomposition of $A$, $$A = L_a \cdot U_a.$$ Now, I want to make LU decomposition of $A'$, $$A' = L_a' \cdot U_a'.$$ is there any relation between $L_a, U_a, L_a',U_a'$? Maybe…
lhc1988
  • 21
2
votes
2 answers

Is there any references to this optimization problem？

Suppose that $\mathbf{a}$ is a $N$-dimentional column vector and $\mathbf{a}=[a_1,a_2,...,a_N]^{T}$, here $a_i \ge 0 $ for $i\in [1,N]$ and $\mathbf{W}$ is a symmetry matrix. I want to ask if there is any reference on how to solve the following…
ai zhongguo
  • 99
2
votes
2 answers

How to compute derivative of a matrix with Hadamard product

I have the matrix $A$ as follows: $$ (\mathbf{X} \odot \mathbf{A}) \mathbf{1}_p $$ where $\odot$ is the Hadamard (elementwise) product, $\mathbf{X}\in \mathbb{R}^{n \times p}$, $\mathbf{A}\in \mathbb{R}^{n \times p}$, and $\mathbf{1}_p$ is a…
MOHAMMED
  • 169
  • 5
2
votes
3 answers

Computing $\frac{\partial(X^Tb)}{\partial X}$

In the matrix cookbook there is an identity $$\frac{\partial (a^TX^T b)}{\partial X} = ba^T$$ I recently ran into a problem where I had to compute $$\frac{\partial (X^T b)}{\partial X}$$ but I couldn't find a formula for this. However it seems that,…
tail_recursion
  • 411
2
votes
2 answers

Derivative using chain rule

I have the following function $$f=\sum_{i=1}^{n} \sum_{j=1}^{p} \bigg \lbrace y_{ij}(\boldsymbol{\lambda_j^{T}m_i}+\beta_{0j}) - \frac{1}{2} (\boldsymbol{\lambda_j^{T}m_i}+\beta_{0j}) - \frac{1}{4} \lbrace…
tata
  • 41
2
votes
0 answers

Possible typo in this paper differentiating a matrix product

I'm reading this paper and it seems to me like there's a typo at the top of page 53. Here's the problematic paragraph: The possible typo is on the third line. $S = R^\dagger \lambda R$ $dS = dR^\dagger \lambda R + R^\dagger d\lambda R + R^\dagger…
Allure
  • 616
2
votes
1 answer

Matrix Calculus - Composition with Differentiable Function

I'm fitting a model to some data, and am trying to take the following derivative: $$\frac{\partial}{\partial V}\|U \phi(VX)-Y\|_F^2$$ where $\phi$ is a differentiable function applied entry-wise. From the matrix cookbook, I've (doubtfully) gotten to…
gfppoy
  • 903
2
votes
0 answers

Matrix calculus proof. Could someone help me please?

For the following proposition could someone perhaps explain to me moving from equation 45 to 46? I would be most grateful if you could do so with an example with simple matrices A and x. Thanks again!
InvestingScientist
  • 433
2
votes
1 answer

Product rule for matrix derivative

For $\nabla_X Y(X) = \nabla_X U(X)V(X)$, is there a general formula to differentiate it? I guess it is something like $\left(\frac{\partial U}{\partial X}\right)^T V(X) + \frac{\partial V}{\partial X} U(X)^T$ (denominator layout). However, when I…
charlieh_7
  • 143
2
votes
1 answer

Gradient of $L=\|Y - X\|_F^2 + \sum_i^I u_i ( \| A_i X_i \|_2^2 - \alpha_i ) + \sum_k^K v_k ( {\rm tr}( (X^* B_k X^T)) - \beta_k)$ w.r.t. $X$

I am not sure how to start with computing the gradient $\frac{\partial L}{\partial X}$ of the following function: \begin{align} L = \| Y - X \|_F^2 + \sum_i^I u_i \left( \| A_i X_i \|_2^2 - \alpha_i \right) + \sum_k^K v_k \left( {\rm tr} \left(…
user550103
  • 2,688
2
votes
0 answers

Implicit Function Theorem for Second Derivatives

Let $F:R^n \times R^m \to R^m$ be a function. Suppose we have $F(x,y)=0$ where $y \in R^m$ is implicitly defined as a function of $x \in R^n$: $y=h(x)$. We have $F(x,h(x))=0$. By the Chain Rule, the Implicit Function Theorem can be derived:…
kx526
  • 145
2
votes
2 answers

Why is $\triangledown_x x^TA^TAx = 2A^TAx$ not $2x^TA^TA$?

A proof I'm looking at shows $\triangledown_x x^TA^TAx = 2A^TAx$. I did Matlab symbolic calculation to verify this, but I found the converse. It should be $2x^TA^TA$. $\frac{d}{dx}x^TA^TAx = \\ [ 2x_1A_{1,1}^2 + 2A_{1,2}x_2A_{1,1} + 2x_1A_{2,1}^2…
drerD
  • 589
1
2
3
…
13 14