Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3828 questions
1
vote
1 answer

Derivative of Matrices

Assuming that the following are matrices with their corresponding dimensions: $\mathbf{X}\in\mathbb{R}^{p\times d},\mathbf{A}\in \mathbb{R}^{p\times p}$ which is symmetric and $\mathbf{B} \in \mathbb{R}^{(p-d)\times (p-d)}$, then what is the…
J.C
  • 11
1
vote
1 answer

Lower bound of the smallest eigenvalue of block hermitian matrix

Let $J$ and $L$ two squared matrix and let $A$ the matrix as follows A=\begin{pmatrix} J^TJ+L^TL&J^T\\ J &I\\ \end{pmatrix} where I denote the identity matrix I ask if there is a lower bounds of the smallest…
aigle81
  • 13
1
vote
1 answer

The derivate formula in Matrix Cookbook

As said in the book, The basic assumptions about matrix derivates can be written in a formula as $ \frac{∂X_{kl}}{∂X_{ij}} = δ_{ik}δ_{lj} $ But I don't know how I can use this formula to calculate matrix derivates. Could anyone give some examples?
Zhe Chen
  • 113
1
vote
1 answer

why the vector derivative of $\frac{d(x^Ta)}{dx} = \frac{d(a^Tx)}{dx} = a^T$, why it's $a^T$ not $a$

$\frac{d(x^Ta)}{dx} = \frac{d(a^Tx)}{dx} = a^T$ I was confused by this simple formula for a few weeks. I thought $x^Ta$ is an scalar, and it's derivative respect to a column vector should be an vector, e.g. $a$ instead of $a^T$. am I missing…
Long
  • 269
1
vote
1 answer

Matrix calculus - matrix derivative

I don't understand how given $X$ is $m \times n$ $\Sigma$ is positive definite $f=\theta^TX(\Sigma^{-1})^TX^T\theta$ How is $df/d\theta = 2X\Sigma^{-1}X^T\theta$.
Lee
  • 811
1
vote
1 answer

matrix calculus product rule confusion

According to The Matrix Cookbook, The gradient of the product is $$\nabla_x(f(X)g(X))=f(X)\nabla_X g(X)+g(X)\nabla_X f(X).$$ But then $$ \nabla_x X^TAX = \nabla_x(X^TA) X+X^TA\nabla_xX$$ $$ =AX+X^TA $$ instead of the supposed answer…
Allan Ruin
  • 111
  • 4
1
vote
2 answers

How to get the derivative of a matrix function?

I want to get the derivative of a matrix function as follow: $$\frac{\partial f(\boldsymbol{AX})}{\partial \boldsymbol{X}}$$ which $f(\cdot)$ is a scalar function, and the result as I think should be the same shape as the matrix $\boldsymbol{X}$
maple
  • 2,883
  • 2
  • 29
  • 37
1
vote
0 answers

Derivative of exponential of a matrix

I want to find the derivative of $$x=\exp{At}$$ with respect to $A$. In this case $A$ is a matrix. Is the solution $$\frac{\mathrm dx}{\mathrm dA}=t\cdot \exp{At}$$ Or is it different?
darwin rajpal
  • 363
1
vote
0 answers

optimality conditions for functions in matrix space

If $f : \mathbb{R}^{n} \rightarrow \mathbb{R}$, then I know that $\nabla f(x) \in \mathbb{R}$ and $\nabla^2 f(x) \in \mathbb{R}^{n \times n}$. If I find a point $z$ such that $\nabla f(z) = 0$ and $\nabla^2 f(z) \succeq 0$, then I know that $z$ is a…
steve
  • 757
1
vote
1 answer

Second derivative of $\sum_i \left(\frac{1}{1+\|Xa_i\|^2}-b_i\right)^2$

Let $A$ be a $n\times m$ matrix, $X$ be a $d \times m$ matrix and $b_i\in \mathbb R$, $i=1\dots n$. $$ f(x) = 4 XA^\top (I+\Delta_X)^{-3}(B+B\Delta_X-I)A $$ where $B=\sum_ie_ib_ie_i^\top$ and $\Delta_X=\sum_i e_ie_i^\top AX^\top XA^\top e_ie_i^\top$…
davcha
  • 1,745
1
vote
0 answers

Using the chain rule for matrix differentation

I have a question regarding the derivative of the $f: X\rightarrow (X'X)^{-1}$ where $X$ is a $m\times n$ matrix. Now the differential of $d(X^{-1})= -X^{-1}(dX)X^{-1}$. To get the derivative $D(X'X^{-1})$ , do I need to use the chain rule as it…
joog
  • 15
1
vote
1 answer

Showing $A(x,y)A(y,z)=A(x,y)$ for matrix valued function A with following properties

Suppose we have two matrix valued functions A, B where: $A:\mathbb{R}^{2}\rightarrow \mathbb{R}^{nxn}$ $B:\mathbb{R}\rightarrow \mathbb{R}^{nxn}$ with $\forall \; x,y \in \mathbb{R}$: $\frac{\delta A}{\delta x}=B(x)A(x,y)$ $\frac{\delta A}{\delta…
guest
  • 13
1
vote
0 answers

Integral over Matrixfunction

I need to work out the Integral $$\int \frac{1}{(x\mathbb{1}-A)^2}B\frac{1}{x\mathbb{1}-C}\ dx$$ Where $A,B,C$ are matrices which generally do not commute and $x$ is real. $\mathbb{1}$ denotes the identity matrix. I am ok with using matrix functions…
Hagadol
  • 401
1
vote
1 answer

matrix kronecker product property

$\det(\mathbf{A}\otimes \mathbf{B})=(\det(\mathbf{A})^m)(\det(\mathbf{B})^n)$. $\mathbf{A}$ is $n\times n$ matrix. $\mathbf{B}$ is $m\times m$ matrix. $\otimes$ is Kronecker product. How can i prove that?
user137231
  • 141
1
vote
1 answer

The spectral norm of the difference of two projective matrices

A $n\times n$ projective matrix $A$ is a Hermitian and idempotent matrix, that is, $$A^*=A=A^2.$$ Suppose that $A,B$ are both projective, then $\|A-B\|\leq 1$, here $\|\cdot\|$ is the spectral norm, $$\|A\|=\max_{\|x\|_2=1}\|Ax\|_2,…
xldd
  • 3,485
1 2 3
…
13 14