Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3828 questions
1
vote
1 answer

How to compute derivative with respect to a matrix?

If I have a function $f = v^TAv$, I can compute its derivative w.r.t. $v$ as follows $$ \begin{align} f &= \sum_i \sum_j A_{ij}v_iv_j \\ \frac{\delta f}{\delta v_k} &= \sum_j A_{kj}v_j + \sum_i A_{ik}v_i \\ &= A_{k,:}v + vA_{:,k} \\ \frac{\delta…
Kong
  • 884
1
vote
1 answer

Matrix derivative of block-diagonal matrix of covariance matrices, with respect to matrix square root

My matrix calculus is entirely self-taught (and the only formal univariate calculus that I had was in high school), so I'm proud to have made it as far as I have. But I am stumped. Here is the problem: $$ y = \mathbf{a^T \Psi b}\\ \mathbf{\Psi} =…
generic_user
  • 133
1
vote
1 answer

Derivative of l2 norm with chain rule

If $X$ is a $n$ by $d$ matrix, $\alpha$ is a $n$ by $1$ vector, let $f(\alpha) = \left\Vert X^\top\alpha \right\Vert_2^2$, what is $\frac{df}{d\alpha}$
ZigZagZebra
  • 115
1
vote
2 answers

(Matrix Calculus) Chain Rule

Let $A \in \mathbb{R}^{n \times n}$ be an invertible matrix, $v \in \mathbb{R}^{n}$ and $\kappa: \mathbb{R}^{n} \rightarrow \mathbb{R} $ . What is $\frac{\partial\ \kappa(A^{-1}v)}{\partial\ A}$? I've been trying all sorts of equations from the…
ASML
  • 31
1
vote
1 answer

Differentiating a matrix wrt to another matrix

I have an orthogonal matrix $R$ and I am trying to differentiate the term $R^{\top}R$ wrt $R$. However, I am stuck at this point because the matrices are not symmetric. Anybody has an idea or trick. I also know that $\text{det}\left(R\right) =…
Kumar
  • 332
1
vote
1 answer

Difficulty in vectorizing a matrix derivative

Consider vectors $x$ and matrix $A$, the vector $z = Ax$ and the scalar $E = z^Tz$. I want to compute the derivative $\frac{dE}{dA}$. This is not too troublesome if we look at just one component of $E$: \begin{align} \frac{\partial E}{\partial…
shimao
  • 202
1
vote
0 answers

derivative of an implicitly given matrix function

Let $K,C,M$ be 3 $n \times n$ real regular matrices. Let $v_i, i = 1,...,k$ an orthonormal basis for a $k$-dimensional subspace of $\mathbb{C}^n$ and define $$V = [v_1, v_2, ...,v_k]$$ Let $F(\omega) = K + i \omega C- \omega^2 M$ and let $z(\omega)$…
Koen
  • 682
1
vote
0 answers

Choose $\lambda$ in ridge regression so that $\beta$ shrinks to unit ball?

Recall ridge regression: $$\beta(\lambda)=(X^TX+\lambda I)^{-1}X^TY.$$ Is there an easy way I can find $\lambda$ so that the solution shrinks to the unit ball: $\beta^T(\lambda)\beta(\lambda)=1$? My first thought is define $f(\lambda) =…
Jirapat Samranvedhya
  • 281
1
vote
1 answer

Question about Matrix Derivative Rule

Let ∇A(x) denote the derivative of X with respect to the matrix A. Let X^T denote the transpose of matrix X. Then the following two rules hold. 1) ∇A (trace of AB) = B^T 2) ∇A (trace of AB A^T C) = CAB + C^T A B^T While both rules are…
user98235
  • 391
  • 3
  • 14
1
vote
3 answers

Proof: $I_n + V$ is invertible if and only if $V=I_n$

So I know there are some related posts that tackle similar questions, but in my case, the conditions are slightly different. I have been thinking about this question, but I don't know where to begin. It is given that $V^2 = I_n$ Prove that $(I_n +…
Emily
  • 21
1
vote
2 answers

Matrix calculus - simplify the formula

I am learning matrix calculus and I would like to understand how the derivative of the following function: $$ \mathit{f}(\mathbf{x}) = \mathbf{x}^T \mathbf{Ax} $$ is calculated. I am able to derive the differentials up to this…
Szpilona
  • 151
1
vote
1 answer

Partial derivative of matrix-vector product

I have to compute the following: $$ \frac{\partial}{\partial \mathbf{A_p}} \sum_{p=1}^P \mathbf{A_p}\mathbf{y}(t-p), $$ with $\mathbf{A}$ being a matrix and $\mathbf{y}$ a vector. Do I have to write the matrix-vector product as a sum?
wrong_path
  • 369
1
vote
1 answer

Chain rule for a matrix derivative

I am trying to derive the following expression: $\frac{\partial} {\partial \theta_i}Tr(A(\theta)^{-1}y (A(\theta)^{-1}y)^{T}B(\theta))$. what I did is : $\frac{\partial} {\partial \theta_i}Tr(A(\theta)^{-1}y (A(\theta)^{-1}y)^{T}B(\theta))= Tr…
Emily W
1
vote
1 answer

Derivative: matrix quadratic form when $\mathbf{B}$ is dependent on $\mathbf{m}$

I have the following equation: $$ f = \mathbf{m^{T}Bm} $$ where B is: $$ \mathbf{B} = \frac{1}{3(m_{x}^{2} + m_{y}^{2})}\begin{pmatrix} 1 & 0.5 & 1\\ 0.5 & 1 & 0.5\\ 1 & 1 & 0.5 \end{pmatrix} $$ and m is: $$ \mathbf{m} = \begin{pmatrix} m_{x}\\…
Johnnylin
  • 135
1
vote
3 answers

Matrix Derivative Problem

Sorry for boring you my friends. I am haunted by a question of matrix derivative. $q$ is a vector of dimension $n\times1$; $A_1$, $A_2$, $...$ $A_n$ and $B$ are matrix with constant coefficients of dimension $n\times n$; $A_1q$ , $A_2q$ $...$ and…
Zihan Shen
  • 127
1 2 3
…
13 14