Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3828 questions
1
vote
0 answers

investor’s loss function gradient calculation

I'm trying to understand a study that recently has been published by Lin et al. (2023). Through this study, an Online Portfolio Selection algorithm has been proposed which takes advantage of mean reversion and meta-learning approaches (in other…
Shayan
  • 113
1
vote
1 answer

Derivative of a matrix given with a root

Let us defined the operator $\boldsymbol{N}(x)=\sqrt{\boldsymbol{M}^{\dagger-1}(x)\,\boldsymbol{M}^{-1}(x)}$. assuming that we know $\frac{d\boldsymbol{M}(x)}{dx}$, how do I compute $\frac{d\boldsymbol{N}(x)}{dx}$?
Yair
  • 501
1
vote
2 answers

Derivative of $a^TA(x)^{-1}b$ with respect to x

Let's say $a$ and $b$ are $n$ by $1$ vectors and $A(x)$ is an $n$ by $n$ matrix that depends on scalar parameter $x$. I would like to take derivative of $$\frac{da^TA(x)^{-1}b}{dx}.$$ I know that $$\frac{da^TA^{-1}b}{dA} = - A^{-T}ab^T A^{-T}$$ from…
user1292919
  • 1,895
1
vote
1 answer

Matrix calculus with row vectors

The rules for matrix calculus I find assume column vectors. Are the rules different for row vectors? (I am having a hard time finding them) I used them when deriving a formula for backpropagation. \begin{gather*} Y\ =\ XW\ +\…
BPDev
  • 113
  • 3
1
vote
2 answers

How to differentiate inverse of a matrix inside frobenius norm?

Basically, I have an equation in the form $$ f = \left \| A^{-1} \right \|^2 $$ I need to differentiate the above equation wrt A. The matrix A is just a 2X2 matrix so I tried to solve it by brute force. It worked but the solution is very long. Not…
Archaic
  • 19
1
vote
0 answers

Derivative of $x^TA$ wrt x

Wikipedia says that given $f(x) = x^TA$: $$\frac{\partial f}{\partial x} = A^T$$ but I am having trouble understanding this result. I tried doing the following: $$f(x+h) - f(x) = h^TA$$ I am now stuck at making the above expression into a linear…
Rainroad
  • 676
1
vote
0 answers

Matrix derivative with respect to vector

Suppose I have a matrix $f = \begin{bmatrix}x_1 & x_2 \\ 1 & 0\end{bmatrix}$. I would like to compute $\frac{dx'fx}{dx}$ with $x = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$. One way is to first compute $x'fx = x_1^3+x_1x_2+x_1x_2^2$, then…
Tianyu
  • 11
1
vote
1 answer

Derivative of matrix $(A^TA)^{-1/2}D(A^TA)^{-1/2}$ w.r.t. $A_{ij}$

I want to find derivative of matrix $(A^TA)^{-1/2}D(A^TA)^{-1/2}$ w.r.t. $A_{ij}$ where D is a diagonal matrix. Alternatively, it is okay too to have $$\frac{\partial}{\partial A_{ij}} a^T(A^TA)^{-1/2}D(A^TA)^{-1/2}b$$ Is there any reference for…
Daniel Li
  • 3,200
1
vote
2 answers

derivative of square root of quadratic form with respect to matrix

How would I go about calculating $ \frac{d\textit{$\alpha$}}{d\boldsymbol{A}}$ for $$\textit{$\alpha$} = \sqrt{\boldsymbol{x}^{\intercal}\boldsymbol{A}\boldsymbol{x}}$$ where $\alpha$ is a scalar, $\boldsymbol{x}\in\mathbb{R}^{n}$, and…
migs
  • 13
  • 2
1
vote
1 answer

How do you calculate a partial derivative of a matrix with respect to a scalar in a vector?

Let's say I have a matrix, $A$: \begin{pmatrix} a_{11} & a_{12} \\\ a_{21} & a_{22} \end{pmatrix} vector $p$: \begin{pmatrix} p_{1} \\\ p_{2} \end{pmatrix} and vector $q$: \begin{pmatrix} q_{1} \\\ q_{2} \end{pmatrix} If I have some function,…
Chrispresso
  • 113
1
vote
1 answer

What is a determinant in matrix ? NOT how to get it but WHAT is it in real life

We know to get determinant in a matrix we use sarrus, laplace, x methods and so on But question : WHAT is it, according to some pages is about permutations the products of element of the matrix , but WHAT IS IT ?
NIN
  • 135
1
vote
1 answer

Derivative Matrix with general function and product

Given $F,f:\mathbb R\to\mathbb R$ such that $F'=f$ and $\pmb a,\pmb b\in\mathbb R^n$, compute $$\frac{d}{d\pmb X}\left(\pmb a^T F\left(\pmb X\right)\pmb b\right)$$ where $\pmb X\in\mathbb R^{n\times n}$. My guess is $\operatorname{diag}(\pmb b)…
foreignvol
  • 926
1
vote
1 answer

Applying the chain rule in matrix form to prove Loss function's derivatives...

I want to prove $\nabla_A J = \nabla_Z J \cdot B^T$ where $Z=AB$. $A$ is a $m \times n$ matrix and $B$ is a $n \times k$ matrix. The function $J$ is not given to me. I began this proof by first writing $B^T$ as $$ B^T = \begin{pmatrix} b_{00} &…
mathnewbie
  • 47
1
vote
1 answer

Derivative of Matrix elements w.r.t. to scalar

First of all, excuse the stupid question. Unfortunately, I couldn't find something similar already asked. Consider the matrix $$ A(r_1, r_2) = \begin{pmatrix} r_1^2 & 3 \\ 2 & r_2 \end{pmatrix}. $$ I'd like to know how I can get the Jacobian of this…
Ronaldinho
  • 243
  • 1
  • 5
1
vote
0 answers

How to prove the matrix is a symmetric

What is the gradient of $\theta^T X^TX \theta$ ? The text book says the gradient is $2\theta^TX^TX$, From this link, I got answer $2X^TX \theta$. This two answers might be the same, if the matrix is a Symmetric matrix that $2X^TX\theta =…
Shark Deng
  • 113
1 2 3
…
13 14