Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3828 questions
0
votes
1 answer

Gradient of squared distance function $\operatorname{dist}^2(z) = \frac{1}{2}\|z - \mathrm{proj}_{\mathcal{C}}\left( z \right) \|_2^2 .$

I am looking for a proof of the gradient of a squared distance function to a set $\mathcal{C}$, where the function can be shown as $$f(z) := \operatorname{dist}^2\left(z\right) = \frac{1}{2}\|z - \operatorname{proj}_{\mathcal{C}}\left( z \right)…
user550103
  • 2,688
0
votes
2 answers

Solving a linear system over $\mathbb{Z}_{5}$

I would like to ask how to solve this matrix, if I substitute for z (x, y, z) z = 4, I can't calculate the matrix. I tried to adjust the matrix to this shape, but I don't know how to proceed. $\left( {\begin{array}{ccc|c} 1 & 2 & 1 & 3\\ 1 & 0 & 0 &…
user886716
0
votes
1 answer

How to take the derivative of $ y = xA^T + b$ w.r.t. $A$ and $b$, where $x,b$ are vectors and $A$ is a matrix

How exactly could I take the derivative of the following expression? $$ y = xA^T + b$$ Let's say that I have $x \in \mathbb{R}^{n}$, $A \in \mathbb{R}^{m,n}$, $y \in \mathbb{R}^{m}$, and $b \in \mathbb{R}^{m}$. And, I wish to take the derivative of…
AlphaBetaGamma96
  • 105
0
votes
2 answers

How to find a derivative with respect to a matrix?

If I have $f: \mathbb{R}^{n \times m} \times \mathbb{R}^{n} \to \mathbb{R}^{m}$ And $f(K,t) = Kt + h$ where $h \in \mathbb{R}$ How would I find $\frac{\partial f}{\partial K}$ and $\frac{\partial f}{\partial t}$?
Shisui
  • 145
0
votes
2 answers

If the absolute values of all the eigenvalues of $C$ are smaller than $1$, then $I-C$ is non-singular.

I am reading "Matrix Calculus"(in Japanese) by Masahiko Saito. There is the following proposition in this book: Let $I$ be an identity matrix. Let $C$ be a square matrix. Let $A=\begin{pmatrix} I&B \\ O&C \end{pmatrix}$. If the absolute values of…
tchappy ha
  • 8,690
0
votes
1 answer

Derivative of function w.r.t to vector where matrix elements are functions itself

Consider the function $f: \mathbb{R}^N \to \mathbb{R}$ with $$ f(r) = ||Ap - b||_2^{2} = p^{\top}A^{\top}Ap - 2p^{\top}A^{\top}b + b^{\top}b $$ where $A \in \mathbb{R}^{N \times N}$ and $p,b \in \mathbb{R}^N$ The elements of the matrix $A$ are…
Ronaldinho
  • 243
  • 1
  • 5
0
votes
1 answer

For a matrix A, is there a function f(A) such that $\frac{df_{ij}}{dA_{kl}}=f_i^k (A) \delta^l_j$?

For a matrix A, is there a function f(A) such that $$\frac{df_{ij}}{dA_{kl}}=f_i^k (A) \delta^l_j$$ ? I am specifically interested in the case of a symmetric matrix $A$ and function f of rank 2. Prior research: I tried $e^A$ but it does not have…
my2cts
  • 148
0
votes
2 answers

On matrix derivation

I have the matrix equation $$ \boldsymbol{\Phi}\boldsymbol{w}=\boldsymbol{y} $$ where $\boldsymbol{y}$ has size $[v\times d]$, $\boldsymbol{\Phi}$ has size $[v\times c]$ and $\boldsymbol{w}$ has size $[c\times d]$. I am trying to compute…
Blade
  • 461
0
votes
1 answer

The steps to calculate the derivative

$\frac{d}{d\mathbf{w}}Xsig(X^T\mathbf{w})=Xdiag(sig(X^T\mathbf{w})\odot (1-sig(X^Tw)))X^T$ I want to know the step of getting the above result, thank you. The $sig$ is the element-wise sigmoid function. The convention is denominator layout.
7337dtd
  • 49
0
votes
1 answer

Derivative of square of a matrix

In this paper I'm reading, they say that if $\Psi(A) = A^2$ then the derivative $D\Psi(A)$ is defined by $D\Psi(A)(X) = AX + XA.$ I'm a bit confused by that line since I know the differential is given by $d(A^2) = A\, dA + dA\, A$. But to me this…
Kashif
  • 1,497
0
votes
1 answer

Matrix of a game

There is a game with 5 positions starting in position 3. two teams have to compete against each other with tasks in math. There cannot be a draw and every round one of the teams wins. One team is stronger than the other team. They start in position…
calculatormathematical
  • 2,222
0
votes
1 answer

Derivative of log determinant

It is known that $$ \frac{\partial}{\partial X} \operatorname{ln} | \operatorname{det}(X)| = (X^{-1})^{T} = (X^{T})^{-1}. $$ (The Matrix Cookbook, Petersen, K., 2012) Let $$ A \in R^{p \times p}, ~ \mu = (\mu_{1}, \cdots, \mu_{p})^{T} \in R^{p}.…
bakgu
  • 55
0
votes
1 answer

What is the matrix derivative of a matrix transformation?

What is $$\frac{\partial Ds}{\partial{D}}$$ where $D \in R^{m \times n}$ and $s \in R^n$? I'm using the "denominator layout", so I know that this has shape $m \times n \times m$. My guess is $$S + \frac{\partial s}{\partial D}D^T$$ where $S_{ijk} =…
Neil G
  • 2,479
0
votes
0 answers

Clarification on the gradient of $f(x) := \frac{1}{2}\left\| X (a+b) \right\|_2^2 + \mu \operatorname{Re}\left\{a^H X b\right\}$

I would like to compute the gradient of the following function \begin{align} f(x) := \frac{1}{2}\left\| X (a+b) \right\|_2^2 + \mu \operatorname{Re}\left\{a^H X b\right\} \end{align} where $X:= \operatorname{Diag}(x)$, $x \in \mathbb{R}^{n \times…
user550103
  • 2,688
0
votes
2 answers

Compute $\int_0^a {\bf e}_i^TA {\bf t} \, {\rm d} t_i$

How to compute the following integral \begin{align} \int_0^a {\bf e}_i^TA {\bf t} \, {\rm d} t_i \end{align} wher ${\bf e}_i$ is standard bases vector and $A$ is some matrix square full rank matrix. It should be a quadratic term. However, I am…
Lisa
  • 2,941
1 2 3
…
13 14