Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3828 questions
1
vote
1 answer

Can I say that $\frac{d}{dl}f=Tr((\frac{\partial}{\partial K}f)^\intercal \frac{\partial K}{\partial l})$?

Let's assume that I know $\frac{\partial}{\partial K}f$, where f is scalar valued, and $K$ is a matrix-valued function of $l\in \mathbb{R}$, i.e. $K=K(l)$. $df=Tr((\frac{\partial}{\partial K}f)^\intercal dK)$, with $dK=\frac{\partial K}{\partial l} …
An old man in the sea.
  • 5,241
1
vote
1 answer

Laplacian of $ \nabla^2 f( A x)$ in terms of Laplacian of $ \nabla^2_y f( y)$

How to find the Laplacian \begin{align} \nabla^2_x f( A x) \end{align} where $n \times m$ matrix and we know know the laplacian of \begin{align} \nabla^2_y f( y) \end{align} This question is about chain rule for the Laplacian. I search but could…
Lisa
  • 2,941
1
vote
1 answer

Derivative of ${\rm Re} \left\{ {\rm tr} \left [Z^H \left( AX \right) \right] \right\}$ w.r.t. $X \in \mathbb{C}^{m \times n}$.

Question Let us say that I have a function $$f = {\rm Re} \left\{ {\rm tr} \left [Z^H \left( AX \right) \right] \right\} \ ,$$ where the matrices are $Z \in \mathbb{C}^{k \times n}$, $A \in \mathbb{C}^{k \times m}$, and $X \in \mathbb{C}^{m \times…
user550103
  • 2,688
1
vote
2 answers

How to find $\frac{\partial}{\partial \mathbf{Q}}\left(x_2^\intercal (\mathbf{I}_T\otimes \mathbf{Q})^{-1}x_2\right)$?

How to find $$\frac{\partial}{\partial \mathbf{Q}}\left(x_2^\intercal (\mathbf{I}_T\otimes \mathbf{Q})^{-1}x_2\right)$$? Q is symmetric I'm thinking we could use some sort of chain rule getting $$ x_2 x_2^\intercal \frac{\partial}{\partial…
An old man in the sea.
  • 5,241
1
vote
1 answer

What's the derivative of this quadratic with respect to matrix A?

What's $$\frac{\partial}{\partial \mathbf{A}}\left((x_1-(\mathbf{I}_T\otimes \mathbf{A})x_2)^\intercal \mathbf{K}^{-1}(x_1-(\mathbf{I}_T\otimes \mathbf{A})x_2)\right)$$? K is symmetric I'm thinking we could use some sort of chain rule getting…
An old man in the sea.
  • 5,241
1
vote
1 answer

Idempotent matrix algebra

Let $M$ be an $n\times n$ idempotent matrix and $a,b$ are $n\times 1$ vectors with $a\le b\le 0$. Then, $$a^\top Mb \ge 0 ?$$ If so, and if $a\ge b\ge 0$, then the above result still hold true? Thanks,
user1292919
  • 1,895
1
vote
1 answer

How does one derive with respect to a matrix when that matrix is part of a summation whose result is inverted?

A and B are matrices, a and b are scalars. How would I derive the following expression with respect to A? $$ \frac{\partial} {\partial A} ((a* A + b * B)^{-1})$$
DeepDeadpool
  • 197
1
vote
1 answer

Derivative of $\text{trace}(U^T x y^T V)$ with respect to $x$

I'm trying to compute the derivative of $\text{trace}(U^T x y^T V)$ with respect to $x$, where $U \in \mathbb{R}^{d_x \times k}$, $V \in \mathbb{R}^{d_y \times k}$, $x \in \mathbb{R}^{d_x}$, and $y \in \mathbb{R}^{d_y}$. I have so far computed the…
Yamaneko
  • 113
1
vote
0 answers

find the derivative of a function that involves complex-valued matrices

Consider the following function $f(\bf x)$: $f(\bf x) = \left\|(\bf A \bf x) \circ {(\bf A \bf x)}^* - \bf c\right\|_2^2$, where $\bf A \in {\mathbb C}^{N\times M}$ (complex-valued matrix) $\bf x \in {\mathbb C}^{M \times1}$ (complex-valued…
Mike
  • 41
1
vote
1 answer

Formula of PRML(3.13) about matrix derivative

My English is not very good, so I am sorry for the inappropriate expression. I have a question in "Pattern Recognition and Machine Learning"(Christopher M. Bishop), and it is a formula which is described in (3.13) of ch 3.1.1 I don't understand why…
komorebi
  • 11
1
vote
1 answer

Derivative of Product of Scalar Equation and Matrix with Respect to a Vector

Consider the follow equation $$f(\vec{x}) = c(\vec{x})B\vec{x}$$ where $c(\vec{x})$ is given by the generalized Rayleigh quotient $$c(\vec{x}) = \frac{\vec{x}^{T}A\vec{x}}{\vec{x}^{T}B\vec{x}}$$ $\vec{x}$ is a vector, and A and B are square…
Mario
  • 37
1
vote
2 answers

A is a hermitian matrix such that $A^2 = \mathbf 0$.Prove that $A = \mathbf 0$

A is a hermitian matrix such that $A^2 = \mathbf 0$.Prove that $ A = \mathbf 0$ I first assumed that $A =(a_{ij} )_{mXn} $ Then $\overline A =(\overline a_{ij} )_{mXn} $ Then $$\overline A^t =(\overline a_{ji} )_{nXm} $$ Now,…
Samya
  • 41
  • 6
1
vote
3 answers

Differentiate a Matrix-Transpose Product

if, $$y = w^T X + b$$ where $w$ is a $[13, 1]$ matrix and $X$ is a $[13, 1]$ matrix, what is $\frac{dy}{dw}$? $\frac{dy}{dX}$ seems to be $w^T$, but $\frac{dy}{dw}$ is not as clear to me. If the transpose wasn't there, it would seem like…
StopReadingThisUsername
  • 1,553
1
vote
1 answer

Matrix by scalar differentiation

Is $$\frac{d}{dx} \left(\mathbf{A}_{3\times3}\mathbf{U}(x)_{3\times3}\mathbf{b}_{3\times1}\right) = \mathbf{A}_{3\times3}\frac{d\mathbf{U}(x)_{3\times3}}{dx}\mathbf{b}_{3\times1}$$ Where $\mathbf{A}$ and $\mathbf{b}$ are independent of $x$. This…
oni
  • 13
  • 2
1
vote
1 answer

Is it possible in this case to calculate the derivative with matrix notation?

$$\tag 1 \frac{1}{2}{(y-Xw)}^T(y-Xw)$$ where: $y$ is $N \times 1$ $X$ is $N \times p$ $\omega$ is $p \times 1$ Is the vectorized way to write $$\tag 2 \frac{1}{2}\sum_{i=1}^n (y^{(i)}-x^{{(i)}^T}\omega)^2$$ I would like to differentiate $(1)$ with…
jacob
  • 113
1 2 3
…
13 14