Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3828 questions
0
votes
1 answer

Matrix Algebra - Deriving Fourier Coefficients using Least Squares

How is the following multiplication performed for this (cost)…
student1
  • 176
  • 1
  • 9
0
votes
1 answer

Obtaining 2nd Derivatives for LogLikelihood Function

L(β;σ^2) = -T/2*(log(2π))-T/2*(log(σ^2))-(1/2σ^2)*(y-Xβ)'(y-Xβ) Can someone clarify that ∂^2 L/∂β∂σ^2 = (1/σ^4)*(X'Xβ-X'y) ?? My tutor has it as -(My answer) but I can't work out why. I'll assume this is a mistake unless this can be proved…
hseager
  • 13
0
votes
1 answer

time derivative of Log(A(t)) for some Hermition square matrix

Assume A(t) is a time dependent n by n Hermition matrix. What is time-derivative of log(A(t)). I remember seeing this somewhere, but now I have a hard time finding it again.
fred
  • 1
0
votes
1 answer

Isotropic points with a jacobian matrix

As definition, in order to find the isotropic points of a jacobian matrix, the matrix's columns become orthogonal and equal to the magnitude. I don't quite understand this definition. if i have a matrix looks like this: a b c d Matrix…
Xitrum
  • 101
0
votes
1 answer

Vector by Matrix Derivatives in Computational Graphs

I'm trying to learn how to implement the back propagation pass given a computational graph. I've been following the following guide: https://homepages.inf.ed.ac.uk/htang2/mlg2022/tutorial-3.pdf In equation (4), page 4 of the PDF, the author writes…
David Yue
  • 225
0
votes
2 answers

Gradient and Hessian of square of Lp-norm $\| \mathbf{x}\|_p^2$, where $1 \leq p<\infty$

What are the gradient and Hessian of square of any Lp-norm $\| \mathbf{x}\|_p^2$, where $1\leq p<\infty$? One could assume that $\mathbf{x} \in \mathbb{C}^n$ (But I think one can also consider $\mathbf{x} \in \mathbb{R}^n$ if it is convenient to…
learning
  • 661
0
votes
1 answer

Derivative of a vector x wrt itself

I am attempting to better understand matrix calculus. Given a vector $x$, I understand that: $\frac{\mathrm{d} x^t}{\mathrm{d} x}= I$ but struggle to think about $\frac{\mathrm{d} x}{\mathrm{d} x}$. This website says it is also the identity matrix,…
Bepop
  • 129
0
votes
1 answer

Definition of Derivative of Matrix

Let's assume A is $n\times 1$ constants, $X$ is $n\times 1$ vector. Does derivative of transpose(A)* X on X should be transpose(A) instead of A? I saw both transpose(A) and A from different resources and would like to confirm the right answer.
Eddy
  • 31
0
votes
1 answer

How to calculate the derivative of the following function?

How would you go about calculating the derivative d/dx of the below function f(x)? a and c are scalars. $f(x)=\phi(x)^T\phi(x)$, $\phi(x)=$$\begin{bmatrix}\sin(ax_1) \\ \sin(acx_2) \\ 2x_1 \\ 2cx_2\end{bmatrix}$ My guess would be the following (i.e.…
user3187119
  • 87
0
votes
1 answer

Funky derivative of trace of matrix with composition

Let's say we have: $z_i$ a vector in $\mathbb{R}^k$ $W$ a matrix in $\mathbb{R}^{d, k}$ and $\Psi$ a invertible diagonal matrix in $\mathbb{R}^{d, d}$ I know that for ex (matrixcookbook): $\frac{\delta Tr\ z_i^T {W}^T \Psi^{-1}Wz_i}{\delta W} = 2…
BrES
  • 1
0
votes
0 answers

Why is the derivative of the identity matrix 0?

I was told in my advanced linear algebra class that $$\partial_t I = 0$$ but i do not see how. I’m not too familiar with matrix calculus but i get that $I$ doesn’t change as $t$ changes but how do you differentiate a matrix? Shouldn’t it be the 0…
John D
  • 130
0
votes
0 answers

matrix-by-matrix derivative in case of x^Tx

I've read somewhere on this forum that if we have three matrices $A,X,B$ that are $m\times n$, $n\times k$ and $k\times l$ respectively, that $$\frac{\partial (AXB)}{\partial X}=B^T \otimes A$$ Now if we apply this to product $x^Tx$ where…
Madara Uchiha
  • 678
0
votes
2 answers

How to compute the Jacobian of a function with an Hadamard division?

I have a function $$ f: \mathbb{R}^n \mapsto \mathbb{R}^n \\ f(x) = A^{-1}(b \ \oslash \ x) $$ where $A \in \mathbb{R}^{n\times n}$ and $b, x \in \mathbb{R}^n$, and I cannot figure out how to compute the Jacobian of $f$ by using Matrix calculus
NoFishLikeIan
  • 65
0
votes
2 answers

Differentiate $()=^{}$ with respect to $A$

Given column vector $x$ and matrix $A$, how can we take the derivative. of $x^{T}Ax$ with respect to $A$?
juliamartin27
  • 21
0
votes
1 answer

Matrix Differentation w.r.t. a Vector

Given $\alpha$ a scalar, $\mathbf{X}$ an $N_x \times 1$ vector that is a function of $\alpha$, $\mathbf{U}$ an $N_u \times 1$ vector that is a function of $\alpha$ and $\mathbf{X}$, and $\mathbf{C}$ an $N_u \times N_u$ matrix that is a function of…
Mohamed Abdelhamid
  • 105
1 2 3
…
13 14