1

In my study, I faced with the following formula and considered shortening the description.

Let $A$ be an $q \times m \times p$ tensor including $m \times p$ matrices $A_0, A_1, \ldots, A_{q-1}$, and $B$ be an $p \times n$ matrix.

We have another $q \times m \times n$ tensor such that $$ C = \begin{bmatrix} A_0 B & A_1 B & \cdots & A_{q-1} B\end{bmatrix}. $$

Then, how can we write $C$ by $A$ and $B$? It is a simple notation C = A @ B in Python/NumPy.

What is the name of this operation?

YuuM
  • 13
  • This is an example of a trace (which is, as you'd guess, a variation of the usual notion of trace of a matrix) or a contraction, in particular of the tensor product $A \otimes B$. If that's useful for you I can promote this comment to an answer. – Travis Willse Mar 02 '23 at 01:33
  • @TravisWillse Thank you for your helpful comment. I'd like to ask you some points.
    1. The meaning of example is, I guess, there are several choices for a trace or a contraction, and I should define the one to use. Is that correct?
    2. According to your helpful advice, I've tried to get the trace of a tensor product $\mathrm{tr} (A\otimes B)$ by np.trace(np.tensordot(A, B, axes=0)), but is not equal to A@B in calculation. If you're familiar with Python/NumPy, would you give me some comments?

    (FYI: The Einstein summation np.einsum('ijk,kl->ijl', A, B) gives the same result as A@B.)

     – YuuM Mar 02 '23 at 03:40
  • That's right. I'm not familiar with NumPy (I should change that---), but what you call an Einstein summation is exactly a trace/contraction. Presumably np.trace() is simply contracting a pair of indices other than the ones you have in mind. In general, you can form a trace for each choice of pair (covariant index, contravariant index). If you have an inner product on the underlying vector space you can identify the two index types and hence contract any pair of indices. – Travis Willse Mar 02 '23 at 20:46
  • N.b. in practice the involved tensors often have additional symmetries, and as a consequence different choices of indices can lead to the same trace operations (or operations identical up to sign), or just the zero operation. For example, forming the trace of the only $2$ indices of an alternating $2$-tensor (using an inner product) gives the zero map. – Travis Willse Mar 02 '23 at 20:49
  • @TravisWillse Thanks a lot! Maybe I got it. Please promote your first comment to be an answer. By the way, in an expression, I think we can write the operation as a contraction by ${C_{ij}}^l = {{A}{ij}}^{k} {{B}{k}}^{l}$. On the other hand, how can we write the trace of tensor with giving the index? – YuuM Mar 03 '23 at 02:57
  • 1
    I should add a comment. The np.trace function accepts axes1 and axes2 parameters where we can specify the axes. Given np.trace(np.tensordot(A, B, axes=0), axis1=2, axis2=3), we get the same result as A@B. – YuuM Mar 03 '23 at 03:06

1 Answers1

0

This operation is an example of a trace, or a contraction (of indices), of the tensor product $A \otimes B$. In general, you can form a trace for each choice of pair (covariant index, contravariant index) of indices on a particular vector space $\Bbb V$. If you have an inner product on $\Bbb V$ you can identify the two index types with one another and hence contract on any pair of indices on $\Bbb V$.

In our situation there is essentially only one trace. Borrowing the choice of covariant and contravariant indices specified in the comments realizes our operation as the contraction $$\operatorname{tr}: (\Bbb A^* \otimes \Bbb B^* \otimes \Bbb C) \times (\Bbb C^* \otimes \Bbb D) \to \Bbb A^* \otimes \Bbb B^* \otimes \Bbb D$$ for vector spaces $\Bbb A, \Bbb B, \Bbb C, \Bbb D$, of respective dimensions $q$, $m$, $p$, $n$, characterized by its definition on pairs $$((\alpha \otimes \beta \otimes c), (\gamma \otimes d)) \mapsto \gamma(c) \alpha \otimes \beta \otimes d .$$

I suspect that NumPy essentially treats tensors as elements of tensor products of $\Bbb R^k$ for various $k$, each equipped with the standard inner product, in which case we can view our contraction map as $$\operatorname{tr}: (\Bbb R^q \otimes \Bbb R^m \otimes \Bbb R^p) \times (\Bbb R^p \otimes \Bbb R^n) \to \Bbb R^q \otimes \Bbb R^m \otimes \Bbb R^n.$$

In situations where there is more than one possible trace of the tensor product, one sometimes sees the notation $\operatorname{tr}_{ab}$ to indicate the contraction of the $a$th and $b$th indices (using a background inner product if necessary), so we might denote our trace as $\operatorname{tr}_{34}$.

Travis Willse
  • 99,363