What's an infinite dimensional or function version of a tensor?

Question

A function $f$ is like an infinite dimensional vector with the norm $|f| = \int^b_a f(t)^2 \, \mathrm{d} t $ and dot product $f \cdot g = \int^b_a f(t) g(t) \, \mathrm{d} t $ where appropriate boundaries have to be chosen or you have to restrict functions so that problems with infinities don't pop up.

A linear operator, integral transform or its kernel $K$ is like an infinite dimensional matrix with the application operation being $Kf = \int^b_a K(s, t) f(t) \, \mathrm{d} t$. Matrix multiplication is $KL = \int^b_a K(s, t) L(t, u) \, \mathrm{d} t$. Again, you have to be careful to deal with problems with infinities.

How do we generalize to the next level? What is the tensor version?

In particular, what does tensor contraction and the tensor product mean?

I know tensors are composed out of linear combinations of tensor products of vectors (which are functions in this case.) But I'm confused about what linear combinations means in this case.

Suppose one has a tensor:

$$ T = \sum^n_{k=0} c_k (u_k \otimes v_k) \otimes w_k $$

What exactly does a linear combination mean?

Answer 1

Any finite-dimensional vector space $V$ is isomorphic to a coordinate vector space $\mathbb{R}^n$ which can be thought of as the vector space of functions $\{1,\cdots,n\}\to \mathbb{R}$.

Generalizing this, we can use a different index set than $\{1,\cdots,n\}$, for instance we can use a continuous interval as an index set. Then a "coordinate vector" whose indices are from $[0,1]$ should be thought of as a function $[0,1]\to\mathbb{R}$.

The dot product $f\cdot g=\sum_i f_i g_i$ (I will put all my indices downstairs) for $\{1,\cdots,n\}$-indexed coordinate vectors can then be generalized to $\langle f,g\rangle=\int_0^1 f(x)g(x)\,\mathrm{d}x$. Similarly, the equation $f=Ag$ (where $f,g$ are column vectors and $A$ is a matrix), written with indices as $f_i=\sum_j a_{ij}g_j$, with integration replacing summation becomes the kernel $f(x)=\int_0^1 A(x,y)g(y)\,\mathrm{d}y$.

With this perspective in mind, tensor contraction should generalize in an obvious way. For instance, the tensor contraction $\sum_{i,j,k} a_{ijk}b_{ir}c_{js}e_{kl}$ is now $\iiint a(i,j,k)b(i,r)c(j,s)e(k,l) \,\mathrm{d}i\,\mathrm{d}j\,\mathrm{d}k$, which is a function of $r$, $s$ and $l$. Indeed, the original kind of tensor contraction with a discrete set of indices can be considered a special case, since summation is just integration over a finite measure space.

Now let's consider tensor products. If $\mathbb{R}^X$ denotes the vector space of functions $X\to\mathbb{R}$, there is a canonical identification $\mathbb{R}^X\otimes\mathbb{R}^Y\cong\mathbb{R}^{X\times Y}$. There is a bilinear map $\mathbb{R}^X\times\mathbb{R}^Y\to\mathbb{R}^{X\times Y}$ where the pair $(f,g)\in\mathbb{R}^X\times\mathbb{R}^Y$ is sent to the function $X\times Y\to\mathbb{R}$ defined by $h(x,y):=f(x)g(y)$, and this extends to an isomorphism $\mathbb{R}^X\otimes\mathbb{R}^Y\to \mathbb{R}^{X\times Y}$. This works even if $X$ and $Y$ aren't discrete.

Pure tensors $u\otimes v$, then, correspond to separable functions $u(x)v(y)$. So then any arbitrary linear combination $\sum_i c_i(u_i\otimes v_i\otimes w_i)$ would correspond to a linear combination $\sum_i c_iu_i(x)v_i(y)w_i(z)$ of separable functions. (For example in differential equations, we sometimes find the separable solutions first, then superpose them to get the whole solution space.)

Answer 2

Disclaimer: this is just a (graduate) educated guess based on this. I've never really worked with such objects.

One way to think of finite dimensional tensors is as multilinear operators

$$ T:V^*\times\cdots\times V^*\times V\times\cdots\times V\rightarrow W $$ where $V,W$ are finite dimensional vector spaces and $V^*$ is the dual space of $V$. So, why not just consider multilinear operators on infinite dimensional spaces $V$? Say we have an order $(0,n)$ multilinear operator. This will eat $n$ functions $f_j\in V$ and spit out a function $g$: $$ g = T(f_1,\ldots,f_n),\quad $$What should the integral representation of this look like? Well, instead of having an integral kernel $K(x,y)$, we would have an integral kernel $K(x_1,x_2,\ldots,x_n,y)$ and so

$$ g(y) = \int\int\cdots\int K(x_1,\ldots,x_n,y)f_1(x_1)\cdots f_n(x_n)dx_1\cdots dx_n $$

I'm pretty sure the study of such beasts is ongoing work, for instance see Tao's work on the subject. For instance I'm not sure if there is a multilinear version of the Schwartz kernel theorem, but I could be wrong. Boundedness of the trilinear Hilbert transform is an open problem. Etc...