2

I'm looking for any theoretical work/paper/development where the notion of Reproducing kernel Hilbert space (RKHS), defined via positive definite symmetric kernel $k(x,y) = k(y,x)$, is extended towards non-symmetrical kernels.

Basically, I'm looking for a similar notion to kernel learning/regression, which is where RKHS is mainly used, but that is based on non-symmetric similarity function $k(x,y) \neq k(y,x)$. Does such mathematical structure exist? Can anyone share a link for any relevant material?

Thanks

Dimka Kopitkov
  • 85
  • If the kernel is not symmetric then you cannot define the associated inner product (it will not be symmetric) and hence you cannot define a Hilbert space structure. The symmetry of the kernel (among other properties) is needed to define a self-adjoint operator and apply the spectral theorem, and without it you lose a lot of the meaningful structure. – snar Feb 23 '20 at 22:58
  • 2
    For example, this paper (https://icml.cc/Conferences/2004/proceedings/papers/392.pdf) talks about RKHS extension where kernel is non-positive. They describe some benefits of such construction. Can we similarly talk about function space that is based in some way on non-symmetrical similarity function $k(x,y)$? – Dimka Kopitkov Feb 24 '20 at 16:47
  • The decomposition of the eigenspace into positive, null and negative parts is fairly straightforward and preserves the main property: roughly speaking, "diagonalization". You could consider a Banach space with the basis given by the non-symmetric kernel, but people don't generally consider this because (among other things) you then cannot use the "kernel trick". – snar Feb 24 '20 at 17:03
  • Ok, thanks a lot for explaining this point! – Dimka Kopitkov Feb 28 '20 at 14:01

1 Answers1

3

RKHS have been generalized to Reproducing Kernel Banach spaces (RKBS). In RKBS the inner product is replaced by a semi-inner product which is not necessarly symmetric anymore or more recently by Bilinearform on a pair of Banach spaces:

Definition Reproducing Kernels for RKBS (Lin et al, 2019, On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of Constructions, p. 3):

Let $B_1$ be an RKBS on a set $Ω_1$. If there exists a Banach space $B_2$ of functions on another set $Ω_2$, a continuous bilinear form $\langle ·, ·\rangle_{B1×B2}$, and a function K on $Ω_1 \times Ω_2$ such that K(x, ·) ∈ $B_2$ for all x ∈ $Ω_1$ and $f(x) = \langle f, K(x, ·)\rangle_{B_1×B_2}$ for all x ∈ $Ω_1$ and all f ∈ $B_1$, then we call K a reproducing kernel for $B_1$.

If, in addition, $B_2$ is also an RKBS on $Ω_2$ and it holds K(·, y) ∈ B1 for all y ∈ $Ω_2$ and g(y) = $\langle K(·,y), g\rangle_{B_1×B_2}$ for all y ∈ $Ω_2$ and all g ∈ $B_2$, then we call $B_2$ an adjoint RKBS of $B_1$ and call $B_1$ and $B_2$ a pair of RKBSs.

In this case, $\tilde{K}(x, y) := K(y, x)$, x ∈ $Ω_2$, y ∈ $Ω_1$, is a reproducing kernel for $B_2$.

The construction with a semi inner product has been applied to machine learning (see Zhang et al, Reproducing Kernel Banach Spaces for Machine Learning). There exist also constructions of this kind where the representer theorem can be applied: G. Song, H. Zhang, and F. J. Hickernell, Reproducing kernel Banach spaces with the ℓ1 norm.

claire_fontaine
  • 68