0

It's a question from HW:

Suppose we have $ \Phi:\mathbb{R}^p \to \mathbb{R}^\infty $ that satisfies:

$$ \Phi\left(x\right)^{T}\Phi\left(y\right)=\exp\left(-\frac{\left\Vert x-y\right\Vert ^{2}}{2\sigma^{2}}\right) $$ Find $ \Phi $.

First I noticed that $ \Phi(x)^T\Phi(x) = \exp(0) = 1 $. Than I substituted $ x = 0 $ and I got:

$ \Phi\left(0\right)^{T}\Phi\left(y\right)=\exp\left(-\frac{\left\Vert y\right\Vert ^{2}}{2\sigma^{2}}\right) $

Hence:

$ \Phi\left(y\right)=\Phi\left(0\right)\Phi\left(0\right)^{T}\Phi\left(y\right)=\Phi\left(0\right)\exp\left(-\frac{\left\Vert y\right\Vert ^{2}}{2\sigma^{2}}\right) $

It seems right to me but when I checked my self I found:

$ \Phi\left(x\right)^{T}\Phi\left(y\right) = \exp\left(-\frac{\left\Vert x\right\Vert ^{2}}{2\sigma^{2}}\right)\exp\left(-\frac{\left\Vert y\right\Vert ^{2}}{2\sigma^{2}}\right) = \exp\left(-\frac{\left\Vert x-y\right\Vert ^{2}}{2\sigma^{2}}-\frac{2x^{T}y}{2\sigma^{2}}\right) = \exp\left(-\frac{2x^{T}y}{2\sigma^{2}}\right)\Phi\left(x\right)^{T}\Phi\left(y\right) $.

which is not the original condition.

HUO
  • 194
  • I used the equality $ |x-y|^2 = |x|^2 + |y|^2 - 2x^ty $ – HUO Jul 20 '15 at 14:12
  • Ok, just subtitute them , I can understand the x=0 ?! – Cardinal Jul 20 '15 at 14:16
  • @john How did you know that $\Phi(y)=\Phi(0)\Phi(0)^T\Phi(y)$? – A.Γ. Jul 20 '15 at 14:17
  • Because $ \Phi(x)^T\Phi(x) = \exp\left(0\right) = 1 $ for any $ x $, in particular for $ x = 0 $. – HUO Jul 20 '15 at 14:19
  • Are x and y real value ? – Cardinal Jul 20 '15 at 14:20
  • @john It does not say that $\Phi(0)\Phi(0)^T$ is identity. – A.Γ. Jul 20 '15 at 14:21
  • Yes, in $ R^p $ as the function $ \Phi $ is defined. – HUO Jul 20 '15 at 14:21
  • @A.G. Sorry, I lost you. what is wrong in the eqation I introduced? – HUO Jul 20 '15 at 14:23
  • @john Try in $\mathbb{R}^2$ first. Let $\Phi(x)=\left[\matrix{\cos x\ \sin x}\right]$ and see what are your quantities here. – A.Γ. Jul 20 '15 at 14:25
  • I guess, It called quadratic form, see here:

    https://en.wikipedia.org/wiki/Quadratic_form

     – Cardinal Jul 20 '15 at 14:27
  • @Cardinal I rather agree that $X^TX=|X|^2$. $XX^T$ is a projection. – A.Γ. Jul 20 '15 at 14:28
  • I think the answer lies in matrix quaderetic form representation – Cardinal Jul 20 '15 at 14:28
  • I'm not sure how to deal with it. Can you tell me please a bit more about the approach? – HUO Jul 20 '15 at 14:41
  • @john, This is quite a complicated question to be asked as HW. See here, here, or here for instance. – Chester Jul 20 '15 at 17:05
  • @Chester, thank you for the links. It seems that my professor was not aware of the difficulty. – HUO Jul 20 '15 at 18:07
  • Maybe, or perhaps he/she wanted to see what you'd come up with! Actually, it's pretty straightforward in the case of $p=1$, as you'll see in one of the links. – Chester Jul 20 '15 at 18:30
  • My conclusion is a bit disappointing. Whenever we have power series $ \Phi(x)^T\Phi(y) = \sum_{n \geq 0} a_nx^n $ with non-negative coefficients, we can take the infinite vector $ \Phi(x) \equiv (\sqrt{a_n}x^n)_n \in R^\infty $ and then we get that $ \Phi(x)^T\Phi(y) $ is exactly the inner product of $ \left\langle \Phi(x), \Phi(y) \right\rangle $. – HUO Jul 20 '15 at 19:51
  • $\Phi(x)^T\Phi(y) = \sum_{n \geq 0} \frac{(-1)^n}{n!2^n\sigma^{2n}} ||x-y||^{2n} \neq \sum_{n \geq 0} a_n x^n$. First, $x$ is vector and you haven't defined what $x^n$ means. I'll assume you meant to put $||x-y||^{2n}$ here, but even so $a_n = \frac{(-1)^n}{n!2^n\sigma^{2n}}$ are not definite in sign. I'm not really sure what you mean. – Chester Jul 20 '15 at 20:14
  • Sorty, I didn't gave the full answer. First we use the formula $ |x-y|^2 = |x|^2+|y|^2-2x^Ty $. Than we can take the power series of $ \exp(x^Ty) $ and use it as I wrote (and use $ \exp(-\frac{1}{2\sigma^{2}}|x|^2-\frac{1}{2\sigma^{2}}|y|^2) $ as a coefficient). – HUO Jul 20 '15 at 20:21
  • @A.G., sorry but I don't understand you. The example you gave is not clear to me. Indeed for $ \Phi(x) = [cosx, sinx] $ it holds $ \Phi(0)^T\Phi(0) = 1 $. – HUO Jul 21 '15 at 07:48
  • @john Right, but the question was about why $\Phi(y)=\Phi(0)\Phi(0)^T\Phi(y)$. I claim that it is wrong. Test with the example your hypothesis whether LHS=RHS. – A.Γ. Jul 21 '15 at 08:16

1 Answers1

0

Not an answer, just an extended comment.

Call $q(x) = \exp(-\frac{1}{2\sigma^{2}}||x||^2)$. In the case of $p=1$, $\exp(\langle x,y \rangle / \sigma^2) = \sum_n \frac{(xy)^n}{n!\sigma^{2n}}$. Define the sequence $\Phi(x) = \left\{ \frac{q(x)x^n}{\sigma^n\sqrt{n!}} \right\}_{n=0}^{\infty}$. Then $\langle \Phi(x), \Phi(y) \rangle = \exp\left(-\frac{\left\Vert x-y\right\Vert ^{2}}{2\sigma^{2}}\right)$. It only works nicely like this I think for $p=1$.

Basically, you would need the $x$ and $y$ components to separate in $\langle x,y \rangle^n$ terms of the taylor expansion of $\exp(\langle x,y \rangle / \sigma^2)$. This doesn't really work for $p\gt 1$, which means the vector space of sequences of $p$-vectors is only the right space to look in the easy case of $p=1$.

Chester
  • 1,409