0

I have a function

$$ f(\mathbf{x}) = \mathbf{x}^\intercal \mathbf{a} $$

where $\mathbf{x}, \mathbf{a} \in \mathbb{R}^d$. Now, suppose that I don't know the values of $\mathbf{a}$. I can fully determine them by taking a set of non-linearly dependent vectors $\mathbf{e}_1, \mathbf{e}_2, \dots, \mathbf{e}_d$ where $\mathbf{e}_i \in \mathbb{R}^d$. If I construct a (square) matrix

$$ E = \left[\begin{matrix} \mathbf{e}^\intercal_1\\ \mathbf{e}^\intercal_2 \\ \dots\\ \mathbf{e}^\intercal_d \end{matrix}\right] $$

I can claim that

$$ \mathbf{a} = E^{-1}\mathbf{y} $$

where

$$ \mathbf{y} = \left[\begin{matrix} f(\mathbf{e}_1)\\ f(\mathbf{e}_2)\\ \dots \\ f(\mathbf{e}_d) \end{matrix}\right]. $$

The first question is: how is this Theorem called?

The second question is: how it is called the Theorem that ensures me that for any $\mathbf{x}$ there is a vector $\mathbf{h} \in \mathbb{R}^d$ such that $\mathbf{x} = E^\intercal \mathbf{h}$.

The third question is: suppose that you have a non-linear mapping $\phi:\mathcal{X}\to\mathbb{R}^d$, therefore the codomain $C$ of $\phi$ is $C \subseteq \mathbb{R}^d$. I still suppose that there is a function $f$

$$ f(\mathbf{x}) = \phi^\intercal(\mathbf{x})\mathbf{a} $$

I want still to say that it exists a set of vectors ${\mathbf{c}_i \in C}_{i=1}^d$ such that every pair of vectors is non-linear, and find a solution, as I did before, by inverting a matrix where each row contains $\mathbf{c}_i^\intercal$.

What is the condition that I need to impose on $\phi$ and on $\mathcal{X}$ to obtain these results? I imagine that for some choice of $\phi$, the codomain does not contain $d$ non-linear vectors, and therefore I cannot build my matrix $C$.

As you see, I miss some nomenclature, and I need it to communicate the assumptions that I want to make in a clean and readable way.

Sam
  • 357
  • 1
    The fact that for all $\mathbf x$, there exists a $\mathbf h$ such that $\mathbf x=E^\top\mathbf h$ is just the definition of a basis, since $E^\top\mathbf h = \sum_{k=1}^d \mathbf e_k \cdot h^k$ for $\mathbf h = (h^1,\dots, h^d)$. For a proof that $d$ linearly independent vectors form a basis, see https://math.stackexchange.com/q/3253877/631742 and https://math.stackexchange.com/q/473853/631742 – Maximilian Janisch Jul 21 '21 at 16:16
  • Okay, thanks. Do you think that saying that "we assume the codomain of $\phi$ is a $d$-dimensional vector space" is enough? (it is another way to say that it contains at least one basis with $d$ vectors, right?) – Sam Jul 21 '21 at 16:28
  • 1
    I guess, sure; I mean, $f$ is always well-defined in the above context, right? – Maximilian Janisch Jul 21 '21 at 16:29
  • What do you mean well-defined? I need to assume that there is a linear mapping $f$ , as I described in the question, and I need to make sure that I can pick up $d$ vectors $\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_d$ such that I can construct a basis $\phi(\mathbf{x}_1), \phi(\mathbf{x}_2), \dots, \phi(\mathbf{x}_d)$ that allows me to compute the vector $\mathbf{a}$ (since, once I know the basis, it is basically sufficient to invert it and to multiply it by $[f(\phi(\mathbf{x}_1)), f(\phi(\mathbf{x}_2)), \dots, f(\phi(\mathbf{x}_d))]^\intercal$). – Sam Jul 21 '21 at 16:35
  • As far as I understand, $f$ is not always linear (consider $d=1$, $\phi:\mathbb R\to\mathbb R, x\mapsto x^2$) and the basis is also not clear, consider $d=1$, $\phi:\mathbb R\to\mathbb R, x\mapsto 0$. – Maximilian Janisch Jul 21 '21 at 16:48
  • I am sorry for the confusion. I should have specified that $f$ is linear w.r.t. ${\phi(\mathbf{x})| \mathbf{x}m \in \mathcal{X}}$ (let's call it the feature space). And the basis vector should also be defined in the feature space, as I wrote in my previous comment, the basis should be formed by $\phi(\mathbf{x}_1), \phi(\mathbf{x}_2), \dots, \phi(\mathbf{x}_d)$. – Sam Jul 21 '21 at 16:55
  • My main problem is that, now, ${\phi(\mathbf{x})| \mathbf{x} \in \mathcal{X}} \subseteq \mathbb{R}^d$, and, therefore, it is not sure that this new space will have a basis formed by $d$ vectors. I wanted to know if there are assumptions that I can make on $\phi$ and $\mathcal{X}$ to ensure that the codomain of $\phi$ has a $d$-dimensional basis – Sam Jul 21 '21 at 17:00
  • Just a heads-up: I'm afraid that I can't stay in this conversation since I will be a bit busy in the near future. – Maximilian Janisch Jul 21 '21 at 17:02
  • 1
    Don't worry, thanks for your help! – Sam Jul 21 '21 at 17:03

0 Answers0