I have a $C$ (matrix $40\times 13$) and $d$ (40). When I use least squares I get $x$. But then when I do $C*x$ the result can be sometimes less than $d$. Is there a way to make sure you always get $C*x \geq d$ ?
If not, what other approach would you recommend please?
Thank you
Zdenek
Start with $$ \mathbf{A}x = b $$ with the matrix $\mathbf{A}\in\mathbb{R}^{m\times n}$ and the data vector $b\in\mathbb{R}^{m}$.
The least squares solution is $$ x_{LS} = \color{blue}{\mathbf{A}^{+}b} + \color{red}{\left( \mathbf{I}_{n} + \mathbf{A}^{+} \mathbf{A} \right) y}, \qquad y \in \mathbb{R}^{n} $$
Consider the full column rank case, the null space $\color{red}{\mathcal{N}\left( \mathbf{A}\right)}$ is trivial. The solution is unique $$ \color{blue}{x_{LS}} = \color{blue}{\mathbf{A}^{+}b} $$ The Moore-Penrose pseudoinverse is a matrix which projects the image of the data onto the range space $\color{blue}{\mathcal{R}\left( \mathbf{A}\right)}$. If we think of the data vector in terms of range and null space components $$ b = \color{blue}{b_{\mathcal{R}}} + \color{red}{b_{\mathcal{R}}}, $$ then the projection of $b$ is $\color{blue}{b_{\mathcal{R}}}$ and we have $$ \boxed{ \lVert \color{blue}{\mathbf{A}^{+}b} \rVert = \lVert \color{blue}{b_{\mathcal{R}}} \rVert \le \lVert b \rVert }. $$
The figure below shows the geometry and helps us understand that the projection of the data vector must always be less than or equal to the data vector.
