1

The input are triples $\left\{ x,y,v\right\}$ where $x,y,v \in \mathbb{R} ^{+}$

I need to find function $f(x,y) = v$ by finding parameters of the following model

$f(x,y) = a + bx^c + dy^e $ where

  1. $f\left( x,y\right) >0,\forall x,y\in \mathbb{R} ^{+}$
  2. $f'\left( x,y\right) \ge 0,\forall x,y\in \mathbb{R} ^{+}$

Ideally I'd like to know how to compute both fitting surface (if any exists) and approximation (regression) with minimal squared error.

Finding regression for given model is trivial.

I'd like to know what is the best way to do it with constraints (here function $f$ must be positive and non-decreasing on given domain, which is positive reals)

Lambder
  • 113
  • What does $f'\ge 0$ exactly mean? Just that each partial derivative $\frac{\partial f}{\partial x}$, $\frac{\partial f}{\partial y}$ is non-negative or something else? – g g May 19 '16 at 11:19
  • It means, its partial derivatives are positive (in $\mathbb{R} ^{+}$ domain) – Lambder May 20 '16 at 08:16

1 Answers1

2

Approximation: As you have observed the problem is just fulfilling the sign constraints. The partial derivatives are: $\frac{\partial f}{\partial x}=cbx^{c-1}$ and $\frac{\partial f}{\partial y}=dey^{d-1}$. Since $x$ and $y$ are positive the derivatives are positive if $cb > 0$ and $de>0$. To make sure that $f>0$ as well, you actually need all coefficients positive $a, b, c, d, e>0$. To see this assume for example $b<0$ and $c<0$: Sufficiently small $x$ values will lead to arbitrarily negative $bx^c$ values, which in turn will make $f$ negative.

So you need to ensure that your least squares optimisation will not run into negative values for the coefficients. The standard way to do this, is to reparameterise the coefficients by making them exponentials. I.e. you write $f(x,y)=\exp(\alpha) + \exp(\beta)x^{\exp(\gamma)} + \exp(\delta)y^{\exp(\eta)}$ and solve the unconstrained problem in terms of the new parameters $\alpha,\ldots,\eta$. This is a non-linear least squares problem which can be solved with standard numerical methods and tools.

Interpolation: If an interpolating solution exists, the least squares should find this, so no extra work required.

g g
  • 2,717
  • @q-q can you explain why cb>, de>0, a>0 => f(x,y) > 0 ? – Lambder May 23 '16 at 15:02
  • Initial wording was incorrect, I edited to clarify. – g g May 23 '16 at 17:10
  • Still, I'm not convinced. $f(x,y) > 0 <= a > bx^c + dy^e$ so if a is positive and large enough $ bx^c + dy^e $ can be negative to have the function positive. – Lambder May 23 '16 at 17:20
  • But you said $f$ shall be positive for ALL $x,y > 0$. A single fixed positive $a$ can compensate negative numbers only up to a certain amount. As an example set $y=1$, $d=1$ and $b,c=-1$. Then $f(x,y)=a + d - x^{-1}$ and $f$ will be negative for all $x<\frac{1}{a+d}$. – g g May 23 '16 at 18:06
  • Ah, of course. Thanks – Lambder May 23 '16 at 21:04
  • How to solve the unconstrained problem $f(x,y)=\exp(\alpha) + \exp(\beta)x^{\exp(\gamma)} + \exp(\delta)y^{\exp(\eta)}$ in terms of the parameters $\alpha,\ldots,\eta$. – Lambder May 24 '16 at 08:32
  • see my edit, if you have never heard about this before start by looking at the respective Wikipedia article: https://en.wikipedia.org/wiki/Non-linear_least_squares.

    Any standard numerical software package (Matlab, R, scipy) should have solvers for this kind of problem.

     – g g May 24 '16 at 11:42
  • Yes, I have but to my knowledge it works if your coefficients are independent. E.g. in $\exp(\beta)x^{\exp(\gamma)}$ $\beta$ and $\gamma$ are not independent. – Lambder May 24 '16 at 15:55
  • I am not sure what you mean by "independent" in this context. The fact that beta and gamma occur within complicated expressions is what makes the problem non-linear. Just put this into your solver, it'll be most likely able to handle it. – g g May 27 '16 at 07:38