2

Suppose you have a set of data $\{x_i\}$ and $\{y_i\}$ with $i=0,\dots,N$. In order to find two parameters $a,b$ such that the line $$ y=ax+b, $$ give the best linear fit, one proceed minimizing the quantity $$ \sum_i^N[y_i-ax_i-b]^2 $$ with respect to $a,b$ obtaining well know results.

Imagine now to desire a fit with a function like $$ y=ax^p+b. $$ After some manipulation one obtain the following relations $$ a=\frac{N\sum_i(y_ix_i^p)-\sum_iy_i\cdot\sum_ix_i^p}{(\sum_ix_i^p)^2+N\sum_i(x_i^p)^2}, $$ $$ b=\frac{1}{N}[\sum_iy_i-a\sum_ix_i^p] $$ and $$ \frac{1}{N}[N\sum_i(y_ix_i^p\ln x_i)-\sum_iy_i\cdot\sum_ix_i^p\ln x_i]=\frac{a}{N}[N\sum_i(x_i^p)^2\ln x_i-\sum_ix_i^p\cdot\sum_ix_i^p\ln x_i. $$ To me it seems that from this it is nearly impossible to extract the exponent $p$. Am I correct?

Ross Millikan
  • 374,822
yngabl
  • 1,024

4 Answers4

2

If you want to fit the function $$y=a+b\:X^c$$ with a set of data $(X_1,y_1),(X_2,y_2),...,(X_k,y_k),...,(X_n,y_n)$, this is possible on various ways. The usual methods are iterative and require guessed initial values $a_0,b_0,c_0$ to start the process.

A non-usual method is described in the paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D , pages 16-17. Since there is no available translation, the next information is sufficient to apply this particular method :

First, let $x=\ln(X)$

Note that $X$ is supposed to be positive because, if not $X^c$ should be complex.

So, the data set becomes $(x_1,y_1),(x_2,y_2),...,(x_k=\ln(X_k),y_k),...,(x_n,y_n)$

and the function to be fitted is : $$y=a+be^{cx}$$ Then see:

Fit exponential with constant

where all the details of computation are given in order to obtain the approximates $a,b,c$.

Community
  • 1
JJacquelin
  • 66,221
  • 3
  • 37
  • 87
1

If you want to determine $p$ instead of assuming a value for $p$ and fitting $a$ and $b$, you have moved from linear curve fitting to non-linear curve fitting. For linear curve fitting it is not required that the curve be a straight line, but that the model be linear in the parameters. Fitting data to $y=ax^2+bx+c$ is linear because $y$ depends linearly on $a,b,c$. Now you need to minimize an error function numerically instead of solving a matrix equation. A discussion is free online in chapter 15 of Numerical Recipes and probably in most other numerical analysis texts.

Ross Millikan
  • 374,822
0

If you fix $p$ and perform a line fitting on the set $(x_i^p,y_i)$, you can compute the residual error (unexplained variance). This defines an objective function $\epsilon(p)$, that you can minimize by means of a numerical method such as the Golden section search.

0

The model being intrisically nonlinear wih respect to its parameter, you will need nonlinear regression.

However, the problem is to provide good estimates. One way to do it is to rewrite the model as $$y=a e^{qx}+b$$ with $q=\log(p)$. Now, a hint already provided by Yves Daoust here : choose three points $(x_1,x_2,x_3)$ such as $x_2\approx \frac{x_1+x_3}{2}$. Now, write $$A=\frac{y_3-y_1}{y_2-y_1}$$ replace and simplify; this leads to $$A=1+e^{q\frac{x_3-x_1}{2} }$$ this gives $$q=\frac{2 }{x_3-x_1}\log (A-1)$$ then $p=e^q$. This now gives $$a=\frac{y_1-y_3}{e^{q x_1}-e^{q x_3}}$$ $$b=y_1-a e^{qx_1}$$ Now, you have consistent estimates for $(a,b,p)$ and the nonlinear regression will converge in very few iterations.

Using the data given in page 18 of JJacquelin's book, let us take (approximate values for the $x$'s) the three points $(-1,0.418)$, $(0,0.911)$, $(1,3.544)$. Applying the above, this gives $q=1.67537$, $p=5.34077$, $a=0.606574$, $b=0.304426$.

Community
  • 1
Claude Leibovici
  • 260,315