1

t --- 0 1 2 3 4 5 6

F(t) 10 15 23 33 45 58 69

Adjust $F$ by a fnction of the type $$g(t) = \frac{100}{1+\alpha e^{-\beta t}}$$ by the discrete least squares method

I'm studying orhotogonal families of polynomials and projection onto subspaces in the context of least squares method.

I think need to see this problem as a projection onto some subspace and use some inner product but I'm lost.

UPDATE:

Shouldn't the function $g(t)$ be a member of a vector space? I tried applying $\ln$ to see if I'd get something from a vector space but it also won't work

Guerlando OCs
  • 6,584

2 Answers2

4

Forgetting (projection/orthogonal families of polynomials), the problem is quite easy to solve using standard nonlinear regression.

As usual, we need good or at least consistent estimates of parameters $(\alpha, \beta)$ and these can be obtained by a linearization of the model. $$g = \frac{100}{1+\alpha e^{-\beta t}} \implies \color{red}{y}=\log \left(\frac{100}{g}-1\right)=\log(\alpha)-\beta\,t=\color{red}{a+b t}$$

Consider the data to be $$\left( \begin{array}{ccc} t & g & y=\log \left(\frac{100}{g}-1\right) \\ 0 & 10 & +2.197225 \\ 1 & 15 & +1.734601 \\ 2 & 23 & +1.208311 \\ 3 & 33 & +0.708185 \\ 4 & 45 & +0.200671 \\ 5 & 58 & -0.322773 \\ 6 & 69 & -0.800119 \end{array} \right)$$ A preliminary linear regression leads to $$\begin{array}{clclclclc} \text{} & \text{Estimate} & \text{Standard Error} & \text{Confidence Interval} \\ a & +2.21599 & 0.01226 & \{+2.18195,+2.25003\} \\ b & -0.50409 & 0.00340 & \{-0.51353,-0.49465\} \\ \end{array}$$ corresponding to $R^2=0.999878$ which is already very good.

This gives as estimates $\alpha=e^a=9.17046$ and $\beta=-b=0.50409$.

Now, we can start the nonlinear regression and obtain $$\begin{array}{clclclclc} \text{} & \text{Estimate} & \text{Standard Error} & \text{Confidence Interval} \\ \alpha & 9.22336 & 0.13438 & \{8.85027,9.59645\} \\ \beta & 0.50576 & 0.00350 & \{0.49603,0.51549\} \\ \end{array}$$ corresponding to $R^2=0.999972$ which is very good. PLease, notice how good are the initial estimates.

Below are reproduced the data as well as the predicted values $$\left( \begin{array}{ccc} t & g & g_{pred} \\ 0 & 10 & 9.782 \\ 1 & 15 & 15.24 \\ 2 & 23 & 22.97 \\ 3 & 33 & 33.08 \\ 4 & 45 & 45.05 \\ 5 & 58 & 57.62 \\ 6 & 69 & 69.27 \end{array} \right)$$

If we had in advance known that the model was good (based on physics for example) and the data in small errors (because of accurate measurements), we could have skipped th first step and used the first and last data points to estimate $(\alpha, \beta)$

$$10=\frac {100}{1+\alpha} \implies \alpha=9$$ $$69=\frac {100}{1+9 e^{-6\beta}}\implies \beta=\frac{1}{6} \log \left(\frac{621}{31}\right)=0.499557$$

Claude Leibovici
  • 260,315
  • +1 for the detailed answer and discussion with me. – farruhota Apr 08 '19 at 10:54
  • 1
    @farruhota. This was my pleasure ! Computing the $SSQ$ from your table gives $0.568$ while, from mine $0.333$. – Claude Leibovici Apr 08 '19 at 11:14
  • If I didn't round, mine would be $0.356$. Agreed, still more than yours. When you say "Now, we can start the nonlinear regression", are you minimizing $SSQ=\sum_{i=0}^6 \left(g_i-\frac{100}{1+\alpha e^{-\beta t}}\right)^2$ (for example, using $\alpha=\beta=0$ as starting values, the Excel solver gives $\alpha =9.223294081; \beta =0.505758705$)? How do you iterate it from the first step (linear regression)? – farruhota Apr 09 '19 at 12:25
  • @farruhota. This problem is very simple. In general, use optimization or Newton-Raphson for solving the partial derivatives equal to zero. – Claude Leibovici Apr 09 '19 at 17:09
3

Make the transformations: $$g(t) = \frac{100}{1+\alpha e^{-\beta t}} \iff \alpha e^{-\beta t}=\frac{100}{g(t)}-1 \iff \underbrace{\ln \left(\frac{100}{g(t)}-1\right)}_{y(x)}=\underbrace{-\beta t}_{ax}+\underbrace{\ln \alpha}_{b}$$ Hence: $$\begin{array}{c|r|r} &x&y(x)&xy&x^2\\ \hline &0&2.20&0.00&0\\ &1&1.73&1.73&1\\ &2&1.21&2.42&4\\ &3&0.71&2.13&9\\ &4&0.20&0.80&16\\ &5&-0.32&-1.60&25\\ &6&-0.80&-4.80&36\\ \hline \text{Total}&21&4.93&0.68&91\\ \end{array}\\ \begin{align}a&=\frac{\sum xy-\frac{\sum x \sum y}{n}}{\sum x^2-\frac{(\sum x)^2}{n}}=\frac{0.68-\frac{21\cdot 4.93}{7}}{91-\frac{21^2}{7}}=-0.5\\ b&=\bar{y}-a\bar{x}=\frac{4.93}{7}-(-0.5)\frac{21}{7}=2.2\\ \ln \alpha&=b=2.2 \Rightarrow \alpha =9.03\\ \beta &=-a=0.5\end{align}$$ So, the final answer: $$g^*(t) = \frac{100}{1+9.03 e^{-0.5t}}\\ \begin{array}{c|c|c} t&g(t)&g^*(t)\\ \hline 0&10&9.97\\ 1&15&15.44\\ 2&23&23.14\\ 3&33&33.17\\ 4&45&45.00\\ 5&58&57.43\\ 6&69&68.99 \end{array}$$

farruhota
  • 31,482
  • You must take care that this is a first step since what is measured is $g$ and not any of its possible transforms. – Claude Leibovici Apr 08 '19 at 04:40
  • @ClaudeLeibovici, thank you for commenting. Am I not measuring $g$? I transformed and relabeled, which is the linearization. We get the same results except rounding discrepancies. – farruhota Apr 08 '19 at 05:11
  • This is exactly what I wrote. You transformed $g$ ! Linearization (as we both did) is very good to get estimates of the parameters. Then, you must use $g$ by itself. This case was not bad because of very marginal errors. – Claude Leibovici Apr 08 '19 at 05:15
  • Yes, $\alpha, \beta$ are estimates of the population parameters calculated from sample data of $7$ observations. Sorry, I’m not getting my mistake if any. – farruhota Apr 08 '19 at 05:51
  • And by calculating $g^*(t)$ (your $g_{pred}$) it is calculated a point estimate, not interval estimate. – farruhota Apr 08 '19 at 05:56
  • I can generate the standard devaition for each predicted value. Just to be clear, you did not make any mistake except that you did not go to the end. What must be fitted is the measured value and nothing else (except is no error at all). – Claude Leibovici Apr 08 '19 at 06:33
  • If by "going to the end" you mean the goodness of fit and SE, right, I calculated the slope and intercept manually. Thank you. – farruhota Apr 08 '19 at 08:07
  • Going to the end stands for doing the nonlinear regression. But, again, be sure that what you did is very good work (I upvoted your answer). Cheers. – Claude Leibovici Apr 08 '19 at 10:49