1

I consider a function $f(t)=\alpha\cdot 2^{-t}+\beta\cdot 2^{-2t}$ and a finite number of measured values $f_0,...,f_n$. The coefficients $ \alpha, \beta $ are calcualted by the least squares method which means $$ \sum\limits_{i=0}^n |f(t_i)-f_i|^2$$ is minimized.

Then $$\sum\limits_{i=0}^n 2^{-t_i}\cdot f(t_i)=\sum\limits_{i=0}^n 2^{-t_i}\cdot f_i.\quad (*)$$

My idea:

I consider the expressions

$$ \begin{aligned}f_0&\stackrel{!}{=}f(t_0)=2^{-t_0}+\beta\cdot 2^{-2t_0}\\f_1&\stackrel{!}{=}f(t_1)=2^{-t_1}+\beta\cdot 2^{-2t_1}\\\vdots\\f_n&\stackrel{!}{=}f(t_n)=2^{-t_n}+\beta\cdot 2^{-2t_n}\end{aligned}$$

which can be reprensented by $$ \underbrace{\begin{pmatrix}2^{-t_0}&2^{-2t_0} \\2^{-t_1}&2^{-2t_1} \\\vdots&\vdots\\2^{-t_n}&2^{-2t_n} \end{pmatrix}}_{=:A}\cdot \underbrace{\begin{pmatrix}\alpha\\\beta \end{pmatrix}}_{=:x}=\underbrace{\begin{pmatrix}f_0\\f_1\\\vdots\\f_n \end{pmatrix}}_{=:f} $$

Then I want to minimize

$$ F((\alpha,\beta)):=\|A\cdot x-f\|_2^2=\sum\limits_{i=0}^n |f(t_i)-f_i|^2=\sum\limits_{i=0}^n \Big(\alpha\cdot 2^{-t_i}+\beta\cdot 2^{-2t_i}-f_i \Big)^2 $$

I calculate the gradiant of $ F $:

$$ \nabla F((\alpha,\beta))= 2\cdot A^T\cdot (A\cdot x-f)\stackrel{!}{=}0$$

which means I have to solve $$ A^T\cdot A\cdot x=A^T\cdot f $$

At first I have $$ A^T\cdot A=\begin{pmatrix}\sum\limits_{i=0}^n 2^{-4t_i}&\sum\limits_{i=0}^n 2^{-3t_i}\\\sum\limits_{i=0}^n 2^{-3t_i}&\sum\limits_{i=0}^n 2^{-2t_i} \end{pmatrix}=:\begin{pmatrix}a&b\\b&c \end{pmatrix} $$ and

$$ A^T\cdot f=\begin{pmatrix}\sum\limits_{i=0}^nf_i\cdot 2^{-2t_i}\\\sum\limits_{i=0}^nf_i\cdot 2^{-t_i} \end{pmatrix}=:\begin{pmatrix}b_1\\b_2 \end{pmatrix} $$

The solution is $$ \alpha=\frac{a\cdot b_2-b\cdot b_1}{a\cdot c-b^2}\qquad \beta=\frac{c\cdot b_1-b\cdot b_2}{a\cdot c-b^2} $$

And here comes the problem. These expressions of the solution are soo complicated that I'm not able to handle them to show the upper identity (*).

Is there annother way to show or what could I do else(*)?

hallo007
  • 545
  • element $c$ of $A^\intercal A$ should be $\sum_{i=0}^n2^{-2t_i}$. Furthermore, $a$ and $c$ should be reversed, i.e. the (2,2) element of $A^\intercal A$ is $\sum_{i=0}^n2^{-4t_i}$, and the (1,1) element $\sum_{i=0}^n2^{-2t_i}$ – Syd Dec 20 '20 at 20:21
  • I have edited it. Thanks. – hallo007 Dec 20 '20 at 20:24

2 Answers2

1

$$\min_{\alpha,\beta}\sum_{i=0}^n(f(t_i)-f_i)^2=\min_{\alpha,\beta}\sum_{i=0}^n(\alpha\,2^{-t_i}+\beta\,2^{-2t_i}-f_i)^2$$ is minimized by

$$\begin{cases}\displaystyle\sum_{i=0}^n2^{-t_i}(f(t_i)-f_i)=0,\\ \displaystyle\sum_{i=0}^n2^{-2t_i}(f(t_i)-f_i)=0.\end{cases} $$

The first equation is the relation you are after.

0

Given that $$ A^T\cdot A=\begin{pmatrix}\sum\limits_{i=0}^n 2^{-2t_i}&\sum\limits_{i=0}^n 2^{-3t_i}\\\sum\limits_{i=0}^n 2^{-3t_i}&\sum\limits_{i=0}^n 2^{-4t_i} \end{pmatrix}=:\begin{pmatrix}a&b\\b&c \end{pmatrix}, $$ note that $$(A^TA)^{-1} = \frac{1}{ac - b^2}\begin{pmatrix} c & -b \\ -b & a \end{pmatrix}.$$ Let $d$ be the first column of $A$ (unfortunate nomenclature...). Then $$d^TA = \begin{pmatrix} \sum_{i=0}^n2^{-2t_i} & \sum_{i=0}^n2^{-3t_i} \end{pmatrix} = \begin{pmatrix} a & b \end{pmatrix},$$ and $$d^TA(A^TA)^{-1} = \frac{1}{ac - b^2}\begin{pmatrix} ac - b^2 & 0 \end{pmatrix} = \begin{pmatrix} 1 & 0 \end{pmatrix}$$ -- et voila! Since now $$d^TA(A^TA)^{-1}A'f = d^Tf,$$ which had to be shown. Note that ($\ast$) can be expressed as $d^TA\hat x = d^Tf$, where $\hat x = (A^TA)^{-1}A^Tf$ is the LS estimate of $x$.

Syd
  • 118
  • 8