2

Let's say I have 2 variables $x$ and $y$ and 4 equations. The parameters in capital are known parameters.

$$I_1=xA_1+yB_1$$ $$I_2=xA_2+yB_2$$ $$I_3=xA_3+yB_3$$ $$I_4=xA_4+yB_4$$

What's the strategy to find the best fitting values for $x$ and $y$? What's the name of this topic so I can google it?

Harry Peter
  • 7,819
Ruts
  • 651
  • 2
    Overdetermined linear system. Often addressed by least-squares. –  Apr 22 '16 at 11:42
  • A similar problem with solution details: http://math.stackexchange.com/questions/1026451/least-squares-problem-find-the-line-through-the-origin-in-mathbbr3/2180128#2180128 – dantopa Mar 13 '17 at 05:07

2 Answers2

2

Let us suppose that you have $n$ data points $(A_i,B_i,I_i)$ ($i=1,\cdots,n)$ and you want to find the "best" $x$ and $y$ for matching the model $$I=x A+y B$$ the genral method, as already answered by G-Cam, is ordinary least square which consist in the minimization of $$F=\sum_{i=1}^n \big(x A_i+y B_i-I_i\big)^2$$ Computing the derivatives and setting them equal to $0$ since we look for the minimum, we get $$F'_x=2\sum_{i=1}^n \big (x A_i+y B_i-I_i\big)A_i=0$$ $$F'_y=2\sum_{i=1}^n \big (x A_i+y B_i-I_i\big)B_i=0$$ Expanding the above equations, we then have $$x \sum_{i=1}^n A_i^2+y\sum_{i=1}^n A_iB_i=\sum_{i=1}^n A_iI_i$$ $$x \sum_{i=1}^n A_iB_i+y\sum_{i=1}^n B_i^2=\sum_{i=1}^n B_iI_i$$ So, two linear equations in $(x,y)$ easy to solve.

For sure, as G-Cam mentionned, you could prefer the matrix formulation.

Claude Leibovici
  • 260,315
  • Does what you've explained litteraly translates in the matrix solution of G-Cam? – Ruts Apr 23 '16 at 08:23
  • The classical formula is $(\mathbf X^\mathrm{T} \mathbf X) \hat{\boldsymbol{\beta}} = \mathbf X^\mathrm{T} \mathbf y$ (https://en.wikipedia.org/wiki/Linear_least_squares_%28mathematics%29) – Claude Leibovici Apr 23 '16 at 08:32
  • But I can say that what you've explained can be written in matrix form to $(\mathbf X^\mathrm{T} \mathbf X) \hat{\boldsymbol{\beta}} = \mathbf X^\mathrm{T} \mathbf y$? Do you maybe know a source where the conversion is explained, so I can refer to it? – Ruts Apr 23 '16 at 08:52
1

You are looking for a technique called "Least-squares regression". This method allows for overdetermined systems (more equations than variables) to be fit to the polynomial that minimizes the sum of the distances (actually the sum of the squared distances) from each point to that polynomial. You specifically want the matrix version that looks like this $$A^TA\vec{x}=A^T\vec{b}$$

G-Cam
  • 278