0

I have a set of prediction scatter data y vs x in black crosses below. enter image description here

For the prediction to be 'accurate', the points ought to fall on the 1:1 line.

I am trying to quantify a correction scheme to transform the data in order to lie on the 1:1 line. I have tried introducing error terms with simple variables but they could not give me the correct result.

For example, the trend below shows y = 0.3 x + 0.5.

A corrected trend may have the form of: y = 0.3 (x + k) + 0.5, where k is an 'error' term. I know this form is incorrect however.

Any one has any ideas how could I do this?

Thank you

Regards Ben

Corse
  • 225
  • 1
    Either I am misunderstanding or its too easy: Subtract $0.5$ from each of the $y$ values obtained and divide by $0.3$. What problem lies in here? – Qwerty Jun 24 '16 at 09:31
  • Yes, but I would like to include the correction as a 1 step process, instead of 2 steps (subtraction and then division) as you suggested? – Corse Jun 24 '16 at 13:00

1 Answers1

1

The most accurate way to correct your data, I believe, would be to rotate it around the intersection of $y=x$ and $y=0.3x+0.5$ until it lines up with $y=x$.


Firstly, note that a line passing through the origin with slope $m$ makes an angle of $\arctan(m)$ with the $x$-axis. Thus, the angle between the two lines on your graph is $$\arctan(1)-\arctan\left(\frac{3}{10}\right)$$

Next, to find the coordinates $(x',y')$ of any point $(x,y)$ rotated by an angle $\phi$ around the origin, you can perform the following matrix multiplication:

$$\begin{bmatrix} x' \\ y' \end{bmatrix} = \begin{bmatrix} \cos(\phi) & -\sin(\phi) \\ \sin(\phi) & \cos(\phi) \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix}$$

Unfortunately, we're not rotating around the origin. That makes things a little more complicated. There's no way to represent that with $2\times2$ matrices, so we need to use $3\times3$ matrices. To find the coordinates $(x',y')$ of any point $(x,y)$ rotated by an angle $\phi$ around the point $(x_0,y_0)$ (lots of different $x$'s and $y$'s there; my apologies), you can perform the following matrix multiplication, which is a little scarier-looking but is much less scary than it looks:

$$\begin{bmatrix} x' \\ y' \\ 1 \end{bmatrix} = \begin{bmatrix} 1 & 0 & x_0 \\ 0 & 1 & y_0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} \cos(\phi) & -\sin(\phi) & 0 \\ \sin(\phi) & \cos(\phi) & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 & -x_0 \\ 0 & 1 & -y_0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x \\ y \\ 1 \end{bmatrix}$$

What's really going on here is that the first matrix to act on the original point (which is the right-most matrix) performs a coordinate transformation that moves the origin to the point around which we want to rotate, the middle matrix performs the rotation around that point, and then the left-most matrix undoes the coordinate transformation.

Multiplying the three matrices together, we find that the above equation simplifies a bit to:

$$\begin{bmatrix} x' \\ y' \\ 1 \end{bmatrix} = \begin{bmatrix} \cos(\phi) & -\sin(\phi) & x_0(1-\cos(\phi))+y_0\sin(\phi)\\ \sin(\phi) & \cos(\phi) & y_0(1-\cos(\phi))-x_0\sin(\phi)\\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x \\ y \\ 1 \end{bmatrix}$$

Performing the final multiplication, simplifying a bit, and getting rid of the now-unnecessary third row, we find that

$$\begin{bmatrix} x' \\ y' \end{bmatrix} = \begin{bmatrix} (x-x_0)\cos(\phi)-(y-y_0)\sin(\phi)+x_0 \\ (x-x_0)\sin(\phi)+(y-y_0)\cos(\phi)+y_0 \end{bmatrix}\tag{$\large\star$}$$


Theoretically speaking, this isn't a one-step process, but it can be sped up greatly by writing a simple program to do this or by spending a bit of time with a calculator. For each point on your line, you just need to plug the $x$- and $y$-coordinates of the point into the above equation to find the coordinates of the transformed point (which should lie on or near the line $y=x$).

The two lines intersect at $\left(\frac{5}{7},\frac{5}{7}\right)$, so $$x_0=y_0=\frac{5}{7},$$ and $$\phi=\arctan\left(\frac{3}{10}\right)-\arctan(1).$$ I know $\phi$ is reversed from what I said above, but just trust me that it needs to be this way for the math to work out. It's the same angle, just in a different direction.


I've done most of the calculations in case you just want a single matrix by which to multiply every point; all you need to do is plug in an $x$ and a $y$ and multiply (unfortunately, the third row is still necessary to make the multiplication possible, even though it's all $0$'s and $1$'s).

$$\begin{bmatrix} x' \\ y' \\ 1 \end{bmatrix} = \frac{1}{\sqrt{218}} \begin{bmatrix} 13 & -7 & \frac{5}{7}\Big(\sqrt{218}-6\Big) \\ 7 & 13 & \frac{5}{7}\Big(\sqrt{218}-20\Big) \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x \\ y \\ 1 \end{bmatrix}$$


Alternatively:

If you want to correct the trend line instead of correcting every point individually, you can replace $y$ in the equation $y=0.3x+0.5$ with $$(x-x_0)\sin(\phi)+(y-y_0)\cos(\phi)+y_0$$ and replace $x$ with $$(x-x_0)\cos(\phi)-(y-y_0)\sin(\phi)+x_0$$ to end up with the equation

$$(x-x_0)\sin(\phi)+(y-y_0)\cos(\phi)+y_0=0.3\Big((x-x_0)\cos(\phi)-(y-y_0)\sin(\phi)+x_0\Big)+0.5$$

Those expressions should look familiar from equation $\left(\large{\star}\right)$. Once you plug in the appropriate values for $x_0$, $y_0$, and $\phi$ and then simplify, you should end up with the line $y=x$.