10

If you have 3 labeled points on a surface of a paper. Like

    1


2       3

This makes a perfect equilateral triangle.

From this perspective I can say that the camera is on top of the paper looking down. We can say the camera is at coordinate $(0,0,100)$. Which is 0 degree rotation in Z axis and 90 degree rotation in Y axis.

Then, I move the camera to some arbitrary spot. Now, the points are like

         1

2                 3

This looks like the camera is farther back from the paper and was lowered, say at location $(0, -100, 50)$. Which is about -90 degree rotation in Z-axis, and 45 degree rotation in Y axis.

So my question is basically, given the $(x_1,y_1), (x_2,y_2), (x_3,y_3)$. Is there some formula that can take these arbitrary points, compare it with the original 3, to know how much of a X,Y,Z rotation it is of the camera?

I can also rotate on angles like this. For example, I can take the second example from above, then I can rotate my head clockwise, making the numbers flip like

2
    1
3

I think getting a normal vector from the center might be better to find.

omega
  • 751
  • Do these points lie on the same plane? – johny Mar 10 '17 at 03:29
  • yes they do lie on a plane. You can think of it like this. The original 4 lie on a paper thats on a flat surface plane, like I guess z=0. But then you can look at the 4 points from any angle and from any position in 3d space. – omega Mar 10 '17 at 03:32
  • so you will have to find the rotation and the translation of the camera. That is you are looking for 9+3 parameters right? – johny Mar 10 '17 at 03:35
  • basically I think so. – omega Mar 10 '17 at 03:38
  • I also updated it to be 3 points instead of 4 without loss of generality. – omega Mar 10 '17 at 03:38
  • I'm looking at the article, but its difficult to understand. What is the up, forward, and right vectors? – omega Mar 10 '17 at 04:13
  • @omega I misread the question. I thought you were trying the find just the rotation of the camera given its position. – KoreanwGlasses Mar 10 '17 at 04:19
  • nope. Its more of comparing 3 2d coordinates with the original 3 2d coordinates, to figure out the position and rotation in 3d space. – omega Mar 10 '17 at 04:21
  • @omega did you figure out the problem? I am just curious, as I wasn't quite sure what is given. There is some ambiguity in the statement of your problem. So my answer relies on some guessing... – Futurologist Mar 21 '17 at 06:56
  • 2
    This is known as the Perspective-Three-Point Problem. It is not so easy and the solution is not unique. In addition, if you don't know the correspondences, six different matchings are possible. –  May 11 '21 at 15:08

3 Answers3

1

Assumption:
I am assuming that the camera is orthographic, since for a perspective projection you would need at least 4 points to determine the position/rotation of the camera.

General Overview of the Method:
Read about the camera transformation here -> https://en.wikipedia.org/wiki/3D_projection#Perspective_projection .

Let the camera transform matrix be $C$. Let the original point be $p$ and the projected point be $b$. Then the orthographic projection is simply $(b_x, b_y)$ where $$ \begin{bmatrix} b_x\\ b_y\\ b_z \end{bmatrix}=C\cdot p $$

Let $v_{1,2,3}$ be the vertices of the original triangle. Let $b_{1,2,3}$ be the vertices of the triangle in the transformed coordinate space. $$C\cdot [v_1\ v_2\ v_3]=[b_1\ b_2\ b_3]$$ Thus $$C=[b_1\ b_2\ b_3]\cdot [v_1\ v_2\ v_3]^{-1}$$

If you fill in values for $v_{1,2,3}$ and only the $x$ and $y$ coordinates of $b_{1,2,3}$, this should leave you with a matrix with some unknowns ($b_{1z}$, $b_{2z}$, and $b_{3z}$), but from there you can deconstruct the camera transform into rotation (and translation), if you make the assumption that the camera is looking at the origin, and that the camera is a fixed distance away.

Good luck. This should be enough to get you started, but I will expand this answer if you need it.

  • It's a bit confusing, Is v1,v2,v3 3 coordinates or 2 coordinates? The input is just 3 2d coordinates for the original triangle, and 3 2d coordinates for the other triangle. And why is z=0 for b? Maybe it would be clearer if you could expand all the vectors fully. – omega Mar 10 '17 at 04:47
  • And yes, we can say the center of the original triangle can be the origin (0,0,0). – omega Mar 10 '17 at 04:54
0

Let, as a model, we use a camera point $K$ and a camera screen $k$, where $k$ is a plane, and let the orthogonal projection $O$ of point $K$ on the plane $k$ be the origin of the two dimensional coordinate system $Oe_1e_2$ on the screen $k$. Attach a three dimensional coordinate system $Ke_1e_2e_3$ to the point $K$ where $e_1$ and $e_2$ are the unit vectors of $Oe_1e_2$ and let $e_3$ be the unit vector orthogonal to $k$ and pointing from $K$ to $k$. Denote by $\nu$ the distance from $K$ to $k$. We may or may not know it.

At first let $A_0B_0C_0$ be an equilateral triangle in a plane $p$ parallel to $k$. Let $G_0 \in p$ be the center of $A_0B_0C_0$ so that $G_0$ lies on the $e_3$ axis, i.e. $G_0$ is the orthogonal projection of $K$ onto $p$. Then we project $A_0B_0C_0$ onto the plane $k$ by drawing the lines $KA_0, \, KB_0, \, KC_0$ and possibly $KG_0$ and denoting by $A, B, C$ and $O$ their intersection points with $k$. Thus we obtain the projected equilateral triangle $ABC$ with center $O$.

Next we move the camera $K$ together with the screen $k$ to a different position and again we project the triangle $A_0B_0C_0$ with center $G_0$ onto the plane $k$ (which is now positioned differently with respect to $p$ and is this possibly not parallel to it) the same way we did before. We draw the lines between point $K$ and the points $A_0, B_0, C_0$ and $G_0$ and look at their respective intersection points $A, B, C$ and $G$ with plane $k$. The resulting triangle $ABC$ is most likely not equilateral.

Think of the coordinate system $Ke_1e_2e_3$, which moved together with the camera and the screen, and recall that $\nu$ is the distance between $K$ and $k$, the orthogonal projection of $K$ onto $k$ is $O$, and $Oe_1e_2$ is the coordinate system on the screen $k$ as before.

If we take a plane $q$ parallel to $p$ passing through point say $C$, then the intersection points $$A_q = KA \cap q, \,\, B_q = KB \cap q, \,\, C_q = KC \cap q \, \text{ and } \, G_q = KG \cap q$$ form an equilateral triangle $A_qB_qC$ with center $G_q$ whose edges are parallel to the edges of $A_0B_0C_0$ (in other words triangle $A_qB_qC$ with center $G_q$ is homothetic image of triangle $A_0B_0C_0$ with center $G_0$ with respect $K$ )

Now, you know the locations of the points $A, \, B, \, C$ and say $G$ with respect to the screen's $k$ coordinate system $Oe_1e_2$ which means you know the vectors $$\vec{OA}, \, \vec{OB}, \, \vec{OC}, \,\, \text{ and } \,\, \vec{OG}$$ so thus, we can form the vectors \begin{align*} \vec{KA} &= \vec{KO} + \vec{OA} = \vec{OA} + \vec{KO} = \vec{OA} + \nu \, e_3\\ \vec{KB} &= \vec{KO} + \vec{OB} = \vec{OB} + \vec{KO} = \vec{OB} + \nu \, e_3\\ \vec{KC} &= \vec{KO} + \vec{OC} = \vec{OC} + \vec{KO} = \vec{OC} + \nu \, e_3\\ \vec{KG} &= \vec{KO} + \vec{OG} = \vec{OG} + \vec{KO} = \vec{OG} + \nu \, e_3\\ \end{align*}
Thus, there exists constants $\lambda, \, \mu$ and $\nu$, if you do not know $\nu$ \begin{align*} \vec{KA_q} &= \lambda \, \vec{KA}= \lambda \, \vec{OA} + \lambda \, \nu \, e_3\\ \vec{KB_q} &= \mu \, \vec{KB}= \mu \, \vec{OA} + \mu \, \nu \, e_3\\ \vec{KG_q} &= \frac{1}{3} \big(\lambda \, \vec{KA} + \mu \, \vec{KB} + \vec{KC}\big)\\ &= \frac{1}{3} \big( \lambda \, \vec{OA} + \mu \, \vec{OA} + \vec{OC} + (\lambda + \mu + 1) \, \nu \, e_3 \big) \end{align*} The restrictions that lead to the equations for the unknowns $\lambda, \, \mu$ and possibly $\nu$ are \begin{align*} |\vec{C_qA_q}| &= |\vec{A_qB_q}|\\ |\vec{C_qB_q}| &= |\vec{A_qB_q}|\\ \vec{KG_q} & \, |\, | \, \vec{KG}\\ \end{align*} where the first two say that the edges of equilateral triangle $A_qB_qC$ are equal while the last one represents the fact that the points $K, \, G_q $ and $G$ are collinear. As equations, these three conditions are

\begin{align*} |\lambda \, \vec{KA} - \vec{KC}|^2 &= |\lambda \, \vec{KA} - \mu \, \vec{KB}|^2 \\ |\lambda \, \vec{KA} - \vec{KC}|^2 &= |\lambda \, \vec{KA} - \mu \, \vec{KB}|^2 \\ |\vec{KG_q}\times \vec{KG}|^2 &= 0\\ \end{align*} which written in terms of $\lambda, \, \mu, \, \nu$ unknown variables turn into

\begin{align*} |\lambda \, \vec{OA} - \vec{OC} + (\lambda \, \nu \, - \nu) \, e_3 |^2 &= |\lambda \, \vec{OA} - \mu \, \vec{OB} + (\lambda \, \nu \, - \mu \, \nu) \, e_3 |^2 \\ |\mu \, \vec{OB} - \vec{OC} + (\mu \, \nu \, - \nu) \, e_3 |^2 &= |\lambda \, \vec{OA} - \mu \, \vec{OB} + (\lambda \, \nu \, - \mu \, \nu) \, e_3 |^2 \\ |\big(\frac{1}{3} \big( \lambda \, \vec{OA} + \mu \, \vec{OA} + \vec{OC} + (\lambda + \mu + 1) \, \nu \, e_3 \big)\big) \times \big(\vec{OG} + \nu \, e_3 \big) |^2 &= 0\\ \end{align*}

If you know $\nu$ then only the first two equations are relevant

\begin{align*} |\lambda \, \vec{OA} - \vec{OC} + (\lambda \, \nu \, - \nu) \, e_3 |^2 &= |\lambda \, \vec{OA} - \mu \, \vec{OB} + (\lambda \, \nu \, - \mu \, \nu) \, e_3 |^2 \\ |\mu \, \vec{OB} - \vec{OC} + (\mu \, \nu \, - \nu) \, e_3 |^2 &= |\lambda \, \vec{OA} - \mu \, \vec{OB} + (\lambda \, \nu \, - \mu \, \nu) \, e_3 |^2 \end{align*}

All these systems of equations may have several solutions though.

I haven't checked for typos and mistakes and I do not guarantee this is the best solution.

Futurologist
  • 9,509
0

Suppose you have three points $p_1, p_2, p_3$ somewhere in space. And you have a perspective camera (simple, pinhole) located at $r_0$ (known), pointing at these three points. An image is generated on the projection plane. The images are $q_1, q_2, q_3$ respectively.

We want to find the rotation matrix associated with the frame attached to the camera.

It is assumed that the projection plane is at a known distance $z_0$ from $r_0$.

Let's create a frame of reference whose origin is point $r_0$ , and related to the world coordinates by

$p = r_0 + R q$

Here $p$ is the world coordinate, $q$ is the camera coordinates, and $R$ is the required rotation matrix.

Let the first column of $R$ extend "horizontally" in the image (projection) plane, and the second column of $R$ extend "vertically". And let the third column extend away from the projection plane which is placed between $r_0$ and the group of points $p_1, p_2$, and $ p_3$.

The equation of the projection plane is $q(3) = -z_0$

To generate the images of $p_i, i = 1 ,2,3$:

Connect $r_0$ with $p_i$ and see where this line intersects the projection plane.

The equation of this line in world coordinates is

$p(t) = r_0 + t (p_i - r_0)$

From $p = r_0 + R q$ it follows that $q = R^T (p - r_0)$

Therefore, the equation of the line of sight in the camera frame is

$q(t) = t R^T (p_i - r_0)$

This line will hit the screen of the camera (the projection plane) when

$q_z(t) = -z_0$

hence the value of t where this happens is

$t_i = - \dfrac{z_0}{ e_3^T R^T (p_i - r_0)}$

where $e_3 = [0, 0, 1]$.

From here, the image of $p_i$ is

$q_i = t_i R^T (p_i - r_0) = - z_0 \dfrac{ R^T (p_i - r_0) }{ e_3^T R^T (p_i - r_0) } $

Multiplying both sides by the denominator of the right hand side,

$(e_3^T R^T (p_i - r_0) ) q_i = - z_0 R^T (p_i - r_0)$

which can be written as

$q_i e_3^T R^T (p_i - r_0) = - z_0 R^T (p_i - r_0)$

let $v_i = R^T (p_i - r_0)$, then

$q_i e_3^T v_i = - z_0 v_i$

which, when re-arranged, becomes

$(q_i e_3^T + z_0 I) v_i = 0 $

We know that this equation has a non-trivial solution $v_i$, (this is because $q_i(3) = -z_0 $)

If $q_i = (a, b, - z_0) $

Then solutions of this equation are

$v_i = \lambda ( - \dfrac{a}{z_0} , - \dfrac{b}{z_0} , 1 ) $

Recall that,

$v_i = R^T (p_i - r_0)$

since $R^T$ does not change the length of the vector it is multiplying, we get

$\lambda = - z_0 \dfrac{ \| p_i - r_0 \| }{\sqrt{a^2 + b^2+z_0^2}} $

The minus sign comes from the fact that $v_i(3)$ is negative.

Now we have $v_1, v_2, v_3$ corresponding to $p_1, p_2, p_3 $.

Then,

$R^T [p_1 - r_0 , p_2 - r_0, p_3 - r_0] = [v_1, v_2, v_3] $

Hence,

$R = [p_1 - r_0, p_2 - r_0, p_3 - r_0] [v_1, v_2, v_3]^{-1} $

Hosam Hajeer
  • 21,978