Finding normal vector of rotated square (parallelogram)

Question

I'm not a mathematician, so please excuse any misuse of terms. :)

I have a 3D model of a cube. Each side of the cube is a different color. A camera is observing the cube. The camera takes a screenshot and passes it to an image processing algorithm.

The algorithm draws around the most prominent side of the cube (i.e. the side of the cube that the camera can see most of). This shape is a parallelogram, as the shape is not necessarily directly facing the camera.

My question is, knowing the four corners of the parallelogram, how do I decide the relative angle that the side is facing compared to the camera? I think I need the normal vector, but how do I calculate this from the corners?

In the screenshot I have provided, the green outline of the red face is where I need to calculate the relative position that the red face is facing.

Answer 1

My other answer is for orthographic projection of the cube. This one is for perspective projection.

It is assumed that the image processing algorithm knows the coordinates of the four corners of the face $P_1, P_2, P_3, P_4$ ordered in clockwise or counter clockwise direction. In addition it is assumed that the two vertices $P_5$ connected to $P_1$ and $P_6$ connected to $P_2$ also have known coordinates.

Each point in the picture corresponds to a line in space. Each of these lines passes through the point $E = (E_x, E_y, 0) $ (unknown) and the point

$Q_i = \begin{bmatrix} P_i \\ z_0 \end{bmatrix} , \ i = 1 , 2, 3, 4$

The focal distance $z_0$ is unknown, and has to be determined.

To determine $E$, we will use the fact that the projection center $E$ is the orthocenter of the three vanishing points in the three directions (left-right, forward-backward, and up-down).

So from the known $6$ vertices we'll be able to determine the projection center $E$.

Having determined $E$, we now have the equation of the lines describing the location of the vertices of the cube face under consideration.

$ L_i(t_i) = E + t_i (Q_i - E) , \ i = 1,2,3,4$

Since we know that the face is a square, then it is a rectangle. A rectangle is a special parallelogram with its corner angles equal to $90^\circ$.

To ensure that the actual points in $3D$ space form a parallelogram, we must have

$ L_2 - L_1 = L_3 - L_4 $

So that,

$ t_2 (Q_2 - E) - t_1 (Q_1 - E) = t_3 (Q_3 - E) - t_4 (Q_4 - E) $

Remember the third coordinate of all the $Q_i$'s is the focal distance $z_0$ and the third coordinate of $E$ is zero. So, from the third component of the above vector equation, we get

$ t_2 - t_1 = t_3 - t_4 $

And using this, $E$ in the first two rows gets eliminated, and the equation for the parallelogram condition becomes

$ t_2 Q_2 - t_1 Q_1 = t_3 Q_3 - t_4 Q_4 $

This is a linear system of $3$ equations in $4$ unknowns. So it can be solved using Gaussian elimination, and it will yields

$ (t_1, t_2, t_3, t_4) = \lambda \mathbf{v} $

Next, we determine $z_0$.

For that, we impose the orthogonality condition on the line segments $L_1 L_2$ and $L_2 L_3$. This means

$ ( t_2 (Q_2 - E) - t_1 (Q_1 - E) ) \cdot ( t_3 (Q_3 - E) - t_2 (Q_2 - E) ) = 0 \hspace{13pt}(1*) $

Using $Q_i = \begin{bmatrix} P_i \\ z_0 \end{bmatrix} $ and $t_i = \lambda v_i $

This becomes

$ ( v_2 (P_2 - E) - v_1 (P_1 - E) ) \cdot ( v_3 (P_3 - E) - v_2 (P2 - E) ) + (t_1 - t_2)(t_2 - t_3) z_0^2 = 0 $

Everything in this equation is known except $z_0$. Solving for $z_0$ , we get

$ z_0 = - \sqrt{ \dfrac{( v_2 (P_2 - E) - v_1 (P_1 - E) ) \cdot ( v_3 (P_3 - E) - v_2 (P2 - E) ) }{ (t_1 - t_2)(t_3 - t_2) }} $

Having determine $E$ and $z_0$, we can now find the normal vector to the face as follows

$N = ( L_3 - L_2 ) \times (L_2 - L_1 ) $

where $\times$ denotes cross multiplication.

Answer 2

Attach the $xy$ plane to the image plane, and let the ordered four corners of the square be $A , B, C , D $. Define

$U = B - A = [U_x, U_y, U_z] $

$V = D - A =[V_x, V_y, V_z]$

The images of $U $ and $V$ are

$ U' = [U_x, U_y, 0] $ and $V' = [V_x, V_y, 0] $

From the image we can determine $U_x, U_y, V_x, V_z$

Now we have only two unknowns which are $U_z $ and $V_z$. There are two equations which these two unknowns satisfy

(Equal length) $ U_x^2 + U_y^2 + Uz^2 = V_x^2 + V_y^2 + V_z^2 $

and

(Perpendicularity) $ U_x V_x + U_y V_y + U_z V_z = 0 $

Solving these two equations determines $U_z$ and $V_z$. Then find the cross product between $U$ and $V$. This gives the normal vector of the face of the cube, with respect to the local coordinate frame of the camera.

There will be two possible solutions, but only one of them is valid. Using the relative location of the face with respect to the rest of the cube it is possible to determine which normal vector is the valid one.