Imagine that there is points of $p_1,p_2,p_3$ in 3D space. So the coordinate components of $p_1$ is $p_{1x},p_{1y},p_{1z}$ Therefore $p_1=[p_{1x},p_{1y},p_{1z}]'$. Similarly; $p_{2}=[p_{2x},p_{2y},p_{2z}]'$, and $p_{3}=[p_{3x},p_{3y},p_{3z}]'$.
$t$ is the translation vector. $t=[t_{x},t_{y},t_{z}]'$. $R$ is the rotation matrix. $R=[r11,r12,r13;r21,r22,r23;r31,r32,r33]$.
The coordinates of $p_1, p_2, p_3$ respect to the new reference frame is: $P_i=R*p_i+t$. If we know the rotation angles(Euler angles) around x,y,z axis, we can calculate $R$ and therefore, we can calculate $P_i=R*p_i+t$.
I would like to ask the opposite form: how we can find the $R$ rotation matrix, when we know the $P_i$ positions?
I tried to do it like this: $P_i-t=R*p_i$.
$P_1-t=R*p_1$.
$P_2-t=R*p_2$.
$P_3-t=R*p_3$.
Then later, $[P_i-t]*inv(p_i)=R$
However I am getting error here. If I do this : $[P_1-t]*inv(p_1)=R$, I can't do it. Because I can't take the inverse of $p_1$. Matrix size of $p_1$ is 3x1. Since it is not square matrix, I can't take the inverse of $p_1$.
If I do this: $[P-t]*inv(p)=R$ , which $p=[p_1,p_2,p_3]=[p_{1x},p_{2x}, p_{3x};p_{1y}, p_{2y}, p_{3y};p_{1z}, p_{2z}, p_{3z}]$, and $P=[P_1,P_2,P_3]=[P_{1x}, P_{2x}, P_{3x};P_{1y}, P_{2y}, P_{3y};P_{1z}, P_{2z}, P_{3z}]$.
Now matrix size of $p$ is 3x3. And inverse of $p$ can be taken. However, I have a problem. $z$ coordinates of the $p_1,p_2,p_3$ is zero, also z coordinates of $p$. Therefore I can't take the inverse of $p$ again. And I am not able to find the $R$ rotation matrix. How can find the $R$ rotation matrix(for the condition of that $t$, $p$, and $P$ is known and given)?
Thanks in advance.