2

May be it is simple, but I'm on Google for hours without finding a clue. I'm reading an article in computer vision where the optical flow equation is $\nabla I\cdot v + {dI \over dt} = 0 $ and for the 3D version it is $\nabla I\cdot [J_\pi V]+{dI \over dt}=0$

I understand most of the paper but this sentence "where the associated 3D displacement of a point is related to the motion of its projection by the $2 \times 3$ Jacobian matrix $J_\pi={\partial p \over \partial P}$ "

$P$ is a 3D point and $p$ is a 2D one, $v$ is 2d vector and $V$ is a 3D one, $\pi$ is the projection matrix (I think), I don't understand this Jacobian part, thanks in advance for any help or guidance.


EDIT: Link to the original article has been added.

Braindead
  • 4,999
Musaab
  • 157
  • 1
    Jacobian matrix usually shows up when you are using the Chain Rule in higher dimensions. They also show up when you change coordinate systems. It might be helpful if you could give us more context, or even link the original article here. – Braindead Jan 22 '14 at 06:15
  • @Braindead Thanks for your reply, I believe it is a change of coordinates as the we are talking about 3D motion vector and it's projection on the 2D plane, but then why the jacobian does contains partial derivatives instead of a normal transformation matrix ? the paper name "Scene flow from depth and color images" Letouzy et al – Musaab Jan 23 '14 at 12:11
  • In the future, if you are asking about an article, try to include the link to the original article in the question post. Looking at the original article clarified a lot of things. – Braindead Jan 23 '14 at 22:27

1 Answers1

2

$I$ is a function on the photograph, $R^2\times R$. It depends both on the position on the 2-D screen and time.

The map $\pi$ in your article is a map from $R^3 \to R^2,$ not necessarily a matrix, or even a "projection" in the sense used by mathematicians.

Technical Note: There is actually a subspace $M\subset R^3$ where $M$ is a 2-dimensional subspace of $R^3$, which is possibly time-dependent. The paper assumes that this map $\pi$ is a smooth diffeomorphism when restricted to the $M$ onto a subspace of $R^2$.

$V$ is a time-dependent vector field on $R^3$. (Or $M$.)

$v$ is the induced vector field on $R^2$ by $\pi.$ (Actually, only on the image of $M$ through $\pi$.)

A vector $V$ at the point $P$ on $M$ can be thought of as a derivative of a path based at the point $P$.

Heuristically,

$V = \left.\dfrac{d}{dt}\right|_{t=0} (P + Vt +O(t^2))$

How does this give you a vector in $R^2$? A path in $M$ turns into a path in $R^2$ by the map $\pi$. And we can take the derivative of the induced path:

$v = \left.\dfrac{d}{dt}\right|_{t=0} \pi(P + Vt +O(t^2)) = (D\pi_P V)$

The right hand side ($D\pi_P$) represents the total derivative of $\pi$ at $P$.

Technical Note 2: The whole bit about $\pi$ being one-to-one was so that you actually get a well-defined vector field on the image. If $\pi$ isn't one to one, then a point $p$ could have two different pre-images, and there could be two different vectors at those two different pre-images.

Because $\pi$ was a function from $R^3$ into $R^2$, $D\pi_P$ can be represented by a Jacobian matrix of dimension $2\times 3$ matrix:

$\pi(x,y,z) = ( u(x,y,z), v(x,y,z))$, where $u$ and $v$ and the horizontal and the vertical components of the function $\pi$. Then,

$J_\pi = \left[\begin{matrix} \dfrac{\partial u}{\partial x} & \dfrac{\partial u}{\partial y} & \dfrac{\partial u}{\partial z} \\ \dfrac{\partial v}{\partial x} & \dfrac{\partial v}{\partial y} & \dfrac{\partial v}{\partial z}\end{matrix}\right]$

Notice that $P = (x,y,z)$ and $p = \pi(P) = (u,v)$. Therefore, the notation $\dfrac{\partial p}{\partial P}$ makes sense.

In short, $D\pi_P = J_\pi = \dfrac{\partial p}{\partial P}$ essentially represent the same exact thing (the latter two expressions are a little bit sloppy with the base point), and

$$ v = J_\pi V$$ by definition.

So basically, the two equations are identical. The only difference is whether you think of the equation in terms of the 2-dimensional vector field $v$ or in terms of the 3-dimensional vector field $V$. But $v$ allows you to recover $V$, so it doesn't make any difference.

Braindead
  • 4,999
  • Thanks for your response, one last question though, what is O(t2) ? – Musaab Jan 25 '14 at 11:29
  • It is called the Big O notation. It is a way of saying things that are like (t^2) or higher order. – Braindead Jan 25 '14 at 18:30
  • Essentially, if I just wrote $P + tV$, that would be a straight line going through $P$ in the direction of vector $V$. However, the straight line might not be actually part of the 3D object you are taking the image of. The $O(t^2)$ portion is some function basically what allows my line to "curve" in a way that it would fit on the 3D object, and it doesn't effect the value of the derivative at $t=0$. – Braindead Jan 26 '14 at 01:15