35

I want to know how the dot product can determine whether two vectors are similar? I know that the formula $$\cos(\theta) = \frac{u \cdot v }{ ||u||\,||v||}$$ means something, but don't know what.

TooTone
  • 6,343
FJam
  • 467

2 Answers2

23

The dot product of two vectors $\mathbf{u}$ and $\mathbf{v}$ is defined as

$$\mathbf{u}\cdot\mathbf{v} = |\mathbf{u}|\,|\mathbf{v}|\cos \theta$$

It's perhaps easiest to visualize its use as a similarity measure when $|\mathbf{v}|=1$, as in the diagram below, where $\cos\theta = \mathbf{u}\cdot\mathbf{v}\,/\,|\mathbf{u}|\,|\mathbf{v}| = \mathbf{u}\cdot\mathbf{v}\,/\,|\mathbf{u}|$.

enter image description here

Here you can see that when $\theta=0$ and $\cos\theta=1$, i.e. the vectors are colinear, the dot product is the product of the magnitudes of the vectors. When $\theta$ is a right angle, and $\cos\theta=0$, i.e. the vectors are orthogonal, the dot product is $0$. In general $\cos\theta$ tells you the similarity in terms of the direction of the vectors (it is $-1$ when they point in opposite directions). This holds as the number of dimensions is increased, and $\cos\theta$ has important uses as a similarity measure in multi-dimensional space.

TooTone
  • 6,343
  • Do we need to normalize the inner product when calculating the 'similarity'? Does the magnitude of cosine matter? – Logan Dec 28 '16 at 10:16
  • @Logan yes, the magnitude of $\cos$ can't be greater than 1 (it's a ratio). – TooTone Dec 28 '16 at 21:18
  • 7
    I get the geometric intuition but not the algebraic one: for example $(5,5,5)\cdot(5,5,5)=75$ and $(5,5,5)\cdot(5,5,6)=80$ while the two vectors in the former case are more similar (identical) . Also $(5,5,5)\cdot(5,5,4)=70$.... What am I missing? – hipoglucido Jul 25 '17 at 22:38
  • 6
    You're not normalizing. You're missing dividing by $|\mathbf{u}|,|\mathbf{v}|$ in $\mathbf{u}\cdot\mathbf{v},/,|\mathbf{u}|,|\mathbf{v}|$. – TooTone Jul 26 '17 at 20:31
15

Here is the geometric intuition.

Dot product between $u$, $v$ can be interpreted as projecting $u$ onto $v$ (or vice-versa), and then taking product of projected length of $u$ ($|u|$) with length of $v$ ($|v|$).

When $u$ is orthogonal to $v$, projection of $u$ onto $v$ is a 0 length vector, yielding 0 product. If you visualize all possible rotations of $u$ while keeping $v$ fixed, the dot product gives:

  • Zero value when $u$ is orthogonal to $v$ as the projection of $u$ onto $v$ yields a vector of zero length. This corresponds to the intuition of zero similarity.
  • Largest value of $|u||v|$ when $u$ and $v$ point in the same direction.
  • Lowest value of $-|u||v|$ when $u$ and $v$ point in opposite direction.

Dividing the $u \cdot v$ by max value of $|u||v|$ nicely limits the range to $[-1, 1]$, making it scale invariant.

ragha
  • 151
  • Now that you've posted this as an answer, please go back and format your answer. For the dot product use \cdot as in $|u| \cdot |v|$ = $|u|\cdot |v|$. Otherwise, please define which operation is $*$. – amWhy Nov 21 '16 at 18:43
  • updated the answer. – ragha Jan 05 '17 at 17:16
  • Nice job! I know it takes awhile to get the hang mathjax formatting, but once you do, it's worth all the effort. – amWhy Jan 05 '17 at 19:25
  • Totally! thanks for the suggestions. – ragha Jan 06 '17 at 09:08