2

I was reading this paper on word vectorization recently and the author says this:

F be a homomorphism between the groups ($\mathbb{R}$;+) and ($\mathbb{R}_{>0}$; $\times$), i.e.,

$F((w_i-w_j)^T\tilde{w}_k) = \frac{F(w_i^T\tilde{w}_k)}{F(w_j^T\tilde{w}_k)}$

Here $w_i$, $w_k$ and $\tilde{w}_k$ are all matrix with real values. And F is an undefined function

I can understand this if he said F is a homomorphism between ($\mathbb{R}$;-) and ($\mathbb{R}_{>0}$; /). But he did not say that. And I can't figure out a way to prove that if F is a homomorphism from plus to multiply it can transform subtraction to division.

Can someone help me prove the author is right?

Here's a link to the paper: GloVe: Global Vectors for Word Representation.

Community
  • 1
lzt
  • 23
  • 1
    How can $F$ take matrices as input if it maps from $\mathbb R$ to $\mathbb R_{>0}$? To answer your second question, since $F$ is a group homomorphism, we have $F(x+y)=F(x)F(y)$ for all $x,y\in\mathbb R$. Note that all group homomorphisms send inverses to inverses, in this case $1=F(0)=F(x-x)=F(x)\cdot F(x)^{-1}$, so $F(-x)=\frac{1}{F(x)}$. – Pink Panther Jul 14 '19 at 19:05
  • @pinkpanther Well those matrices are just collections of $\mathbb{R}$ right? That's how I'm thinking about it at least. But your comment definitely helped me a lot. I was stuck on how $1 = F(0)$ for a bit so here's my version of your explanation. Since $F(n) = F(n + 0) = F(n)F(0)$, $F(0)$ is the identity in the image of F. Anyways, I think I got it now. Thanks a lot! – lzt Jul 15 '19 at 18:54
  • 1
    Your question is not self-contained, one must consult the paper to understand it. Please try to be very precise in future questions. – Paul Frost Jul 15 '19 at 23:06

1 Answers1

1

First note that $w_i, w_j, \tilde{w}_k$ are elements of $\mathbb R^d$, i.e. vectors. Obviously they are regard as column vectors (a column vector is a $d \times 1$-matrix). In this interpretation, for $a, b \in \mathbb R^d$ the expression $a^Tb$ is in fact a matrix product - but it is nothing else than the usual dot product $a \cdot b$ in $\mathbb R^d$.

The authors begin by considerung very general functions $F$, but then

To avoid this issue, we can first take the dot product of the arguments, $$F((w_i - w_j)^T\tilde{w}_k ) = \frac{P_{ik}}{P_{jk}}$$ which prevents F from mixing the vector dimensions in undesirable ways.

A little later they get even more restrictive:

We require that $F$ be a homomorphism between the groups ($\mathbb{R}$;+) and ($\mathbb{R}_{>0}$; $\times$).

This means that we get $$F((w_i - w_j)^T\tilde{w}_k ) = F(w^T_i \tilde{w}_k - w^T_j \tilde{w}_k ) = \frac{F(w^T_i \tilde{w}_k)}{F(w^T_j \tilde{w}_k)} .$$ Here we used $F(a-b) = F(a+(-b)) = F(a)(F(-b)$ and $F(-b) = 1/F(b)$. The latter holds because $1 = F(0) = F(b + (-b)) = F(b)F(-b)$.

Paul Frost
  • 76,394
  • 12
  • 43
  • 125
  • Sorry if my question wasn't clear. What I didn't understand was why the very last equation you wrote stands true since by the definition of F I only have F(x+y) = F(x)F(y). – lzt Jul 17 '19 at 04:01
  • I edited my question! Note that for any group homomorphism $f : G \to H$ one has $f(ab^{-1}) = f(a)f(b)^{-1}$. – Paul Frost Jul 17 '19 at 07:40