0

First, I would like to know how to calculate entropy of cosine similarity.

We have $H(X)=-\sum P(x_i)log_bP(x_i)$ for entrpy, and

similarity $ = \sum \frac{A_iB_i}{\sqrt{\sum(A_i^2)}\sqrt{\sum(B_i^2)}} $,

but we have two variable A,B for cosine distance, whereas we have only x for entropy. How could I calculate entropy?

cosmiccapsule
  • 51
  • $A$ and $B$ are random variables, and the cosine similarity is a function of these two and hence another random variable, let's call it $X$. No need to consider two random variables simultaneously when calculating entropy of $X$. – mikkola Dec 12 '17 at 21:59

1 Answers1

2

Let $A,B$ be random vectors of $n$ componentes.

Let the similarity be denoted by $S = \sum \frac{A_iB_i}{\sqrt{\sum(A_i^2)}\sqrt{\sum(B_i^2)}}$. This variables has a discrete support assuming than $A$ and $B$ are discrete.

Now we want to compute the entropy of $S$; that is,

$$ H(S) = -\sum P(x_i)log_bP(x_i) $$

How can we compute it? Well, we cannot. Not without knowing the distributions of $A$ and $B$ at least.

Jsevillamol
  • 4,668