1

In the paper "The continuous categorical: a novel simplex-valued exponential family", the supplementary material shows that the normalizing constant of the continuous categorical density function can be written as

$$\left(-1\right)^{K+1}\left[\sum\limits_{k=1}^K\frac{\lambda_k}{\prod\limits_{i\ne k}\log\frac{\lambda_i}{\lambda_k}}\right]^{-1}$$

which is defined if $\lambda_i\ne\lambda_k\,\,\forall i\ne k$.

What is the general form of the normalizing constant if two or more of the $\lambda$ parameters are equal?

If only two of the $\lambda$s are equal $\left(K\ge 2\text{ , }\lambda_1=\lambda_2\text{ , and }\lambda_k\ne\lambda_1\,\,\forall k\ne 1,2\right)$, then I can use L'hopital's rule applied to the above equation, to derive that the normalizing constant should have the form of

$$\left(-1\right)^{K+1}\left[\sum\limits_{k\ne 1,2}\frac{\lambda_k}{\prod\limits_{i\ne k}\log\frac{\lambda_i}{\lambda_k}}-\frac{\lambda_1}{\prod\limits_{i\ne 1,2}\log\frac{\lambda_i}{\lambda_1}}\left(1+\sum\limits_{i\ne 1,2}\frac{1}{\log\frac{\lambda_i}{\lambda_1}}\right)\right]^{-1}$$

I may be wrong here, as my maths is not perfect.

If I only have three parameters, such that $\left(K=3\text{ and }\lambda_1=\lambda_2=\lambda_3\right)$, then I can again use many many differentiation rounds of L'Hopital's rule to get a normalizing constant of

$$\left(-1\right)^{K+1}\left[-\frac{\lambda_1}{2}\right]^{-1}=6$$

With any more parameters, the L'Hopital differentiation of the numerator and denominator gets to many multiple pages of long algebra, in which it is very easy to make a mistake that is difficult to catch.

What is the general form of the continuous categorical density function's normalizing constant when any number of the $\lambda$ parameters are equal to each other?

Is there an easier way to derive the general form of the normalization constant when several of the $\lambda$s are equal, other than by using L'Hopital's rule?

I appreciate any help that you can give.

1 Answers1

1

A general form is given here (Section 3): https://arxiv.org/abs/2204.13290

I won't reproduce the mathematical detail from this link, but the idea is basically as follows:

Suppose that some elements of $\lambda$ are repeated. Then the normalizing constant corresponding to this $\lambda$ can be related to the normalizing constant corresponding to a lower-dimensional $\lambda'$, where $\lambda'$ ''collapses'' the repeated values of $\lambda$ onto a single coordinate. This relationship can be worked out by writing out the integral definition of the normalizing constant and integrating out the dimension corresponding to the repeated coordinate. Applying this relationship repeatedly lets us collapse all repeated coordinates of $\lambda$, eventually obtaining an expression that depends only on a continuous categorical distribution where there are no repeated $\lambda$s.

egr95
  • 46
  • Hi egr. It is preferable that you write something here for this site (stackexchange.com), rather than simply steering people to another site. Can you summarize the relevant bit from Section 3? – 311411 Apr 30 '22 at 18:34
  • While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – 311411 Apr 30 '22 at 18:34
  • Done, hope this helps – egr95 Jun 14 '22 at 03:40