2

The pdf for Dirichlet distribution seems to be

$$ Dir(\alpha_1,\alpha_2,\ldots\alpha_k) \text{ is defined as} $$ $$ pdf(θ_1,θ_2,\ldots,θ_k )= \frac{Γ(\alpha_0)}{Γ(∝_1 )Γ(∝_2 )\cdots Γ(∝_k )} θ_1^{∝_1-1} θ_2^{∝_2-1} \cdots θ_k^{∝_k-1} $$ $$ \text{ over the region where } θ_i \gt 0 \text{ and } θ_1 + θ_2 + ... + θ_k = 1 $$ $$ \text{ and } ∝_0 = ∝_1+ ∝_2+\cdots+∝_k $$

The formula for a multinomial seems to be $$ pmf(φ_1,φ_2,\ldots, φ_k;n_1,n_2,… n_k)= \frac{Γ(n_0+1)}{Γ(n_1+1)Γ(n_2+1)\cdots Γ(n_k+1)} φ_1^{n_1} φ_2^{n_2}\cdots φ_k^{n_k} $$ $$ \text{ over the region where } φ_i \ge 0 \text{ and } φ_1 + φ_2 + ... + φ_k = 1 $$ $$ \text{ and } n_0 = n_1+ n_2+\cdots+n_k $$ I am unable to see the difference in the formulas? Did I miss some other conditions?

(I am aware that Dirichlet is a conjugate-prior to multinomial)

Ranjan
  • 55
  • 4

1 Answers1

2

The pdf for the multinomial distribution is a discrete probability mass function, supported on the set of tuples $(n_1,\ldots,n_k)$ of non-negative integers satisfying the constraint $n_1+\cdots+n_k=n$. The notation you use obscures that. Which distribution it is depends on parameters $(\alpha_1,\ldots,\alpha_k)$. The input to the function is a tuple $(n_1,\ldots,n_k)$ and the output is a probablity.

The pdf for the Dirichlet distribution is a continuous probability density function, supported on the set of tuples $(\theta_1,\ldots,\theta_k)$ satisfying the constraint $\theta_1+\cdots+\theta_k=1$. Obviously these are real arguments, not integer arguements, and they vary continuously, not discretely. The output is not a probabilty, and in particular it can be far more than $1$.

A lot of confusion can be caused by messy notation, and yours is pretty bad. You're actually using the same letter, $p$ to refer to two quite different functions, and then you're using $p$ with subscripts to refer to something altogether different, and you don't at all make it clear that the input to the probability mass function for the multinomial distribution is a tuple of integers $n_1,\ldots,n_k$ rather than a tuple of real numbers, nor do you make clear the relationship between the arguments $p_1,\ldots,p_k$ and the parameters $\theta_1,\ldots,\theta_k$.

Michael Hardy
  • 1
  • Thanks. As a feedback - After reading your answer, I understood the subtle difference between a integer valued and real-valued functions, and read the literature again. Now, it all made sense. The fact that these distributions share similar formulas, actually helps in becoming "conjugate priors". This was not obvious to me, as I was reading the literature for understanding LDA, and not as part of a course. – Ranjan Mar 10 '15 at 07:02