0

This post talks about "the difference between multinomial and categorical distribution" without any concrete examples, and the answer have not been accepted yet.

section 3.9 of the "Deep Learning Book" Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning distinguishes the difference:

The multinoulli distribution is a special case of the multinomial distribution

without any concrete examples either.

It seems that this CMU Machine Learning Course consider rolling dice as multinomial distribution while I think might be multinoulli distribution per the "Deep Learning Book".multinomial

Am I missing something?

in other words, could someone give an concrete example to illustrate the scenario in which rolling dice could be treated as a multinomial distribution and some other scenarios in which rolling dice could be treated as a multinoulli distribution

JJJohn
  • 1,436
  • Your question is hard to follow...I think you are jumbling your terms. Please edit for clarity. – lulu Nov 09 '19 at 12:06
  • @lulu I've updated my question, would you please take a loot at that? – JJJohn Nov 09 '19 at 12:48
  • I think your confusion is purely semantic. For a two state outcome, a Bernoulli process is the same as a one trial Binomial process. Same thing here. One roll of a die is either a multinoulli process or a one trial multinomial process. – lulu Nov 09 '19 at 12:50
  • @lulu thanks for your reply. in this context, shall I treat 2 rolls of a die a 2 trial multinomial process which is not a multinoulli process, right? – JJJohn Nov 09 '19 at 12:54
  • Just think about the two state case. A two trial binomial process is not the same as a bernoulli process. But...don't get hung up on vocabulary. People have given names to some standard distributions that seem to arise a lot, but the important thing is the underlying distribution...not the name we give to it. – lulu Nov 09 '19 at 12:56
  • consider rolling a die 2 times, is this a multinomial process which consists of 2 single multinoulli processes, right? – JJJohn Nov 09 '19 at 13:00
  • It depends what you do with the outcome! In a standard multinomial process, you count the occurrences of each outcome. If, say, you roll a $1$ and then a $6$, that would be $(1,0,0,0,0,1)$ with, I think, obvious notation. Same if you roll a $6$ and then a $1$. However, you could also keep track of the order. That would be another multinoulli process, since you'd now have $36$ disjoint outcomes each with its own probability (uniform in this case, assuming a fair die). – lulu Nov 09 '19 at 13:03
  • But you see the semantics problem. I could view the first process as mulinoulli as well, just assign a probability to each of the $21$ possible outcomes. That would be non-uniform even with a fair die as $(1,0,0,0,0,1)$ has twice the probability as $(2,0,0,0,0,0)$. This is why getting hung up on vocabulary is a bad idea. – lulu Nov 09 '19 at 13:07
  • @lulu Assume I use the outcome to compute the MAP estimate, as that CMU Machine Learning Course do. then I got (1,6,...), which consists of 2 single multinoulli processes, right? – JJJohn Nov 09 '19 at 20:37
  • Sorry, I'm not going to watch an hour + video . It still seems to me that you are hung up on vocabulary. The usual multinomial process just keeps tracks of the running results of independent trials of a multinoulli process. That's a very standard thing to do. – lulu Nov 09 '19 at 20:45

0 Answers0