1

I know that $X$ is a Poisson random variable with parameter $\lambda$, and need to find

$$E[X \log X].$$

How do I approach it?

In general, how do we approach the expectation of some arbitrary expressions on random variables (if the question makes any sense)?

This particular problem arose as a part of a computer science problem I'm working on.

yujaiyu
  • 138
  • 1
    This particular question is a bit more difficult, but you need to know combinatorial tricks in general. Shifting indices of summation, changes of variable, recognizing standard forms, differentiation of power series etc. will all hold you in good stead. This particular question is a bit hard from definition. It will help if you add your references for this question. Which textbooks do you consult for probability? Where did you learn the Poisson variable from? Is this question from that book? Allowing others to access your resources allows tailoring of an answer to your comforts. – Sarvesh Ravichandran Iyer Jul 14 '21 at 03:14
  • @TeresaLisbon This particular question does not come from a textbook (to the best of my knowledge). It arose in a practical research problem I am solving as a part of my computer science studies. I've learned about Poisson distribution from "Understanding Probability" by Henk Tijms, and from Wikipedia. – yujaiyu Jul 14 '21 at 03:28
  • That is very useful information, thank you. I request you to edit this into your question post , or give me the permission to do so (Note : it could remain in the comments, but occasionally people miss it. I'll look up Tijms' book anyway). – Sarvesh Ravichandran Iyer Jul 14 '21 at 03:31
  • @foki: This posting may be of interest to you. – Mittens Jul 20 '21 at 01:49

1 Answers1

3

Since this is a discrete distribution you would sum over all possibilities:

$$E = \sum_i Pr(X_i) X_i \log X_i.$$

However, there's no guarantee that this will have a simple closed-form solution. Even if it was a continuous distribution so that you had an integral, generic integral don't yield simple closed-form solutions. In this case you'll have a term that's a logarithm of a factorial, so I'm pretty confident you won't get a very nice expression.

David
  • 1,609
  • Hi David, I think this might have a closed form. I wanted to mark this question as a duplicate first, but then I saw only one question which actually didn't have an answer, but instead a nice hint. See here. – Sarvesh Ravichandran Iyer Jul 14 '21 at 03:15
  • @TeresaLisbon oh cool! I'm not sure the hint quite makes sense to me, but I do think the identity $\log Z = \lim_{n \rightarrow 0^+} d Z^n/dn$ might be useful. – David Jul 14 '21 at 03:22
  • It has to do with differentiation under the integral. Let $f(s) = E[X^s]$. Then, $f$ is apparently differentiable. What should the derivative of $f$ be? Well, we expect , if we can exchange the derivative with the integral, that $f'(s) = E[g'(s)]$ where $g(s) = X^s$. Now, $g'(s) = X^s \log X$, so $f'(1) = g'(1) = E[X \log X]$. So we need to find $f'(1)$. – Sarvesh Ravichandran Iyer Jul 14 '21 at 03:25
  • @TeresaLisbon Ah, got it, nice! I was just thinking about $\log X$, I hadn't realized that $X \log X$ was actually simpler. – David Jul 14 '21 at 03:27
  • 1
    I know, right! It's a really smart trick. – Sarvesh Ravichandran Iyer Jul 14 '21 at 03:31
  • @David where does that identity come from? Any reference? – yujaiyu Jul 14 '21 at 21:52
  • 1
    foki I kind of think of the definition of a discrete random variable $X$ as being that $X$ takes the value $x_i$ with probability $p_i$, with the probabilities nonnegative and summing to one. If that's the case, the value of a function $f(X)$ averaged over the probability distribution is $E[f] = \sum_i f(x_i) p_i$. This extends conceptually to multidimensional and continuous variables. Wikipedia's article on "Expected value" covers it pretty well. Note also that per @TeresaLisbon my skepticism that the sum could be evaluated was probably unwarranted. – David Jul 15 '21 at 21:10