1

Following this question, I have the PDF of a gamma-exponential compound distribution as

$$f(y) = \frac{\alpha\beta^{\alpha}} {(y+\beta)^{(\alpha+1)}} $$

For my application I need the entropy of this distribution. So far I've evaluated this numerically using quadrature, but it's too slow. Is there a closed-form expression for the entropy of this distribution? I've attempted to derive it myself by evaluating

$$\int_{-\infty}^{\infty} f(y) \log(f(y)) dy$$

as a sum of two improper integrals $$\int_{-\infty}^{0} ... dy + \int_{0}^{\infty} ... dy$$ but I get lost when trying to evaluate the infinite limits:

$$\int f(y)\log(f(y)) dy = \frac{ \beta^{\alpha} (\beta+y)^{-\alpha} (1+\alpha-\alpha\cdot{}\log(\alpha \beta^{\alpha} (\beta+y)^{-1-\alpha})) }{ \alpha }$$ (according to Wolfram)

$$\int_{-\infty}^{\infty} f(y) \log(f(y)) dy = \lim_{m\to\infty}[ \frac{ \beta^{\alpha} (\beta+m)^{-\alpha} (1+\alpha-\alpha\cdot{}\log(\alpha \beta^{\alpha} (\beta+m)^{-1-\alpha})) }{ \alpha } ] - \lim_{n\to-\infty}[\frac{ \beta^{\alpha} (\beta+n)^{-\alpha} (1+\alpha-\alpha\cdot{}\log(\alpha \beta^{\alpha} (\beta+n)^{-1-\alpha})) }{ \alpha }] $$

(Skipped a step or two there.)

But here I get stuck because I'm not sure how to evaluate the limits when the logarithms are part of the function, as I haven't found (here or here) a clear limits of logarithms rule that I can apply here.

Sorry my typesetting is ugly. I'd love to know if there's a known entropy for this distribution or if somebody can help me past where I'm stuck in the derivation of it. Thanks

Community
  • 1
Josh Hansen
  • 113
  • That distribution makes no sense for $y\lt-\beta$. In the question you linked to, it's only defined for $y\gt0$. In that case, you'd need to integrate from $0$. I think the antiderivative you got from Wolfram|Alpha evaluates to $0$ at infinity, so the entropy would be its value at $0$. – joriki Feb 26 '16 at 00:38
  • Ah yes, the distribution does have non-negative support. In that case and assuming the limit does go to $0$ as you suggest, I think the result would be: $H(Y)=\frac{-\alpha \log(\alpha \beta) + \alpha + 1}{\alpha}$. I could still use some help seeing why the limit goes to $0$, though. – Josh Hansen Feb 26 '16 at 00:55
  • I think it's $\log(\alpha/\beta)$? (BTW, in the question you have $a$ there; I guess it's meant to be $\alpha$?) – joriki Feb 26 '16 at 00:56
  • The limit goes to zero because $(\beta+m)^{-\alpha}$ goes to zero polynomially whereas the logarithm it's multiplied only increases logarithmically. – joriki Feb 26 '16 at 00:58
  • Good catch on both counts. I fixed the alphas. And yes, I think the complete result is $H(Y)=\frac{-\alpha \log(\frac{\alpha}{\beta})+\alpha+1}{\alpha}$ – Josh Hansen Feb 26 '16 at 01:08
  • You can slightly simplify that as $\frac{\alpha \log\left(\frac{\beta}{\alpha}\right)+\alpha+1}{\alpha}$. – joriki Feb 26 '16 at 01:15
  • Which is $\log\beta - \log\alpha + 1 + \alpha^{-1}$. – joriki Feb 26 '16 at 01:16

1 Answers1

0

As commented in the original question, this is a Pareto distribution with parameter $\alpha$ (it's shifted, but that's irrelevant - you can consider instead the variable $z=y+\beta$). It's entropy is then

$$H(y)=\log\left(\frac{\beta}{\alpha}\right)+1 + 1/\alpha$$

as computed here

Community
  • 1
leonbloy
  • 63,430