0

In the PRML equation 2.239, for a non-informative prior of a scale parameter, the probability mass for interval $A \leq\sigma \leq B$ and $A/c\leq \sigma \leq B/c$ for any choice of all A and B should be the same. Hence $p(\sigma) \propto \frac{1}{\sigma} $.

In the following, it said it is convenient to consider prior distribution in its log form and have $p(\ln\sigma)=\text{const}$

But what's the usage of $p(\ln\sigma)=\text{const}$ ?

Sumanta
  • 9,534
Kelvin_neverKnow
  • 67
  • Is the very next sentence unclear to you? PRML p. 119 (139 of 758): "Thus, for this prior there is the same probability mass in the range $1 \leq \sigma \leq 10$ as in the range $10 \leq \sigma \leq 100$ and in $100 \leq \sigma \leq 1000$." – Eric Towers Dec 10 '20 at 08:11
  • @EricTowers Hi, I just found hard to understand the conclusion $p(ln\sigma) = const$. Why think in for of $ln\sigma$ will be convenient and what could we do about the conclusion. – Kelvin_neverKnow Dec 10 '20 at 08:17

1 Answers1

0

The cited passage goes on to explicitly mention that the same probability mass falls in each decade: $[1,10]$, $[10,100]$, $[100,1000]$, ... Equivalently, the same probability mass falls in each octave: $[1,2]$, $[2,4]$, $[4,8]$, ... Equivalently, the same probability mass falls in each partition by integer powers of a positive constant different from $1$.

In the text between (2.236) and (2.237), we have the text

The parameter $\sigma$ is known as a scale parameter, and the density exhibits scale invariance.

Scale invariance (in dimension $1$) is a synonym for the previous sentences about power partitions.

Further, at the top of p. 118, we have

If we have a distribution $p(x\mid \lambda)$ governed by a parameter $\lambda$, we might be tempted to propose a prior distribution $p(\lambda) = \mathrm{const}$ as a suitable prior.

The text then explains that might be fine for a discrete parameter, but is (generally) incorrect for a continuous parameter. Ending with

If, however, we are to choose a prior distribution that is constant, we must take care to use an appropriate representation for the parameters.

Taking these together in the context of (2.239), we should be looking for a (re-)parametrization leading to a constant probability density. Since $\sigma > 0$ by assumption, one may parametrize by $ \lambda = \sigma$, $\lambda = \sigma +1$, $\lambda = \sigma^3$, $\lambda = \mathrm{e}^\sigma$, $\lambda = \sqrt{\sigma}$, or any of several other functions of $\sigma$. There is one simple choice for which the probability density is constant for the new choice of parameter : $\lambda = \ln \sigma$. (There are "silly" variations, like $\log_7 \sigma$ or $\ln(2\sigma)$, but these are just obscurations of the plain logarithmic relation.)

Eric Towers
  • 67,037
  • Thank you for your answer, could you elaborate how you make the choice of parameter to be $\lambda = \ln \sigma$? – Kelvin_neverKnow Dec 11 '20 at 05:55
  • @Kelvin_neverKnow : ALready answered in the last paragraph. It is (one of) the choice(s) of (re-)parametrization that produces a constant probability density. How does one know that? By having absorbed the information in this section about scale invariance and parametrization choices that are convenient for expressing that property. – Eric Towers Dec 11 '20 at 12:57