Unbiased estimate and distribution of the maximum of a uniform random variable given a sample maximum

Question

Consider a uniform continuous random variable $x$ with minimum $0$ and unknown maximum $x_\text{max}$. The task is to find an unbiased estimate of $x_\text{max}$ given only the maximum of a sample of $N$ points, $x_\text{max,s}$. The discrete version of this problem is called the German tank problem. An unbiased estimate was given in a previous question, however, I do not understand why the estimate is unbiased.

For a single piece of data ($N = 1$) it is obvious that the probability that the sample maximum $x_\text{max,s}$ is less than $x$ is $$P(x_\text{max,s} < x | x_\text{max}) = \frac{x}{x_\text{max}}.$$

For $N$ independent samples this becomes $$P(x_\text{max,s} < x | x_\text{max}) = \left(\frac{x}{x_\text{max}}\right)^N.$$

This can be converted to a density function: $$p(x_\text{max,s} | x_\text{max}) = \frac{N}{x} \left(\frac{x}{x_\text{max}}\right)^N.$$

One can then calculate the expected value: $$E(x_\text{max,s} | x_\text{max}) = \int_0^{x_\text{max}} x \, p \,\mathrm{d}x = \left(\frac{N}{N + 1}\right) x_\text{max}.$$

This is my understanding of the derivation given in the previous question here.

The answer in the previous question then suggests that the equation above can be "rearranged" into the following estimate, which is claimed to be unbiased: $$\hat{x}_\text{max} = \left(\frac{N + 1}{N}\right) x_\text{max,s}.$$

Why is this estimate unbiased? I assume it is meant that $E(x_\text{max} - \hat{x}_\text{max}) = 0$, but I can't see how that is satisfied here.

I am also interested in the distribution of $x_\text{max}$. I've tried a Bayesian approach to get $p(x_\text{max} | x_\text{max,s})$, but unfortunately the normalizing integral diverges for all priors I've tried, because the power on $x_\text{max}$ is strongly negative in $p(x_\text{max,s} | x_\text{max})$.

If these answers can not be found exactly, I'd also be interested in approximations. Thanks to whoever can help.

Answer 1

Your choice of notation is a bit unfortunate, because it is less immediately obvious to ascertain what is a parameter, versus random variable, versus observation, versus estimate, etc. To address this, we will use lowercase Greek letters for parameters, capital English letters for random variables, and lowercase English letters for data/observations.

Now, if $X \sim \operatorname{Uniform}(0,\theta)$, for a parameter $\theta > 0$, and a sample $\boldsymbol x = (x_1, \ldots, x_n)$ of size $n$ is realized, then the sample maximum, also known as the largest order statistic $$x_{(n)} = \max (x_1, \ldots, x_n) = \max_i x_i$$ has the cumulative distribution $$\Pr[X_{(n)} \le x] \overset{\text{ind}}{=} \prod_{i=1}^n \Pr[X_i \le x] = \left(\frac{x}{\theta}\right)^n,$$ hence the density is $$f_{X_{(n)}}(x) = \frac{nx^{n-1}}{\theta^n}, \quad 0 \le x \le \theta.$$ This much is consistent with what you have written. Now the expectation is straightforward to calculate and yields $$\operatorname{E}[X_{(n)}] = \frac{n}{n+1} \theta.$$ Again, we are in agreement. Now, if we choose the estimator for $\theta$ $$\hat \theta = x_{(n)},$$ namely, we estimate the parameter $\theta$ by taking the largest observation in our sample, then the expectation of this estimator is simply $$\operatorname{E}[\hat \theta] = \operatorname{E}[X_{(n)}] = \frac{n}{n+1}\theta$$ as we previously computed. Thus the bias is $$\operatorname{E}[\hat \theta - \theta] = \operatorname{E}[\hat \theta] - \theta = \frac{n}{n+1} \theta - \theta = - \frac{1}{n+1},$$ so $\hat \theta$ is biased but asymptotically unbiased (the bias tends to $0$ as $n$ increases).

We can now see how we can trivially correct this bias, since $n$ is the sample size and is neither a parameter nor a random variable--it is a known and fixed quantity. We simply apply a multiplicative factor of $(n+1)/n$ to the estimator: $$\tilde \theta = \frac{n+1}{n}\hat \theta,$$ and this bias-corrected estimator obviously has expectation $$\operatorname{E}[\tilde\theta] = \operatorname{E}\left[\frac{n+1}{n} \hat \theta\right] = \frac{n+1}{n} \operatorname{E}[\hat \theta] = \frac{n+1}{n} \cdot \frac{n}{n+1}\theta = \theta.$$ There's nothing more to it.

Your final question about the distribution of $\theta$ (or $x_{\text{max}}$ as you call it) is inherently a Bayesian one, not a frequentist one as we have up to this point treated it. That is to say, you want the posterior distribution of $\theta$ given the sample maximum $x_{(n)}$. But you cannot answer this question without an assumption on the prior distribution of $\theta$, since $$f(\theta \mid x_{(n)}) \propto f(x_{(n)} \mid \theta) p(\theta).$$ Without $p(\theta)$ you cannot compute the posterior likelihood on the right hand side.

Let's give an example. Suppose the prior is exponentially distributed with rate hyperparameter $\lambda$; i.e., $$p(\theta) = \lambda e^{-\lambda \theta}, \quad \lambda, \theta > 0.$$ Then we have $$f(\theta \mid x_{(n)}) \propto \frac{nx_{(n)}^{n-1}}{\theta^n} \lambda e^{-\lambda \theta} \propto \theta^{-n} e^{-\lambda \theta}.$$ Note we remove all factors which are constants of proportionality with respect to $\theta$. We now recognize this expression as being proportional to a gamma distribution with shape and rate hyperparameters $a = n-1$, $b = \lambda$, thus the posterior distribution of $\theta$ is gamma, and this also suggests that a conjugate prior is gamma. Interestingly, this result also shows that for this choice of prior, the posterior doesn't depend on the data you observed, only the number of observations.