Proving empirical distribution function is the effective estimator for cumulative distribution function

Question

Let $\hat{F}_{n}(x)$ be the Empirical Distribution Function (EDF), where $X_{1}...X_{n}$ is an iid sample:

$$ \hat{F}_n(x) = \frac{1}{n}\sum_{i=1}^n1_{\{X_i\leq x\}} \quad \forall x\in\mathbb{R}$$

It can be shown that $\forall x$ $E\hat{F}_{n}(x)=F(x)$, where $F(x)$ is a CDF. So the EDF is an unbiased estimator for CDF at any point.

Question: Can it be shown that it is also an "effective" estimator? Or what I mean (English isn't my 1st language so I'm afraid of using wrong terminology), for any other estimator $G_{n}(x)$ such that $EG_{n}(x)=F(x)$ we get $$Var(G_{n}(x))\ge Var(\hat{F}_{n}(x)) $$

Answer 1

The following shows that EDF is not always the min variance unbiased estimator, though it is optimal over a certain class of estimators, and also optimal when $\{X_i\}$ are Bernoulli.

---

Assume that $\{X_i\}$ are iid with CDF $F_X(x)=P[X\leq x]$, where $X=X_1$. Fix $x \in \mathbb{R}$. Define $p=F_X(x)$. We want an unbiased estimator of $p$ based on $n$ i.i.d. samples $\{X_i\}_{i=1}^n$.

To this end, define the "EDF" estimator: $$ \hat{F}_n(x) = \frac{1}{n}\sum_{i=1}^n 1_{\{X_i\leq x\}} $$ where $1_{\{X_i\leq x\}}$ is an indicator function that is 1 if $X_i\leq x$ and $0$ else. Define $$Y_i=1_{\{X_i\leq x\}} \quad \forall i \in \{1, 2, 3, ...\}$$ Then $\{Y_i\}$ are i.i.d. $Bern(p)$ and $\hat{F}_n(x)=\frac{1}{n}\sum_{i=1}^nY_i$. It is clear that $\hat{F}_n(x)$ is an unbiased estimator of $p$ with variance $p(1-p)/n$.

Unbiased estimators based only on $\{Y_i\}_{i=1}^n$:

If we restrict attention only to estimators based on $\{Y_1, ..., Y_n\}$, rather than more general estimators based on $\{X_1, ..., X_n\}$, then it can be shown $\frac{1}{n}\sum_{i=1}^nY_i$ is the min variance unbiased estimator of $p$. This is because its variance meets the Cramer-Rao bound (CRB) with equality. See "Example: Single-Parameter Bernoulli experiment" here: https://en.wikipedia.org/wiki/Fisher_information

Counter-example for estimators based only on $\{X_i\}_{i=1}^n$:

We can get an "academic" counter-example by crafting a family of distributions parameterized by an unknown value $q\in (0,1)$.

Define a discrete random variable $Z\in\{0,10\}$ with $P[Z=0]=q$, $P[Z=10]=1-q$.

Define $X=Z+q$.

Let $\{X_i\}_{i=1}^{\infty}$ be i.i.d. with the same distribution as $X$. Fix $x=5$. We want to estimate $p=P[X\leq 5]$ based on $n$ observations of $\{X_i\}$. We have $$p=P[X\leq 5] = P[Z+q\leq 5] = P[Z=0] = q$$ The estimator $\frac{1}{n}\sum_{i=1}^n 1_{\{X_i\leq 5\}}$ is unbiased and has variance $q(1-q)/n$.

However, the estimator $X_1-\lfloor X_1\rfloor$ is unbiased and has variance 0.