5

We know that for sample/empirical distribution function $F_n(x)$ we have that

a) $F_n(x)\xrightarrow[p]{}F(x)$ (pointwise convergence)

b) $\dfrac{\sqrt{n}(F_n(x)-F(x))}{\sqrt{F(x)(1-F(x))}}\xrightarrow[d]{}N(0,1)$

c) $F_n$ converges uniformly in probability to F.

My question is how do we prove that the sample moments of order $k$, and sample central moments of order $k$ converge to $E(X^k)$ and $E(X-E(X))^k$ respectively? (I think need to use the above empirical distribution function properties, but I do not know how, or which...)

Any help would be appreciated.

If you know how to explain that the sample statistics converge, without using any property of the empirical distribution, I would also be thankful.

Michael Hardy
  • 1
An old man in the sea.
  • 5,241
  • It doesn't just follow from the formula? –  Aug 09 '14 at 18:54
  • @Hurkyl what formula? From the empirical distribution definition?

    How do we do that?

     – An old man in the sea. Aug 09 '14 at 19:01
  • At first glance, this looks like the sort of problem where you write down the formula relating the sample moment to the samples, work out the distribution of the sample moment, then use $F_n \to F$ to get your answer. But I could be wrong. –  Aug 09 '14 at 19:05
  • Ok. But how do I relate $1/n \sum X^k_i = g(F_n( \cdot))$, where $F_n(x)=\frac{\sum I[X_i\leq x]}{n}$ and g continuous? – An old man in the sea. Aug 09 '14 at 19:17
  • Well, I was thinking of doing something like $1/n \sum X^k_i = g(F_n())$ with $g$ continuous. But you're suggesting to find the distribution of $1/n \sum X^k_i $? The only thing I know is that $X_i \sim F$. This seems really hard to find. Could you hint me how to do it? – An old man in the sea. Aug 09 '14 at 19:27
  • 1
    My first thought is to write $\displaystyle\lim_{n\to\infty} \int_{-\infty}^\infty x^k , dF_n(x)$. My second thought is that I could be lazy and instead of doing the whole thing from scratch I could look at Serfling's Limit Theorems of Mathematical Statistics and see if it's done there. ${}\qquad{}$ – Michael Hardy Aug 21 '14 at 12:57
  • I might be ignorant, but what part of you question is not answered here? en.wikipedia.org/wiki/Central_limit_theorem – sheß Dec 02 '14 at 14:25

1 Answers1

1

I'm assuming that your sample consists of iid random variables $X_1,\dots, X_n$. The law of large numbers says that if a random variable $X$ has a finite expected value, then then the sample mean $\overline X=\frac1n\sum_{i=1}^n X_i$ converges to $E(X)$ in probability. Replacing $X$ by $X^k$, this implies that the sample moments of order $k$ converge in probability to $E(X^k)$, as long as $E(X^k)$ exists.

For the same reason, $\frac1n\sum_{i=1}^n (X_i-E(X))^k$ converges in probability to the central moment of order $k$ if it exists; however, to get convergence of the sample central moments, we need to prove a modification of this statement where $E(X)$ is replaced by $\overline X$. We can do this by expressing the $k$th sample central moment as a polynomial in terms of the $j$th sample moments, $j=1,\dots,k$ (e.g., see http://mathworld.wolfram.com/SampleCentralMoment.html), and then applying the convergence of sample moments.

If we have an infinite iid sequence $X_1,X_2,\dots$, and if we form a sequence of samples where the $n$th sample consists of the first $n$ variables in this sequence, $X_1,\dots,X_n$, then we can use the strong law of large numbers to strengthen the conclusions above: namely, we will get almost sure convergence (which is stronger than convergence in probability) of both the sample moments and sample central moments.

Brent Kerby
  • 5,539