0

One of the ways to state the (strong) Law of Large Numbers is this:

Given a probability space $(\Omega, \mathcal{A}, P)$,

$$\lim_{n \to \infty} \left( \frac{1}{n} \sum_{i=1}^{n} X_i \right) \overset{a.s.}{\longrightarrow} \mu$$

Where $(X_i)_{1 \leq i < n}$ are independent, identically distributed random variables with mean $\mu$.

But the random variables are (measurable) functions $X : \Omega \mapsto \mathbb{R}$. So, my question is: in this (and similar) presentations of the LLN, what do the $X_i$ represent in terms of the events $\omega \in \Omega$? Are we assuming the same event for all of them (the $X_i$ are really $X_i(\omega)$ for a fixed $\omega$)? Or are we summing over all the events for the same random variable (the $X_i$ are really $X(\omega_i)$ for the same $X$ and different $\omega_i \in \Omega$)? Or something else?

I understand that this is a notation confusion on my part, but I want to know what exactly is being stated here.

For example, I know that whenever we write $P(X_i = x_i)$, we really mean $P(\{\omega\})$, where $\{\omega \in \Omega : X_i(\omega) = x_i\}$, since $P: \mathcal{A} \mapsto [0,1]$. I would like a similar break down of the $X_i$ in the LLN (or Markov Chain analysis etc.)

usernameiwantedwasalreadytaken
  • 103
  • 4
  • 5
    Yes, the $X_i$ are measurable functions. So the conclusion of the LLN means that $\frac1n\sum_{i=1}^n X_i(\omega)\to\mu$ for almost every $\omega\in\Omega$. – David C. Ullrich May 25 '18 at 11:39
  • 1
    @DavidC.Ullrich: That looks like a full answer. – hmakholm left over Monica May 25 '18 at 11:39
  • @HenningMakholm I guess. I'd prefer that if someone is browsing my Answers they see things with more content. If I made that comment an Answer it would be expanded: (i) I'd want to mention the Rademacher functions as a concrete example, (ii) I'd want to explicitly address the OP's confusion instead of just saying how things are: A priori the $"X_i(\omega_i)$" doesn't really make much sense, but given that the $X_i$ are iid it's possible to "realize" things in such a way that $\Omega=\prod\Omega_j$ and in fact $X_i(\omega)=\omega_i$. – David C. Ullrich May 25 '18 at 12:12
  • @DavidC.Ullrich Your comment did clarify things to me. I'm just a bit uncomfortable with the "almost every $\omega$" part... – usernameiwantedwasalreadytaken May 25 '18 at 12:20
  • @usernameiwantedwasalreadytaken: "almost every $\omega$" is standard jargon for "a set of $\omega$s whose probability measure is $1$". – hmakholm left over Monica May 25 '18 at 12:21
  • @usernameiwantedwasalreadytaken Well I don't know what you mean by saying you're uncomfortable with that. Saying something happens for almost every $\omega$ means that $P(A)=1$, if $A$ is the set of $\omega$ such that the something happens. So here if $A={\omega\in\Omega:\frac1n\sum X_i(\omega)\to\mu}$ then $P(A)=1$. – David C. Ullrich May 25 '18 at 12:23

2 Answers2

1

The $X_i$s are measurable functions, but it is common in probability calculations not to write the argument to those functions explicitly because it's always $\omega$.

(We can explain that either as a notational convention, or we can say that we're extending every mathematical operator to work pointwise on the space of measurable functions from $\Omega$).

The "a.s." on the arrow means "almost surely", standard jargon for "in an event of measure $1$".

With these conventions unfolded, $$\lim_{n \to \infty} \left( \frac{1}{n} \sum_{i=1}^{n} X_i \right) \overset{a.s.}{\longrightarrow} \mu$$ means precisely $$ P\left(\biggl\{\omega\in\Omega \Biggm| \lim_{n \to \infty} \biggl( \frac{1}{n} \sum_{i=1}^{n} X_i(\omega) \biggr) = \mu \biggr\}\right) = 1 $$

hmakholm left over Monica
  • 286,031
  • Ok. So suppose we perform an experiment and a measurement encoded in the random variable $X$ and that we repeat this experiment $n$ times each time in the same conditions and each time independently of each other. Each time we may observe a different outcome of the experiment, $\omega$. When @DavidC.Ullrich first commented, I thought we could just sum $X(\omega)$ for all the experiments (which is how a Monte Carlo simulation is made, if I'm not mistaken). But the way you put this, seems that we will have a set of $\omega$'s and, for each of them, the emprical average will tend to $\mu$. – usernameiwantedwasalreadytaken May 25 '18 at 12:39
  • 1
    @usernameiwantedwasalreadytaken: In the usual formalism there is a single $\omega$ that encodes the result of all the (countably many) experiments in your super-experiment. Each of the $X_i$s is a different function from $\Omega$ to (say) $\mathbb R$, such that $X_i(\omega)$ can depend on $i$ even when we hold $\omega$ fixed. That's why we're not saying that they're the same random variable, but merely identically distributed random variables. – hmakholm left over Monica May 25 '18 at 12:49
  • Before we start the sequence of experiments the law promises us that no matter how the sequence turns out -- that is, no matter which single $\omega$ the gods of chance pick -- we can be pretty damn sure that the sequence of sample averages will converge to $\mu$, where "pretty damn sure" means "with probability $1$". – hmakholm left over Monica May 25 '18 at 12:54
  • Ok. It makes sense now. Thank you (by the way, I didn't mean to sound rude before, I'm really thankful for your explanations). – usernameiwantedwasalreadytaken May 25 '18 at 12:55
  • @usernameiwantedwasalreadytaken: I don't think you sounded rude anywhere. – hmakholm left over Monica May 25 '18 at 12:57
1

Each $X_i$ is a map e.g. $X_i : \Omega \to R$. Then you could consider $\prod_i X_i$ a map from $\prod_i\Omega_i$ to $\prod_iR_i$ which just means that for one event $\prod_i w_i$ you have a sequence of real numbers the convergance in probability means that $$ (\prod_i P_i)(\{\prod_iw_i : \frac{1}{n}\sum_{i=1}^{\infty}X_i(w_i) \to \mu\}) = 1 $$ You need to be a bit careful in how to define these, see references below.

To read more see http://planetmath.org/infiniteproductmeasure

Infinite product of measurable spaces

Now for the Markov chain example, which I missed, sorry!, The sequence is dependent and for those you need to model dependance, one example to do this is to consider models of the kind $$ X_{i+1} = F(X_{i}) + Z_{i}, X_0 = 0. $$ with $Z_i$ i.i.d. and hence defined in the product sense above and then from this i.i.d. and a matematical derivation apply the measure. One could say that a common theme in processes modeling is to from a set of independent processes construct a model with dependency that accurately fit the real world example. e.g. see ARMA Models and Kalman filters.

Stefan
  • 889