6

I have a question about a statement which intuitively seems like it should be a canonical fact, but which I cannot find in any common textbook on probability. Namely, does there exist a probability space $(\Omega,\mathcal F,\mathbb P)$ such that for any probability measure $\mu$ on $\mathbb R^d$, there exists a random variable $X:\Omega\to\mathbb R^d$ such that $X$ has law $\mu$, i.e. $X_\#\mathbb P=\mu$? Can we do this on a single probability space as $d$ ranges over the natural numbers?

I suspect that the space $$([0,1]^\omega,\mathcal B_{[0,1]}^{\otimes\omega},\mathcal L^\omega)$$ should do the trick, where $\mathcal L$ is the Lebesgue measure on $[0,1]$, which I see can simply reduce to proving that $$([0,1]^d,\mathcal B_{[0,1]}^{\otimes d},\mathcal L^d)$$ works for $\mathbb R^d$, but the details of this last part are a bit beyond me.

Dominic Wynter
  • 6,159
  • 2
    From deterministic functions of a single random variable $U$ that is uniform over $[0,1]$, you can generate an infinite sequence of i.i.d. uniform variables, and also generate a random vector in $\mathbb{R}^d$ with any (valid) joint distribution you like. – Michael Mar 19 '18 at 05:46
  • 1
    How can that be done? – Dominic Wynter Mar 19 '18 at 05:53
  • 1
    It is easiest to generate a 1-d random variable $X$ from $U$ via a generalized inverse of the CDF $F_X(x)$, see here: https://math.stackexchange.com/questions/1204378/existence-of-a-random-variable-given-a-cdf – Michael Mar 19 '18 at 05:55
  • 2
    For the infinite sequence ${Z_1, Z_2, Z_3, ...}$ of i.i.d. uniform variables generated from $U$, take the decimal expansion $U=0.U_1U_2U_3...$ and use that to fill in the decimal expansions of ${Z_i}$ by "diagonally" filling in a grid of points $(i,j)$, $i\geq 1, j \geq 1$. The $j$th decimal expansion value of $Z_i$ is on the grid point $(i,j)$. Or just use your favorite bijection $\mathbb{N}\leftrightarrow \mathbb{N}\times \mathbb{N}$ between the indices ${i}$ and ${(i,j)}$ for $0.U_1U_2...$ and $0.Z_{i1}Z_{i2}...$. – Michael Mar 19 '18 at 05:57
  • 1
    To sum up, $$([0,1]^\omega,\mathcal B_{[0,1]}^{\otimes\omega},\mathcal L^\omega)$$ is not necessary since $$([0,1],\mathcal B_{[0,1]},\mathcal L)$$ suffices. – Did Mar 19 '18 at 07:51

1 Answers1

1

Suppose we have a (valid) joint distribution $F_{X_1, ..., X_d}(x_1, ..., x_d)=P[(X_1, ..., X_d)\leq (x_1,..., x_d)]$. We want to generate a random vector $(X_1, ..., X_d)$ with that joint CDF.

Method 1 (assuming we can compute conditional distributions):

  • Start with a single random variable $U$ that is uniform over $[0,1]$.

  • From that, generate i.i.d. uniform variables $Z_1, ..., Z_d$ as I mentioned in comments above.

  • Using $Z_1$, generate $X_1$ from the marginal $F_{X_1}(x_1)$ using the 1-d method in my comments above. Call $u_1$ the particular value of $X_1$ that is chosen.

  • Using $Z_2$, generate $X_2$ from the conditional CDF $P[X_2\leq x_2 |X_1=u_1]$ via the same 1-d method. Call $u_2$ the particular value of $X_2$ that is chosen.

  • Using $Z_3$, generate $X_3$ from the conditional CDF $P[X_3 \leq x_3 | X_1=u_1, X_2=u_2]$.

and so on.

Method 2:

  • Start with $U$ uniform over $[0,1]$.

  • From $U$, generate the infinite i.i.d. sequence $\{Z_i\}_{i=1}^{\infty}$ of uniform variables.

  • Chop up $\mathbb{R}^d$ into a countable collection of disjoint unit hypercubes $\{S_i\}$. Let $p_i = P[(X_1, ..., X_d) \in S_i$]. Use $Z_1$ to randomly choose a hypercube $i \in \{1, 2, 3, ...\}$ with prob $p_i$. This is like determining the integer parts of the random vector $(X_1, ..., X_d)$. To get the remaining fractional parts do the next steps.

  • Refine the chosen (hyper)cube by chopping into $2^d$ subcubes, use $Z_2$ to randomly choose a subcube (according to the appropriate conditional probabilities given you are already in the larger cube).

  • Refine the chosen subcube by chopping into $2^d$ sub-subcubes, use $Z_3$ to randomly choose a sub-subcube.

  • And so on, so each $Z_i$ refines our (binary) expansion of the random vector $(X_1, ..., X_d)$.

One disadvantage of this method is that it involves an infinite number of steps. The nice thing is that the conditional probabilities needed are always well defined as the denominators are always nonzero. Indeed, if at any step the prob 0 event occurs that you end up in a cube with mass 0, just stop the process and define $(X_1, ..., X_d)=(0,0,...,0)$ (or, just start again if you like). This will not affect the distribution as it happens with prob 0. That is, with prob 1, at each of the infinite number of steps, we choose a subcube with positive probability mass.

Michael
  • 23,905