How to prove that geometric distributions converge to an exponential distribution?
To solve this, I am trying to define an indexing $n$/$m$ and to send $m$ to infinity, but I get zero, not some relevant distribution. What is the technique or approach one must use here?
Recall pmf of the geometric distribution: $\mathbb{P}(X = k) = p (1-p)^k$ for $k \geq 0$.
The geometric distribution has the interpretation of the number of failures in a sequence of Bernoulli trials until the first success.
Consider a regime when the probability of success is very small, such that $n p = \lambda$, and consider $x = \frac{k}{n}$. Then, in the large $n$ limit: $$ 1 = \sum_{k=0}^\infty \mathbb{P}(X = k) = \sum_{k=0}^\infty \lambda \left(\left(1 - \frac{\lambda}{n} \right)^{n \cdot k/n} \frac{1}{n} \right) \stackrel{n \to \infty}\rightarrow \int_0^\infty \lambda \mathrm{e}^{-\lambda x} \mathrm{d} x $$
Alternatively, you could look at the moment generating function for the geometric distribution: $$ \mathcal{M}(p, t) = \frac{p}{1-\mathrm{e}^t (1-p)} $$ To recover the mgf of the exponential distribution consider the limit: $$ \lim_{n \to \infty} \mathcal{M}\left( \frac{\lambda}{n}, \frac{t}{n} \right) = \lim_{n \to \infty} \frac{\lambda}{n - \mathrm{e}^{t/n}\left(n - \lambda\right)} = \lim_{n \to \infty} \frac{\lambda}{n \left(1 - \mathrm{e}^{t/n}\right) + \lambda \mathrm{e}^{t/n} } = \frac{\lambda}{\lambda-t} $$ which is the mgf of the exponential distribution with rate $\lambda$.
It may be useful to construct the relevant distributions the other way round, that is, to start from a single exponential random variable $X_\lambda$ with density $\lambda\mathrm e^{-\lambda x}$ on $x\geqslant0$, for a given positive $\lambda$, and to consider the integer part $Y_a$ of $X_\lambda/a$, for every positive $a$.
Then, $[Y_a=n]=[na\leqslant X_\lambda\lt (n+1)a]$ hence $\mathrm P(Y_a=n)=\mathrm e^{-\lambda na}-\mathrm e^{-\lambda (n+1)a}$ for every nonnegative integer $n$, that is, $$\mathrm P(Y_a=n)=(p_a)^n(1-p_a),$$ with $$ p_a=\mathrm e^{-\lambda a}. $$ This proves that each $Y_a$ is geometric with parameter $p_a$. Note that, for a given $\lambda$, every parameter $p$ in $(0,1)$ is realized as $p=p_a$ for some $a$, hence, starting from a unique random variable $X_\lambda$, this construction yields a whole family of geometric random variables $(Z_p)_{p\in(0,1)}$ such that each $Z_p$ is geometric with parameter $p$ and such that $Z_p\leqslant Z_q$ with full probability, for every $p\geqslant q$ in $(0,1)$: simply define each $Z_p$ by $Z_p=Y_{\alpha(p)}$, with $$ \alpha(p)=-\log(p)/\lambda. $$ Coming back to your question, $X_\lambda\leqslant aY_a\lt X_\lambda+a$ with full probability, hence $aY_a\to X_\lambda$ almost surely when $a\to0$, in particular $\alpha(p)Z_p\to X_\lambda$ when $p\to1$. Thus, $-\log(p)Z_p$ converges almost surely (hence in distribution) to the standard exponential random variable $X_1=\lambda X_\lambda$.
Since $-\log(p)\sim1-p$ when $p\to1$, this shows that $(1-p)Z_p$ converges almost surely (hence also in distribution) to $X_1$. Now, the convergence in distribution of some random variables does not depend on their realization hence, by the remarks above, for every family $(T_p)_{p\in(0,1)}$ of random variables such that each $T_p$ is geometrically distributed with parameter $p$, the random variables $(1-p)T_p$ converge in distribution to the standard exponential distribution when $p\to1$.
Note Such a simultaneous and almost sure construction of geometric random variables $Z_p$, for every $p$ in $(0,1)$, from a single (exponentially distributed) random variable $X_\lambda$ is called a coupling. For more about this powerful idea, see the WP page on coupling in probability.
The waiting time $T$ until the first success in a sequence of independent Bernoulli trials with probability $p$ of success in each one has a geometric distribution with parameter $p$: its probability mass function is $P(x) = p (1-p)^{x-1}$ and cumulative distribution function $F(x) = 1 - (1-p)^x$ for positive integers $x$ (note that some authors use a different convention where the random variable is $T-1$ rather than $T$, but that won't make a difference in the limit). The scaled version $p T$ converges in distribution as $p \to 0+$ to an exponential random variable with rate $1$, as for $x \ge 0$ $$ P(p T \le x) = F(x/p) = 1 - (1-p)^{\lfloor x/p\rfloor} \to 1 - e^{-x} $$
The geometric distribution is given by $$P(n) = (1-p)^n p = q^np$$ Here $P(n)$ is the probability of a succes after $n$ failures.
If we define the chance of succes $p=X/n$, where $X$ is a bernouilli random variable then we get after substitution in the above equation:
$$P(n) = \left(1-\frac{X}{n}\right)^n \left(\frac{X}{n}\right) $$
In taking the limit we get:
$$ \lim_{n\rightarrow \infty} dP(n) = \lim_{n\rightarrow \infty} \left(1-\frac{X}{n}\right)^n \left(\frac{X}{n}\right) = e^{-X} dX$$
Where we see $e^{-X}$ which is the exponential probability density function.
I think we can just expand 1-p in Taylor series for small p we get the exponential function.:) 1-p=exp(-p) for very small p (around p=0)such that np=constant. so we find:
p(x)=(1-p)^x.p is becoming p exp(-px) for a continuous random variable x, this in part has the continuous exponential distribution function:)
