2

Recently, I dealt with the Cramér theorem, see here, Theorem 1 on page 1.

In this version of the theorem, it is needed that

  • (i) the moment-generating function $M$ is finite on a neighborhood $B_0$ of $0$ and, additionally, that
  • (ii) the supremum in the definition of the rate function I(x) is obtained at some interior point in the neighboorhood $B_0$.

In books and other sources of the internet, the theorem is usually only formulated with (i), that is, it is only needed that there is such a neighborhhod $B_0$ on which the moment generating function is finite, for example here on page 40, Theorem 4.1.5 and (4.2).

So I am really (!) wondering, why (ii) is supposed/ needed in the first linked version since this makes things more difficult:

If I have found a neighborhood that fulfills (i) then up to this version, I have to show additionally that the supremum in the definition of the rate function $I$ is obtained in an interior point of this found neighborhood. Whereas up to the second linked version of the theorem, I can apply the theorem without checking this.

Do you see, if (ii) is in fact needed or that it maybe follows anyway when having found a neighborhood fulfilling (i)?

Rhjg
  • 2,029
  • Note that later in the proof, they differentiate $\log M(\theta)$ at $\theta^$. That doesn't make proper sense unless $M$ is defined in a neighborhood of $\theta^$. – Ian Aug 18 '15 at 17:15
  • 1
    Nonetheless, it seems to me that this comes for free; take $B_0={ \theta \geq 0 : M(\theta) < \infty }$, then for $\theta$ to the right of $B_0$ you have $\theta x - \Lambda(\theta) = -\infty$, which can never be the supremum since $0 - \Lambda(0)=0>-\infty$. – Ian Aug 18 '15 at 17:20
  • @Ian But there might be some $B_0'$ with $B_0\subset B_0'$ and the supremum might be obtained on $B_0'\setminus B_0$? – Rhjg Aug 18 '15 at 17:34
  • In principle yes, that would be the problem. But can that happen where $M$ is infinite? I think it cannot. If I'm right, then the only possible concern is that my $B_0$ might not be a valid neighborhood. (I suppose there is some possible issue in the degenerate case of a constant random variable.) – Ian Aug 18 '15 at 17:38
  • (1.) Why are you defining $B_0$ only for non-negative $\theta$? (2.) Why should it not be possible to have such a $B_0'$ when M is finite? – Rhjg Aug 18 '15 at 17:45
  • 1
    Check the proof of the upper bound to see that if $\theta < 0$ then $\theta x - \log M(\theta) \leq 0$, so the supremum can never be there. – Ian Aug 18 '15 at 17:47
  • Ok. Your $B_0$ is - so to say - maximal, but what if I have found some $B_0'=\left{0 < \theta < c: M(\theta)<\infty\right}$? There might be a neighboorhood that contains $B_0'$ and on which it is $M(\theta)<\infty$, too. For me it seems, that the supremum must not be obtained on $B_0'$ then. – Rhjg Aug 18 '15 at 17:56
  • It might help to do some visualization, which we can do with a toy example. Consider an unbiased Bernoulli r.v. $X$. Then $M(t)=E[e^{tX}]=\frac{1+e^t}{2}$. So $\theta x - \log(M(\theta))=\theta x - \log(1+e^\theta) + \log(2)$. Try graphing this function with $0<x<1$, especially $0.5<x<1$. You should see that it increases to zero at zero, then increases a bit more, then drops. This is the typical situation. The main thing that can change is that the moment generating function might only be finite on some interval, in which case there could be an abrupt drop to $-\infty$. – Ian Aug 18 '15 at 18:06
  • The point of this is that the supremum might be on your $B'_0$, but (as far as I can tell) it is guaranteed to be on my $B_0$. – Ian Aug 18 '15 at 18:11
  • My example actually seems to have exposed something interesting: if $x$ is too large then the supremum can fail to be attained. Notably, the function $\theta-\log(1+e^\theta)+\log(2)$ is increasing forever. So perhaps this is part of the issue? – Ian Aug 18 '15 at 18:15
  • I always thought in order to apply the theorem, it suffices to find any neighborrhood of 0 on which the moment generating function is finite. But as far as I do understand you, this is not enough? – Rhjg Aug 18 '15 at 18:18
  • The problems I can see: $x$ is so large that $\theta \mapsto \theta x - \log(M(\theta))$ is not bounded above; or $\theta x - \log(M(\theta))$ is increasing up until the value of $\theta$ where $M$ blows up. The former is a very real issue: I suspect that whenever $X$ is bounded above and $x$ is greater than or equal to the maximum of $X$, it will be relevant. I do not think the latter can occur at all, but I do not have a proof. – Ian Aug 18 '15 at 18:22
  • But choosing a "good" neighborhood is no obstacle; if the moment generating function is finite on some neighborhood and the supremum is attained somewhere, then the proof goes through. – Ian Aug 18 '15 at 18:24
  • But up to the cited theorem, if I find some neighborhood on which the moment-generating function is finite, I additionally have to show that on it the supremum is obtained, or? Otherwise I cannot apply the theorem – Rhjg Aug 18 '15 at 18:42
  • I think that's right. Think about my Bernoulli example. If you take a sum of unbiased iid Bernoulli rvs, the probability that the sum is greater than or equal to $n$ is just the probability that the sum is equal to $n$, which is $2^{-n}$. So you "should" have $I(1)=\log(2)$ and $I(x)=+\infty$ for $x>1$ (where we define $I$ through the large deviation principle, not through the moment generating function). I don't think the Cramer theorem, at least as formulated in your link, can detect this "boundary" effect. – Ian Aug 18 '15 at 18:49
  • But I emphasize that this is truly a boundary effect; in this example the Cramer theorem, as formulated in your link, tells you what $I$ is for $x \in (-\infty,1)$. – Ian Aug 18 '15 at 18:52
  • I am bit confused to be honest. Maybe that theorem is a version I do not really understand. Do you know another version of it where it is indeed enough to find ANY neighborhood of 0 on which the moment-generating function is 0? – Rhjg Aug 18 '15 at 19:11
  • A more general one is stated here: http://ocw.mit.edu/courses/sloan-school-of-management/15-070j-advanced-stochastic-processes-fall-2013/lecture-notes/MIT15_070JF13_Lec3.pdf You get one of the bounds from your link by taking $F=[x,\infty)$ and the other by taking $U=(x,\infty)$, provided that $I$ is right-continuous at $x$. If $I$ is not continuous at $x$ then the infimum over $U$ could be infinite while the infimum over $F$ is finite. – Ian Aug 18 '15 at 19:26
  • This seems to encapsulate the problem that I mentioned with the Bernoulli r.v. example: in this case $I$ is not right-continuous at $1$ so the LDP does not take the nice form from your link. You still get $\limsup \frac{1}{n} P(S_n \geq 1) \leq -I(1)$ but the other bound becomes $\liminf \frac{1}{n} P(S_n > 1) \geq -\infty$ which is of course useless. – Ian Aug 18 '15 at 19:29
  • Maybe I only have a mistake in my thinking. I have an example with $(Z_i)$ are i.i.d. and $P(Z_i>k)\leqslant (8/9)^{k/2}=e^{-ck}$ with $c=\frac{1}{2}\log(9/8)$. This implies that $M_{Z_i}(t)<\infty$ for $t\in (0,c)$. That is my neighboorhood is here $B_0=(0,c)$ resp. $[0,c)$ since at $0$ the moment generating function is always finite. Isn't then this $[0,c)$ the "maximal" interval on which the moment generating function is finite? So that I can apply the linked theorem? – Rhjg Aug 18 '15 at 19:32
  • I expect that will have the same problem as my Bernoulli r.v. example: for $x=c$ you get a nontrivial upper bound and a trivial lower bound, and then for $x>c$ you get nothing. Pay careful attention: $B_0$ is not defined in terms of $x$, but whether the supremum in the definition of $I$ is attained on $B_0$ does depend on $x$. – Ian Aug 18 '15 at 19:33
  • @Ian Is it always true that $a\theta-\log M(\theta)>0$ for $a>E(X_1)$ and small $\theta>0$? Or only if $a\in Int(D)$? – Rhjg Aug 24 '15 at 20:14
  • For small $\theta$, under the preceding assumptions (that $M$ is finite on some interval around zero), yes. But the supremum may not be finite. – Ian Aug 25 '15 at 00:06
  • Ok. So to apply the linked nice version of Cramer, one has again to have additionalky that x is in int (D). – Rhjg Aug 25 '15 at 08:06
  • Resp.: for small eps> 0 it works again on (m,m+eps) since the continuity of $x\mapsto \theta (x)$. – Rhjg Aug 25 '15 at 08:16
  • @Ian By the way: Chernoff's bound says that $P(S_n\geqslant na)\leqslant e^{-n I(a)}$ for $a>E(X_1)$. Since $\theta a-\log M_{X_1}(\theta)>0$ for small $\theta>0$, it follows that $I(a)>0$ (not necessarily finite). Choosing $1<d<e^{I(a)}$, we get $P(S_n\geqslant na)\leqslant e^{-nI(a)}\leqslant d^{-n}$. This always works, no matter if $I(a)$ is finite or not. - - -Am I right? – Rhjg Aug 26 '15 at 12:35

1 Answers1

1

Most of this answer is in the comments.

The most general LDP for the iid variable case says this:

$$-\inf_{x \in Int(A)} I(x) \leq \liminf \frac{1}{n} \log P(S_n \in A) \leq \limsup \frac{1}{n} \log P(S_n \in A) \leq -\inf_{x \in Cl(A)} I(x)$$

whenever $A$ is Borel. Here $Int$ denotes interior, $Cl$ denotes closure, and $I$ is the rate function. When applied to tails of such averages, you consider $A=[y,\infty)$.

Now assume the $X_i$ have mean $\mu$ and $y \geq \mu$. Then $I$ is monotone increasing on $[y,\infty)$. Therefore the infimum over $[y,\infty)$ is $I(y)$ and the infimum over $(y,\infty)$ is $\lim_{x \to y^+} I(x)$. In the case where $I$ is right continuous at $y$, these are the same. In this case, you get the nice version of the LDP*:

$$\lim \frac{1}{n} \log P(S_n \geq y) = -I(y).$$

But when $I$ is not right continuous at $y$, we become forced to fall back on the form of the LDP that I stated initially.

One way that this can occur is if $X_i$ are unbiased Bernoulli variables, in which case $I(x)=\sup_{\theta \in \mathbb{R}} \theta x - \log(1+e^\theta) + \log(2)$. This has a relative extremum when $x-\frac{e^\theta}{1+e^\theta}=0$. But this ratio of exponentials can only be in $(0,1)$. Consequently if $x \geq 1$, then the supremum is not attained at any finite value. For $x>1$ the supremum is infinite, essentially because $\theta x - \log(1+e^\theta)$ grows linearly at infinity (since $\frac{e^\theta}{1+e^\theta} \to 1$). But exactly at $x=1$, the supremum is actually finite, and is equal to $\log(2)$. Thus $I$ is not right-continuous at $1$.

In this example, since $I(1)$ is finite, the upper bound is still useful; we get

$$\limsup \frac{1}{n} P(S_n \geq 1) \leq -\log(2)$$

which is actually exact, as we know, but the LDP cannot detect this. The lower bound becomes trivial:

$$\liminf \frac{1}{n} P(S_n > 1) \geq -\infty$$

which is completely useless (but is, in fact, exact again, as we know). The LDP suffers from these types of "boundary effects" in many situations. For example, this is one of the subtle difficulties in properly describing how the solution to an SDE with small noise stays "close" to the trajectory whose Freidlin-Wentzell action is minimal, even though the action of the trajectories which are actually chosen is infinite with probability 1.

The hypotheses in your link provide a way to avoid these technical issues that arise in the general case, at the cost of some generality. You can prove that it avoids these issues by using the fact that $I$ is always continuous on $Int(D)$ where $D=\{ x : I(x) < \infty \}$. (Cf. den Hollander p. 8)

* This might not work when $P(S_n=y)$ remains bounded away from zero, such as in the boring case where $X_i=y$ a.s. But this is the generic situation.

Ian
  • 101,645
  • Is $I(x)=\theta^x-\Lambda(\theta^)$ in the link meant to be valid for all $x\in\mathbb{R}$? Or is it to be read as: For each $x\in Int(D)$ there exists some $\lambda\in Int(E)$, $E=\left{\theta: \log M(\theta)<\infty\right}$, such that $I(x)=\theta x-\Lambda(\theta)$? – Rhjg Aug 20 '15 at 17:09
  • @Rhjg The latter (except you have one typo). For instance, the theorem in the link applies to my Bernoulli example for $x<1$, but not for $x=1$. – Ian Aug 20 '15 at 17:12
  • Where are typos? - OK, for the latter; I found the following Lemma: $I$ is convex in $D$, strictly convex in $Int(D)$ and $I\in C^{\infty}(Int(D))$. Furthermore, for any $\hat{x}\in Int(D)$, there exists an unique $\hat{\lambda}\in Int(E)$ with $I(\hat{x})=\hat{\lambda}x-\Lambda(\hat{\lambda})$. --- – Rhjg Aug 20 '15 at 17:18
  • You should have had either $\theta \in Int(E)$ or $I(x)=\lambda x - \Lambda(\lambda)$. But yes, that's the key. – Ian Aug 20 '15 at 17:18
  • If there is some $\lambda^>0$ with $M(\lambda)<\infty$ for $\lvert\lambda\rvert < \lambda^$, that is, $B_0=\left{\lambda: \lvert\lambda\rvert < \lambda^*\right}$ then $B_0\subset Int(E)$ and hence this $B_0$ does what is desired by the link? – Rhjg Aug 20 '15 at 17:22
  • @Rhjg Looks like it. – Ian Aug 20 '15 at 17:38
  • Last question: This holds, too, if I have found some neighborhood $(0,c)$ on which the moment-generating function is finite? – Rhjg Aug 20 '15 at 18:14
  • @Rhjg For some $x$, yes. But in general, no. In my Bernoulli example, the moment generating function is finite everywhere, but the supremum is only attained if $x \in (0,1)$. The supremum then exists and is finite but is "attained" at $\pm \infty$ for $x=0,1$, and is infinite otherwise. – Ian Aug 20 '15 at 18:19
  • But is "for some" x enough to apply the linked theorem? – Rhjg Aug 20 '15 at 19:12
  • @Rhjg I'm not sure you quite understand. The statement of the theorem depends on $x$. The hypotheses of the theorem apply to certain values of $x$; for these, the conclusion of the theorem (which also depends on $x$) follows. For other values of $x$, the hypotheses do not apply, and so the conclusion need not apply either. – Ian Aug 20 '15 at 19:13
  • The statement is for fixed $x>E(X_1)$. How do I know that for this x, there is a theta in $(0,c)$ such that $I(x)=\theta x-\Lambda(\theta)$? – Rhjg Aug 20 '15 at 19:18
  • @Rhjg At the end of the day it comes down to knowing whether $x \in Int(D)$. The way that it is convenient to check that will depend on the problem. Note that the general form of Cramer's theorem still applies even when $x \not \in Int(D)$, it's just that one or both inequalities might not say anything. For instance, it would be useful to work out what happens when $X$ is exponential with parameter $1$ in detail. – Ian Aug 20 '15 at 19:28
  • It is $I(E(X_1))=0$ and on $x\geq E(X_1)$, $I(x)$ is non-decreasing, hence for fixed $x>E(X_1)$, we have that $I(x)<\infty$, hence $x\in D$. But isn't then this fixed $x$ in $Int(D)$ since $x\in (m,x+1)\subset Int(D)$? – Rhjg Aug 20 '15 at 19:32
  • @Rhjg You are not guaranteed that $I(x)<\infty$ even necessarily for any $x>E(X_1)$. In the degenerate case where $X_1$ is always equal to $a$, $I(a)=0$ and $I(x)=+\infty$ otherwise. That said, if the MGF is finite on some interval around zero and you are not in this trivial situation, then it will work for some $x>E(X_1)$, but not necessarily all of them. – Ian Aug 20 '15 at 19:35
  • This is because there might be a point of non-continuity to the left of $E(X_1)$, where I(x) then can be $+\infty$ suddenly, right? – Rhjg Aug 20 '15 at 19:38
  • @Rhjg Right. $I$ will be $+\infty$ on $(-\infty,a)$, then finite and smooth on $[a,b]$, then $+\infty$ again on $(b,\infty)$. I'm pretty sure that this is actually completely general (allowing for $a=-\infty$ and/or $b=+\infty$), but I haven't written it out. – Ian Aug 20 '15 at 19:40
  • But when we express this with $\varepsilon$, then one could say that for small $\varepsilon$, it is $I(E(X_1)+\varepsilon)<\infty$ and so the theorem can be applied for $x\in (m,m+\varepsilon)$? - In the link this is the reason why they SUPPOSE this for x.; so their "fix ANY $x>E(X_1)$" is just due to the fact that they SUPPOSE the supremum is attained for x. (They do not differ between if it is in fact the case for all $x>E(X_1)$, the SUPPOSE is. – Rhjg Aug 20 '15 at 19:41
  • Then the formulation is a bit vague, isn't it? Since they do not say that $x>E(X_1)$ has to be in $Int(D)$. - They just say that there has to be an interior point out of the $\theta$-neighboorhood. – Rhjg Aug 20 '15 at 19:51
  • @Rhjg Their formulation implies $x \in Int(D)$, since they assume $x$ is such that the supremum is attained at an interior point of the neighborhood. – Ian Aug 20 '15 at 19:52
  • The remaining problem for me is, why it seems to be enough (in other books) only to suppose that there is some neighborhood of $0$ on which the mgf is finite, say on (0,c). How can they then claim that for $x>m:=E(X_1)$, the statement $\lim_n \frac{1}{n}\log P(S_n\geq x)=-I(x)$ follows? Since (1) we only know that $(0,c)\subset Int(E)$ and (2) we do not know if for $x>m$ it is $x\in Int(D)$. – Rhjg Aug 20 '15 at 20:08
  • @Rhjg If $x \in Int(D)$, then you get the nice formulation, where the tail probabilities are decaying exponentially (maybe with a lower order multiplicative factor). If $x \in Ext(D)$, then the lower bound and upper bound are both $-\infty$, which means that the tail probability is decaying faster than exponentially. You only get a weird case when $x \in \partial D$, in which case the LDP gives a trivial lower bound and a nontrivial upper bound. In this case the tail probabilities are decaying at least exponentially but perhaps faster. – Ian Aug 20 '15 at 20:13
  • @Rhjg My Bernoulli example illustrates this. In that case, $D=[0,1]$. For $x \in (0,1)$, you get the nice formulation. For $x \in (-\infty,0) \cup (1,\infty)$, the probabilities are actually just zero, so they are certainly decaying faster than exponentially, and the LDP tells you that much. For $x=0$ or $x=1$, the probabilities are decaying exponentially, and the LDP only tells you that might or might not be happening. – Ian Aug 20 '15 at 20:20
  • Okay, but what is the role of the (0,c), the neighborhood of 0 on which the mgf is finite? If $x\in Int(D)$, okay, then there is an $\theta\in Int(E)$ such that $I(x)=\theta x-\Lambda(\theta)$. But what has (0,c) to do with that? Same when $x\in Ext(D)$ or $x\in\partial D$: What has $(0,c)$ to do with that? – Rhjg Aug 20 '15 at 20:26
  • @Rhjg Let $E={ \theta : M(\theta) < \infty }$. Then there exists $\theta^* \in Int(E)$ such that the supremum in the definition of $I$ is attained at $\theta^*$ if and only if $x \in Int(D)$. I don't think this is that hard to prove, and I think it will clear up your confusion. – Ian Aug 20 '15 at 20:28
  • Yes, if $x\in Int(D)$, then the supremum in the definition is attained for a $\theta\in Int(E)$, I got this. But $(0,c)\subset Int(E)$, or? I do not see which exact role this (0,c) plays... since for some $x\in Int(D)$ the associated $\theta\in Int(E)$ might be in $Int(D)\setminus (0,c)$. – Rhjg Aug 20 '15 at 20:31
  • @Rhjg You can forget about $(0,c)$. Everything goes through in the maximum possible generality if you replace that with $Int(E)$. – Ian Aug 20 '15 at 20:32
  • And exactly this is my problem! In many versions it is just claimed it needs to have ANY neighborhood on which the mgf is finite, f.e. an interval (0,c)$. – Rhjg Aug 20 '15 at 20:33
  • My problem is: In a text I found an interval (0,c) and wrote that because of this I can apply cramer. But is it true to formulate it like this? – Rhjg Aug 20 '15 at 20:34
  • If you know of some (perhaps non-maximal) $(0,c)$ where the MGF is finite, then this is contained in $Int(E)$, so you can apply the theorem for the $x$'s such that $\theta^* \in (0,c)$. – Ian Aug 20 '15 at 20:35
  • For the $x's>E(X_1)$ in $Int(D)$ such that $\theta^*\in (0,c)$? – Rhjg Aug 20 '15 at 20:39
  • @Rhjg Right. (It still applies for $x \leq E(X_1)$ but the infimum will always be at $E(X_1)$, which is just telling you that the weak law of large numbers applies.) – Ian Aug 20 '15 at 20:42
  • In my text I wrote that since on (0,c) the MGF is finite, by Cramer it follows that $\lim_n\frac{1}{n}\log P(S_n>\varepsilon+E(X_1))=-I(\varepsilon+E(X_1))$ for all $\varepsilon >0$. I now think this was wrong. Since I do not know if for all $\varepsilon >0$ I have (i) $E(X_1)+\varepsilon\in Int(D)$ and (ii) if the associated $\theta^*$ is in $(0,c)$. – Rhjg Aug 20 '15 at 20:49
  • @Rhjg I agree. But outside the fully degenerate case, it will work for $x \in (E(X_1),E(X_1)+\varepsilon)$ for some small $\varepsilon>0$. – Ian Aug 20 '15 at 20:52
  • How do you know that for $(E(X_1),E(X_1)+\varepsilon)$, $\varepsilon$ small, the associated $\theta^*$'s are in $(0,c)$? – Rhjg Aug 20 '15 at 20:53
  • @Rhjg $\theta^$ is continuous on $Int(D)$, and $\theta^(E(X_1))=0$. (Note that there is no contradiction in the degenerate case, since then $Int(D)=\emptyset$.) – Ian Aug 20 '15 at 20:55
  • Later in my text, there will be $\varepsilon\to 0$ anyway. Since if I wrote: "Since the MGF is finite on $(0,c)$, it is $\lim_n\frac{1}{n}\log P(S_n\geq E(X_1)+\varepsilon)=-I(x)$ for $\varepsilon\to 0$ , then this should be okay? Since if $\varepsilon\to 0$, $x$ is in $(E(X_1),E(X_1)+\varepsilon)$.... (maybe better to write that in another way since the intervall wil be empty then...) – Rhjg Aug 20 '15 at 21:06
  • So, I guess the following to be right. Let $m:=E(X_1)$. Given, that for some c>0, on (0,c) the mgf is finite, it follows that, by Cramer, $lim_n \frac{1}{n}\log P(\frac{1}{n}S_n\geq m+\varepsilon)=-I(m+\varepsilon)$ for $\varepsilon>0$ small. In particular, the identity holds for $\varepsilon\to 0$. -- Am I finally right? – Rhjg Aug 20 '15 at 22:27
  • @Rhjg Except in the degenerate case, yes. – Ian Aug 20 '15 at 22:27
  • Ok. So my last comment is right, great. Sorry that I made this thread so confusing. What was the degenerate case? – Rhjg Aug 20 '15 at 22:34
  • @Rhjg When $X_i=x$ with probability $1$, $I(x)=0$ and $I(y)=+\infty$ for $y \neq x$. Well...I guess then it still works, it's just that the right side is $-\infty$. So disregard that (provided the right side being infinite is acceptable for your needs). – Ian Aug 20 '15 at 22:39
  • Ok! Think I got it now. Since in my text, in which I do not have the degenerate case and in which I show for a (0,c) that the mgf is finite and I later consider eps to 0,, I think my mistake to say that the identity holds for all eps>0 is not too bad. The main thing for my text is that the identity holds for m+eps with eps small resp. for m+eps as eps tends to 0. As you confirmed, this is true. So only a small mistake in my text in the end :) – Rhjg Aug 20 '15 at 22:45
  • One thing: why does it hold for eps to 0? Then I have m+eps to m... does the identity hold for m?! – Rhjg Aug 20 '15 at 22:55
  • @Rhjg It definitely applies for $m$. Note that $I(m)=0$, so this is saying that $\frac{1}{n} \log P(S_n \geq m) \to 0$. But in fact $P(S_n \geq m)$ converges to $1/2$ by the central limit theorem. The result follows. – Ian Aug 20 '15 at 23:07
  • It follows by weak law of large numbers, alternatively. By the latter, $\log P(S_n=m)\to 0$, hence $0=\lim_n\frac{1}{n}\log P(S_n=m)\leq \lim_n\frac{1}{n}\log P(S_n\geq m)=-I(m)=0$. Hence $\lim_n\frac{1}{n}\log P(S_n\geq m)=0$ and $-I(m)=0$. – Rhjg Aug 21 '15 at 10:02
  • What did you mean above when writing that $\theta^*$ is continuous on $Int(D)$? I guess you mean that $x\mapsto \theta x-\Lambda(\theta)$ is continuous on $Int(D)$? – Rhjg Aug 21 '15 at 10:10
  • @Rhjg No, the weak law doesn't help you, it deals with $(m-\varepsilon,m+\varepsilon)$. And I mean that $x \mapsto \arg \max_\theta \theta x - \Lambda(\theta)$ is continuous. – Ian Aug 21 '15 at 10:15
  • Ok. Why is it continuous? $\Lambda(\theta)$ is continous on $int(E)$. – Rhjg Aug 21 '15 at 10:20
  • @Rhjg By a concavity argument, $\theta^*$ is the solution to $x-\frac{M'(\theta)}{M(\theta)}=0$. But $\frac{M'(\theta)}{M(\theta)}$ is smooth. So you apply the implicit function theorem (possibly repeatedly, if you need to extend the domain of the implicit function). – Ian Aug 21 '15 at 10:26
  • Since for each $x\in Int(D)$ there is an (even unique) $\theta^\in Int(E)$ such that $I(x)=\theta^ x-\Lambda(\theta^)$, I guess your comment says that $x\mapsto \frac{M'(\theta^)}{M(\theta^)}\cdot\theta^-\Lambda(\theta^)$, that is the right side only depends on $\theta^$. Moreover, it is continuous on $Int(E)$. – Rhjg Aug 21 '15 at 10:38
  • But wouldn't it be better to have something like $x\mapsto \theta^*(x)$ is continuous on $Int(D)$? Do I get that by implicit function theorem?! – Rhjg Aug 21 '15 at 11:01
  • @Rhjg Yes, I meant that $x \mapsto \theta^*(x)$ is continuous on $Int(D)$. But yes: the implicit function theorem gives you the result until you leave the region where $\frac{d}{d \theta} \frac{M'(\theta)}{M(\theta)} \neq 0$. That should be all of $Int(D)$, if I'm not missing something. – Ian Aug 21 '15 at 11:10
  • Ok, and since $m\mapsto \theta^(m)=0$, and one can suppose that $\theta^(x)\geq 0$, one has that for small eps>0, and for $x\in (m,m+eps)$ we have that $\theta^*(x)\in (0,c)$. - - - Why does the weak law of large numbers not help to show that the identity holds for m? ISince it only deals with the probability that $S_n$ is within $(m-eps,m+eps)$ and not with $S_n\geq m$? How do you get with central limit theorem that it converges to 1/2? – Rhjg Aug 21 '15 at 11:16
  • 1
    @Rhjg The weak law says that $P(S_n \in (m-\varepsilon,m+\varepsilon)) \to 1$. It doesn't tell you how that's divided between $(m-\varepsilon,m)$ and $[m,m+\varepsilon)$. The central limit theorem does, because the Gaussian distribution is symmetric about the mean. – Ian Aug 21 '15 at 11:25
  • Ok, $P(S_n\geq m)\Leftrightarrow P(Z_n\geq 0)$ with $Z_n:=\frac{S_n-m}{\sigma}$. The CLT says that the latter is 1/2. – Rhjg Aug 21 '15 at 12:52
  • Still do not see why $x\mapsto\theta^*(x)$ is continuous on $int(D)$, sorry. – Rhjg Aug 21 '15 at 12:55
  • Your implicit function on which you apply the theorem about implicit functions is $F\colon Int(D)\times int(E)\to\mathbb{R}$, $F(x,y):=yx-\Lambda(y)$, right? – Rhjg Aug 21 '15 at 15:20
  • Rather, the implicitly defined function is $y(x)$, and it arises from the equation $F(x,y)=0$ where $F(x,y)=x-\Lambda'(y)$, with the base point $(m,0)$. – Ian Aug 21 '15 at 15:29
  • ...So actually, it is just the inverse of $\Lambda'$. Neat! – Ian Aug 21 '15 at 15:48
  • Ok. Choose $U:=Int(D), V:=Int(E), F\colon U\times V\to\mathbb{R}, F(x,y):=x-\Lambda'(y)$. Then $(0,m)\in U\times V, F(0,m)=0, F\in C^1(U\times V,\mathbb{R})$, since $\frac{\partial F}{\partial x}=1$ and $\frac{\partial F}{\partial y}=-\Lambda''(y)$ are both continuously differentiable (in fact, $\Lambda\in C^{\infty}$ on $V$). Moreover, $\frac{\partial F}{\partial y}<0$. Then the theorem says that there are open neighborhhods $U_0$ of $m$ and $V_0$ of $0$ and an unique continously differentiable function $g\colon U_0\to V_0$ with $g(m)=0$ s.t. $F(x,g(x))=0$ for all $x\in $U_0$.. – Rhjg Aug 21 '15 at 17:12
  • If I choose $U_0=U$ and $V_0=V$ then the only function with $g(m)=0$ so that $F(x,g(x))=0$ for all $x\in U$, is indeed $g(x):=\theta^*(x)$. So the theorem says that this function $g$ is continously differentiable - and therefore continous on $U$. - - Right? – Rhjg Aug 21 '15 at 17:14
  • @Rhjg That's right. However, if $F$ is globally $C^1$, then you can take $U_0$ with a point where $F_y=0$ on its boundary. Then when $F_y$ is actually zero you get a bifurcation. (Again, think of the circle.) – Ian Aug 21 '15 at 17:16
  • For the circle, $U_0=(-1,1)$. What do you mean? – Rhjg Aug 21 '15 at 17:30
  • @Rhjg You stop being able to define the implicit function when you hit the vertical tangents on the sides of the circle. – Ian Aug 21 '15 at 17:33
  • Anyway, what was your intention to remark this after my application of the implicit function theorem above? – Rhjg Aug 21 '15 at 17:34
  • How do you get from theorem of implicit functions that the implicit function g is the inverse of $\Lambda'$? The theorem says that $g'=-1/\Lambda''$ – Rhjg Aug 21 '15 at 18:38
  • You want $\frac{\partial}{\partial y}(xy-\Lambda(y))=0$, hence $x=\Lambda'(y)$, so $y=(\Lambda')^{-1}(x)$. – Ian Aug 21 '15 at 18:43
  • Does the theorem state that the unique function which fulfills F(x,g(x))=0 for all x in U_0 (here U_0=int(D)) is contoinously diff? Hence, here. $g(x):=\theta(x)$ is THE ONLY FUNCTION on int(D) which fulfills this (this follows by the lemma i gave above at the beginning) - so the theorem says:g(x) is contonoussly differentiable? - i.e. continous? – Rhjg Aug 21 '15 at 18:49
  • There is only one function on a sufficiently small neighborhood; you need to do some more work to show that this neighborhood can be taken to be $Int(D)$. But yes, the implicit function theorem tells you that the implicitly defined function is $C^1$. It even tells you how to calculate its derivative. – Ian Aug 21 '15 at 18:50
  • This was the claim of a lemma: That for each $x\in Int(D)$ there is an unique $\theta\in int(E)$, such that $x=\Lambda'(\theta)$. KNOWING this one knowst that this is the only function fulfilling F(x,g(x))=0 for all $U_0=int(D)$, the theorem of implicit function implies that $g(x)$ is continous on int(D). - - -If I can not presuppose this lemma, one would have to show additionally that $g(x):=(\Lambda')^{-1}$ is THE ONLY function fulfilling $F(x,g(x))=0$ for all $x\in int(D)$, right? assuming this lemma, it is clear this is the only function doing this. continuity on int(D) follows ? – Rhjg Aug 21 '15 at 18:56
  • Yes, there is only one possibility until you hit a bifurcation. – Ian Aug 21 '15 at 19:06
  • That is, anytime one can show that on a chosen $U_0$ a function $g(x)$ fulfills $F(x,g(x))=0$ one knows by the theorem (1) its the only function doing this (2) moreover, it is $C^1$? For our example: $g=(\Lambda')^{-1}$ fulfills $F(x,g(x))=0$, on $Int(D)$, hence (1) its the only one doing this (2) its continous. THAT one can choose $int(D)$ is not clear without knowing that $x=\Lambda'(\theta^)$ with $\theta^\in int(E)$ unique for $x\in Int(D)$. This is what you meant that (without the lemma I cited) one would have to show that? – Rhjg Aug 21 '15 at 19:12
  • Is my last comment correct? If yes, I think I can stop asking anymore. – Rhjg Aug 22 '15 at 13:11