4

Suppose I have a sequence $X_1, \dots, X_n$ of Bernoulli RVs with the property that for all $i = 1, \dots, n$, the function $$f(x_1, \dots, x_{i - 1}) := \Pr[X_i = 1 \mid X_1 = x_1, \dots, X_{i - 1} = x_{i - 1}]$$ is (non-strictly) decreasing. Is there an exponential tail bound for $X := \sum_{i = 1}^n X_i$?

Intuitively, there should be, because the dependence between the $X_i$'s only helps concentration. That is, if there are many 1's among the first $i-1$ variables, $X_i$ is more likely to be 0, and vice versa. But I can't figure out how to show such a bound. If you find one, you get an acknowledgement on my learning theory paper!

Approaches I tried: (1) coupling arguments (2) showing that the $X_i$'s are negatively associated, which implies (exponential) Chernoff bounds. For an example of (1), a random walk whose step sizes (depending on position) are biased towards the origin is concentrated, via coupling to an unbiased random walk (martingale). I think (2) is more promising, but there are not many tools for proving negative association. I did manage to prove negative association for a coupon-collector type scenario, where the variables clearly have the property above.

Claim: Suppose each day, each of $m$ species of butterflies appears independently (with probability, say, $1 / m$, it doesn't matter). Let $X_t = 1$ if on day $t$, you see a new species of butterfly you haven't seen before, and $X_t = 0$ otherwise. Then the variables $X_1, \dots, X_n$ are negatively associated (implying their sum obeys Chernoff bounds).

Proof: We make use of the following three lemmas about negatively associated random variables, which appear to be the only tools for proving negative association (NA). (1) If $\sum_{i = 1}^m Y_i \leq 1$ with probability 1, then the $Y_i$ are NA. (2) If I have several NA collections of random variables that are independent of each other, then their union is also NA. (3) If $Y_1, \dots, Y_n$ are NA, and I apply functions $f_1, \dots, f_m$ that are all non-decreasing to disjoint subsets of the $Y_i$, then the resulting random variables are NA. To prove the claim, let $X_{tj} = 1$ if butterfly species $j$ is seen on day $t$. Let $Y_{tj} = 1$ iff $X_{tj} = 1$ but $X_{sj} = 0$ for all $s < t$; that is, iff you see species $j$ for the first time on day $t$. For fixed $j$, the variables $\{Y_{tj}\}_{1 \leq t \leq n}$ satisfy (1), thus are NA. Similarly, for $j \neq k$, the collections $\{Y_{tj}\}_{1 \leq t \leq n}$ and $\{Y_{tk}\}_{1 \leq t \leq n}$ are independent. Hence by (2), all the $Y_{tj}$ are NA (e.g., $\{Y_{tj}\}_{1 \leq t \leq n, 1 \leq j \leq m}$ is NA). Now for $1 \leq t \leq n$, let $X_t = f(Y_{t1}, \dots, Y_{tm})$, where $f$ is the OR function, e.g., $f(Y_{t1}, \dots, Y_{tm}) = 1$ iff there exists a $j$ for which $Y_{tj} = 1$. That is $X_t = 1$ iff you see a new species on day $t$. Clearly, $f$ is a non-decreasing function; hence the $X_t$ are NA. $\square$

Thanks in advance! :)

  • I think you can use Theorem 3.4 of this paper. Will that be too loose? – Clement C. Oct 13 '21 at 02:32
  • 1
    That theorem looks perfect, thanks so much! My condition should imply that paper's "1-correlation" and therefore concentration. When I get a chance I'll check this and add it to my question. – spencerjpeters Oct 13 '21 at 14:21
  • Darn. Unfortunately my condition does not imply x-correlation for any x > 0, and in fact it does not imply any concentration. Consider the uniform distribution on the two sequences 101...1, 010...0. This satisfies my condition, since 01 and 10 (and also all the postfixes 01x, 10y) are incomparable. But ofc, the sums are n - 1 and 1 respectively. As for x-correlation, $\Pr[X_3 = X_4 = ... = X_n = 1] = 1/2$ but $\Pr[X_3 = 1] * ... * \Pr[X_n = 1] = 1/2^{n - 2}$, violating $2^{n - 1}$-correlation. Next I'll check if my application satisfies the stronger condition in the paper Clement C. linked. – spencerjpeters Oct 13 '21 at 20:11

2 Answers2

1

Here is a counter-example that appears to work. Define the two infinite sequences A = 1011011011011011011011 ... and B = 0100100100100100100100 ... and put a probability mass of 1/2 on each. The resulting process satisfies the OP condition, but A averages out to 2/3 while B to 1/3, hence no concentration.

Aryeh
  • 440
  • 1
    Thanks, this is correct! In fact putting probability 1/2 each on 101^n, 010^n also works--the key point is that 10 and 01 (and all continuations 10x, 01y of these) are incomparable. – spencerjpeters Oct 13 '21 at 20:15
  • I don't think this works either... The fourth bit is positively correlated with the sum of the first three.

    More generally, I don't think you'll be able to have a distribution that is uniform over two strings as a counterexample...

    – Noah Stephens-Davidowitz Oct 14 '21 at 01:10
  • Yes, but that’s not the condition in the OP. For f to fail to be decreasing you need to find two prefixes where one dominates the other, right? – Aryeh Oct 14 '21 at 04:43
  • 1
0

UPDATE: this answer is wrong, see the correct (accepted) one. Leaving this up for pedagogical reasons only.

Since you allowed non-strictly decreasing functions, constant ones are allowed in particular. Thus, the process where $P(X_1=X_2=\ldots=X_n=0)=P(X_1=X_2=\ldots=X_n=1)=1/2$ is covered by this setting. In this case, you obviously don't get any non-trivial concentration. I'm afraid imposing strict decreasing won't make things any better: just infinitesimally perturb my example.

Aryeh
  • 440
  • I'm confused. I don't think this satisfies the condition. E.g., $\Pr[X_2=1 \ | \ X_1 = x_1]$ seems to be an increasing function of $x_1$, rather than a decreasing (or constant) function, since it equals $0$ if $x_1$ is $0$ and $1$ if $x_1$ is $1$. Intuitively, OP's condition is a form of anti-correlation while your example seems to be maximally correlated. – Noah Stephens-Davidowitz Oct 13 '21 at 12:41
  • Oops, I guess you're right. Will have to think about it some more. So you expect the Chernoff bound to just hold verbatim? – Aryeh Oct 13 '21 at 12:46
  • No worries! "So you expect the Chernoff bound to just hold verbatim?" Maybe? – Noah Stephens-Davidowitz Oct 13 '21 at 12:48
  • OK, here's what I'm currently thinking. Both NAND and NOR are decreasing functions on 2 bits. What if the process always generates either 00 or 11 as the initial 2 bits (w/prob 1/2). If it generated 00, then each subsequent bit is determined by the NOR of the last 2 bits, and the sequence will be A=001001001001... . If it generated 11, then each subsequent bit is determined by the NAND of the last 2 bits, and the sequence will be B=110110110... . In case of A, the average tends to 1/3, and for B, it tends to 2/3, so no concentration. Just need to check that I didn't mess up the monotonicity. – Aryeh Oct 13 '21 at 13:34
  • I think this still satisfies Noah's first observation--e.g., $\Pr[X_2 = 1 \mid X_1 = 1] = 1$, but $\Pr[X_2 = 1 \mid X_1 = 0] = 0$. In fact, I think the theorem Clement C. pointed to in his comment does show concentration under a (possibly strictly) weaker condition--when I get the chance I'll check this carefully and add it to my question. – spencerjpeters Oct 13 '21 at 14:18
  • Great, would love to see this worked out. I don't think this is equivalent to negative association in its full generality, but maybe you don't actually need that... – Aryeh Oct 13 '21 at 14:25
  • OK, last attempt before I throw in the towel. Take sequence B to be as above, but modify A to be 100100100... . That is, the first bit is always 1, the 2nd bit is a fair coin flip, sequences starting with 10 follow the NOR rule, while those starting with 11 follow NAND. – Aryeh Oct 13 '21 at 14:34
  • I think when conditioning on positive-probability prefixes, your condition is satisfied. When conditioning on 0-probability events, you can assign the conditional probabilities arbitrarily. – Aryeh Oct 13 '21 at 15:09
  • So with probability 1/2 the sequence is 100100100... and otherwise it is 110110110...? I like the idea, but this still doesn't work, because $$\Pr[X_5 = 1 \mid X_1 = 1, X_2 = 1, X_3 = 0, X_4 = 1] = 1

    \Pr[X_5 = 1 \mid X_1 = 1, X_2 = 0, X_3 = 0, X_4 = 1] = 0.$$

    – spencerjpeters Oct 13 '21 at 19:11