Order in a Biased Coin

Question

Situation: Consider the classic coin tossing experiment. We want to explore if the coin is biased. Coin 1: Coin is tossed $50$ times. We get $20$T and then $30$H, in that sequence Coin 2: Coin is tossed $50$ times. We get $4$T $6$H, $4$T $6$H, $4$T $6$H, $4$T $6$H, $4$T $6$H in that sequence.

Q1 : Is the probability of Coin 1 and Coin 2 being biased is the same? My gut feel is that yes, because each event is independent so the order of the events doesn't matter at all.

Q2: I have a coin toss where observations are not independent. P(H | Previous toss is tail) = $0.6$ and P(T | Previous Head) = $0.5$ What kind of statistical test can I use to check the probability of coin being biased?

For 1, just compute the probability of each of the specific outcomes. For 2, what is your definition of biased considering the correlation? You have the specific probabilities. The coin does not appear biased to me because of the symmetry. In the long run, you should still get half heads, half tails. Long runs of heads and tails should be more common than with a coin where the tosses are independent. — Ross Millikan, Jul 18 '18 at 00:30
Now you can just flip a lot of times. Compute the long term average fraction of heads and tails. It will not be $0.5$ each. — Ross Millikan, Jul 18 '18 at 00:53
Ross - the question is that no matter how many flips I do, the probability of coin being biased is not going to be 100%, how do ascertain that probability? Could I simply flips coins and do a t-test and estimate that? — Penrose, Jul 18 '18 at 01:53
Your null hypothesis is that the long term average is half heads. When the offset from half gets large enough you declare the coin biased. I think that is the t-test, but I don't remember. You could also look at TT vs TH, which is more biased. — Ross Millikan, Jul 18 '18 at 02:02
I don't see how your edits have clarified Question $2$. You're specifying all the probabilities, so it's unclear (at least to me) what you want to test and what you would consider a bias (or lack thereof) of such a fully specified coin. — joriki, Jul 18 '18 at 08:40

score 2 · Accepted Answer · answered Jul 18 '18 at 04:12

Your goal here is to test the marginal probability of a head in your coins. However, you need to be careful with your assumptions. You say in your question that the coin tosses are independent, but the data for the coins clearly falsifies this. The standard coin-toss model with independent outcomes is based on an assumption of exchangeability of the outcomes, which can be tested via a permutation test (e.g., a runs test). For a binary process with twenty observed tails and thirty observed heads, the distribution of the number of runs is shown in the plot below (R code for this plot below).

In your data, Coin 1 has two runs and Coin 2 has ten runs. Two runs is so far in the tails that we do not get a single random generation of this in $10^6$ simulations, yielding a simulated p-value of zero. Ten runs is so far in the tails that we get a value as or more extreme than this only seven times in $10^6$ simulations, yielding a simulated p-value close to zero. In short, for both coins ( but especially the first), there is extremely strong evidence that exchangeability does not hold, so the tosses are not independent (even when we condition on the marginal probability of a head).

Given that your coin tosses show evidence of non-exchangeability, you need to base your analysis on some kind of more general model (e.g., a binary auto-regression (BAR) model). Now, you might still find that the evidence for the marginal probability of heads is the same in both cases, under a more general model. However, you cannot base this on an assumption of independence of tosses, which is clearly falsified by the observed data.

R code for this plot:

#Define a function to calculate the runs for an input vector
RUNS <- function(x) { n <- length(x);
                      R <- 1;
                      for (i in 2:n) { if(x[i] != x[i-1]) { R <- R+1; } }
                      R }

#Simulate the runs statistic for k permutations
k <- 10^6;
set.seed(12345);
RR <- rep(0, k);
for (i in 1:k) { x_perm <- sample(x, length(x), replace = FALSE);
                 RR[i] <- RUNS(x_perm); }

#Generate the frequency table for the simulated runs
FREQS <- as.data.frame(table(RR));

#Plot estimated distribution of runs
library(ggplot2);
ggplot(data = FREQS, aes(x = RR, y = Freq/k, colour = 'Red', fill = 'Red')) +
       geom_bar(stat = 'identity') +
       theme(legend.position = 'none') +
       labs(title ='Plot of Distribution of Runs',
       subtitle = '(Simulation using 1,000,000 generated values)',
       x = 'Runs', y = 'Estimated Probability');

Ben, Thanks for this answer. Can I clarify what do you mean by Coin 1 has 2 runs and Coin 2 has 10 runs? Both have 50 tosses — Penrose, Jul 18 '18 at 04:27
For a binary sequence, the number of "runs" is defined as the number of series of the same value. So for the first coin you have an initial "run" of twenty tails, and then a second "run" of thirty heads. For the second coin, you have ten "runs" consisting of sequences of four tails, then six heads, etc. — Ben, Jul 18 '18 at 04:31

score 2 · Answer 2 · answered Jul 18 '18 at 08:49

A detailed answer has already been given under the assumption that the independence of the coin tosses is in question. I interpreted the question to mean that the independence assumption stands and you merely want to consider the possibility of the coin being biased away from the uniform distribution for $p=\frac12$.

In this case, the answer to Question $1$ is affirmative. In this experiment, the number of heads is a sufficient statistic for the parameter $p$ of interest, the probability of the coin to yield heads. That is, it contains all the information about this parameter that the entire run of data contains. Formally, the likelihood of the parameter conditional on the statistic is the same as the likelihood of the parameter conditional on the entire data. Thus, the order in which the results occur makes no difference in your posterior beliefs about the parameter.

Thanks for the answer. This would also mean that the order will be important if the observations were not to be independent. What kind of statistic should one use in such case and what kind of test can be used to check for statistical significance? — Penrose, Jul 18 '18 at 11:38
@Penrose: I don't know much about that sort of thing -- but I get the impression that Ben (the author of the other answer) does. — joriki, Jul 18 '18 at 12:08

score 1 · Answer 3 · 2018-07-18T03:05:07.863

Suppose I have n independent Bernoulli trials which can be simulated with the Binomial distribution. That is I have a random variable $X \sim Bin(n,p)$. The pmf for this is given by $f(k,n,p) = \binom{n}{k} (p)^{k}(1-p)^{n-k} $

Now...if I want to simulate say 50 coins flips..let me do that..in say Python like the following...

import numpy as np import random

random.seed(42)

n_flips = 25
p =.50
coin_flips = np.random.choice([0,1],n_flips,[(1-p),p])

what's this look like

Out[6]: 
array([0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1,
       1, 1, 1])

lucky for us we know something about the mean of the binomial distribution.

$$ E(X) = np$$

So if I have a hypothesis that this is a fair coin. How can we use that to test it? If the coin is biased then we would be able to observe that the mean is different on average. So if you have 2 coins then if you subtract the two means then you would observe that the difference would be greater than you would expect. Then you compare this with a p value. It's generally called an A/B test

This is valid for independent observation. Would it hold when observations are not independent? — Penrose, Jul 18 '18 at 03:31
@Penrose this above-assumed independence. Bayes rules always gave me trouble. But it would be Bayes rule to do some kind of conditional test. I'd have to go through my book. Someone smarter than me could explain that one. — , Jul 18 '18 at 03:38

Order in a Biased Coin

3 Answers3