## Monday Math 160

Suppose we have four identical-looking coins. Three are fair, but one is biased, with a probability of coming up heads of 3/5. We select one of the four coins at random.

1. If we flip the selected coin twice, and it comes up heads both times, what is the probability that our coin is the biased one?

2. If we flip the selected coin three times, and it comes up heads all three times, what, then, is the probability that our coin is the biased one?

3. Generalize: We have m fair coins and one identical-looking biased coin with probability p of getting heads. If we select one coin at random, and obtain k heads in n flips, what is the probablility P(m,p,n,k) that we have the biased coin?

1. Let us denote selecting a biased coin by B, selecting an unbiased coin by U, and getting two heads in two flips by H2. Then we are looking for P(B|H2), the probability that our coin is biased given that we got two heads in two flips. According to Bayes’ Theorem,
$\text{P}\left(\text{B}|\text{H}^2\right)=\frac{\text{P}\left(\text{H}^2|\text{B}\right)\cdot\text{P}\left(\text{B}\right)}{\text{P}\left(\text{H}^2\right)}$

and the law of total probability tells us that
$\text{P}\left(\text{H}^2\right)=\sum\limits_{n} \text{P}\left(\text{H}^2|\text{C}_n\right)\text{P}\left(\text{C}_n\right)=\text{P}\left(\text{H}^2|\text{B}\right)\text{P}\left(\text{B}\right)+\text{P}\left(\text{H}^2|\text{U}\right)\text{P}\left(\text{U}\right)$

Now, we know that $\text{P}\left(\text{B}\right)=\frac14$ and $\text{P}\left(\text{U}\right)=\frac34$. Now, the probability of getting two heads in a row with the biased coin is $\text{P}\left(\text{H}^2|\text{B}\right)=\left(\frac35\right)^2=\frac{9}{25}$, and the probability with an unbiased coin is $\text{P}\left(\text{H}^2|\text{U}\right)=\left(\frac12\right)^2=\frac14$.

Thus, we see that the total probability of getting two heads in a row is given by the law of total probability as:
$\text{P}\left(\text{H}^2\right)=\frac{9}{25}\cdot\frac{1}{4}+\frac{1}{4}\cdot\frac{3}{4}=\frac{111}{400}$

Thus, Bayes’ theorem tells us:
$\text{P}\left(\text{B}|\text{H}^2\right)=\frac{\frac{9}{25}\cdot\frac{1}{4}}{\frac{111}{400}}=\frac{12}{37}$

2. Now, with H3 being the outcome of getting three heads in three flips, we see that the probability of H3 with the biased coin is $\text{P}\left(\text{H}^3|\text{B}\right)=\left(\frac35\right)^3=\frac{27}{125}$, and the probability with an unbiased coin is $\text{P}\left(\text{H}^3|\text{U}\right)=\left(\frac12\right)^3=\frac18$.

Thus the total probability of getting three heads in a row is
$\text{P}\left(\text{H}^3\right)=\frac{27}{125}\cdot\frac{1}{4}+\frac{1}{8}\cdot\frac{3}{4}=\frac{591}{4000}$

and the probability that our coin is biased is
$\text{P}\left(\text{B}|\text{H}^3\right)=\frac{\frac{27}{125}\cdot\frac{1}{4}}{\frac{591}{4000}}=\frac{72}{197}$

3. Let H[n,k] denote the outcome where we have k heads in n flips. The probability of k successes in n successive Bernoulli trials, each with probability p, is given by the binomial distribution as:
$\text{P}\left(X=k\right)=\binom{n}{k}p^k(1-p)^{n-k}$
So, we see that the probability of H[n,k] given that our coin is biased is
$\text{P}\left(\text{H}[n,k]|\text{B}\right)=\binom{n}{k}p^k(1-p)^{n-k}$
and given an unbiased coin,
$\text{P}\left(\text{H}[n,k]|\text{U}\right)=\binom{n}{k}\left(\frac12\right)^k\left(1-\frac12\right)^{n-k}=\binom{n}{k}\frac{1}{2^n}$

Now, with $\text{P}\left(\text{B}\right)=\frac1{m+1}$ and $\text{P}\left(\text{U}\right)=\frac{m}{m+1}$, we see that the total probability of getting k heads in n flips is
$\begin{array}{rcl}\text{P}\left(\text{H}[n,k]\right)&=&\binom{n}{k}p^k(1-p)^{n-k}\frac1{m+1}+\binom{n}{k}\frac{1}{2^n}\frac{m}{m+1}\\&=&\frac{1}{m+1}\binom{n}{k}\left(p^k(1-p)^{n-k}+\frac{m}{2^n}\right)\end{array}$

Plugging these into Bayes’ theorem, we obtain:
$\begin{array}{rcl}\text{P}(m,p,n,k)&=&\text{P}\left(\text{B}|\text{H}[n,k]\right)\\&=&\frac{\text{P}\left(\text{H}[n,k]|\text{B}\right)\text{P}\left(\text{B}\right)}{\text{P}\left(\text{H}[n,k]\right)}\\&=&\frac{\binom{n}{k}p^k(1-p)^{n-k}\frac1{m+1}}{\frac{1}{m+1}\binom{n}{k}\left(p^k(1-p)^{n-k}+\frac{m}{2^n}\right)}\\&=&\frac{p^k(1-p)^{n-k}}{p^k(1-p)^{n-k}+m\frac{1}{2^n}}\\\text{P}(m,p,n,k)&=&\frac{1}{1+m\frac{1/2^n}{p^k(1-p)^{n-k}}}\end{array}$

(Or, to put it in a form with more visible parallelism between the probability p of the biased coin and the 1/2 probability of the fair coins,
$\text{P}(m,p,n,k)=\frac{1}{1+m\left(\frac{1/2}p\right)^k\left(\frac{1-1/2}{1-p}\right)^{n-k}}$)

Using our original m=3, p=3/5 example, we obtain

$\begin{array}{rcl}\text{P}(3,\frac35,n,k)&=&\frac{1}{1+3\frac{1/2^n}{\left(\frac35\right)^k\left(1-\frac35\right)^{n-k}}}\\&=&\frac{1}{1+3\left(\frac56\right)^k\left(\frac54\right)^{n-k}}\end{array}$

So, if we were to flip our coin 100 times, and we get 59 heads, then the probability that we have the biased coin is $\frac{1}{1+3\left(\frac56\right)^{59}\left(\frac54\right)^{41}}$≈0.625. If instead we get 55 heads, the probability drops to $\frac{1}{1+3\left(\frac56\right)^{55}\left(\frac54\right)^{45}}$≈0.247, and if we get 51 heads, then the probability that we have the biased coin is only $\frac{1}{1+3\left(\frac56\right)^{51}\left(\frac54\right)^{49}}$≈0.061.

Considering our getting all heads, as in (1) and (2), we have in the case k=n that $\text{P}(3,\frac35,n,n)=\frac{1}{1+3\left(\frac56\right)^n}$. We can, by solving for n, see that to obtain $\text{P}(3,\frac35,n,n)>\frac{1}{2}$, we must have n≥7, to get $\text{P}(3,\frac35,n,n)>0.75$ requires n≥13, and to get $\text{P}(3,\frac35,n,n)>0.9$ requires n≥19 flips all heads!

One final note: when we divided both the numerator and denomenator by $p^k(1-p)^{n-k}$, we were assuming that 0<p<1, so that this term is not zero. If the coin is fixed so as to always come up heads; that is to say, if p=1, then obviously if our coin ever comes up tails, we have a fair coin: P(m,1,n,k)=0 for any k<n. If, however, we have n flips all heads, then we can’t be sure we have the biased coin (rather than an unlikely “streak” on the biased coins):
$\text{P}(m,1,n,n)=\frac{1}{1+\frac{m}{2^n}}$
For example, if we have 99 fair coins, one “fixed” coin, and get ten flips all heads, we still have a probability of 1-P(99,1,10,10)≈8.8% chance of having a fair coin.