Monday Math 124

The central limit theorem tells us that for sufficiently large n, the mean of n samples of any random variable for which the first and second moments (and thus mean and variance) exist has a distribution that approaches, and can thus be approximated by, a normal distribution (Gaussian distribution).

Now, consider the case of the probability of getting exactly k heads in n (fair) coin flips. This is described by the binomial distribution with p=q=1/2
,
where
.
The mean is .

Now, suppose we have 2n flips (so that the mean is n), and we consider the probability of getting  heads (so that x is the difference from the mean). What is the Gaussian approximation for the probability distribution when n is large, and what are the limits on x for which the approximation is valid?

From our binomial distribution, we have

and
.
Now, recall Stirling’s approximation

for large n. Then


and

so


Now, to approximate the  and  terms, we use
.
Now, we expand to second order in the exponent by expanding the logarithm via the Mercator series:
,
and  gives us
.
Substituting these into our approximation of the binomial coefficient,
.
The exponential tells us that only x up to order  have a significant contribution, so we can use , obtaining
,
and so
.
Note that since , the above Gaussian distribution does integrate to unity, as expected of a proper distribution.
To test the validity of this approximation, let us expand the exponents in the  and  terms to fourth order (third order terms, like the first order terms, will cancel):
,
and
,
so we now get
.
and so
;
this correction term  differs significantly from unity only when x approaches the order of ; at that point, however, the probability is already negligibly small for large n ().