5.2.2 Bernoulli, binomial and geometric distributions

Bernoulli trials

A Bernoulli trial is an experiment with two outcomes, called success and failure. Success has probability $p$ , a number from 0 to 1, and failure $1 - p$ .

"Success" is only a label for the outcome being counted. A coin landing heads ( $p = \frac{1}{2}$ ), a die showing a six ( $p = \frac{1}{6}$ ) and a part failing its test are all Bernoulli trials.

Rule: Bernoulli distribution

\begin{gathered} p_X(1) = p, \quad p_X(0) = 1 - p \\[4pt] p_X(x) = p^x (1 - p)^{1 - x} \end{gathered}

$X$ is 1 for a success and 0 for a failure, so it counts successes in one trial. The second line packs both cases into one formula, for $x = 0$ or 1.

With $p = 0$ or $p = 1$ there is no chance involved: the trial always fails, or always succeeds.

The binomial distribution

Now repeat the trial $n$ times. The trials are independent and each has the same $p$ . Let $X$ be the number of successes; it can be any whole number from 0 to $n$ .

Take $n = 3$ and $p = 0.4$ . By independence (lesson 5.1.3), the probability of a sequence is the product of its trials' probabilities: $SSF$ has $0.4 \times 0.4 \times 0.6 = 0.096$ .

sequences	successes $k$	each sequence	$P(X = k)$
$FFF$	0	$0.6^3 = 0.216$	0.216
$SFF$ , $FSF$ , $FFS$	1	$0.4 \times 0.6^2 = 0.144$	$3 \times 0.144 = 0.432$
$SSF$ , $SFS$ , $FSS$	2	$0.4^2 \times 0.6 = 0.096$	$3 \times 0.096 = 0.288$
$SSS$	3	$0.4^3 = 0.064$	0.064

Every sequence with $k$ successes has the same probability, $p^k(1 - p)^{n - k}$ . The number of such sequences is the number of ways to choose which $k$ of the $n$ trials succeed: $\binom{n}{k}$ , from lesson 5.1.2. Here $\binom{3}{1} = \binom{3}{2} = 3$ .

Rule: Binomial distribution

\begin{gathered} P(X = k) = \binom{n}{k} p^k (1 - p)^{n - k} \\[4pt] k = 0, 1, \ldots, n \end{gathered}

The count of successes in $n$ independent trials, each with success probability $p$ . We write $X \sim \operatorname{Bin}(n, p)$ , read " $X$ has the binomial distribution with $n$ trials and probability $p$ ". A Bernoulli trial is the case $n = 1$ .

Ten trials with $p = 0.3$

For $n = 10$ and $p = 0.3$ , three successes have probability $\binom{10}{3} \times 0.3^3 \times 0.7^7 = 120 \times 0.027 \times 0.08235 = 0.2668$ . The whole pmf, to four decimals:

$k$	0	1	2	3	4	5	6	7	8
$P(X = k)$	0.0282	0.1211	0.2335	0.2668	0.2001	0.1029	0.0368	0.0090	0.0014

$k = 9$ has probability 0.0001, and $k = 10$ rounds to 0. The distribution function of lesson 5.2.1 adds up from the left: $P(X \le 2) = 0.0282 + 0.1211 + 0.2335 = 0.3828$ . At the other end, $P(X \ge 5) = 1 - F(4) = 0.1503$ .

n = 3

import math import numpy as np from scipy.stats import binom print([round(float(v), 4) for v in binom.pmf(range(4), 3, 0.4)]) n, p = 10, 0.3 print(round(math.comb(n, 3), 4), round(p**3, 4), round((1 - p)**7, 5)) print(round(math.comb(n, 3) * p**3 * (1 - p)**7, 4)) print([round(float(v), 4) for v in binom.pmf(range(n + 1), n, p)]) print(round(binom.cdf(2, n, p), 4), round(binom.sf(4, n, p), 4)) rng = np.random.default_rng(53) trials = rng.random((100_000, n)) < p X = trials.sum(axis=1) print(round((X == 3).mean(), 4), round((X <= 2).mean(), 4), round((X >= 5).mean(), 4))

⌘/Ctrl + Enter runs the cell

binom.sf(4, n, p) is $P(X > 4)$ , which is $P(X \ge 5)$ . To get $P(X \ge k)$ , ask for binom.sf(k - 1, n, p).

None, all and at least one

Three cases come up often and need no counting. No successes means every trial fails, and all successes means every trial succeeds. "At least one" is the complement of none (lesson 5.1.1).

Rule: None, all, at least one

\begin{gathered} P(X = 0) = (1 - p)^n \\[4pt] P(X = n) = p^n \\[4pt] P(X \ge 1) = 1 - (1 - p)^n \end{gathered}

For 10 trials with $p = 0.3$ , none has probability $0.7^{10} = 0.0282$ , and at least one $0.9718$ .

A tempting shortcut is wrong. In 10 rolls of a die, the chance of at least one six is $1 - \left(\frac{5}{6}\right)^{10} = 0.8385$ , not $10 \times \frac{1}{6}$ , which is more than 1.

The shape

With $p = \frac{1}{2}$ the pmf is symmetric: $P(X = k) = P(X = n - k)$ , because $\binom{n}{k} = \binom{n}{n - k}$ . For $n = 10$ , both $k = 3$ and $k = 7$ have probability 0.1172.

With $p$ below one half, the bars pile up on the left, with a long tail to the right. Above one half, the picture is the mirror image.

The failures are a binomial count too: if $X \sim \operatorname{Bin}(n, p)$ , then $n - X \sim \operatorname{Bin}(n, 1 - p)$ . So $P(X = 7)$ for $p = 0.8$ equals $P(X = 3)$ for $p = 0.2$ : both are 0.2013.

p = \frac{1}{2}

from scipy.stats import binom print(round(binom.pmf(3, 10, 0.5), 4), round(binom.pmf(7, 10, 0.5), 4)) print(round(binom.pmf(7, 10, 0.8), 4), round(binom.pmf(3, 10, 0.2), 4)) print(round(1 - (5 / 6)**10, 4), round(10 / 6, 4))

⌘/Ctrl + Enter runs the cell

Try it below. Set the interval from 0 to 2 and check $P(X \le 2) = 0.3828$ ; then move $p$ from 0.1 to 0.9 and watch the bars slide across. The dashed line marks the long-run average of $X$ , which lesson 5.3.1 defines; lesson 5.3.2 shows it is $np$ .

Explore

Each bar is the probability of one value. Change the parameters, then set an interval: the shaded bars add up to its probability.

The dashed line is the mean.

n, trials = 10

p, success probability = 0.3

Interval from a = 2

to b = 4

Probability: P(2 ≤ X ≤ 4) = 0.7004
Mean: 3
Variance: 2.1, so the standard deviation is 1.4491

When the binomial does not apply

The binomial needs four things: a fixed number of trials $n$ , two outcomes per trial, the same $p$ every time, and independent trials. Check each before using it.

Draw two cards from a shuffled deck of 52 without putting the first back, and count the aces. The chance of an ace changes after the first card, so the trials are not independent. By counting (lesson 5.1.2), $P(\text{two aces}) = \binom{4}{2} \div \binom{52}{2} = \frac{1}{221} = 0.004525$ .

The binomial with $n = 2$ and $p = \frac{4}{52} = \frac{1}{13}$ would give $\left(\frac{1}{13}\right)^2 = 0.005917$ : right if the first card goes back, wrong here. Counts from draws without replacement follow the hypergeometric distribution, named here only.

Try it Cards 0 to 3 are the aces. Sorting 52 random numbers shuffles the deck, one row per shuffle, and the first two columns are the cards drawn. rng.integers instead draws each card from the full deck, as if the first went back. In this seeded run the shares are 0.00453 and 0.00552, against the exact 0.004525 and 0.005917.

import math import numpy as np print(round(math.comb(4, 2) / math.comb(52, 2), 6), round((1 / 13)**2, 6)) rng = np.random.default_rng(55) hands = rng.random((100_000, 52)).argsort(axis=1)[:, :2] print(round((hands < 4).all(axis=1).mean(), 6)) back = rng.integers(0, 52, size=(100_000, 2)) print(round((back < 4).all(axis=1).mean(), 6))

⌘/Ctrl + Enter runs the cell

When the number of trials is not fixed in advance, the next section's distribution may be the one you need.

The geometric distribution

Repeat independent trials with success probability $p$ until the first success. Let $X$ be the number of trials this takes, the success included. $X$ can be $1, 2, 3, \ldots$ , with no upper limit.

$X = k$ means $k - 1$ failures and then a success. By independence, that has probability $(1 - p)^{k - 1}p$ .

Rule: Geometric distribution

\begin{gathered} P(X = k) = (1 - p)^{k - 1} p \\[4pt] k = 1, 2, 3, \ldots \end{gathered}

The number of trials up to and including the first success, for $0 < p \le 1$ . We write $X \sim \operatorname{Geom}(p)$ . Each extra trial multiplies the probability by $1 - p$ .

Roll a die until a six appears, so $p = \frac{1}{6}$ . The first six comes on roll 1 with probability $\frac{1}{6} = 0.1667$ , on roll 2 with $\frac{5}{6} \times \frac{1}{6} = \frac{5}{36} = 0.1389$ , and on roll 3 with $\frac{25}{216} = 0.1157$ .

The probabilities add up to 1. They form a geometric series with first term $p$ and ratio $1 - p$ , whose sum is $\frac{p}{1 - (1 - p)} = 1$ by lesson 1.5.3. With $p = 0$ the series is all zeros: a success never comes.

The tail

$X > k$ means the first $k$ trials all fail. So the tail has a short formula, and the distribution function follows from it.

Rule: Geometric tail

\begin{gathered} P(X > k) = (1 - p)^k \\[4pt] F_X(k) = 1 - (1 - p)^k \end{gathered}

For a whole number $k \ge 0$ . The second line is the complement rule, as in lesson 5.2.1.

With a die, $P(X > 6) = \left(\frac{5}{6}\right)^6 = 0.3349$ : about one time in three, six rolls bring no six. And $F(4) = 1 - \left(\frac{5}{6}\right)^4 = 0.5177$ , so a six arrives within four rolls about half the time: lesson 5.1.3's chance of at least one six in 4 rolls.

The number of rolls up to the first six. Each bar is $\frac{5}{6}$ of the one before. The highlighted tail, which goes on past 24, holds $\left(\frac{5}{6}\right)^6 = 0.335$ of the probability.

How many rolls make you 95% sure of a six? You need $\left(\frac{5}{6}\right)^k \le 0.05$ , so $k \ge \frac{\ln 0.05}{\ln(5/6)} = 16.43$ by lesson 1.4.4. Seventeen rolls are enough: $\left(\frac{5}{6}\right)^{17} = 0.0451$ , while $\left(\frac{5}{6}\right)^{16} = 0.0541$ is not yet below 0.05.

Trials or failures?

Some books and libraries count the failures before the first success instead, $Y = X - 1$ , with values $0, 1, 2, \ldots$ and $P(Y = j) = (1 - p)^j p$ . Check which one a formula uses. scipy.stats.geom and NumPy's rng.geometric both count trials, like this lesson.

P(Y = 0) = P(X = 1) = 0.25

import math import numpy as np from scipy.stats import geom p = 1 / 6 print(np.round(geom.pmf([1, 2, 3], p), 4)) print(round(geom.sf(6, p), 4), round((5 / 6)**6, 4), round(geom.cdf(4, p), 4)) print(round(math.log(0.05) / math.log(5 / 6), 2), round((5 / 6)**16, 4), round((5 / 6)**17, 4)) print(geom.pmf(0, 0.25, loc=-1), geom.pmf(1, 0.25)) rng = np.random.default_rng(54) X = rng.geometric(p, size=100_000) print(round((X > 6).mean(), 4), X.min())

⌘/Ctrl + Enter runs the cell

Lesson 5.2.3 lets the number of trials grow while successes become rare, which gives the Poisson distribution. Lesson 5.4.2 meets the continuous version of a waiting time, and case study 5.2.5 counts defaults in a loan portfolio with the binomial.

Common mistakes

Do not forget $\binom{n}{k}$ . Exactly two sixes in five rolls has probability $10 \times \left(\frac{1}{6}\right)^2\left(\frac{5}{6}\right)^3 = 0.1608$ , not 0.0161: there are ten orders.
"At least one" is $1 - (1 - p)^n$ , not $np$ . The product $np$ can exceed 1.
sf(k) is $P(X > k)$ . For $P(X \ge k)$ use sf(k - 1).
Check that the trials are independent with the same $p$ . Cards drawn without replacement are not binomial.
Trials or failures? A geometric formula may count trials (from 1) or failures before the first success (from 0).
The geometric tail has $k$ in the power. $P(X > k) = (1 - p)^k$ ; $(1 - p)^{k - 1}p$ is the single value $P(X = k)$ .

Bernoulli, binomial and geometric distributions

0 of 12 correct

1.
A Bernoulli trial has $p = 0.3$ . Find $P(X = 0)$
2.
How many sequences of 6 trials have exactly 2 successes?
3.
$X \sim \operatorname{Bin}(4, 0.5)$ . Find $P(X = 2)$
4.
Roll a die 5 times. Find the probability of exactly two sixes, as a fraction
5.
$X \sim \operatorname{Bin}(10, 0.3)$ . Find the probability of at least one success (type powers with ^)
6.
$X \sim \operatorname{Bin}(8, 0.5)$ and $P(X = 3) = \frac{7}{32}$ . Find $P(X = 5)$
7.
$X \sim \operatorname{Bin}(10, 0.8)$ counts successes. The failures $10 - X$ are binomial with 10 trials and what probability?
8.
binom.sf(4, 10, 0.3) gives $P(X \ge k)$ for which $k$ ?
9.
Roll a die until the first six. Find $P(X = 2)$
10.
$X \sim \operatorname{Geom}(0.2)$ . Find $P(X > 3)$
11.
$X \sim \operatorname{Geom}(0.5)$ . Find $P(X \le 3)$
12.
What is the smallest number of rolls that gives at least a 90% chance of a six?

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

p

Bernoulli, binomial and geometric distributions

Bernoulli trials

Rule: Bernoulli distribution

The binomial distribution

Rule: Binomial distribution

Ten trials with $p = 0.3$

None, all and at least one

Rule: None, all, at least one

The shape

When the binomial does not apply

The geometric distribution

Rule: Geometric distribution

The tail

Rule: Geometric tail

Trials or failures?

Common mistakes

Bernoulli, binomial and geometric distributions

Summary card

Free further reading

Bernoulli trials

Rule: Bernoulli distribution

The binomial distribution

Rule: Binomial distribution

Ten trials with p=0.3p = 0.3p=0.3

None, all and at least one

Rule: None, all, at least one

The shape

When the binomial does not apply

The geometric distribution

Rule: Geometric distribution

The tail

Rule: Geometric tail

Trials or failures?

Common mistakes

Summary card

Free further reading

Ten trials with $p = 0.3$