5.1.1 Sample spaces, events and the axioms of probability

Experiments and sample spaces

An experiment is anything with an uncertain result: rolling a die, tossing a coin, drawing a card. Each possible result is an outcome. The set of all outcomes is the sample space, written $\Omega$ (the Greek capital omega).

One die: $\Omega = \{1, 2, 3, 4, 5, 6\}$ , six outcomes.
A coin tossed twice: $\Omega = \{HH, HT, TH, TT\}$ , with H for heads and T for tails. $HT$ (heads, then tails) and $TH$ are different outcomes.
Two dice, one red and one blue: each outcome is a pair $(a, b)$ , red first. There are 36 pairs.

Each time the experiment runs, exactly one outcome happens. So the sample space must list every possible result, and no two of its outcomes can happen together.

Rule: Sample space

\Omega = \{\text{all possible outcomes}\}

Each run of the experiment produces exactly one element of $\Omega$ . Keep different results apart: with two dice, $(1, 2)$ and $(2, 1)$ are two outcomes.

\Omega

from itertools import product die = set(range(1, 7)) coins = set(product("HT", repeat=2)) dice = set(product(range(1, 7), repeat=2)) print(sorted(die)) print(sorted(coins)) print(len(dice), (1, 2) in dice, (2, 1) in dice)

⌘/Ctrl + Enter runs the cell

Events are sets of outcomes

An event is a set of outcomes: a subset of $\Omega$ , which means every element of it is in $\Omega$ . An event happens when the outcome of the run is one of its elements.

With one die, "the roll is even" is the event $E = \{2, 4, 6\}$ . A roll of 4 makes $E$ happen; a roll of 3 does not.

Two events are special. $\Omega$ always happens: it is the certain event. The empty set $\varnothing$ has no elements and never happens: the impossible event.

A single outcome is an event too, such as $\{6\}$ .

With two dice, set-builder notation (lesson 0.2.1) describes events by a rule:

\begin{gathered} A = \{(a, b) \in \Omega : a + b = 7\} \\ B = \{(a, b) \in \Omega : a = 6 \text{ or } b = 6\} \end{gathered}

$A$ , "the sum is 7", has 6 outcomes, from $(1, 6)$ to $(6, 1)$ . $B$ , "at least one six", has 11. Six have the red die on 6 and six the blue die, but $(6, 6)$ counts once.

The 36 outcomes of two dice, each labelled with its sum. Event $A$ , a sum of 7, is the outlined diagonal of 6 cells; event $B$ , at least one six, is the 11 shaded cells. They share 2 cells.

And, or, not

The words that combine events become operations on sets.

Rule: Union, intersection, complement

\begin{gathered} A \cup B:\ A \text{ or } B \text{ or both} \\ A \cap B:\ A \text{ and } B \\ A^c:\ \text{not } A \end{gathered}

The union $A \cup B$ holds every outcome in $A$ , in $B$ or in both. The intersection $A \cap B$ holds the outcomes in both. The complement $A^c$ holds every outcome of $\Omega$ that is not in $A$ .

With one die, take $E = \{2, 4, 6\}$ , "even", and $G = \{4, 5, 6\}$ , "at least 4".

$E \cup G = \{2, 4, 5, 6\}$ : even, or at least 4, or both.
$E \cap G = \{4, 6\}$ : even and at least 4.
$E^c = \{1, 3, 5\}$ : odd.

In probability, "or" always includes "both". Two events with no outcome in common, $A \cap B = \varnothing$ , are disjoint, also called mutually exclusive: they cannot happen together. "Even" and "odd" are disjoint, and so is any event with its complement; together the two make up $\Omega$ .

On the grid, $A \cap B = \{(1, 6), (6, 1)\}$ , and $A \cup B$ has $6 + 11 - 2 = 15$ outcomes: the overlap must not be counted twice.

\cup

from itertools import product Omega = set(range(1, 7)) E = {2, 4, 6} G = {4, 5, 6} print(E | G, E & G, Omega - E) print(E & {1, 3, 5}) dice = set(product(range(1, 7), repeat=2)) A = {w for w in dice if w[0] + w[1] == 7} B = {w for w in dice if 6 in w} print(len(A), len(B), sorted(A & B), len(A | B))

⌘/Ctrl + Enter runs the cell

The three axioms

A probability gives every event $A$ a number $P(A)$ , read "the probability of $A$ ". It runs from 0 (the event never happens) to 1 (it always does) and has no units. It is often written as a percentage: $P(A) = 0.25$ is 25%.

One way to read it: run the experiment many times, and $A$ happens in a share of the runs close to $P(A)$ . The last section tests this with a simulation.

Any way of giving events numbers counts as a probability if it obeys three rules, the axioms. They are taken as the starting point, and every other rule follows from them.

Rule: The axioms of probability

\begin{gathered} 1.\ P(A) \ge 0 \\ 2.\ P(\Omega) = 1 \\ 3.\ P(A \cup B) = P(A) + P(B) \\ \text{if } A \cap B = \varnothing \end{gathered}

No probability is negative, some outcome is certain to happen, and the probabilities of disjoint events add. The third axiom also holds for a list of disjoint events $A_1, A_2, \ldots$ that never ends, which lesson 5.2.3 needs.

The outcomes need not be equally likely. A loaded die might show 6 half the time and each other face with probability 0.1. These six numbers are not negative and add up to 1, so they obey the axioms.

Single outcomes are disjoint, so axiom 3 adds them up for any event. For the loaded die, $P(\text{even}) = P(\{2\}) + P(\{4\}) + P(\{6\}) = 0.1 + 0.1 + 0.5 = 0.7$ .

Two rules that follow

Not. $A$ and $A^c$ are disjoint and together make up $\Omega$ . So axioms 2 and 3 give $P(A) + P(A^c) = 1$ .

Rule: Complement rule

P(A^c) = 1 - P(A)

The probability that $A$ does not happen. With $A = \Omega$ it gives $P(\varnothing) = 0$ . Use it when "not $A$ " is easier to count, as for "at least one".

With two fair dice, each of the 36 outcomes has probability $\frac{1}{36}$ ; a later section says why. "At least one six" is the complement of "no six", and no six leaves $5 \times 5 = 25$ of the 36 outcomes. So $P(B) = 1 - \frac{25}{36} = \frac{11}{36}$ .

Or. When $A$ and $B$ overlap, $P(A) + P(B)$ counts the outcomes of $A \cap B$ twice. Subtract them once.

Rule: Addition rule

P(A \cup B) = P(A) + P(B) - P(A \cap B)

For any two events. When they are disjoint, $P(A \cap B) = 0$ and the rule is axiom 3.

On the grid, $P(A \cup B) = \frac{6}{36} + \frac{11}{36} - \frac{2}{36} = \frac{15}{36} = \frac{5}{12}$ . For the loaded die, $P(\text{even or at least } 4) = 0.7 + 0.7 - 0.6 = 0.8$ , since the overlap $\{4, 6\}$ has probability 0.6.

The axioms also fix the range. Since $P(A^c) \ge 0$ , no probability is above 1. And if every outcome of $A$ is in $B$ , then $P(A) \le P(B)$ .

The Proof Gym proves these rules from the axioms.

Try it The loaded die is a dictionary from faces to probabilities. The cell checks that none is negative and that they add up to 1, then adds single outcomes to get each event. round(\dots, 10) hides float noise such as 0.7999999999999999.

p = {1: 0.1, 2: 0.1, 3: 0.1, 4: 0.1, 5: 0.1, 6: 0.5} def prob(event): return round(sum(p[k] for k in event), 10) print(all(v >= 0 for v in p.values()), prob(p.keys())) E, G = {2, 4, 6}, {4, 5, 6} print(prob(E), prob(G), prob(E & G)) print(prob(E | G), round(prob(E) + prob(G) - prob(E & G), 10)) print(prob({1, 2, 3, 4, 5}), round(1 - prob({6}), 10))

⌘/Ctrl + Enter runs the cell

Equally likely outcomes: probability by counting

Often every outcome is equally likely: a fair die, a fair coin, a well-shuffled deck. With $n$ outcomes, each then has probability $\frac{1}{n}$ , because the probabilities must add up to 1. So a probability becomes a count.

Rule: Equally likely outcomes

P(A) = \frac{|A|}{|\Omega|}

$|A|$ is the number of outcomes in $A$ , and $|\Omega|$ the number in the sample space. This works only when every outcome is equally likely.

Two fair dice have 36 equally likely outcomes. Counting the cells of the grid for each sum gives a table:

sum	2	3	4	5	6	7	8	9	10	11	12
outcomes	1	2	3	4	5	6	5	4	3	2	1

So $P(\text{sum is } 7) = \frac{6}{36} = \frac{1}{6}$ , the most likely sum, and $P(\text{sum is } 2) = \frac{1}{36}$ . The algebra agrees: for each red value $a$ , exactly one blue value, $7 - a$ , makes 7, so there are 6 outcomes. A sum of 2 needs both dice on 1.

The sums themselves are not equally likely. Treating the 11 sums as equally likely outcomes would give each $\frac{1}{11}$ , which is wrong. The same trap hides in the number of heads in two tosses: one head comes from $HT$ or $TH$ , so $P(\text{exactly one head}) = \frac{2}{4} = \frac{1}{2}$ , not $\frac{1}{3}$ .

Lesson 5.1.2 counts outcomes in sample spaces too large to list.

Try it Counter counts the pairs that give each sum, which rebuilds the table. Fraction keeps the answers exact. The last line counts the tosses with exactly one head. True counts as 1 in a sum.

from collections import Counter from fractions import Fraction from itertools import product dice = list(product(range(1, 7), repeat=2)) counts = Counter(a + b for a, b in dice) print(sorted(counts.items())) print(Fraction(counts[7], len(dice)), Fraction(counts[2], len(dice))) coins = list(product("HT", repeat=2)) print(Fraction(sum(w.count("H") == 1 for w in coins), len(coins)))

⌘/Ctrl + Enter runs the cell

Simulation: probability as a long-run share

A computer can run an experiment many times, and the share of runs in which $A$ happens estimates $P(A)$ . NumPy's random numbers come from a generator, made by np.random.default_rng(seed). The same seed gives the same numbers every time, so anyone can repeat the run.

rng.integers(1, 7, size=(n, 2)) rolls two dice $n$ times: whole numbers from 1 to 6, stopping before 7 as range does. With the seed 1, the share of sums equal to 7 came out as:

rolls	100	10,000	1,000,000
share with sum 7	0.19	0.1652	0.1665

The exact answer is $\frac{1}{6} \approx 0.1667$ . After 100 rolls the share is off by more than 0.02; after a million it is within 0.0002. These numbers come from one seeded run: another seed gives slightly different shares.

The running share of sevens in one seeded run of a million rolls. It wanders at first, then settles on $\frac{1}{6}$ .

Lesson 5.7.2 explains why the share settles, and how fast. For now a simulation is a check: if a million runs land far from your exact answer, one of the two is wrong.

\frac{11}{36} \approx 0.3056

import numpy as np rng = np.random.default_rng(1) rolls = rng.integers(1, 7, size=(1_000_000, 2)) s = rolls.sum(axis=1) for n in [100, 10_000, 1_000_000]: print(n, round((s[:n] == 7).mean(), 4)) six = (rolls == 6).any(axis=1) print(round(six.mean(), 4), round(((s == 7) | six).mean(), 4)) loaded = rng.choice([1, 2, 3, 4, 5, 6], size=100_000, p=[0.1, 0.1, 0.1, 0.1, 0.1, 0.5]) print(round(np.isin(loaded, [2, 4, 6]).mean(), 4))

⌘/Ctrl + Enter runs the cell

Case study 5.1.6 uses these rules on a table of credit ratings, where each row of probabilities must add up to 1.

Common mistakes

"Or" includes "both". $P(A \cup B) = P(A) + P(B)$ only for disjoint events. Otherwise subtract $P(A \cap B)$ , or the overlap counts twice.
Counting needs equally likely outcomes. The 11 sums of two dice are not equally likely, and neither are 0, 1 and 2 heads. Count the equally likely pairs instead.
Order counts in the sample space. With two dice, $(1, 2)$ and $(2, 1)$ are different outcomes, so a sum of 3 has probability $\frac{2}{36}$ , not $\frac{1}{36}$ .
The complement of "at least one" is "none", not "exactly one". $P(\text{at least one six}) = 1 - P(\text{no six})$ .
A probability lies between 0 and 1. An answer outside that range means a mistake, often an overlap counted twice. Six rolls of a die do not give a six with probability $6 \times \frac{1}{6} = 1$ ; lesson 5.1.2 finds the true value.
A simulated share is an estimate. It changes with the seed and the number of runs, so quote it rounded and beside the exact answer.

Sample spaces, events and axioms

0 of 13 correct

1.
A coin is tossed three times. How many outcomes does the sample space have?
2.
One die is rolled. How many outcomes are in the event 'the roll is a multiple of 3'?
3.
One die: $E = \{2, 4, 6\}$ and $G = \{4, 5, 6\}$ . How many outcomes are in $E \cup G$ ?
4.
$P(A) = 0.35$ . Find $P(A^c)$
5.
$A$ and $B$ are disjoint, with $P(A) = 0.2$ and $P(B) = 0.45$ . Find $P(A \cup B)$
6.
$P(A) = 0.5$ , $P(B) = 0.4$ and $P(A \cap B) = 0.2$ . Find $P(A \cup B)$
7.
$P(A) = 0.6$ , $P(B) = 0.5$ and $P(A \cup B) = 0.8$ . Find $P(A \cap B)$
8.
A loaded die shows 6 with probability 0.5, and the other five faces are equally likely. Find the probability of a 1
9.
Two fair dice are rolled. Find $P(\text{sum is } 8)$
10.
Two fair dice are rolled. Find the probability that at least one shows a 1
11.
Two fair dice are rolled. Find the probability of a double (both the same) or a sum of 8
12.
A fair coin is tossed twice. Find the probability of exactly one head
13.
The share of sevens in many rolls of two dice settles near $\frac{1}{6}$ . About how many sevens should 600 rolls give?

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

\Omega

Sample spaces, events and the axioms of probability

Experiments and sample spaces

Rule: Sample space

Events are sets of outcomes

And, or, not

Rule: Union, intersection, complement

The three axioms

Rule: The axioms of probability

Two rules that follow

Rule: Complement rule

Rule: Addition rule

Equally likely outcomes: probability by counting

Rule: Equally likely outcomes

Simulation: probability as a long-run share

Common mistakes

Sample spaces, events and axioms

Summary card

Free further reading