5.2.1 Random variables, PMFs and CDFs

A number for each outcome

Lesson 5.1.1 described an experiment by its sample space $\Omega$ (omega), the set of all its outcomes. Often what matters is a number that each outcome produces: a count, a total, a score. A random variable is that number.

Toss a coin twice. The sample space is $\{HH, HT, TH, TT\}$ , four equally likely outcomes. Let $X$ be the number of heads.

outcome	$HH$	$HT$	$TH$	$TT$
$X$ , the number of heads	2	1	1	0
$Y$ , heads minus tails	2	0	0	$-2$

Rule: Random variable

X : \Omega \to \mathbb{R}

A rule that gives a real number to each outcome: a function (lesson 1.1.1) from $\Omega$ to the real numbers $\mathbb{R}$ . A random variable is written with a capital letter, $X$ ; a value it can take is written in lower case, $x$ .

The same experiment can carry many random variables. $Y$ above is heads minus tails, and it takes the values $-2$ , 0 and 2. A random variable has the units of what it counts or measures: heads, points on a die, dollars won.

Each statement about $X$ is an event, a set of outcomes. The event $\{X = 1\}$ is $\{HT, TH\}$ , and $\{X \le 1\}$ is $\{HT, TH, TT\}$ . So the probability rules of module 5.1 apply to them.

A random variable is discrete when its values can be listed: finitely many, or $0, 1, 2, \ldots$ without end. This module is about discrete random variables. Module 5.4 treats continuous ones, which can take any value in an interval.

X

from fractions import Fraction from itertools import product outcomes = ["".join(o) for o in product("HT", repeat=2)] for o in outcomes: print(o, o.count("H"), o.count("H") - o.count("T")) pmf = {} for o in outcomes: x = o.count("H") pmf[x] = pmf.get(x, 0) + Fraction(1, 4) print({x: str(p) for x, p in sorted(pmf.items())})

⌘/Ctrl + Enter runs the cell

The probability mass function

The probability mass function (pmf) of $X$ gives the probability of each value. To find $P(X = x)$ , add the probabilities of all the outcomes that produce $x$ .

For two tosses, $X = 1$ comes from two outcomes, $HT$ and $TH$ . So $P(X = 1) = \frac{2}{4} = \frac{1}{2}$ , while $P(X = 0) = P(X = 2) = \frac{1}{4}$ .

Rule: Probability mass function

\begin{gathered} p_X(x) = P(X = x) \\[4pt] p_X(x) \ge 0, \quad \sum_x p_X(x) = 1 \end{gathered}

The sum runs over every value of $X$ . The events $\{X = x\}$ for different values never overlap and together cover $\Omega$ , so their probabilities add up to 1 (the axioms of lesson 5.1.1). At any number that is not a value, $p_X(x) = 0$ .

A pmf can be given by a formula. $p(k) = \frac{k}{10}$ for $k = 1, 2, 3, 4$ is a pmf: the four values are positive and add up to $\frac{10}{10} = 1$ . The same formula for $k = 1, \ldots, 5$ is not one, because the five values add up to $\frac{15}{10}$ .

The larger of two dice

Roll two fair dice and let $M$ be the larger of the two numbers; on a double, it is that number. The 36 outcomes are equally likely. Each cell shows $M$ , with the first die down the side and the second across the top.

	1	2	3	4	5	6
1	1	2	3	4	5	6
2	2	2	3	4	5	6
3	3	3	3	4	5	6
4	4	4	4	4	5	6
5	5	5	5	5	5	6
6	6	6	6	6	6	6

Count the cells for each value. $M = 1$ in 1 cell, $M = 2$ in 3, and so on up to $M = 6$ in 11.

$m$	1	2	3	4	5	6
cells	1	3	5	7	9	11
$p_M(m)$	$\frac{1}{36}$	$\frac{3}{36}$	$\frac{5}{36}$	$\frac{7}{36}$	$\frac{9}{36}$	$\frac{11}{36}$

The counts are the odd numbers, so $p_M(m) = \frac{2m - 1}{36}$ . They add up to 36, so the pmf adds up to 1. The next section finds the same formula by algebra.

M

from fractions import Fraction from itertools import product import numpy as np count = {m: 0 for m in range(1, 7)} for a, b in product(range(1, 7), repeat=2): count[max(a, b)] += 1 for m in range(1, 7): print(m, count[m], Fraction(count[m], 36)) print(sum(count.values())) rng = np.random.default_rng(52) rolls = rng.integers(1, 7, size=(100_000, 2)).max(axis=1) print(np.round([(rolls == m).mean() for m in range(1, 7)], 4)) print(np.round([(2 * m - 1) / 36 for m in range(1, 7)], 4))

⌘/Ctrl + Enter runs the cell

The distribution function

The distribution function of $X$ , written $F_X$ , gives $F_X(x) = P(X \le x)$ : the probability that $X$ is at most $x$ . It is also called the cumulative distribution function, or CDF. It is defined at every real number $x$ , not only at the values of $X$ .

For $M$ , add the pmf from the left. $F_M(1) = \frac{1}{36}$ , $F_M(2) = \frac{1 + 3}{36} = \frac{4}{36}$ , and so on.

$m$	1	2	3	4	5	6
$F_M(m) \times 36$	1	4	9	16	25	36

The totals are squares, and algebra says why. $M \le m$ means both dice show at most $m$ . Each does so with probability $\frac{m}{6}$ , and the dice are independent, so $F_M(m) = \left(\frac{m}{6}\right)^2$ by lesson 5.1.3.

Rule: Distribution function

\begin{gathered} F_X(x) = P(X \le x) \\[4pt] = \sum_{k \le x} p_X(k) \end{gathered}

The sum runs over the values $k$ of $X$ up to $x$ . $F_X$ never decreases, is 0 to the left of every value and 1 to the right of them all. It is flat between values and jumps at each value $k$ by $p_X(k)$ .

Between two values nothing is added, so the graph of $F$ is a staircase. $F_M(2.5) = F_M(2) = \frac{4}{36}$ , because $M$ cannot be 2.5. Below the smallest value $F_M(x) = 0$ , so $F_M(0.5) = 0$ ; from 6 on, $F_M(x) = 1$ .

At each value the staircase takes the upper level. $F_M(2)$ counts $M = 2$ itself, because $\le$ includes the end. On the graph, a filled dot marks the level at the value and an open dot the level just before it.

The pmf of $M$ and its distribution function. The two highlighted bars, $\frac{5}{36} + \frac{7}{36}$ , are the rise of the staircase from $F(2) = \frac{4}{36}$ to $F(4) = \frac{16}{36}$ : both give $P(2 < M \le 4) = \frac{1}{3}$ .

From $F$ back to the pmf

The jump at each value is its probability. For a random variable whose values are whole numbers:

p_X(k) = F_X(k) - F_X(k - 1)

For $M$ , the algebra gives $p_M(m) = \frac{m^2 - (m - 1)^2}{36} = \frac{2m - 1}{36}$ . That is the formula the counting found.

Try it np.cumsum adds up the pmf from the left: the partial sums of lesson 1.5.2. Multiplied by 36, they are the squares of 1 to 6. np.diff takes the jumps back. With a 0 in front, it returns the pmf.

import numpy as np m = np.arange(1, 7) pmf = (2 * m - 1) / 36 F = np.cumsum(pmf) print(np.round(F * 36, 6)) print(np.round((m / 6)**2 * 36, 6)) print(np.round(np.diff(F, prepend=0) * 36, 6))

⌘/Ctrl + Enter runs the cell

Probabilities of intervals

$F(b)$ counts every value up to $b$ . Subtract $F(a)$ , everything up to $a$ , and what is left are the values above $a$ and up to $b$ .

Rule: Probability of an interval

\begin{gathered} P(a < X \le b) = F_X(b) - F_X(a) \\[4pt] P(X > a) = 1 - F_X(a) \end{gathered}

The left end is left out and the right end is kept. The second line is the complement rule of lesson 5.1.1: $X > a$ is "not $X \le a$ ".

For $M$ , $P(2 < M \le 4) = \frac{16}{36} - \frac{4}{36} = \frac{12}{36} = \frac{1}{3}$ . The bars at 3 and 4 give the same, $\frac{5}{36} + \frac{7}{36}$ . And $P(M > 4) = 1 - \frac{16}{36} = \frac{5}{9}$ , the chance that at least one die shows 5 or 6.

Which ends are included

For a discrete random variable, $<$ and $\le$ give different answers: the end can carry probability. When the values are whole numbers, move to the neighbouring whole number.

probability	from $F$	for $M$
$P(X \le b)$	$F(b)$	$P(M \le 4) = \frac{16}{36}$
$P(X < b)$	$F(b - 1)$	$P(M < 4) = \frac{9}{36}$
$P(X > a)$	$1 - F(a)$	$P(M > 4) = \frac{20}{36}$
$P(X \ge a)$	$1 - F(a - 1)$	$P(M \ge 4) = \frac{27}{36}$
$P(a \le X \le b)$	$F(b) - F(a - 1)$	$P(3 \le M \le 4) = \frac{12}{36}$

For a continuous random variable (module 5.4) a single value has probability 0, and the ends stop mattering.

p_X

import numpy as np from scipy.stats import rv_discrete m = np.arange(1, 7) M = rv_discrete(values=(m, (2 * m - 1) / 36)) print(round(M.cdf(4) - M.cdf(2), 4), round(M.sf(4), 4)) print(round(M.cdf(2.5), 4), M.cdf(0.5), M.cdf(9)) print(round(M.cdf(3) * 36, 6), round((1 - M.cdf(3)) * 36, 6)) rng = np.random.default_rng(52) rolls = rng.integers(1, 7, size=(100_000, 2)).max(axis=1) print(round(((rolls > 2) & (rolls <= 4)).mean(), 4))

⌘/Ctrl + Enter runs the cell

Lesson 5.2.2 meets the first named distributions, whose pmfs are formulas, and scipy.stats has them ready-made with the same .pmf, .cdf and .sf. Lesson 5.3.1 uses the pmf to define the average value of $X$ .

Common mistakes

$X$ is the rule; $x$ is a value. Write $P(X = 3)$ , the probability that the random variable equals 3, not $P(3)$ .
Count every outcome that gives the value. $M = 2$ comes from three outcomes, $(1, 2)$ , $(2, 1)$ and $(2, 2)$ , not two.
A pmf adds up to 1. If the values add up to anything else, a value is missing or a probability is wrong.
$F$ is defined between values too. It is flat there: $F_M(3.7) = F_M(3)$ .
Watch the ends. $P(X < 4) = F(3)$ , not $F(4)$ , and $F(b) - F(a)$ leaves $a$ out.
$P(X \ge a)$ is $1 - F(a - 1)$ . $1 - F(a)$ is $P(X > a)$ , which leaves out $X = a$ .

Random variables, PMFs and CDFs

0 of 12 correct

1.
Toss a coin twice and let $X$ be the number of heads. Find $P(X = 1)$
2.
Toss a coin three times and let $X$ be the number of heads. Find $P(X = 2)$
3.
For two tosses, $Y$ is the number of heads minus the number of tails. Find $P(Y = 0)$
4.
$p(k) = ck$ for $k = 1, 2, 3, 4, 5$ is a pmf. Find $c$
5.
$X$ takes the values 1, 2 and 3, with $p_X(1) = 0.2$ and $p_X(2) = 0.5$ . Find $p_X(3)$
6.
$M$ is the larger of two dice. Find $P(M = 4)$
7.
For the larger of two dice, find $F_M(3) = P(M \le 3)$
8.
For the larger of two dice, find $F_M(3.7)$
9.
For the larger of two dice, find $P(2 < M \le 5)$
10.
For the larger of two dice, find $P(M \ge 3)$
11.
$X$ takes the values 0, 1, 2 and 3, with $F(0) = 0.1$ , $F(1) = 0.4$ , $F(2) = 0.8$ and $F(3) = 1$ . Find $P(X = 2)$
12.
For the same $X$ , find $P(X < 2)$

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

X

Random variables, PMFs and CDFs

A number for each outcome

Rule: Random variable

The probability mass function

Rule: Probability mass function

The larger of two dice

The distribution function

Rule: Distribution function

From $F$ back to the pmf

Probabilities of intervals

Rule: Probability of an interval

Which ends are included

Common mistakes

Random variables, PMFs and CDFs

Summary card

Free further reading

A number for each outcome

Rule: Random variable

The probability mass function

Rule: Probability mass function

The larger of two dice

The distribution function

Rule: Distribution function

From FFF back to the pmf

Probabilities of intervals

Rule: Probability of an interval

Which ends are included

Common mistakes

Summary card

Free further reading

From $F$ back to the pmf