5.3.1 Expectation and LOTUS

The expected value

A random variable $X$ attaches a number to each outcome of an experiment (lesson 5.2.1). Its probability mass function (pmf), $p_X(x) = P(X = x)$ , gives the chance of each value $x$ . The expected value of $X$ weights each value by its chance and adds up.

Rule: Expected value

\mathbb{E}[X] = \sum_x x\,p_X(x)

The sum runs over every value $X$ can take. $\mathbb{E}[X]$ is also called the mean of $X$ and written $\mu$ (mu). It has the units of $X$ : dollars, days, dots on a die.

For a fair die, each face from 1 to 6 has probability $\frac{1}{6}$ :

$x$	1	2	3	4	5	6
$p_X(x)$	1/6	1/6	1/6	1/6	1/6	1/6
$x\,p_X(x)$	1/6	2/6	3/6	4/6	5/6	6/6

The last row adds up to $\frac{21}{6} = 3.5$ . No roll ever shows 3.5: an expected value need not be a value $X$ can take.

A game with a negative expected value

Roll a die. You win $4 on a six and lose $1 on anything else. Your winnings $X$ are 4 with probability $\frac{1}{6}$ and $-1$ with probability $\frac{5}{6}$ , so

\mathbb{E}[X] = 4 \cdot \tfrac{1}{6} + (-1) \cdot \tfrac{5}{6} = -\tfrac{1}{6}

That is a loss of about $0.17 a game: the game loses money on average. A prize of $5 would make it fair, with $\mathbb{E}[X] = 0$ . A constant $c$ , which takes one value with probability 1, has $\mathbb{E}[c] = c$ .

Try itexpected is the rule: a sum of value times probability. Fraction keeps exact fractions, so the die prints 7/2.The last line averages the two possible winnings without their probabilities. That gives 1.5, which is wrong (common mistake 1).

from fractions import Fraction def expected(values, probs): return sum(x * p for x, p in zip(values, probs)) sixth = Fraction(1, 6) print(expected([1, 2, 3, 4, 5, 6], [sixth] * 6)) game = expected([4, -1], [sixth, 5 * sixth]) print(game, float(game)) print(expected([5, -1], [sixth, 5 * sixth])) print((4 + -1) / 2)

⌘/Ctrl + Enter runs the cell

The balance point and the long-run average

Picture the pmf as bars standing on a ruler, each as heavy as its probability. The expected value is the point where the ruler balances. A fair die balances in the middle, at 3.5.

Each pmf balances at its expected value. The die is symmetric, so it balances in the middle. The waiting time is most often 1, but its long tail pulls the balance point out to 3.

Now count the rolls until a 5 or a 6 first appears. That count is geometric with $p = \frac{1}{3}$ (lesson 5.2.2), and one roll is the most likely count. Yet the mean is 3: the balance point is not the tallest bar.

The long-run average

Roll the die $n$ times and average the results. Each face turns up in about $\frac{1}{6}$ of the rolls, so the average is close to $\sum_x x \cdot \frac{1}{6} = 3.5$ . As $n$ grows, the share of each face settles on its probability, and the average settles on $\mathbb{E}[X]$ .

That is what an expected value means in practice: the average result per try over a great many tries. Lesson 5.7.2 states it precisely as the law of large numbers, and case study 5.3.5 averages 1,000 daily returns this way.

n

import numpy as np rng = np.random.default_rng(531) rolls = rng.integers(1, 7, size=100_000) for n in [10, 100, 1000, 10_000, 100_000]: print(n, rolls[:n].mean()) winnings = np.where(rolls == 6, 4, -1) print("game:", winnings.mean())

⌘/Ctrl + Enter runs the cell

When the sum is infinite

The sum in the rule can fail to settle. Toss a coin until the first head. If that takes $k$ tosses, which has probability $\left(\frac{1}{2}\right)^k$ , you win $2^k$ dollars.

Each term of the sum is $2^k \cdot \left(\frac{1}{2}\right)^k = 1$ , so the expected value is $1 + 1 + 1 + \cdots$ . That series diverges (lesson 1.5.3), so the game has no finite expected value. If it paid nothing after 10 tails in a row, the expected value would be exactly $10: one dollar for each toss allowed.

A simulation shows the symptom: averages never settle. Rare long runs of tails pay huge amounts, so the more games you play, the larger the typical average.

n

import numpy as np rng = np.random.default_rng(5310) for n in [10, 100, 1000, 10_000]: tosses = rng.geometric(0.5, size=(100, n)) averages = (2.0 ** tosses).mean(axis=1) print(n, round(float(np.median(averages)), 2))

⌘/Ctrl + Enter runs the cell

The means of module 5.2's distributions

The same sum gives the mean of each distribution of module 5.2.

distribution	values	$\mathbb{E}[X]$
Bernoulli( $p$ )	0 or 1	$p$
$\operatorname{Bin}(n, p)$	0 to $n$	$np$
$\operatorname{Geom}(p)$ , trials to the first success	1, 2, 3, …	$\frac{1}{p}$
$\operatorname{Poisson}(\lambda)$	0, 1, 2, …	$\lambda$

A Bernoulli variable is 1 with probability $p$ and 0 otherwise, so $\mathbb{E}[X] = 0 \cdot (1 - p) + 1 \cdot p = p$ . The cell checks the other three, and lesson 5.3.2 shows why the binomial gives $np$ .

Each mean reads naturally. Ten trials with $p = 0.3$ give 3 successes on average, and waiting for a six takes 6 rolls on average. A Poisson count of rare events with rate $\lambda = 2.5$ averages 2.5.

Try it The binomial sum has 11 terms. The geometric and Poisson sums have infinitely many, so the cell adds enough terms for the rest to be tiny. For the geometric, the sums of the first 10, 20, 50 and 200 terms climb to 6. SciPy's .mean() gives each answer directly.

import numpy as np from scipy import stats k = np.arange(0, 11) print(round(np.sum(k * stats.binom.pmf(k, 10, 0.3)), 10), stats.binom(10, 0.3).mean()) for terms in [10, 20, 50, 200]: k = np.arange(1, terms + 1) print(terms, round(np.sum(k * stats.geom.pmf(k, 1 / 6)), 4)) print(stats.geom(1 / 6).mean()) k = np.arange(0, 60) print(round(np.sum(k * stats.poisson.pmf(k, 2.5)), 10), stats.poisson(2.5).mean())

⌘/Ctrl + Enter runs the cell

The expected value of a function: LOTUS

A function of a random variable is another random variable. If $X$ is a die roll, its square $Y = X^2$ is one, and so were the game's winnings. To find $\mathbb{E}[Y]$ you could work out the pmf of $Y$ first, but you do not need to.

Rule: LOTUS

\mathbb{E}[g(X)] = \sum_x g(x)\,p_X(x)

Weight each value $g(x)$ by the probability of $x$ . The name is short for the law of the unconscious statistician, because people use it without noticing that it is a rule.

Here is the same answer found both ways. Let $X$ take the values $-1$ , 0, 1 and 2, each with probability $\frac{1}{4}$ , and let $Y = X^2$ .

$x$	$-1$	0	1	2
$p_X(x)$	1/4	1/4	1/4	1/4
$g(x) = x^2$	1	0	1	4

With LOTUS: $\mathbb{E}[X^2] = \frac{1}{4}(1 + 0 + 1 + 4) = \frac{3}{2}$ .
Through the pmf of $Y$ : $Y$ is 0 with probability $\frac{1}{4}$ , 1 with probability $\frac{1}{2}$ (from $x = -1$ and $x = 1$ ), and 4 with probability $\frac{1}{4}$ . So $\mathbb{E}[Y] = 0 + \frac{1}{2} + 1 = \frac{3}{2}$ , the same.

The second way had to merge the two values of $x$ that give $y = 1$ . LOTUS never needs that step.

For a fair die, LOTUS gives $\mathbb{E}[X^2] = \frac{1 + 4 + 9 + 16 + 25 + 36}{6} = \frac{91}{6} \approx 15.17$ . With $g(x) = \frac{1}{x}$ it gives $\mathbb{E}\left[\frac{1}{X}\right] = \frac{49}{120} \approx 0.4083$ .

p_X

from fractions import Fraction import numpy as np def expect_g(g, values, probs): return sum(g(x) * p for x, p in zip(values, probs)) quarter = Fraction(1, 4) print(expect_g(lambda x: x**2, [-1, 0, 1, 2], [quarter] * 4)) print(0 * quarter + 1 * (2 * quarter) + 4 * quarter) faces, sixths = [1, 2, 3, 4, 5, 6], [Fraction(1, 6)] * 6 print(expect_g(lambda x: x**2, faces, sixths), expect_g(lambda x: Fraction(1, x), faces, sixths)) rng = np.random.default_rng(531) rolls = rng.integers(1, 7, size=100_000) print(round((rolls**2).mean(), 4), round((1 / rolls).mean(), 4))

⌘/Ctrl + Enter runs the cell

The mean of $g(X)$ is not $g$ of the mean

The mean of the squares is not the square of the mean. For the die, $\mathbb{E}[X^2] = \frac{91}{6} \approx 15.17$ , but $(\mathbb{E}[X])^2 = 3.5^2 = 12.25$ . They differ by $\frac{35}{12} \approx 2.92$ , a number lesson 5.3.2 calls the variance.

The same happens with $\frac{1}{X}$ : $\mathbb{E}\left[\frac{1}{X}\right] \approx 0.4083$ , but $\frac{1}{\mathbb{E}[X]} = \frac{2}{7} \approx 0.2857$ . Putting the mean into $g$ works only for straight lines, $g(x) = ax + b$ (lesson 5.3.2).

Case study 2.4.5 met this with $g = \ln$ . A fair flip of $20,000 on $100,000 has expected log wealth 11.4925, below $\ln 100{,}000 = 11.5129$ . Lesson 5.7.1 shows which way the gap goes for curves that bend up or down: Jensen's inequality.

\mathbb{E}[g(X)]

from fractions import Fraction def expect_g(g, values, probs): return sum(g(x) * p for x, p in zip(values, probs)) faces, sixths = [1, 2, 3, 4, 5, 6], [Fraction(1, 6)] * 6 mean = expect_g(lambda x: x, faces, sixths) for name, g in [("x^2", lambda x: x**2), ("1/x", lambda x: 1 / Fraction(x)), ("10x - 20", lambda x: 10 * x - 20)]: print(name, expect_g(g, faces, sixths), g(mean))

⌘/Ctrl + Enter runs the cell

Common mistakes

Weight by the probabilities. Averaging the game's two outcomes, 4 and $-1$ , gives 1.5; the expected value is $-\frac{1}{6}$ .
Check that the probabilities add up to 1 before trusting an expected value.
Do not round a mean to a possible value. A die's mean is 3.5, not 3 or 4.
The mean of a function is not the function of the mean. $\mathbb{E}[X^2]$ is larger than $(\mathbb{E}[X])^2$ unless $X$ never varies.
In LOTUS, weight by the probabilities of $X$ . If you build the pmf of $Y$ instead, add up the probabilities of every $x$ that gives the same $y$ .
A simulated average can mislead. When the sum for $\mathbb{E}[X]$ diverges, averages never settle. Check the sum, not only a simulation.

Expected values and LOTUS

0 of 12 correct

1.
$X$ is 0, 1 or 2 with probabilities 0.5, 0.3 and 0.2. Find $\mathbb{E}[X]$
2.
Find the expected value of a fair die roll, as a fraction
3.
A game pays 4 dollars on a six and costs 1 dollar on any other roll. Find the expected winnings per game, as a fraction
4.
The game pays $w$ dollars on a six and costs 1 dollar on any other roll. Which prize $w$ makes it fair?
5.
$X$ is binomial with $n = 20$ and $p = 0.15$ . Find $\mathbb{E}[X]$
6.
On average, how many rolls of a fair die does it take to get the first six?
7.
$X$ is $-1$ , 0, 1 or 2, each with probability $\frac{1}{4}$ . Find $\mathbb{E}[X^2]$
8.
For a fair die, find $\mathbb{E}[X^2]$ as a fraction
9.
$X$ is 1, 2 or 3 with probabilities $\frac{1}{4}$ , $\frac{1}{2}$ and $\frac{1}{4}$ . Find $\mathbb{E}[(X - 2)^2]$
10.
$X$ is 1, 2 or 4, each with probability $\frac{1}{3}$ . Find $\mathbb{E}[1/X]$ as a fraction
11.
For a fair die, how far is $\mathbb{E}[X^2]$ above $(\mathbb{E}[X])^2$ ? Give a fraction
12.
The doubling game pays $2^k$ dollars when the first head comes on toss $k$ , and nothing if the first 10 tosses are all tails. Find its expected value in dollars

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

\mathbb{E}[X] = \sum_x x\,p_X(x)

Expectation and LOTUS

The expected value

Rule: Expected value

A game with a negative expected value

The balance point and the long-run average

The long-run average

When the sum is infinite

The means of module 5.2's distributions

The expected value of a function: LOTUS

Rule: LOTUS

The mean of $g(X)$ is not $g$ of the mean

Common mistakes

Expected values and LOTUS

Summary card

Free further reading

The expected value

Rule: Expected value

A game with a negative expected value

The balance point and the long-run average

The long-run average

When the sum is infinite

The means of module 5.2's distributions

The expected value of a function: LOTUS

Rule: LOTUS

The mean of g(X)g(X)g(X) is not ggg of the mean

Common mistakes

Summary card

Free further reading

The mean of $g(X)$ is not $g$ of the mean