5.4.1 Densities, distribution functions and the uniform distribution

Probability as area

A continuous random variable can take any value in an interval: a waiting time, a weight, a measurement error. Lesson 5.2.1 gave each value of a discrete variable its own probability. With a continuum of values that no longer works, so probabilities belong to intervals instead.

A density $f_X$ says how probability is spread along the line. The probability that $X$ lands between $a$ and $b$ is the area under $f_X$ between them (lesson 2.5.1). With one variable in play, write $f$ for short.

Rule: Density

\begin{gathered} P(a < X < b) = \int_a^b f(x)\,dx \\[4pt] f(x) \ge 0, \quad \int_{-\infty}^{\infty} f(x)\,dx = 1 \end{gathered}

A density is never negative, and its total area is 1: $X$ lands somewhere. Its units are probability per unit of $x$ . If $X$ is in hours, $f$ is per hour.

Take $f(x) = \frac{x}{2}$ for $x$ from 0 to 2, and $f(x) = 0$ elsewhere. The region under it is a triangle of base 2 and height 1, so its area is 1. Values near 2 are the likeliest: there the density is tallest.

Split $[0, 2]$ into four pieces of width 0.5. An antiderivative of $\frac{x}{2}$ is $\frac{x^2}{4}$ (lesson 2.5.2), so each area is a difference of $\frac{x^2}{4}$ (lesson 2.5.3).

interval	0 to 0.5	0.5 to 1	1 to 1.5	1.5 to 2
probability	0.0625	0.1875	0.3125	0.4375
share of 100,000 draws	0.0622	0.1873	0.3125	0.438

The four probabilities add to 1. The last row comes from one seeded simulation in the last cell of this lesson, and agrees to about two decimal places.

A density is a rate, not a probability

Over a short interval of width $\Delta x$ , the area is close to a rectangle: $P(x < X < x + \Delta x) \approx f(x)\,\Delta x$ . For $x = 1$ and $\Delta x = 0.01$ that is $0.5 \times 0.01 = 0.005$ , against the exact 0.005025.

So a density can exceed 1. Spread all the probability evenly over $[0, 0.25]$ and the height must be 4, to make the area 1. What can never exceed 1 is an area.

\frac{x}{2}

from scipy.integrate import quad def f(x): return x / 2 if 0 <= x <= 2 else 0.0 print(round(quad(f, 0, 2)[0], 6), round(quad(f, 0.5, 1.5)[0], 6)) for a, b in [(0, 0.5), (0.5, 1), (1, 1.5), (1.5, 2)]: print(a, b, round(quad(f, a, b)[0], 6)) print(round(quad(f, 1, 1.01)[0], 6), f(1) * 0.01)

⌘/Ctrl + Enter runs the cell

One value has probability zero

The probability of one exact value is an integral with equal limits, $\int_a^a f(x)\,dx = 0$ (lesson 2.5.3). So for a continuous variable $P(X = a) = 0$ , and it does not matter whether an interval includes its ends: $P(a \le X \le b) = P(a < X < b)$ .

That is the opposite of lesson 5.2.1, where a die gives $P(X = 3) = \frac{1}{6}$ . A probability of 0 does not mean impossible: $X = 1$ can happen, but no single value carries any weight.

Making a curve into a density

Any curve with $g \ge 0$ and a finite area becomes a density once it is divided by that area. Take $g(x) = x^2$ on $[0, 3]$ : its area is $\left[\frac{x^3}{3}\right]_0^3 = 9$ , so $f(x) = \frac{x^2}{9}$ is a density. Dividing by the area is called normalising, and the 9 is the normalising constant.

x^2

from scipy.integrate import quad area = quad(lambda x: x**2, 0, 3)[0] print(round(area, 6), round(quad(lambda x: x**2 / area, 0, 3)[0], 6)) print(quad(lambda x: x / 2, 1, 1)[0])

⌘/Ctrl + Enter runs the cell

The distribution function

The distribution function $F_X(x) = P(X \le x)$ is the area under the density from $-\infty$ up to $x$ . It is the area function of lesson 2.5.3, started at $-\infty$ as an improper integral (lesson 2.6.3).

Rule: Distribution function

\begin{gathered} F(x) = \int_{-\infty}^{x} f(t)\,dt \\[4pt] F'(x) = f(x) \\[4pt] P(a < X \le b) = F(b) - F(a) \end{gathered}

$F$ rises from 0 to 1 and never falls. Its slope is the density, wherever $f$ is continuous (lesson 2.5.3). Any interval's probability is a difference of two values of $F$ .

For $f(x) = \frac{x}{2}$ on $[0, 2]$ , $F(x) = 0$ below 0, $F(x) = \frac{x^2}{4}$ from 0 to 2, and $F(x) = 1$ above 2. So $P(0.5 < X \le 1.5) = F(1.5) - F(0.5) = 0.5625 - 0.0625 = 0.5$ .

The same probability twice. It is the shaded area under the density from 0.5 to 1.5, and the rise of the distribution function between the same points.

The distribution function of a discrete variable is a staircase (lesson 5.2.1): it jumps at each value with positive probability. A continuous variable's $F$ has no jumps, which is another way of saying $P(X = a) = 0$ .

Expectation and variance as integrals

Lesson 5.3.1 defined $\mathbb{E}[X] = \sum x\,p(x)$ , each value weighted by its probability. For a density the sum becomes an integral: each $x$ is weighted by $f(x)\,dx$ , the probability of a thin strip around it.

Rule: Expectation and variance

\begin{gathered} \mathbb{E}[X] = \int_{-\infty}^{\infty} x\,f(x)\,dx \\[4pt] \mathbb{E}[g(X)] = \int_{-\infty}^{\infty} g(x)\,f(x)\,dx \\[4pt] \operatorname{Var}(X) = \mathbb{E}[X^2] - \mu^2 \end{gathered}

$\mu = \mathbb{E}[X]$ is the mean, in the units of $X$ ; the variance is in those units squared (lesson 5.3.2). The middle line is LOTUS, the law of the unconscious statistician of lesson 5.3.1, with an integral for the sum.

For $f(x) = \frac{x}{2}$ on $[0, 2]$ :

\begin{gathered} \mathbb{E}[X] = \int_0^2 \frac{x^2}{2}\,dx = \left[\frac{x^3}{6}\right]_0^2 = \frac{4}{3} \\[4pt] \mathbb{E}[X^2] = \int_0^2 \frac{x^3}{2}\,dx = \left[\frac{x^4}{8}\right]_0^2 = 2 \end{gathered}

So $\operatorname{Var}(X) = 2 - \frac{16}{9} = \frac{2}{9}$ , and the standard deviation is $\sqrt{2/9} \approx 0.471405$ . The mean $\frac{4}{3}$ sits right of the middle of $[0, 2]$ , where most of the area is.

As in lesson 5.3.1, $\mathbb{E}[g(X)]$ is not $g(\mathbb{E}[X])$ in general. Here $\mathbb{E}\left[\frac{1}{X}\right] = \int_0^2 \frac{1}{x}\cdot\frac{x}{2}\,dx = 1$ , but $\frac{1}{\mathbb{E}[X]} = 0.75$ .

A mean exists only if its integral converges (lesson 2.6.3). Lesson 5.4.4 meets a bell-shaped density whose tails are too heavy for that.

\mathbb{E}[X^2]

from scipy.integrate import quad def f(x): return x / 2 mean = quad(lambda x: x * f(x), 0, 2)[0] second = quad(lambda x: x**2 * f(x), 0, 2)[0] print(round(mean, 6), round(second, 6)) print(round(second - mean**2, 6), round((second - mean**2) ** 0.5, 6)) print(round(quad(lambda x: (1 / x) * f(x), 0, 2)[0], 6), round(1 / mean, 6))

⌘/Ctrl + Enter runs the cell

The uniform distribution

The simplest density is flat. $X$ is uniform on $[a, b]$ , written $X \sim U(a, b)$ , when every stretch of the same length is equally likely. The symbol $\sim$ reads "has the distribution".

Rule: Uniform distribution

\begin{gathered} f(x) = \frac{1}{b - a} \text{ on } [a, b] \\[4pt] \mathbb{E}[X] = \frac{a + b}{2} \\[4pt] \operatorname{Var}(X) = \frac{(b - a)^2}{12} \end{gathered}

The height is 1 over the width, so the area is 1. A probability is the share of the length: $F(x) = \frac{x - a}{b - a}$ on $[a, b]$ . The mean is the midpoint.

A bus every 12 minutes. You arrive at a random moment, so your wait $W$ is uniform on $[0, 12]$ , in minutes. Then $P(W > 9) = \frac{3}{12} = 0.25$ , the mean wait is 6 minutes, and the variance is $\frac{144}{12} = 12$ minutes squared. The standard deviation is $\sqrt{12} \approx 3.4641$ minutes.

A rounding error. A reading rounded to the nearest 0.1 is off by an error uniform on $[-0.05, 0.05]$ . The ends can be negative: the mean is 0 and the standard deviation is $\frac{0.1}{\sqrt{12}} \approx 0.028868$ .

The variance follows from the rule above. $\mathbb{E}[X^2] = \int_a^b \frac{x^2}{b - a}\,dx = \frac{a^2 + ab + b^2}{3}$ , and subtracting $\left(\frac{a + b}{2}\right)^2$ leaves $\frac{(b - a)^2}{12}$ .

Set the width to 4 and the height drops to 0.25. Then slide the interval: its probability is always its length divided by 4, wherever it sits inside $[a, b]$ .

Explore

The curve is a density: probabilities are areas under it. Change the parameters, then set an interval and read the shaded area.

The dashed line is the mean.

a, lower end = 0

width b − a = 1

Interval from a = 0.2

to b = 0.5

Probability: P(0.2 ≤ X ≤ 0.5) = 0.3
Mean: 0.5
Variance: 0.0833, so the standard deviation is 0.2887

a

from scipy.stats import uniform W = uniform(loc=0, scale=12) print(W.sf(9), W.mean(), W.var(), round(W.std(), 4)) E = uniform(loc=-0.05, scale=0.1) print(round(E.mean(), 6), round(E.std(), 6)) print(uniform(loc=2, scale=6).mean(), uniform(loc=2, scale=4).mean())

⌘/Ctrl + Enter runs the cell

Quantiles

A quantile answers the reverse question: below which value does $X$ fall with probability $p$ ? The $p$ -quantile $x_p$ solves $F(x_p) = p$ . The 0.5-quantile is the median; the 0.25- and 0.75-quantiles are the quartiles.

Rule: Quantile

F(x_p) = p, \quad 0 < p < 1

$x_p$ is in the units of $X$ . For $U(a, b)$ , $x_p = a + p(b - a)$ . SciPy calls the quantile function ppf, the percent point function.

For the bus, the 0.9-quantile is $0 + 0.9 \times 12 = 10.8$ : nine waits in ten are shorter than 10.8 minutes.

For $f(x) = \frac{x}{2}$ , solve $\frac{x_p^2}{4} = p$ to get $x_p = 2\sqrt{p}$ . The median is $\sqrt{2} \approx 1.414214$ and the quartiles are 1 and $\sqrt{3} \approx 1.732051$ . The median is above the mean, $\frac{4}{3}$ : the thin side of this density is on the left, and it pulls the mean down.

Simulating with a quantile

The quantile function also makes random draws. Draw $U$ uniform on $[0, 1]$ ; then $x_U$ has the distribution $F$ . Each $p$ is equally likely, and $x_p$ sends it to the matching place under $f$ .

This is the inverse transform method; lesson 5.6.1 shows why it works. For our density, $2\sqrt{U}$ is a draw of $X$ .

U

import numpy as np rng = np.random.default_rng(1) U = rng.uniform(0, 1, 100_000) X = 2 * np.sqrt(U) print(round(np.mean(X < 1), 4), round(X.mean(), 4), round(X.var(), 4)) print([round(float(np.mean((X > a) & (X < b))), 4) for a, b in [(0, 0.5), (0.5, 1), (1, 1.5), (1.5, 2)]])

⌘/Ctrl + Enter runs the cell

Lesson 5.4.2 meets a density over the whole half-line $x \ge 0$ , the exponential. Case study 5.4.6 uses quantiles to compare return data with a model.

Common mistakes

A density is not a probability. It can exceed 1; probabilities are areas under it.
One value has probability 0 for a continuous variable, so $\le$ and $<$ give the same answer. For a discrete one they do not.
Subtract values of $F$ . $P(a < X \le b) = F(b) - F(a)$ , not $F(b - a)$ .
Check that the area is 1. If it is not, divide by the normalising constant.
SciPy's uniform takes a width. uniform(loc=a, scale=b - a).
$\mathbb{E}[X^2]$ is not $(\mathbb{E}[X])^2$ . Their difference is the variance.
The median and the mean differ unless the density is symmetric.

Densities, distribution functions and the uniform distribution

0 of 12 correct

1.
Find the constant $c$ that makes $f(x) = cx$ on $[0, 4]$ a density
2.
$X$ has density $\frac{x}{2}$ on $[0, 2]$ . Find $P(X \le 1)$
3.
$X$ has density $\frac{x}{2}$ on $[0, 2]$ . Find $P(1 < X < 2)$
4.
$X$ is continuous with density $\frac{x}{2}$ on $[0, 2]$ . Find $P(X = 1)$
5.
$X$ is uniform on $[0, 0.25]$ . What is the height of its density?
6.
$X$ has distribution function $F(x) = \frac{x^3}{8}$ on $[0, 2]$ . Find its density at $x = 1$
7.
$X$ has distribution function $F(x) = \frac{x^3}{8}$ on $[0, 2]$ . Find $P(1 < X \le 2)$
8.
$X$ has density $3x^2$ on $[0, 1]$ . Find $\mathbb{E}[X]$
9.
$X$ has density $\frac{x}{2}$ on $[0, 2]$ , with $\mathbb{E}[X] = \frac{4}{3}$ and $\mathbb{E}[X^2] = 2$ . Find $\operatorname{Var}(X)$
10.
A bus comes every 12 minutes and your wait is uniform on $[0, 12]$ . Find the variance of the wait, in minutes squared
11.
Find the 0.9-quantile of the uniform distribution on $[2, 6]$
12.
$X$ has distribution function $F(x) = \frac{x^3}{8}$ on $[0, 2]$ . Find its median (type 2^(2/3) or a decimal)

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

f \ge 0

Densities, distribution functions and the uniform distribution

Probability as area

Rule: Density

A density is a rate, not a probability

One value has probability zero

Making a curve into a density

The distribution function

Rule: Distribution function

Expectation and variance as integrals

Rule: Expectation and variance

The uniform distribution

Rule: Uniform distribution

Quantiles

Rule: Quantile

Simulating with a quantile

Common mistakes

Densities, distribution functions and the uniform distribution

Summary card

Free further reading