5.4.2 The exponential distribution

Waiting at a constant rate

Some events arrive at a steady rate $\lambda$ (lambda) per unit of time: calls to a help desk, or failures of a machine part. Each moment carries the same small chance of the event, whatever has happened before. The waiting time $X$ until the next event then has the exponential distribution, written $X \sim \text{Exp}(\lambda)$ .

Rule: Exponential distribution

\begin{gathered} f(x) = \lambda e^{-\lambda x}, \quad x \ge 0 \\[4pt] F(x) = 1 - e^{-\lambda x} \\[4pt] P(X > x) = e^{-\lambda x} \end{gathered}

$\lambda > 0$ is a rate, in events per unit of time; $x$ is a time, in the same unit. The density is 0 for $x < 0$ : a wait cannot be negative. The tail $P(X > x)$ , the chance of waiting longer than $x$ , falls by the factor $e^{-\lambda}$ each unit of time.

The total area is 1, the exponential tail of lesson 2.6.3: $\int_0^\infty \lambda e^{-\lambda x}\,dx = \lambda \cdot \frac{1}{\lambda} = 1$ . $F$ is the area from 0 to $x$ , $\big[-e^{-\lambda t}\big]_0^x = 1 - e^{-\lambda x}$ (lesson 2.5.3), and its slope is $\lambda e^{-\lambda x}$ again.

A machine part fails at a rate of 0.2 a year. Its life $X$ in years has $P(X > 5) = e^{-1} \approx 0.367879$ and $P(X \le 2) = 1 - e^{-0.4} \approx 0.32968$ .

A help desk gets 3 calls an hour. The wait for the next call has $\lambda = 3$ per hour, which is $0.05$ per minute. Waiting more than 30 minutes has probability $e^{-3 \times 0.5} = e^{-0.05 \times 30} \approx 0.22313$ : the same answer in either unit, as long as $\lambda$ and $x$ agree.

The density starts at its peak, $f(0) = \lambda$ , so short waits are the likeliest. With $\lambda = 3$ per hour that peak is 3, above 1: a density, not a probability (lesson 5.4.1).

Case study 2.5.5 used this density for the time until a borrower defaults, with $\lambda$ the hazard rate.

Mean and variance by parts

The mean wait is an improper integral (lesson 2.6.3) that needs integration by parts (lesson 2.6.2). Take $u = x$ and $dv = \lambda e^{-\lambda x}\,dx$ , so $du = dx$ and $v = -e^{-\lambda x}$ :

\begin{aligned} \mathbb{E}[X] &= \int_0^\infty x\,\lambda e^{-\lambda x}\,dx \\ &= \big[-x\,e^{-\lambda x}\big]_0^\infty + \int_0^\infty e^{-\lambda x}\,dx \\ &= 0 + \frac{1}{\lambda} \end{aligned}

The bracket is 0 at both ends: at 0 because of the factor $x$ , and at infinity because $e^{\lambda x}$ outgrows $x$ (lesson 2.1.3). Parts again gives $\mathbb{E}[X^2] = \frac{2}{\lambda}\mathbb{E}[X] = \frac{2}{\lambda^2}$ .

Rule: Mean and variance of the exponential

\begin{gathered} \mathbb{E}[X] = \frac{1}{\lambda} \\[4pt] \operatorname{Var}(X) = \frac{2}{\lambda^2} - \frac{1}{\lambda^2} = \frac{1}{\lambda^2} \end{gathered}

A rate of 0.2 failures a year means a mean life of 5 years. The standard deviation, $\frac{1}{\lambda}$ , equals the mean.

\mathbb{E}[X^2]

import math from scipy.integrate import quad lam = 0.2 def f(x): return lam * math.exp(-lam * x) print(round(quad(f, 0, math.inf)[0], 10)) print(round(quad(lambda x: x * f(x), 0, math.inf)[0], 10), round(1 / lam, 10)) print(round(quad(lambda x: x**2 * f(x), 0, math.inf)[0], 10), round(2 / lam**2, 10))

⌘/Ctrl + Enter runs the cell

Quantiles and the median

Solve $1 - e^{-\lambda x_p} = p$ with logs (lesson 1.4.4) to get the quantile.

Rule: Exponential quantiles

\begin{gathered} x_p = -\frac{\ln(1 - p)}{\lambda} \\[4pt] \text{median} = \frac{\ln 2}{\lambda} \approx \frac{0.693}{\lambda} \end{gathered}

The median is about 0.69 of the mean: the long right tail pulls the mean up. For a decaying atom the median is called the half-life.

For the machine part, the median life is $\frac{\ln 2}{0.2} \approx 3.465736$ years, shorter than the mean of 5. Only $e^{-1} \approx 0.367879$ of parts outlive the mean, whatever $\lambda$ is.

Raise $\lambda$ : the curve starts higher and falls faster, and the dashed mean $\frac{1}{\lambda}$ moves left. The interval starts as $[0, \frac{1}{\lambda}]$ ; it holds 0.6321 at every $\lambda$ , since $1 - e^{-1} \approx 0.632121$ .

Explore

The curve is a density: probabilities are areas under it. Change the parameters, then set an interval and read the shaded area.

The dashed line is the mean.

λ, rate = 1

Interval from a = 0

to b = 1

Probability: P(0 ≤ X ≤ 1) = 0.6321
Mean: 1
Variance: 1, so the standard deviation is 1

= \frac{1}{\lambda}

from scipy.stats import expon X = expon(scale=1 / 0.2) print(X.sf(5), X.cdf(2)) print(X.mean(), X.std(), round(X.median(), 6)) print(X.ppf([0.25, 0.5, 0.9])) print(expon(3).mean(), expon(scale=1 / 3).mean())

⌘/Ctrl + Enter runs the cell

Memorylessness

A part that has already run for 3 years fails at the same rate as a new one. So the chance it lasts 2 more years is the chance that a new part lasts 2. By conditional probability (lesson 5.1.3), the event "more than 5" is inside "more than 3":

\begin{aligned} P(X > 5 \mid X > 3) &= \frac{P(X > 5)}{P(X > 3)} \\ &= \frac{e^{-0.2 \times 5}}{e^{-0.2 \times 3}} = e^{-0.2 \times 2} \\ &\approx 0.67032 \end{aligned}

That is $P(X > 2)$ exactly. The 3 years already survived drop out.

Rule: Memorylessness

P(X > s + t \mid X > s) = P(X > t)

For $s, t \ge 0$ . Having waited $s$ tells you nothing about the further wait. The exponential is the only continuous distribution with this property; the geometric (lesson 5.2.2) is its discrete cousin.

The exponential density with $\lambda = 0.5$ , and its tail beyond 2 moved back to 0 and divided by its area, 0.368. The result is the original density: beyond one more unit lies 0.607 of it, as for a new start.

The rule only fits a constant rate. A part that wears out fails more often as it ages, so its life is not exponential. The formula would understate the risk for old parts.

Try it One seeded run draws a million lifetimes with mean 5 and keeps the parts still working at 3 years. Among those, the share that lasts beyond 5 is close to 0.67032, the share of new parts lasting beyond 2. Their mean remaining life is close to 5 years, the mean of a new part.

import numpy as np rng = np.random.default_rng(2) life = rng.exponential(scale=5, size=1_000_000) old = life[life > 3] print(len(old), round(np.mean(old > 5), 4), round(np.mean(life > 2), 4)) print(round(np.mean(old - 3), 3), round(life.mean(), 3))

⌘/Ctrl + Enter runs the cell

Exponential gaps make Poisson counts

Calls arrive at rate $\lambda$ , so the gaps between them are independent exponential waits. Count the calls in a window of length $t$ and you get a Poisson count with mean $\lambda t$ (lesson 5.2.3). The two descriptions fit together: no call in the window is the same event as a first wait longer than $t$ .

Rule: Exponential and Poisson

\begin{gathered} N_t \sim \text{Poisson}(\lambda t) \\[4pt] P(N_t = 0) = e^{-\lambda t} = P(X > t) \end{gathered}

$N_t$ is the number of events in a window of length $t$ , and $X$ the wait for the first. The rate links the two: the mean count is $\lambda t$ , and the mean gap is $\frac{1}{\lambda}$ .

For the help desk, 3 calls an hour, simulate 100,000 hours. Each hour adds exponential gaps of mean $\frac{1}{3}$ hour until it passes 1, and counts the calls.

calls in the hour	0	1	2	3	4	5	6
share of simulated hours	0.05	0.1502	0.2225	0.2254	0.1667	0.1003	0.0517
Poisson(3) pmf	0.0498	0.1494	0.224	0.224	0.168	0.1008	0.0504

The shares are from one seeded run and agree with the pmf to about two decimal places. The mean count is 2.99969, close to $\lambda t = 3$ .

10^{-10}

import numpy as np from scipy.stats import poisson rng = np.random.default_rng(3) gaps = rng.exponential(scale=1 / 3, size=(100_000, 20)) times = gaps.cumsum(axis=1) counts = (times <= 1).sum(axis=1) for k in range(7): print(k, round(float(np.mean(counts == k)), 4), round(poisson.pmf(k, 3), 4)) print(counts.mean())

⌘/Ctrl + Enter runs the cell

Lesson 5.4.3 turns to the normal distribution, the bell curve of lesson 2.6.3.

Common mistakes

SciPy and NumPy take the mean, not the rate. expon(scale=1/lam) and rng.exponential(scale=1/lam); expon(3) is a shifted density.
Match the units. A rate per hour needs times in hours: 30 minutes is $x = 0.5$ .
Half the waits are shorter than the median, $\frac{\ln 2}{\lambda}$ , not the mean. Only 0.3679 of them exceed the mean.
The past does not count. $P(X > 5 \mid X > 3)$ is $P(X > 2)$ , not $P(X > 5)$ .
The variance is $\frac{1}{\lambda^2}$ , not $\frac{1}{\lambda}$ ; the standard deviation equals the mean.
Only a constant rate gives an exponential. Wear-out, or a rate that changes through the day, breaks memorylessness.

The exponential distribution

0 of 12 correct

1.
A part fails at rate 0.2 a year. Find the probability it lasts more than 5 years (type e^(…) or a decimal)
2.
A part fails at rate 0.2 a year. Find the probability it fails within 2 years
3.
Customers arrive at 4 an hour. What is the mean wait for the next one, in minutes?
4.
Find the variance of an exponential wait with rate $\lambda = 0.5$
5.
Find $\mathbb{E}[X^2]$ for an exponential with rate $\lambda = 3$
6.
Find the median of an exponential with rate $\lambda = 0.5$ (type ln for the natural log)
7.
Find the 0.9-quantile of an exponential with rate $\lambda = 2$
8.
For any rate $\lambda$ , what is $P(X > \frac{1}{\lambda})$ , the chance of waiting longer than the mean?
9.
Calls come at 3 an hour. Find the probability of no call in the next 20 minutes
10.
A bulb's life is exponential with rate 0.25 a year. It has worked for 4 years. Find the probability it works 2 more
11.
Events arrive at 2 a day. What is the mean number of events in 3 days?
12.
In SciPy, expon(scale=5) is an exponential with which rate $\lambda$ ?

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

\lambda

The exponential distribution

Waiting at a constant rate

Rule: Exponential distribution

Mean and variance by parts

Rule: Mean and variance of the exponential

Quantiles and the median

Rule: Exponential quantiles

Memorylessness

Rule: Memorylessness

Exponential gaps make Poisson counts

Rule: Exponential and Poisson

Common mistakes

The exponential distribution

Summary card

Free further reading