The normal distribution

The normal density

Lesson 2.6.3 scaled the bell curve $e^{-z^2/2}$ to area 1 and called it $\varphi$. Shift it to centre $\mu$ and stretch it sideways by $\sigma$, and you get the normal distribution with mean $\mu$ and standard deviation $\sigma$.

The peak is $\frac{1}{\sigma\sqrt{2\pi}}$: a wider bell is lower, so that its area stays 1. The curve bends from concave down to concave up (lesson 2.4.2) at $\mu - \sigma$ and $\mu + \sigma$, which is where to look for $\sigma$ on a graph.

Filling bottles. A machine fills bottles with $X \sim \mathcal{N}(500, 4^2)$ millilitres: mean 500 ml, standard deviation 4 ml, variance 16 ml². The peak of its density is $\frac{1}{4\sqrt{2\pi}} \approx 0.099736$ per ml.

The bell never reaches zero, so a normal variable can take any value. Values far from $\mu$ are possible but rare: the tails fall like $e^{-z^2/2}$, much faster than the exponential's $e^{-\lambda x}$ of lesson 5.4.2.

Standardising

Lesson 5.3.2 standardised any random variable to mean 0 and variance 1. For a normal variable the result is again normal: the standard normal. The z-score of a value counts how many standard deviations it lies from the mean.

$Z$ has mean 0, since $\varphi$ is symmetric about 0. Its variance is $\int z^2\varphi(z)\,dz$; by parts (lesson 2.6.2) with $u = z$ and $dv = z\,\varphi(z)\,dz$, so that $v = -\varphi(z)$, it is $\big[-z\varphi(z)\big]_{-\infty}^{\infty} + \int \varphi(z)\,dz = 0 + 1 = 1$.

Then lesson 5.3.2's rules for $X = \mu + \sigma Z$ give $\mathbb{E}[X] = \mu$ and $\operatorname{Var}(X) = \sigma^2$: the parameters are the mean and the variance.

For the bottles, a fill below 495 ml has z-score $\frac{495 - 500}{4} = -1.25$, so $P(X < 495) = N(-1.25) \approx 0.10565$. About one bottle in ten is under 495 ml.

Exam scores. Scores are $\mathcal{N}(70, 10^2)$. A score above 85 is 1.5 standard deviations up, so $P(X > 85) = 1 - N(1.5) \approx 0.066807$.

$\varphi$ and $N$ as probabilities

Lesson 2.6.3 computed areas under $\varphi$ and called the area up to $z$ $N(z)$. A density's areas are probabilities (lesson 5.4.1), so those areas now answer questions about $Z$. $N$ is the distribution function of $Z$.

• $N(1) \approx 0.841345$ is $P(Z \le 1)$.
• The band from $-1.96$ to 1.96 held 0.950004 of the area, so $P(-1.96 < Z < 1.96) \approx 0.95$.
• The band from $-1$ to 1 held 0.683, so $P(|Z| < 1) \approx 0.682689$.
• $N(0) = \frac{1}{2}$: half the time $Z$ is negative.

The 68–95–99.7 rule

Three numbers are worth knowing by heart. They hold for every normal, since standardising turns "within $k$ standard deviations of $\mu$" into $|Z| < k$.

For the bottles: 68.27% of fills lie between 496 and 504 ml, 95.45% between 492 and 508, and 99.73% between 488 and 512.

The bell is the same shape for every $\mu$ and $\sigma$. Move $\mu$ and $\sigma$, then set the interval to one standard deviation either side of the mean: it holds 0.6827 every time. Then try two standard deviations.

Quantiles

The $p$-quantile of $Z$ is $z_p = N^{-1}(p)$, norm.ppf(p) in SciPy. Like $N$ itself, it has no formula and is computed numerically.

By symmetry $z_{1-p} = -z_p$: the 0.01-quantile is $-2.326348$.

The lightest 1% of bottles hold less than $500 + 4 \times (-2.326348) \approx 490.69$ ml.

Setting the machine. To make only 1% of bottles fall under 495 ml, the mean must sit 2.326348 standard deviations above 495: $\mu = 495 + 4 \times 2.326348 \approx 504.31$ ml. This is the same equation, solved for $\mu$.

The top 10% of exam scores start at $70 + 10 \times 1.281552 \approx 82.82$.

Linear changes and sums

A linear change of a normal variable is normal again. Lesson 5.3.2's rules give its mean and variance.

A temperature in °F. If a temperature is $\mathcal{N}(20, 3^2)$ in °C, then $1.8C + 32$ is $\mathcal{N}(68, 5.4^2)$ in °F.

A pack of six bottles filled independently holds $\mathcal{N}(3000, 6 \times 16)$ ml, so its standard deviation is $\sqrt{96} \approx 9.798$ ml, not $6 \times 4 = 24$. A pack under 2,980 ml has z-score $-2.0412$ and probability 0.020613.

A difference behaves the same way: $X_1 - X_2$ is $\mathcal{N}(0, 32)$ for two bottles, with standard deviation $\sqrt{32} \approx 5.657$. The variances still add, because $\operatorname{Var}(-X_2) = \operatorname{Var}(X_2)$.

Shape, and where the normal does not fit

Lesson 5.3.3 measured shape with skewness and kurtosis. The normal is the benchmark: skewness 0, because it is symmetric, and kurtosis 3. SciPy reports the excess kurtosis, kurtosis minus 3, so the normal's is 0: norm.stats(moments="mvsk") gives mean 0, variance 1, skewness 0 and excess kurtosis 0.

A normal model can mislead in two ways.

• It allows every value. Model a lifetime as $\mathcal{N}(2, 1.5^2)$ years and $P(X < 0) = N(-1.3333) \approx 0.091$: a negative lifetime one time in eleven. The lognormal of lesson 5.4.4 stays positive.
• Its tails are thin. Beyond $4\sigma$ it puts only 0.0000633. Many data sets, daily returns among them, have more values out there. Lesson 5.4.4 meets the Student t, and case study 5.4.6 compares both with return data.

Lesson 5.4.4 builds three distributions from the normal: the lognormal, the Student t and the chi-squared.

Common mistakes

1. The second slot is the variance. $\mathcal{N}(500, 16)$ has $\sigma = 4$, and SciPy wants norm(500, 4).
2. Standardise before reading $N$. $P(X < 495)$ is $N(-1.25)$, not $N(495)$.
3. Use symmetry for the upper side. $P(Z > z) = 1 - N(z) = N(-z)$; for far tails use norm.sf.
4. Add variances, not standard deviations, for independent sums and for differences.
5. Sums of normals are normal when they are independent (or, lesson 5.5.3, jointly normal). Normality alone is not enough.
6. A normal model gives every value some probability, including impossible negative ones, and it understates extreme values in fat-tailed data.