5.3.3 Skewness and kurtosis

Moments

The mean and the variance describe where a distribution sits and how widely it spreads. Two more numbers describe its shape. All of them are moments.

The $k$ -th moment of $X$ is $\mathbb{E}[X^k]$ , found by LOTUS (lesson 5.3.1) with $g(x) = x^k$ . The first moment is the mean $\mu$ . The $k$ -th central moment is $\mathbb{E}[(X - \mu)^k]$ , and the second is the variance.

Central moments carry units: $\mathbb{E}[(X - \mu)^3]$ is in units cubed, so it grows $100^3 = 10^6$ times when dollars become cents. Standardising first removes the units.

Rule: Standardised moments

\begin{gathered} \mathbb{E}[Z^k] \\[4pt] Z = \frac{X - \mu}{\sigma} \end{gathered}

$Z$ counts standard deviations $\sigma$ from the mean (lesson 5.3.2). Its moments have no units, and they do not change when $X$ is shifted or multiplied by a positive number.

The first two are always the same: $\mathbb{E}[Z] = 0$ and $\mathbb{E}[Z^2] = 1$ . So the new information starts with the third and the fourth.

z^k

import numpy as np def std_moment(values, probs, k): values, probs = np.asarray(values, float), np.asarray(probs, float) mu = values @ probs sigma = np.sqrt(((values - mu) ** 2) @ probs) return ((values - mu) / sigma) ** k @ probs faces, sixths = np.arange(1, 7), np.full(6, 1 / 6) print([round(float(std_moment(faces, sixths, k)), 4) + 0.0 for k in range(1, 5)]) print([round(float(std_moment(10 * faces + 3, sixths, k)), 4) + 0.0 for k in range(1, 5)])

⌘/Ctrl + Enter runs the cell

Skewness: which tail is longer

Cubing keeps the sign of each deviation and magnifies the large ones. If the large deviations lie mostly above the mean, the average cube is positive; if mostly below, negative. That average is the skewness.

Rule: Skewness

\operatorname{skew}(X) = \mathbb{E}\big[Z^3\big]

Positive: a long tail to the right, made of rare values far above the mean. Negative: a long tail to the left. A symmetric distribution has skewness 0.

Take a Bernoulli variable with $p = 0.2$ : $X = 1$ with probability 0.2 and 0 otherwise. Its mean is 0.2 and its standard deviation is $\sqrt{0.2 \times 0.8} = 0.4$ (lesson 5.3.2).

$x$	0	1
$p_X(x)$	0.8	0.2
$z = \frac{x - 0.2}{0.4}$	$-0.5$	2
$z^3$	$-0.125$	8

So $\operatorname{skew}(X) = 0.8 \times (-0.125) + 0.2 \times 8 = -0.1 + 1.6 = 1.5$ . The rare 1, two standard deviations above the mean, outweighs the common 0 just below it. In general a Bernoulli variable has skewness $\frac{1 - 2p}{\sqrt{p(1 - p)}}$ : 1.5 at $p = 0.2$ , 0 at $p = \frac{1}{2}$ and $-1.5$ at $p = 0.8$ .

Three binomial distributions with 10 trials. A long tail to one side gives skewness of that sign, and pulls the mean towards it, away from the most likely value.

The mean follows the tail. With $p = 0.15$ the most likely count is 1, but the mean is 1.5. Lesson 5.3.1's wait for a 5 or a 6 is the same: peak at 1, mean 3, skewness 2.0412.
A mirror flips the sign. $-X$ has skewness $-\operatorname{skew}(X)$ ; shifting $X$ or multiplying it by a positive number changes nothing.
Same mean and spread, different shape. Lesson 4.6.1's three bets all have mean 0 and standard deviation 0.2. Their skewness is 0, 1.5 and $-1.5$ , and the negative one, the rare big loss, was the one a concave utility minded most.

Try it The first line is the Bernoulli table, then its formula. The loop gives the three binomials of the figure. The last lines are the wait of lesson 5.3.1, summed over 400 terms, and lesson 4.6.1's three bets.

import numpy as np from scipy import stats def std_moment(values, probs, k): values, probs = np.asarray(values, float), np.asarray(probs, float) mu = values @ probs sigma = np.sqrt(((values - mu) ** 2) @ probs) return ((values - mu) / sigma) ** k @ probs print(round(float(std_moment([0, 1], [0.8, 0.2], 3)), 4), round((1 - 2 * 0.2) / np.sqrt(0.2 * 0.8), 4)) k = np.arange(0, 11) for p in [0.15, 0.5, 0.85]: print(p, round(float(std_moment(k, stats.binom.pmf(k, 10, p), 3)), 4) + 0.0) k = np.arange(1, 401) print(round(float(std_moment(k, stats.geom.pmf(k, 1 / 3), 3)), 4)) bets = [([0.2, -0.2], [0.5, 0.5]), ([0.4, -0.1], [0.2, 0.8]), ([-0.4, 0.1], [0.2, 0.8])] for values, probs in bets: print(values, round(float(std_moment(values, probs, 3)), 4) + 0.0)

⌘/Ctrl + Enter runs the cell

Kurtosis: how heavy the tails are

Fourth powers magnify large deviations even more. A value 3 standard deviations from the mean adds $3^4 = 81$ , while one at half a standard deviation adds $0.5^4 = 0.0625$ . So the fourth moment is driven by the rare, large deviations.

Rule: Kurtosis

\operatorname{kurt}(X) = \mathbb{E}\big[Z^4\big]

How much of the spread comes from rare values far from the mean, compared with common values near it. It is never below 1, and it is the same for $X$ , $-X$ and any $aX + b$ with $a \neq 0$ .

A family of examples shows it best. Let $X$ be 0 with probability $1 - q$ , and $a$ or $-a$ with probability $\frac{q}{2}$ each, where $a = \frac{1}{\sqrt{q}}$ . Then the mean is 0 and the variance is $q a^2 = 1$ for every $q$ , so $Z = X$ and the kurtosis is $q a^4 = \frac{1}{q}$ .

$q$	the values and their chances	kurtosis
1	$-1$ or 1, each $\frac{1}{2}$	1
$\frac{1}{3}$	0 with $\frac{2}{3}$ ; $-1.73$ or 1.73, each $\frac{1}{6}$	3
0.1	0 with 0.9; $-3.16$ or 3.16, each 0.05	10

All three have mean 0 and variance 1. With $q = 0.1$ , $X$ is usually 0 but now and then jumps 3.16 standard deviations, and the kurtosis is 10 times the coin's.

Three distributions with the same mean and variance. The rarer and further out the jumps, the larger the kurtosis.

Kurtosis is often described as how peaked a distribution is. The $q = 0.1$ case has both a tall spike at 0 and far-out jumps, but it is the jumps that make the number large. Read kurtosis as a measure of the tails.

A fair die has kurtosis 1.7314: it cannot stray far from its mean.

The normal's values, and excess kurtosis

The normal distribution, the bell curve of lesson 2.6.3, has skewness 0 and kurtosis 3 (lesson 5.4.3). Kurtosis is therefore often quoted relative to 3.

Rule: Excess kurtosis

\operatorname{kurt}(X) - 3

0 for a normal. Positive means heavier tails than a normal, often called fat tails; negative means lighter tails.

distribution	skewness	excess kurtosis
the $\pm 1$ coin	0	$-2$
a fair die	0	$-1.2686$
a normal	0	0
Bernoulli, $p = 0.2$	1.5	0.25
wait for a 5 or a 6	2.0412	6.1667
the family with $q = 0.1$	0	7

In the family, $q = \frac{1}{3}$ matches the normal's kurtosis exactly. Case study 5.3.5 measures the excess kurtosis of daily returns.

q

import numpy as np from scipy import stats def std_moment(values, probs, k): values, probs = np.asarray(values, float), np.asarray(probs, float) mu = values @ probs sigma = np.sqrt(((values - mu) ** 2) @ probs) return ((values - mu) / sigma) ** k @ probs for q in [1, 1 / 3, 0.1]: a = 1 / np.sqrt(q) print(round(q, 4), round(float(std_moment([-a, 0, a], [q / 2, 1 - q, q / 2], 4)), 4)) print(round(float(std_moment(np.arange(1, 7), np.full(6, 1 / 6), 4)), 4)) rng = np.random.default_rng(533) x = rng.standard_normal(1_000_000) print(round(np.mean(((x - x.mean()) / x.std()) ** 3), 4), round(np.mean(((x - x.mean()) / x.std()) ** 4), 4)) print([float(v) for v in stats.norm.stats(moments="mvsk")]) for dist in [stats.bernoulli(0.2), stats.geom(1 / 3)]: print([round(float(v), 4) for v in dist.stats(moments="sk")])

⌘/Ctrl + Enter runs the cell

Sample skewness and kurtosis

For data $x_1, \ldots, x_n$ , treat each value as equally likely, with probability $\frac{1}{n}$ , and apply the same formulas. That uses the sample mean $\bar{x}$ and the standard deviation that divides by $n$ . scipy.stats.skew and scipy.stats.kurtosis do exactly this by default.

-1.2678

import numpy as np from scipy import stats rng = np.random.default_rng(533) rolls = rng.integers(1, 7, size=10_000) print(round(stats.skew(rolls), 4), round(stats.kurtosis(rolls), 4), round(stats.kurtosis(rolls, fisher=False), 4)) z = (rolls - rolls.mean()) / rolls.std() print(round(np.mean(z**3), 4), round(np.mean(z**4) - 3, 4))

⌘/Ctrl + Enter runs the cell

A sample kurtosis is noisy

A sample's kurtosis rests on its few most extreme values, so short samples give very different answers. Draw 100 values at a time from the family with $q = 0.1$ , whose kurtosis is 10. Five seeded samples give 12.5, 14.25, 5.79, 8.99 and 9.94.

Only big samples pin it down: 1,000,000 draws give 9.9996. The higher the moment, the more data it needs.

Try it Each line of the loop is a separate sample of 100 draws. rng.choice draws from the three values with the given chances. fisher=False asks for the plain kurtosis, so the target is 10.

import numpy as np from scipy import stats a = 1 / np.sqrt(0.1) rng = np.random.default_rng(533) for _ in range(5): sample = rng.choice([-a, 0, a], p=[0.05, 0.9, 0.05], size=100) print(round(stats.kurtosis(sample, fisher=False), 2)) big = rng.choice([-a, 0, a], p=[0.05, 0.9, 0.05], size=1_000_000) print(round(stats.kurtosis(big, fisher=False), 4))

⌘/Ctrl + Enter runs the cell

Where the measures break

No spread. If every value is the same, $\sigma = 0$ and $Z$ is $\frac{0}{0}$ . SciPy returns nan, "not a number", with a warning.
An infinite kurtosis. The sum for $\mathbb{E}[Z^4]$ can diverge even when the variance is finite. The Student $t$ distribution with 4 degrees of freedom (lesson 5.4.4) is an example. Its sample kurtosis never settles, however long the sample.

t

import warnings from scipy import stats with warnings.catch_warnings(): warnings.simplefilter("ignore") print(stats.skew([2.0, 2.0, 2.0]), stats.kurtosis([2.0, 2.0, 2.0])) print([float(v) for v in stats.t(4).stats(moments="vk")])

⌘/Ctrl + Enter runs the cell

Common mistakes

SciPy's kurtosis is excess kurtosis by default. Normal data gives about 0, not 3.
Positive skewness means a long right tail. Most of the values then sit below the mean, as in the binomial with $p = 0.15$ .
Standardise before cubing. $\mathbb{E}[(X - \mu)^3]$ has units cubed and changes with the units; the skewness does not.
A sign flip changes the skewness, not the kurtosis. $-X$ has skewness $-\operatorname{skew}(X)$ and the same kurtosis.
Kurtosis measures the tails, not the peak.
Do not trust the kurtosis of a short sample. A few extreme values decide it.

Skewness and kurtosis

0 of 11 correct

1.
Find the skewness of a Bernoulli variable with $p = 0.2$
2.
Find the skewness of a Bernoulli variable with $p = 0.9$ , as a fraction
3.
Find the skewness of a fair die roll
4.
$X$ has skewness 1.2. Find the skewness of $7 - 3X$
5.
$X$ has kurtosis 4.5. Find the kurtosis of $5X - 2$
6.
$X$ is 0 with probability 0.75, and $-2$ or 2 with probability 0.125 each. Its mean is 0 and its variance 1. Find its kurtosis
7.
Find the excess kurtosis of that same $X$
8.
Find the kurtosis of a coin that is $-1$ or 1 with equal chances
9.
Find the kurtosis of a Bernoulli variable with $p = 0.2$
10.
A data set has an excess kurtosis of 4.5. What is its kurtosis $\mathbb{E}[Z^4]$ ?
11.
scipy.stats.kurtosis is given a million draws from a normal distribution. About what does it return?

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

k

Skewness and kurtosis

Moments

Rule: Standardised moments

Skewness: which tail is longer

Rule: Skewness

Kurtosis: how heavy the tails are

Rule: Kurtosis

The normal's values, and excess kurtosis

Rule: Excess kurtosis

Sample skewness and kurtosis

A sample kurtosis is noisy

Where the measures break

Common mistakes

Skewness and kurtosis

Summary card

Free further reading