The Markov, Chebyshev and Jensen inequalities

A guarantee from the mean alone

Often you know a mean, or a mean and a variance, but not the whole distribution. An inequality turns those numbers into a bound: a probability that cannot be exceeded, whatever the distribution. It is a guarantee, not an estimate.

Waits in a queue are never negative and average 2 minutes. Could half of all customers wait 10 minutes or more? No: those customers alone would push the mean to at least $0.5 \times 10 = 5$ minutes.

At most a fifth can wait that long, since $0.2 \times 10 = 2$ already uses up the whole mean. This argument is Markov's inequality.

Markov's inequality

A fair die. $X$ is the number shown, with mean 3.5 (lesson 5.3.1). The bound $\frac{3.5}{a}$ against the truth:

The bound always holds, but loosely: at $a = 6$ it allows 0.5833 where the truth is $\frac{1}{6}$. Up to $a = 3$ it is above 1 and says nothing at all.

An exponential wait. With mean 1 (lesson 5.4.2), $P(X \ge a) = e^{-a}$. At $a = 3$ that is 0.0498, against the bound $\frac{1}{3} \approx 0.3333$; at $a = 5$ it is 0.0067 against 0.2.

Three cases complete the picture.

• The bound can be reached. Let $X$ be 0 with probability 0.8 and 10 with probability 0.2. Its mean is 2, and $P(X \ge 10) = 0.2 = \frac{2}{10}$ exactly. So no better bound can come from the mean alone.
• For $a \le \mathbb{E}[X]$ it says nothing. The bound is then 1 or more, and every probability is at most 1 anyway.
• Negative values break it. Let $X$ be $-10$ or $10$ with equal chances. Its mean is 0, so the "bound" would say $P(X \ge 10) \le 0$, yet that probability is $\frac{1}{2}$. The negative values cancel the large ones in the mean.

Chebyshev's inequality

Markov's inequality uses only the mean. Add the variance and you can bound how far $X$ strays on either side.

The squared distance $(X - \mu)^2$ is never negative, and its mean is the variance $\sigma^2$ (lesson 5.3.2). Markov's inequality applied to it, with $a = \varepsilon^2$, gives the rule below. Here $\mu$ is the mean, $\sigma$ the standard deviation and $\varepsilon$ (epsilon) any positive distance.

So at least $1 - \frac{1}{4} = 75\%$ of any distribution lies within 2 standard deviations of its mean, and at least $\frac{8}{9} \approx 88.89\%$ within 3. For the normal the shares are 95.45% and 99.73% (lesson 5.4.3): the guarantee is weaker because it must hold for every distribution.

The die again. $\mu = 3.5$ and $\sigma = \sqrt{35/12} \approx 1.7078$. The faces 1 and 6 lie 2.5 from the mean, which is $k \approx 1.4639$ standard deviations.

So $P(|X - 3.5| \ge 2.5) = \frac{2}{6} = \frac{1}{3}$, and Chebyshev allows $\frac{35/12}{2.5^2} \approx 0.4667$.

Three distributions at once. The two-sided tail $P(|X - \mu| \ge k\sigma)$ for a normal, a uniform and an exponential with mean 1, against the bound:

A uniform never strays more than $\sqrt{3} \approx 1.73$ standard deviations from its mean, so its tail is 0 from $k = 2$ on. The exponential's long right tail keeps it above the normal far out.

The cases to know:

• $k \le 1$ says nothing. The bound is then at least 1.
• The bound can be reached. Let $X$ be $-1$, 0 or 1 with probabilities $\frac{1}{8}$, $\frac{3}{4}$, $\frac{1}{8}$. Then $\mu = 0$ and $\sigma = \frac{1}{2}$, and $P(|X| \ge 1) = \frac{1}{4}$: exactly $\frac{1}{k^2}$ with $k = 2$.
• It needs a finite variance. A Student t with $\nu \le 2$ degrees of freedom has none (lesson 5.4.4), and Chebyshev's inequality says nothing about it.

Chebyshev's inequality is loose, but it holds without knowing the shape of the distribution. Lesson 5.7.2 uses it to show that averages settle on the mean.

Jensen's inequality

Lesson 5.3.1 found that for a die the mean of the squares, $\mathbb{E}[X^2] \approx 15.17$, is above the square of the mean, $3.5^2 = 12.25$. It also found $\mathbb{E}\left[\frac{1}{X}\right] \approx 0.4083$, above $\frac{1}{\mathbb{E}[X]} \approx 0.2857$. Both curves, $x^2$ and $\frac{1}{x}$ for $x > 0$, bend up.

Four examples, two of each kind:

• $x^2$, convex. $15.1667 \ge 12.25$. The gap is exactly the variance, $\frac{35}{12} \approx 2.9167$: for $g(x) = x^2$, Jensen's inequality says a variance is never negative.
• $\frac{1}{x}$, convex for $x > 0$. $0.4083 \ge 0.2857$.
• $\ln x$, concave. $\mathbb{E}[\ln X] \approx 1.0965$, below $\ln 3.5 \approx 1.2528$. The same holds for $\sqrt{x}$: 1.8053 against 1.8708.
• $e^x$, convex. Case study 5.6.5 took $Y = vZ$ with $Z$ standard normal and $v \approx 0.141421$. Then $\mathbb{E}[e^{Y}] = e^{v^2/2} \approx 1.01005$, above $e^{\mathbb{E}[Y]} = e^0 = 1$. That excess is why its model subtracts $\frac{\sigma^2}{2}$ in the exponent.

Where the rule says less, or nothing:

• Equality holds when $X$ is a constant or $g$ is a straight line over the values of $X$.
• Convex over every value. If $X$ is $-1$ or 1 with equal chances, $\frac{1}{x}$ is not convex across 0. Then $\mathbb{E}\left[\frac{1}{X}\right] = 0$, while $\frac{1}{\mathbb{E}[X]} = \frac{1}{0}$ does not exist.
• A curve that bends both ways, such as $x^3$, can give either order. With $X$ equal to 0 or 2, $\mathbb{E}[X^3] = 4 \ge 1^3$; with $X$ equal to $-2$ or 0, $\mathbb{E}[X^3] = -4 \le (-1)^3$.

How big the gap is

For a small spread the gap is about half the curvature times the variance:

This is the quadratic term of the Taylor polynomial about $\mu$ (lesson 2.7.1), averaged. For $x^2$ it is exact: $\frac{1}{2} \cdot 2 \cdot \sigma^2 = \sigma^2$. For $e^{vZ}$ it gives $\frac{v^2}{2} = 0.01$, against the exact 0.01005.

For the die the spread is too wide for the estimate to be close. The sign of $g''$ still tells you the direction.

Risk aversion

Case study 2.4.5 valued wealth $w$ with the concave utility $u(w) = \ln w$. A fair flip of $20,000 on $100,000 gave an expected utility of 11.4925, below $\ln 100{,}000 \approx 11.5129$.

Jensen's inequality says this happens for every gamble, not only that flip. For any concave $u$ and any random wealth $W$, $\mathbb{E}[u(W)] \le u(\mathbb{E}[W])$.

So the certainty equivalent, the sure wealth with the same utility, is never above the expected wealth, and the risk premium is never negative. A convex utility reverses this: that person is risk seeking, as in case study 2.4.5's $u(w) = w^2$.

Take wealth of $50,000, $100,000 or $200,000 with probabilities 0.25, 0.5 and 0.25. The expected wealth is $112,500, but with log utility the certainty equivalent is exactly $100,000. This person would give up $12,500 to avoid the risk.

Common mistakes

1. Markov's inequality needs $X \ge 0$. For $-10$ or $10$ with equal chances it would claim $P(X \ge 10) \le 0$.
2. A bound is not an estimate. For a die, Markov allows $P(X \ge 6) \le 0.5833$; the truth is $\frac{1}{6}$.
3. Chebyshev's bound covers both sides together, and needs $k > 1$ to say anything.
4. Keep the two forms apart. $\frac{\sigma^2}{\varepsilon^2}$ uses the variance and a distance in the units of $X$; $\frac{1}{k^2}$ counts standard deviations. With $\sigma = 2$ and $\varepsilon = 4$, both give $\frac{1}{4}$.
5. Jensen's direction. Convex means the mean of $g$ is the larger side. Check with $x^2$: $\mathbb{E}[X^2] \ge (\mathbb{E}[X])^2$.
6. The curve must bend one way over every value of $X$. $\frac{1}{x}$ is convex only for $x > 0$.