Functions of random variables

A function of a random variable

Lesson 5.2.1 defined a random variable as a number attached to each outcome. Apply a function $g$ to that number and you attach a new number, $y = g(x)$, to the same outcome. So $Y = g(X)$ is a random variable too, and its probabilities come from those of $X$.

Take a fair die $X$ and its squared distance from 3, $Y = (X - 3)^2$.

Two faces give $y = 1$ and two give $y = 4$, so those values collect two sixths each. The pmf of $Y$ is

For the mean alone, LOTUS (lesson 5.3.1) skips $p_Y$: $\mathbb{E}[(X - 3)^2] = \frac{4 + 1 + 0 + 1 + 4 + 9}{6} = \frac{19}{6} \approx 3.1667$. The pmf of $Y$ gives the same, $0 \cdot \frac{1}{6} + 1 \cdot \frac{1}{3} + 4 \cdot \frac{1}{3} + 9 \cdot \frac{1}{6} = \frac{19}{6}$. It also answers questions LOTUS does not, such as $P(Y \ge 4) = \frac{1}{2}$.

Continuous $X$: the distribution-function method

A continuous random variable gives each single value probability 0 (lesson 5.4.1), so adding up probabilities of values fails. Work with the distribution function instead. $F_Y(y) = P(Y \le y)$ is the probability that $g(X) \le y$, and that is an event about $X$.

Take $U$ uniform on $[0, 1]$, with density 1 there, and $Y = U^2$. For $y$ between 0 and 1, $U^2 \le y$ means $U \le \sqrt{y}$, because $U$ is never negative. So $F_Y(y) = P(U \le \sqrt{y}) = \sqrt{y}$.

Half the values of $U^2$ lie below 0.25: squaring a number between 0 and 1 pushes it towards 0. Differentiating gives the density, $f_Y(y) = F_Y'(y) = \frac{1}{2\sqrt{y}}$ for $0 < y \le 1$, and 0 elsewhere.

The density $\frac{1}{2\sqrt{y}}$ is 5 at 0.01, 1 at 0.25 and 0.5 at 1. It blows up at 0, yet its area is still 1: an improper integral of the kind in lesson 2.6.3.

A monotone $g$: the density rule

When $g$ is increasing, the method collapses into one formula. Write $x = g^{-1}(y)$ for the value that $g$ sends to $y$. Then $g(X) \le y$ exactly when $X \le x$, so $F_Y(y) = F_X(x)$, and the chain rule (lesson 2.3.3) gives $f_Y(y) = f_X(x)\frac{dx}{dy}$.

The picture behind it is a thin strip. The strip of width $dx$ around $x$ holds probability about $f_X(x)\,dx$, and $g$ carries it onto a strip of width $dy$ around $y$. The probability is the same, so $f_Y(y)\,dy = f_X(x)\,dx$: where $g$ stretches the strip, the density thins.

Take $Y = e^U$, with $U$ uniform on $[0, 1]$ in the part of $X$. It rises from 1 to $e$, and its inverse is $u = \ln y$, so $\frac{du}{dy} = \frac{1}{y}$. The density is $f_Y(y) = 1 \cdot \frac{1}{y}$ for $1 \le y \le e$.

The strips in numbers:

Probabilities follow by integrating the density. $P(Y \le 1.5) = \int_1^{1.5} \frac{dy}{y} = \ln 1.5 \approx 0.405465$, and $P(Y \ge 2) = 1 - \ln 2 \approx 0.306853$. One seeded run of 100,000 draws gives 0.4014 and 0.3088.

The mean is $\int_1^e y \cdot \frac{1}{y}\,dy = e - 1 \approx 1.718282$. LOTUS gives the same, $\int_0^1 e^u\,du = e - 1$. It is not $e^{\mathbb{E}[U]} = e^{0.5} \approx 1.648721$ (lesson 5.3.1).

Quantiles carry over

An increasing $g$ keeps values in order, so it carries quantiles across. If half of $U$ lies below $\frac{1}{2}$, half of $e^U$ lies below $e^{0.5}$: the median of $Y$ is $e^{0.5} \approx 1.648721$, and the simulation gives 1.6569.

The mean does not carry over in this way: it is $e - 1$, larger than the median. Case study 5.6.5 meets the same gap in stock prices.

Linear functions: moving and stretching

For $Y = aX + b$ with $a \neq 0$, the inverse is $x = \frac{y - b}{a}$ and $\left\lvert \frac{dx}{dy} \right\rvert = \frac{1}{\lvert a \rvert}$. So

The shape is the same, moved by $b$ and stretched by $\lvert a \rvert$, and its height is divided by $\lvert a \rvert$ so the area stays 1. A negative $a$ also flips it: $1 - U$ has density $f_U(1 - y) = 1$ on $[0, 1]$, uniform again.

The normal. Take $Z$ standard normal, with density $\varphi$, and $X = \mu + \sigma Z$ with $\sigma > 0$. The rule gives $f_X(x) = \frac{1}{\sigma}\varphi\!\left(\frac{x - \mu}{\sigma}\right)$, the $\mathcal{N}(\mu, \sigma^2)$ density of lesson 5.4.3: standardising read backwards. With $\mu = 2$ and $\sigma = 3$, $f_X(2) = \frac{\varphi(0)}{3} \approx 0.132981$ and $f_X(5) = \frac{\varphi(1)}{3} \approx 0.080657$.

The lognormal. Exponentiate that normal $X$: as for $e^U$, the inverse is $x = \ln y$ with $\frac{dx}{dy} = \frac{1}{y}$, so $e^X$ has density $\frac{1}{\sigma y}\varphi\!\left(\frac{\ln y - \mu}{\sigma}\right)$ for $y > 0$. That is the lognormal density of lesson 5.4.4; its $\frac{1}{y}$ is the stretch of $e^x$. With $\mu = 0$ and $\sigma = 0.5$ it is 0.38287 at $y = 1.5$.

Units. A density is probability per unit of the variable. If $X$ is uniform on $[0, 2]$ dollars, its density is 0.5 per dollar. In cents, $100X$ is uniform on $[0, 200]$ with density 0.005 per cent: a hundredth as tall.

A decreasing $g$: from uniform to exponential

$Y = -\ln U$ falls as $U$ rises: $u$ near 1 gives $y$ near 0, and $u$ near 0 gives large $y$. Its inverse is $u = e^{-y}$, with $\frac{du}{dy} = -e^{-y}$, which is negative. The absolute value gives $f_Y(y) = 1 \cdot e^{-y}$ for $y \ge 0$: the exponential distribution with $\lambda = 1$ (lesson 5.4.2).

Without the absolute value, the "density" $-e^{-y}$ would be negative. For any decreasing $g$, $F_Y(y) = 1 - F_X(x)$, and the minus sign of that $1 - F$ cancels the minus sign of $\frac{dx}{dy}$.

Dividing by $\lambda$ gives $-\frac{1}{\lambda}\ln U$, exponential with rate $\lambda$ and mean $\frac{1}{\lambda}$. A computer can turn uniform draws into exponential ones this way.

Why the inverse transform works

Lesson 5.4.1 drew $X$ as $x_U$: the quantile function of $X$ applied to a uniform $U$, the inverse transform method. The distribution-function method shows why the draws have the distribution $F$. For an increasing, continuous $F$, $x_U \le x$ exactly when $U \le F(x)$, so $P(x_U \le x) = P\big(U \le F(x)\big) = F(x)$.

For the exponential, $F(x) = 1 - e^{-\lambda x}$ gives the quantile $x_u = -\frac{1}{\lambda}\ln(1 - u)$. Since $1 - U$ is uniform too, $-\frac{1}{\lambda}\ln U$ does the same job.

When $g$ folds: add the branches

If several values of $x$ give the same $y$, each sends probability to $y$, and the contributions add. Take $Y = Z^2$ with $Z$ standard normal. Both $z = -1$ and $z = 1$ give $y = 1$.

By the distribution-function method, $Z^2 \le y$ means $-\sqrt{y} \le Z \le \sqrt{y}$, so $F_Y(y) = N(\sqrt{y}) - N(-\sqrt{y})$ for $y > 0$. Differentiate with $N' = \varphi$ and the chain rule:

The two terms are the monotone rule on each branch, $z = \sqrt{y}$ and $z = -\sqrt{y}$, each with $\left\lvert \frac{dz}{dy} \right\rvert = \frac{1}{2\sqrt{y}}$. The normal is symmetric, so they are equal.

This is the chi-squared distribution with 1 degree of freedom of lesson 5.4.4, chi2(1) in SciPy; at $y = 1$ both give 0.241971. Its probabilities come from the normal: $P(Z^2 \le 1) = P(\lvert Z \rvert \le 1) \approx 0.682689$, the 68% of lesson 5.4.3. Since $1.96^2 = 3.8416$, $P(Z^2 \le 3.8416) \approx 0.950004$.

Where the density rule needs care

• The range. $f_Y$ is 0 outside the values $g$ can take: $e^U$ lives on $[1, e]$, while $-\ln U$ and $Z^2$ live on $[0, \infty)$.
• Flat points. Where $g'(x) = 0$, as for $U^2$ and $Z^2$ at 0, the factor $\left\lvert \frac{dx}{dy} \right\rvert$ is infinite and the density blows up. Its area stays finite, so this does no harm.
• Flat stretches. If $g$ is constant on an interval, $Y$ piles probability onto one value, and there is no density there.

$Y = \max(Z, 0)$ is such a function: it is 0 whenever $Z \le 0$, so $P(Y = 0) = \frac{1}{2}$. Above 0 it has density $\varphi(y)$, whose area is the other half. An option's payoff, $\max(S_T - K, 0)$, has this shape (case study 2.6.5).

Lesson 5.6.2 describes a distribution by a different function, its moment generating function. Case study 5.6.5 uses this lesson's rule to find the density of a future stock price.

Common mistakes

1. Keep the factor $\left\lvert \frac{dx}{dy} \right\rvert$. For $e^U$, $f_U(\ln y) = 1$ alone on $[1, e]$ has area $e - 1 \approx 1.718282$, not 1.
2. Divide by the stretch, do not multiply. The factor is $\frac{dx}{dy} = \frac{1}{g'(x)}$, not $\frac{dy}{dx}$.
3. Take the absolute value. For a decreasing $g$ such as $-\ln U$, $\frac{dx}{dy}$ is negative, and without $\lvert\;\rvert$ the density would be too.
4. Add every branch. One branch of $Z^2$ gives half the density, with area $\frac{1}{2}$.
5. Answer in $y$, with its range. Put $x = g^{-1}(y)$ into $f_X$, and say where $f_Y$ is 0.
6. The mean does not pass through $g$. The median of $e^U$ is $e^{0.5}$, but its mean is $e - 1$.