Case study: the minimum-variance portfolio and the efficient frontier

The two-asset minimum by Lagrange

A portfolio puts a fraction $w_i$ of the money in asset $i$. Its variance is $\mathbf{w}^\top\Sigma\mathbf{w}$, with $\Sigma$ the covariance matrix (case study 3.2.5).

The weights must add up to 1, so that all the money is invested: $\mathbf{1}^\top\mathbf{w} = 1$, where $\mathbf{1}$ is the vector of ones. A negative weight is a short sale (case study 1.2.4).

So the safest portfolio solves: minimise $f(\mathbf{w}) = \mathbf{w}^\top\Sigma\mathbf{w}$ subject to $g(\mathbf{w}) = \mathbf{1}^\top\mathbf{w} = 1$. Lesson 4.4.1's quadratic $\frac{1}{2}\mathbf{w}^\top A\mathbf{w}$ has gradient $A\mathbf{w}$, and here $A = 2\Sigma$, so $\nabla f = 2\Sigma\mathbf{w}$. The budget has $\nabla g = \mathbf{1}$.

Lagrange's condition $\nabla f = \lambda\nabla g$ reads $2\Sigma\mathbf{w} = \lambda\mathbf{1}$: every entry of $\Sigma\mathbf{w}$ is the same number, $\frac{\lambda}{2}$. For case study 1.2.4's two assets, with $\Sigma = \begin{pmatrix} 0.04 & 0.012 \\ 0.012 & 0.09 \end{pmatrix}$, the two entries are

Subtracting gives $0.028\,w_A = 0.078\,w_B$. With $w_B = 1 - w_A$, that is $0.106\,w_A = 0.078$, so $w_A = \frac{0.078}{0.106} \approx 0.7358$. That is case study 1.2.4's vertex, $\frac{0.156}{0.212}$, the same number found a second way.

Case study 3.2.5 noticed that at case study 1.2.4's weight both entries of $\Sigma\mathbf{w}$ equal 0.0326, the variance itself. The rule says why: it holds at every minimum-variance portfolio, with any number of assets.

The second line follows from the first. $\Sigma\mathbf{w} = \frac{\lambda}{2}\mathbf{1}$ gives $\mathbf{w} = \frac{\lambda}{2}\Sigma^{-1}\mathbf{1}$, and the budget fixes $\frac{\lambda}{2}$ so that the weights add up to 1. In code, solve $\Sigma\mathbf{y} = \mathbf{1}$ with np.linalg.solve (lesson 3.2.3) and divide $\mathbf{y}$ by the sum of its entries.

Why the entries must agree

Entry $i$ of $\Sigma\mathbf{w}$ is the covariance of asset $i$ with the portfolio (case study 3.2.5). Move a little money from asset $j$ to asset $i$, along $\mathbf{e}_i - \mathbf{e}_j$ (standard basis vectors, lesson 3.5.1): the budget stays the same. The variance changes at the rate $\nabla f \cdot (\mathbf{e}_i - \mathbf{e}_j) = 2\big((\Sigma\mathbf{w})_i - (\Sigma\mathbf{w})_j\big)$ (lesson 4.2.1).

If two entries differ, moving money towards the smaller one lowers the variance. Only when all entries agree does no move help.

Three assets

The three assets of case study 3.2.5 have expected returns $\boldsymbol{\mu} = (0.06, 0.08, 0.11)$, volatilities 12%, 18% and 25%, and correlations $\rho_{AB} = 0.3$, $\rho_{AC} = 0.1$, $\rho_{BC} = 0.5$. The same rule gives the minimum-variance portfolio:

Its variance is 0.012036, the common value, so its volatility is 10.97%. Its expected return is $\mathbf{w} \cdot \boldsymbol{\mu} = 6.81\%$.

The 50/30/20 portfolio of case study 3.2.5 has $\Sigma\mathbf{w} = (0.009744, 0.01746, 0.02075)$. The entries differ, so it is not the minimum. Moving money from C to A lowers the variance at the rate $2(0.009744 - 0.02075) = -0.022012$ per unit.

Case study 3.2.5's risk contribution of asset $i$ is $w_i(\Sigma\mathbf{w})_i$. At the minimum every $(\Sigma\mathbf{w})_i$ equals the variance, so each asset's share of the risk equals its share of the money. The shares are 74.65%, 15.11% and 10.23%.

Is this the lowest variance, or only a candidate? $\Sigma$ is positive definite, with eigenvalues 0.011741, 0.022502 and 0.075058 (lesson 3.7.1). So $\mathbf{w}^\top\Sigma\mathbf{w}$ is strictly convex and the budget is linear: lesson 4.5.3 says the Lagrange point is the one global minimum.

A target return

The minimum-variance portfolio expects only 6.81%. An investor who wants more return asks for the safest portfolio that expects a target $m$: a second constraint, $\boldsymbol{\mu}^\top\mathbf{w} = m$.

Two equality constraints bring two multipliers, as in lesson 4.5.2's rule for two constraints:

That is five linear equations in $w_A$, $w_B$, $w_C$, $\lambda_1$ and $\lambda_2$, solved as one system like lesson 4.5.1's. For $m = 9\%$ the answer is 29.87% in A, 16.88% in B and 53.25% in C, with volatility 15.99%.

The multiplier $\lambda_2 = 1.2393$ is a rate, as in lesson 4.5.1: the variance per unit of target return. From 9% to 9.1% the variance rises by 0.001268, close to $\lambda_2 \times 0.001 = 0.001239$. Extra return costs risk, and $\lambda_2$ says how much.

The efficient frontier

Solving for every target at once gives a formula, with $a = \frac{\lambda_1}{2}$ and $b = \frac{\lambda_2}{2}$. The first equation becomes $\Sigma\mathbf{w} = a\mathbf{1} + b\boldsymbol{\mu}$, so $\mathbf{w} = a\,\Sigma^{-1}\mathbf{1} + b\,\Sigma^{-1}\boldsymbol{\mu}$. Every frontier portfolio mixes the same two vectors.

Put that $\mathbf{w}$ into the two constraints. With three numbers from the data,

they read $ak_1 + bk_2 = 1$ and $ak_2 + bk_3 = m$. This $2 \times 2$ system (lesson 3.2.3) gives $a = \frac{k_3 - k_2m}{d}$ and $b = \frac{k_1m - k_2}{d}$, with $d = k_1k_3 - k_2^2$.

The variance is short to find too: $\mathbf{w}^\top\Sigma\mathbf{w} = \mathbf{w}^\top(a\mathbf{1} + b\boldsymbol{\mu}) = a + bm$, since $\mathbf{w}^\top\mathbf{1} = 1$ and $\mathbf{w}^\top\boldsymbol{\mu} = m$.

Here $k_1 = 83.0812$, $k_2 = 5.6611$, $k_3 = 0.421027$ and $d = 2.93109$. The vertex is at $\frac{k_2}{k_1} = 6.81\%$, with variance $\frac{1}{k_1} = 0.012036$: the minimum-variance portfolio again.

Plotted with volatility $\sigma$ across and expected return up, the frontier is a hyperbola, the shape lesson 3.7.1 met as a level curve. Only its upper half is worth holding: the efficient frontier. A 6% target needs a volatility of 11.80%, and so does 7.63% on the upper half: same risk, more return.

The 50/30/20 portfolio lies inside the curve. For its 7.6% the frontier needs only 11.74%, against its 11.94%.

Two funds are enough

$a$ and $b$ are straight-line functions of $m$, so the frontier weights are too. Each extra point of target return moves the weights by the same amounts: $-20.49$ points in A, $+0.81$ in B and $+19.68$ in C. In the figure they are three straight lines.

Take the 8% and 12% frontier portfolios as the two funds. Then 75% of the first and 25% of the second gives exactly the 9% portfolio, (29.87%, 16.88%, 53.25%). The minimum-variance portfolio and the 11% portfolio, mixed with $t = 0.5222$, give it too.

No short sales

Many funds may not sell short, so every weight must satisfy $w_i \ge 0$. Each is an inequality constraint of lesson 4.5.2, with its own multiplier $\nu_i$ (nu). A $\ge$ constraint has $\nu_i \ge 0$.

Two assets. Case study 1.2.4 met $\rho = 0.8$, where the rule gives 123.53% in A and $-23.53\%$ in B. Long-only, try all in A, where $\Sigma\mathbf{w} = (0.04, 0.048)$.

B is left out, and its entry 0.048 is above A's 0.04, so the conditions hold. The problem is convex (lesson 4.5.3), so all in A is the best long-only portfolio: case study 1.2.4's "end nearest the vertex".

Three assets, no target. The minimum-variance weights are all positive, so the ban changes nothing.

Three assets, with a target. The frontier weights are all positive from 6.29% to 10.46%, the band in the figure. There the long-only frontier is the same.

Above 10.46%, the formula would sell A short. Hold A at 0 instead: B and C alone must meet both constraints, which fixes their weights.

At 10.8% that gives 6.67% in B and 93.33% in C, with volatility 23.96%, against 23.89% with short sales. Its $\nu_A = 0.009042 \ge 0$, so the conditions hold.

The sign test also catches a wrong guess. At 10%, holding A at 0 gives $\nu_A = -0.012124$, the wrong sign: A should be held. There the short-sales portfolio, (9.39%, 17.69%, 72.92%), is already long-only.

Below 6.29% it is C that would be sold short; at 6.2% the long-only portfolio is 90% A and 10% B. Long-only targets run only from 6% (all A) to 11% (all C), the lowest and highest expected returns. With these assets the ban costs at most 0.20 points of volatility, at the 6% end: 12.00% against 11.80%.

With more assets, guessing which weights are 0 gets slow, and a solver does the search. SLSQP (lesson 4.5.2) takes the two equations as "eq" constraints and $w_i \ge 0$ as bounds. Check that the target lies between the lowest and highest expected return first, and check r.success after.

The explorer draws the frontier for the three assets. Set the target to 10.8% and read the weights with short sales allowed; then turn short sales off and watch A drop to 0. Move the target below 6.29% to see C do the same.

On the desk

import numpy as np

MU = np.array([0.06, 0.08, 0.11])
SIGMA = np.array([0.12, 0.18, 0.25])
R = np.array([[1.0, 0.3, 0.1], [0.3, 1.0, 0.5], [0.1, 0.5, 1.0]])
COV = np.diag(SIGMA) @ R @ np.diag(SIGMA)
TWO = np.array([[0.04, 0.012], [0.012, 0.09]])


def _weights(value, what):
    assert value is not None, f"{what} returned None: return the array of weights"
    return np.asarray(value, dtype=float)


def _raises_value_error(fn, *args):
    try:
        fn(*args)
    except ValueError:
        return True
    return False


def test_two_assets():
    """Case study 1.2.4's two assets: 73.58% in A and 26.42% in B"""
    w = _weights(min_variance(TWO), "min_variance")
    expect_close(w, [0.156 / 0.212, 1 - 0.156 / 0.212], what="min_variance for case study 1.2.4", rel=1e-9)


def test_three_assets():
    """The three assets: about 74.65% in A, 15.11% in B and 10.23% in C"""
    w = _weights(min_variance(COV), "min_variance")
    expect_close(w, [0.7465402, 0.1511109, 0.1023489], what="min_variance for A, B and C", rel=1e-5)
    expect_close(w.sum(), 1.0, what="sum of the weights")


def test_equal_entries():
    """At the minimum, every entry of Σw equals the variance, for a random 4-asset Σ too"""
    rng = np.random.default_rng(455)
    M = rng.normal(size=(6, 4))
    cov = M.T @ M / 20
    w = _weights(min_variance(cov), "min_variance")
    expect_close(cov @ w, np.full(4, w @ cov @ w), what="Σw against wᵀΣw in every entry", rel=1e-9)


def test_target_return():
    """The 9% frontier portfolio is about (29.87%, 16.88%, 53.25%), with volatility 15.99%"""
    w = _weights(frontier_portfolio(MU, COV, 0.09), "frontier_portfolio")
    expect_close(w, [0.2987268, 0.1687887, 0.5324845], what="frontier_portfolio(mu, cov, 0.09)", rel=1e-5)
    expect_close(np.sqrt(w @ COV @ w), 0.1599427, what="volatility at 9%", rel=1e-5)


def test_constraints_hold():
    """A frontier portfolio sums to 1 and earns its target, here 12%"""
    w = _weights(frontier_portfolio(MU, COV, 0.12), "frontier_portfolio")
    expect_close(w.sum(), 1.0, what="sum of the weights")
    expect_close(w @ MU, 0.12, what="expected return")


def test_no_lower_variance():
    """No other portfolio with the same budget and return has a lower variance"""
    w = _weights(frontier_portfolio(MU, COV, 0.09), "frontier_portfolio")
    # (3, −5, 2) keeps both the budget and the return: 3 − 5 + 2 = 0 and 0.18 − 0.40 + 0.22 = 0
    d = np.array([3.0, -5.0, 2.0])
    for step in [-0.01, 0.01, 0.1]:
        other = w + step * d
        assert other @ COV @ other > w @ COV @ w, f"moving by {step} × (3, −5, 2) lowered the variance: not the minimum"


def test_minimum_on_frontier():
    """The frontier portfolio at the minimum-variance return is the minimum-variance portfolio"""
    w0 = _weights(min_variance(COV), "min_variance")
    w = _weights(frontier_portfolio(MU, COV, float(w0 @ MU)), "frontier_portfolio")
    expect_close(w, w0, what="frontier_portfolio at 6.81%", rel=1e-8)


def test_two_funds():
    """Two-fund theorem: 75% of the 8% portfolio plus 25% of the 12% portfolio is the 9% portfolio"""
    w8 = _weights(frontier_portfolio(MU, COV, 0.08), "frontier_portfolio")
    w12 = _weights(frontier_portfolio(MU, COV, 0.12), "frontier_portfolio")
    w9 = _weights(frontier_portfolio(MU, COV, 0.09), "frontier_portfolio")
    expect_close(0.75 * w8 + 0.25 * w12, w9, what="the mix of two frontier portfolios", rel=1e-9)


def test_percent_units():
    """Volatilities typed in percent give the same weights"""
    pct = np.diag(100 * SIGMA) @ R @ np.diag(100 * SIGMA)
    expect_close(_weights(min_variance(pct), "min_variance"), _weights(min_variance(COV), "min_variance"), what="weights from Σ in percent", rel=1e-9)


def test_not_symmetric():
    """A covariance matrix with a typo in one triangle raises ValueError"""
    typo = COV.copy()
    typo[0, 1] = 0.0648
    assert _raises_value_error(min_variance, typo), "a matrix that is not symmetric should raise ValueError"


def test_not_positive_definite():
    """Impossible correlations, with a negative eigenvalue, raise ValueError"""
    bad = np.array([[1.0, 0.9, 0.9], [0.9, 1.0, -0.9], [0.9, -0.9, 1.0]])
    assert _raises_value_error(min_variance, np.diag(SIGMA) @ bad @ np.diag(SIGMA)), "Σ with a negative eigenvalue has no minimum: raise ValueError"


def test_equal_returns():
    """Equal expected returns leave no frontier: ValueError"""
    assert _raises_value_error(frontier_portfolio, [0.08, 0.08, 0.08], COV, 0.09), "with every μᵢ = 8%, no portfolio earns 9%: raise ValueError"


def test_wrong_length():
    """Two expected returns for three assets raise ValueError"""
    assert _raises_value_error(frontier_portfolio, [0.06, 0.08], COV, 0.07), "three assets need three expected returns"