Case study: the covariance matrix and portfolio variance

The covariance matrix

The data are invented but typical, in annual decimals: 0.12 means 12% a year.

The correlations are $\rho_{AB} = 0.3$, $\rho_{AC} = 0.1$ and $\rho_{BC} = 0.5$. As in case study 1.2.4, the volatility $\sigma_i$ is how widely asset $i$'s yearly return typically swings. The correlation $\rho_{ij}$, between $-1$ and 1, says how closely two assets move together.

The covariance of two assets is their correlation times both volatilities, $\rho_{ij}\sigma_i\sigma_j$. Module 5.5 defines it from probability; here this product is all you need. An asset's covariance with itself is its variance $\sigma_i^2$, because $\rho_{ii} = 1$.

Multiplying by $D$ on the left scales row $i$ of $R$ by $\sigma_i$; multiplying by $D$ on the right scales column $j$ by $\sigma_j$ (lesson 3.2.1). So entry $(i, j)$ of $DRD$ is $\sigma_i\,\rho_{ij}\,\sigma_j$. For the three assets:

The diagonal holds $0.12^2$, $0.18^2$ and $0.25^2$. The entry $\Sigma_{AB} = 0.3 \times 0.12 \times 0.18 = 0.00648$ appears twice, in row A and in row B.

$\Sigma$ is symmetric (lesson 3.2.3), because $\rho_{ij} = \rho_{ji}$. The transpose rule says the same: $(DRD)^\top = D^\top R^\top D^\top = DRD$, since $D$ and $R$ are symmetric. Its entries are annual variances and covariances, in squared decimals.

Portfolio variance, three ways

First way: the double sum

The portfolio's return is $R_p = \mathbf{w} \cdot \mathbf{r} = w_A r_A + w_B r_B + w_C r_C$, where $r_A$, $r_B$ and $r_C$ are the assets' returns over the year (case study 3.1.4). Its variance adds one term for every pair of assets, an asset paired with itself included:

Here $i$ and $j$ each run over A, B and C, and $\sigma_p$ is the portfolio's volatility. Module 5.5 derives the formula from the rules of variance; take it as given, as case study 1.2.4 did for two assets. With three assets there are nine terms $w_iw_j\Sigma_{ij}$:

The three diagonal terms, each asset on its own, add up to 0.009016. The six off-diagonal terms, the pairs, add 0.005244. The total is $\sigma_p^2 = 0.01426$, a volatility of $\sqrt{0.01426} \approx 0.119415$, or 11.94%.

Second way: the matrix product

The double sum is a row times a matrix times a column, as in lesson 3.2.3. First $\Sigma\mathbf{w}$: each entry is a row of $\Sigma$ dotted with $\mathbf{w}$, giving $(0.009744, 0.01746, 0.02075)$. Then $\mathbf{w}^\top(\Sigma\mathbf{w}) = 0.5 \times 0.009744 + 0.3 \times 0.01746 + 0.2 \times 0.02075 = 0.01426$.

The correlations decide where between two extremes the answer lands.

If every correlation were 0, only the diagonal would count: a variance of 0.009016 and a volatility of 9.50%. If every correlation were 1, the volatility would be the weighted average $\mathbf{w} \cdot \boldsymbol{\sigma} = 0.164$ from the hook. The actual 11.94% sits between: that gap is diversification.

Two assets: case study 1.2.4 again

Case study 1.2.4 had two assets with $\sigma_A = 0.2$, $\sigma_B = 0.3$ and $\rho = 0.2$, and weights $w$ in A and $1 - w$ in B. Its covariance matrix is $2 \times 2$:

Multiplying out $\mathbf{w}^\top\Sigma\mathbf{w}$ gives back case study 1.2.4's formula. The two off-diagonal entries, both $\rho\sigma_A\sigma_B$, are where its factor 2 comes from:

The numbers match case study 1.2.4's table:

At its minimum-variance weight $w^* \approx 0.7358$, the product gives a variance of 0.0326 and a volatility of 0.1806, as before. The parabola of case study 1.2.4 is the two-asset case of $\mathbf{w}^\top\Sigma\mathbf{w}$.

Third way: from a sample of returns

On a desk, $\Sigma$ is not handed to you: it is estimated from past returns. Here is an invented sample of 120 monthly returns, ten years' worth, drawn with a fixed seed from the model above. Its monthly means are $\boldsymbol{\mu}/12$ and its monthly covariance matrix is $\Sigma/12$.

Module 3.7 shows how such samples are made. They are illustrative, not market data.

Put the returns in a $120 \times 3$ matrix $X$: row $t$ holds the returns of A, B and C in month $t$. Then $X\mathbf{w}$ dots each row with $\mathbf{w}$: the 120 monthly returns of the portfolio. The first three are 0.0095, $-0.0369$ and 0.0311.

The sample covariance of two columns is the dot product of their centred vectors divided by $n - 1$, for $n$ months, as in case study 3.1.4. The sample covariance matrix $S$ collects these for every pair of columns: it does for the sample what $\Sigma$ does for the model. np.cov computes it.

The sample variance of the portfolio's returns then equals $\mathbf{w}^\top S\mathbf{w}$, exactly, for any sample: both are 0.000973 a month.

Times 12, that is 0.011679 a year, a volatility of 10.81%: not 11.94%. The sample is only an estimate of the model, and ten years are not enough to pin $\Sigma$ down. Compare the annualised $S$ with $\Sigma$:

The sample volatilities are 11.13%, 16.91% and 20.59%, against 12%, 18% and 25%. np.corrcoef gives correlations 0.308, 0.181 and 0.419, against 0.3, 0.1 and 0.5; each is the cosine of an angle between centred return vectors (case study 3.1.4).

Risk contributions

Which asset carries the risk? The last step of $\mathbf{w}^\top(\Sigma\mathbf{w})$ is a dot product, so the variance splits into one term per asset.

Here $\Sigma\mathbf{w} = (0.009744, 0.01746, 0.02075)$, so the contributions are 0.004872, 0.005238 and 0.00415, which add up to 0.01426. As shares of the variance: 34.17%, 36.73% and 29.10%.

Divided by $\sigma_p$, the contributions add up to the volatility instead: 4.08, 4.39 and 3.48 percentage points of the 11.94%.

A contribution can be negative. Sell C short, with $\mathbf{w} = (0.7, 0.5, -0.2)$, which still adds up to 1. The variance is then 0.016852, a volatility of 12.98%.

C still moves with the rest of the portfolio, $(\Sigma\mathbf{w})_C = 0.00085$, but its weight is negative. So its contribution is $-0.00017$: the short position offsets part of the risk of A and B.

One more pattern links back to case study 1.2.4. At its minimum-variance weight, both entries of $\Sigma\mathbf{w}$ equal 0.0326, the variance itself. Module 4.5 shows that every minimum-variance portfolio has this property.

On the desk

So treat $\Sigma$ as an estimate with error bars, and a portfolio built on it as no more precise than its inputs. Module 3.5 uses covariances to find the minimum-variance hedge, and module 3.6 finds the main directions of risk in the covariance matrix of yield changes.

import numpy as np

SIGMA = [0.12, 0.18, 0.25]
R = [[1.0, 0.3, 0.1], [0.3, 1.0, 0.5], [0.1, 0.5, 1.0]]
W = [0.5, 0.3, 0.2]
EXPECTED = [[0.0144, 0.00648, 0.003], [0.00648, 0.0324, 0.0225], [0.003, 0.0225, 0.0625]]


def _raises_value_error(fn, *args):
    try:
        fn(*args)
    except ValueError:
        return True
    return False


def test_covariance_matrix():
    """The three assets: Σ has variances 0.0144, 0.0324, 0.0625 and covariances 0.00648, 0.003, 0.0225"""
    expect_close(covariance_matrix(SIGMA, R), EXPECTED, what="covariance_matrix(sigma, R)")


def test_two_assets():
    """Case study 1.2.4's two assets give [[0.04, 0.012], [0.012, 0.09]]"""
    expect_close(covariance_matrix([0.2, 0.3], [[1, 0.2], [0.2, 1]]), [[0.04, 0.012], [0.012, 0.09]], what="two-asset covariance matrix")


def test_portfolio_variance():
    """The portfolio (0.5, 0.3, 0.2) has variance 0.01426"""
    expect_close(portfolio_variance(W, EXPECTED), 0.01426, what="portfolio_variance(w, Sigma)")


def test_case_study_124():
    """With σ_A = 0.2, σ_B = 0.3, ρ = 0.2: variance 0.032625 at w = 0.75, and case study 1.2.4's minimum at w*"""
    cov = [[0.04, 0.012], [0.012, 0.09]]
    expect_close(portfolio_variance([0.75, 0.25], cov), 0.032625, what="variance at w = 0.75")
    w_star = 0.156 / 0.212
    expect_close(portfolio_variance([w_star, 1 - w_star], cov), 0.09 - 0.156**2 / (4 * 0.106), what="variance at w*")


def test_contributions():
    """The risk contributions are 0.004872, 0.005238 and 0.00415, and add up to the variance"""
    rc = np.asarray(risk_contributions(W, EXPECTED), dtype=float)
    expect_close(rc, [0.004872, 0.005238, 0.00415], what="risk_contributions(w, Sigma)")
    expect_close(rc.sum(), 0.01426, what="sum of the risk contributions")


def test_short_position():
    """Short 20% of C: C's risk contribution is negative, about −0.00017"""
    rc = np.asarray(risk_contributions([0.7, 0.5, -0.2], EXPECTED), dtype=float)
    expect_close(rc, [0.008904, 0.008118, -0.00017], what="risk contributions with C sold short")


def test_sample():
    """For a sample of returns, the variance of the portfolio's returns equals wᵀSw with S from np.cov"""
    rng = np.random.default_rng(5)
    X = rng.normal(0.0, 0.05, size=(60, 3))
    w = np.array(W)
    expect_close(portfolio_variance(w, np.cov(X, rowvar=False)), np.var(X @ w, ddof=1), what="wᵀSw against the sample variance", rel=1e-9)


def test_correlation_above_one():
    """A correlation of 1.2 raises ValueError"""
    bad = [[1.0, 1.2, 0.1], [1.2, 1.0, 0.5], [0.1, 0.5, 1.0]]
    assert _raises_value_error(covariance_matrix, SIGMA, bad), "a correlation of 1.2 should raise ValueError"


def test_correlation_not_symmetric():
    """A correlation matrix with ρ_AB = 0.3 in one place and 0.03 in the other raises ValueError"""
    bad = [[1.0, 0.3, 0.1], [0.03, 1.0, 0.5], [0.1, 0.5, 1.0]]
    assert _raises_value_error(covariance_matrix, SIGMA, bad), "a correlation matrix that is not symmetric should raise ValueError"


def test_diagonal_not_one():
    """A correlation matrix with 0.9 on its diagonal raises ValueError"""
    bad = [[0.9, 0.3, 0.1], [0.3, 1.0, 0.5], [0.1, 0.5, 1.0]]
    assert _raises_value_error(covariance_matrix, SIGMA, bad), "a diagonal entry other than 1 should raise ValueError"


def test_sizes_differ():
    """Three volatilities with a 2 × 2 correlation matrix raise ValueError"""
    assert _raises_value_error(covariance_matrix, SIGMA, [[1, 0.3], [0.3, 1]]), "three volatilities need a 3 × 3 correlation matrix"


def test_negative_volatility():
    """A negative volatility raises ValueError"""
    assert _raises_value_error(covariance_matrix, [0.12, -0.18, 0.25], R), "a volatility of -0.18 should raise ValueError"


def test_covariance_not_symmetric():
    """portfolio_variance refuses a covariance matrix with a typo in one triangle"""
    typo = [row[:] for row in EXPECTED]
    typo[0][1] = 0.0648
    assert _raises_value_error(portfolio_variance, W, typo), "a covariance matrix that is not symmetric should raise ValueError"


def test_weights_wrong_length():
    """Two weights with a 3 × 3 covariance matrix raise ValueError"""
    assert _raises_value_error(portfolio_variance, [0.5, 0.5], EXPECTED), "two weights cannot go with a 3 × 3 matrix"