3.6.1 Eigenvalues and eigenvectors

Directions a matrix only stretches

Lesson 3.2.2 showed that a matrix moves almost every vector to a new direction. A few directions are special: the matrix only stretches them. This lesson finds them.

Take the symmetric matrix of lesson 3.2.3,

S = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}

and see where it sends a few vectors:

$\mathbf{x}$	$(1, 0)$	$(1, 2)$	$(1, 1)$	$(2, 2)$	$(-1, -1)$	$(1, -1)$	$(-2, 2)$
$S\mathbf{x}$	$(2, 1)$	$(4, 5)$	$(3, 3)$	$(6, 6)$	$(-3, -3)$	$(1, -1)$	$(-2, 2)$

The vector $(1, 0)$ lands on $(2, 1)$ , a new direction. But $(1, 1)$ lands on $(3, 3) = 3\,(1, 1)$ : the same direction, three times as long. Every point of that line does the same, such as $(2, 2)$ and $(-1, -1)$ .

On the line through $(1, -1)$ , $S$ changes nothing at all: $(1, -1)$ lands on itself, and so does $(-2, 2)$ .

Rule: Eigenvector and eigenvalue

A\mathbf{v} = \lambda\mathbf{v}, \quad \mathbf{v} \neq \mathbf{0}

A nonzero vector $\mathbf{v}$ with $A\mathbf{v} = \lambda\mathbf{v}$ is an eigenvector of the square matrix $A$ , and the number $\lambda$ (lambda) is its eigenvalue. $A$ keeps the line through $\mathbf{v}$ and multiplies every vector on it by $\lambda$ .

So $S$ has the eigenvector $(1, 1)$ with eigenvalue 3, and $(1, -1)$ with eigenvalue 1. "Eigen" is German for "own": these are the matrix's own directions.

The eigenvalue says what happens along the line:

$\lambda > 1$ stretches it, $0 < \lambda < 1$ shrinks it, and $\lambda = 1$ leaves it alone.
$\lambda < 0$ reverses it, and stretches or shrinks by $\lvert \lambda \rvert$ .
$\lambda = 0$ squashes the whole line onto $\mathbf{0}$ .

The zero vector never counts, because $A\mathbf{0} = \mathbf{0} = \lambda\mathbf{0}$ for every $\lambda$ . Any nonzero multiple of an eigenvector is an eigenvector with the same $\lambda$ , so an eigenvector is really a whole line, an eigenvector line.

An eigenvalue is a scale factor, so it has the units of the entries of $A$ . Here those are plain numbers; in case study 3.6.5 the entries are variances, and so are the eigenvalues.

$S$ turns $(1, 0)$ through 26.6°, but it keeps the two dashed lines. Along one it stretches by 3, along the other by 1.

The explorer starts on $S$ , with its two eigenvector lines dashed, and the last line of its readout lists the eigenvalues, 3 and 1. Drag the probe $\mathbf{x}$ from $(1, 2)$ onto $(1, 1)$ : the readout says $\mathbf{x}$ is an eigenvector, stretched by 3.

Try $(-1, 1)$ , then press Shear and Rotate 90°. Count the dashed lines that remain, and read the eigenvalues each time.

Explore

Pick a matrix or set its entries. Its first column is where e₁ lands and its second where e₂ lands; the whole grid follows. Then drag the probe x and look for directions that A only stretches.

a (row 1, column 1) = 2

b (row 1, column 2) = 1

c (row 2, column 1) = 1

d (row 2, column 2) = 2

Probe x = (1, 2) (drag its tip)Show eigenvector lines

Matrix: $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$
Columns: Ae₁ = (2, 1), Ae₂ = (1, 2)
det A: 2 × 2 − 1 × 1 = 3: every area is multiplied by 3.
Probe: Ax = (4, 5): a different direction.
Eigenvalues: λ = 3 and 1

S\mathbf{x}

import numpy as np S = np.array([[2, 1], [1, 2]]) for p in [(1, 0), (1, 2), (1, 1), (2, 2), (-1, -1), (1, -1), (-2, 2)]: x = np.array(p) y = S @ x same_line = x[0] * y[1] - x[1] * y[0] == 0 print(x, y, same_line)

⌘/Ctrl + Enter runs the cell

Finding the eigenvalues

Start from $A\mathbf{v} = \lambda\mathbf{v}$ . Write $\lambda\mathbf{v}$ as $\lambda I\mathbf{v}$ , since $I\mathbf{v} = \mathbf{v}$ (lesson 3.2.3), and move it to the left: $(A - \lambda I)\mathbf{v} = \mathbf{0}$ .

So $A - \lambda I$ must send a nonzero vector to $\mathbf{0}$ . A matrix that does that flattens the plane, so its determinant is 0 (lesson 3.4.1). That gives an equation for $\lambda$ alone.

Rule: The eigenvalue equation

\det(A - \lambda I) = 0

$\lambda$ is an eigenvalue of $A$ exactly when $A - \lambda I$ is singular. $A - \lambda I$ subtracts $\lambda$ from the diagonal entries only.

For a $2 \times 2$ matrix with entries $a$ , $b$ , $c$ , $d$ , multiply out the determinant:

\begin{aligned} &\det(A - \lambda I) \\ &= (a - \lambda)(d - \lambda) - bc \\ &= \lambda^2 - (a + d)\lambda + (ad - bc) \end{aligned}

This quadratic in $\lambda$ is the characteristic polynomial of $A$ . Its roots are the eigenvalues: factorise it, or use the quadratic formula of lesson 1.2.2.

Example 1. For $S$ , the polynomial is $(2 - \lambda)^2 - 1 = \lambda^2 - 4\lambda + 3 = (\lambda - 1)(\lambda - 3)$ . So the eigenvalues are 3 and 1, the stretch factors of the table.

A table of $\det(S - \lambda I)$ shows the same roots:

$\lambda$	0	1	2	3	4
$\det(S - \lambda I)$	3	0	$-1$	0	3

At $\lambda = 2$ , $S - 2I = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}$ has determinant $-1$ , not 0. So no vector is doubled by $S$ .

Example 2. Take

A = \begin{pmatrix} 1 & 2 \\ 3 & 2 \end{pmatrix}

Its polynomial is $(1 - \lambda)(2 - \lambda) - 6 = \lambda^2 - 3\lambda - 4 = (\lambda - 4)(\lambda + 1)$ . The eigenvalues are 4 and $-1$ .

\det(S - \lambda I)

import numpy as np S = np.array([[2, 1], [1, 2]]) I = np.eye(2) for lam in [0, 1, 2, 3, 4]: print(lam, round(np.linalg.det(S - lam * I), 10) + 0.0) A = np.array([[1, 2], [3, 2]]) for M in (S, A): coefficients = [1, -int(np.trace(M)), round(np.linalg.det(M))] roots = np.sort(np.roots(coefficients))[::-1] print(coefficients, np.round(roots, 6) + 0.0)

⌘/Ctrl + Enter runs the cell

Finding the eigenvectors

With an eigenvalue $\lambda$ in hand, solve $(A - \lambda I)\mathbf{v} = \mathbf{0}$ . Because $A - \lambda I$ is singular, its two rows are multiples of each other, and only one equation is left.

One equation in two unknowns has one free variable, so its solutions form a line through $\mathbf{0}$ (lesson 3.3.2). That line is the null space of $A - \lambda I$ (lesson 3.5.1), and every nonzero point on it is an eigenvector.

matrix	$\lambda$	$A - \lambda I$	equation	eigenvector
$S$	3	$\begin{pmatrix} -1 & 1 \\ 1 & -1 \end{pmatrix}$	$v_2 = v_1$	$(1, 1)$
$S$	1	$\begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix}$	$v_2 = -v_1$	$(1, -1)$
$A$	4	$\begin{pmatrix} -3 & 2 \\ 3 & -2 \end{pmatrix}$	$3v_1 = 2v_2$	$(2, 3)$
$A$	$-1$	$\begin{pmatrix} 2 & 2 \\ 3 & 3 \end{pmatrix}$	$v_2 = -v_1$	$(1, -1)$

Check by multiplying: $A(2, 3) = (8, 12) = 4\,(2, 3)$ . And $A(1, -1) = (-1, 1)$ : the line through $(1, -1)$ is reversed, as an eigenvalue of $-1$ says.

Rule: Eigenvectors of a 2 × 2 matrix

\begin{gathered} (A - \lambda I)\,\mathbf{v} = \mathbf{0} \\[4pt] \mathbf{v} = (b,\ \lambda - a) \end{gathered}

Row 1 of $A - \lambda I$ is $(a - \lambda,\ b)$ , and $\mathbf{v} = (b, \lambda - a)$ solves it. If that gives $\mathbf{0}$ , use row 2, $(c,\ d - \lambda)$ , the same way: $\mathbf{v} = (\lambda - d,\ c)$ .

For $A$ with $\lambda = 4$ the shortcut gives $(2, 3)$ ; with $\lambda = -1$ it gives $(2, -2)$ , a multiple of $(1, -1)$ .

The two eigenvectors of $A$ are not perpendicular: $(2, 3) \cdot (1, -1) = -1$ (lesson 3.1.2). Those of $S$ are, since $(1, 1) \cdot (1, -1) = 0$ . Lesson 3.6.3 shows that every symmetric matrix has perpendicular eigenvectors.

\lambda

import numpy as np def eigenvector_2x2(M, lam): """A solution of (M - lam I) v = 0, from row 1: v = (b, lam - a).""" a, b = M[0] return np.array([b, lam - a]) S = np.array([[2, 1], [1, 2]]) A = np.array([[1, 2], [3, 2]]) for M, lam in [(S, 3), (S, 1), (A, 4), (A, -1)]: v = eigenvector_2x2(M, lam) print(lam, v, M @ v, lam * v) print(np.array([2, 3]) @ np.array([1, -1]), np.array([1, 1]) @ np.array([1, -1]))

⌘/Ctrl + Enter runs the cell

Trace and determinant

The trace of a square matrix, $\operatorname{tr} A$ , is the sum of its diagonal entries: $a + d$ for a $2 \times 2$ matrix. So the characteristic polynomial is $\lambda^2 - (\operatorname{tr} A)\lambda + \det A$ .

If its roots are $\lambda_1$ and $\lambda_2$ , it also equals $(\lambda - \lambda_1)(\lambda - \lambda_2) = \lambda^2 - (\lambda_1 + \lambda_2)\lambda + \lambda_1\lambda_2$ . Matching the two forms gives a check on any answer.

Rule: Trace and determinant

\begin{gathered} \operatorname{tr} A = \lambda_1 + \lambda_2 \\[4pt] \det A = \lambda_1 \lambda_2 \end{gathered}

The eigenvalues add up to the trace and multiply to the determinant. The same holds for an $n \times n$ matrix, with all $n$ eigenvalues and a repeated one counted twice; lesson 3.6.3 checks a $3 \times 3$ one.

matrix	$\operatorname{tr}$	$\det$	eigenvalues	sum	product
$S$	4	3	3 and 1	4	3
$A$	3	$-4$	4 and $-1$	3	$-4$
$Z = \begin{pmatrix} 1 & 2 \\ 0.5 & 1 \end{pmatrix}$	2	0	2 and 0	2	0

The product is the area factor of lesson 3.4.1. $S$ stretches by 3 along one eigenvector line and by 1 along the other, so it multiplies every area by 3. A negative product, as for $A$ , means one line is reversed: the plane is flipped.

A zero eigenvalue. $\det Z = 0$ , so one eigenvalue of the singular matrix $Z$ (lesson 3.2.3) is 0, and the other is the trace, 2. Its eigenvectors are $(2, 1)$ , sent to $(4, 2)$ , and $(2, -1)$ , sent to $\mathbf{0}$ : the direction that $Z$ flattens.

(2, 1)

import numpy as np matrices = { "S": [[2, 1], [1, 2]], "A": [[1, 2], [3, 2]], "Z": [[1, 2], [0.5, 1]], } for name, M in matrices.items(): M = np.array(M) values = np.sort(np.linalg.eig(M).eigenvalues.real)[::-1] values = np.round(values, 10) + 0.0 print(name, np.trace(M), round(np.linalg.det(M), 10) + 0.0, values, values.sum(), values.prod()) Z = np.array([[1, 2], [0.5, 1]]) print(Z @ np.array([2, 1]), Z @ np.array([2, -1]))

⌘/Ctrl + Enter runs the cell

Repeated and complex eigenvalues

The characteristic polynomial is a quadratic, so its discriminant (lesson 1.2.2) sorts the cases. Here it is $(\operatorname{tr} A)^2 - 4\det A$ .

Rule: Three cases for a 2 × 2 matrix

\begin{gathered} (\operatorname{tr} A)^2 - 4\det A > 0: \text{two real} \\ = 0: \text{one, repeated} \\ < 0: \text{a complex pair} \end{gathered}

For $S$ the discriminant is $16 - 12 = 4$ : two real eigenvalues. A repeated eigenvalue can come with one eigenvector line or with every line; a complex pair comes with no real line at all.

One eigenvalue, twice

The shear of lesson 3.2.2, $H = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$ , has polynomial $(1 - \lambda)^2$ , so $\lambda = 1$ twice. Then $H - I = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$ says only $v_2 = 0$ : the $x_1$ -axis. The shear has one eigenvector line, and every other direction leans over.

$H$ is triangular: every entry on one side of its diagonal, here below it, is 0. The determinant of a triangular matrix is the product of its diagonal (lesson 3.4.1), so its eigenvalues are its diagonal entries.

A repeated eigenvalue can also keep every direction. $2I$ has polynomial $(2 - \lambda)^2$ too, but $2I - 2I$ is the zero matrix, so every nonzero vector is an eigenvector: $2I$ stretches the whole plane evenly.

No real direction: rotations

The quarter turn $R = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ has trace 0 and determinant 1, so its polynomial is $\lambda^2 + 1$ . That has no real root: every vector turns through 90°, so no line is kept.

With complex numbers (lesson 1.7.2), $\lambda^2 = -1$ has the roots $i$ and $-i$ . Multiplying by $i$ is a quarter turn, which is exactly what $R$ does.

A turn through any angle $\theta$ has trace $2\cos\theta$ and determinant $\cos^2\theta + \sin^2\theta = 1$ . Its eigenvalues are $\cos\theta \pm i\sin\theta = e^{\pm i\theta}$ (lesson 1.7.3): the angle of the turn is the angle of the eigenvalues. For 30° they are $0.866 \pm 0.5i$ .

Complex eigenvalues of a real matrix always come as a conjugate pair, like the roots of a real quadratic (lesson 1.7.2). A turn has real eigenvalues only at 0°, where it is $I$ , and at 180°, where it is $-I$ .

2I

import cmath import numpy as np def discriminant(M): return np.trace(M) ** 2 - 4 * np.linalg.det(M) H = np.array([[1, 1], [0, 1]]) # shear R = np.array([[0, -1], [1, 0]]) # quarter turn t = np.radians(30) R30 = np.array([[np.cos(t), -np.sin(t)], [np.sin(t), np.cos(t)]]) for name, M in [("shear", H), ("2I", 2 * np.eye(2)), ("quarter turn", R), ("30 degrees", R30)]: values = np.linalg.eig(M).eigenvalues.astype(complex) print(name, round(discriminant(M), 10) + 0.0, np.round(values, 4) + 0.0) values, V = np.linalg.eig(H) V = V.real * np.sign(V.real[0]) # make each first entry positive print(np.round(V, 4) + 0.0) print(np.round(cmath.exp(1j * t), 4))

⌘/Ctrl + Enter runs the cell

Eigenvalues in NumPy

np.linalg.eig(A) returns the eigenvalues of $A$ , and a matrix whose columns are eigenvectors: column $j$ goes with eigenvalue $j$ . Each column has length 1.

Three habits make its answers safe to print and compare:

Fix the order. eig lists the eigenvalues in no particular order, so sort them, largest first, and reorder the columns to match.
Fix the sign. An eigenvector line has two directions of length 1, and another computer may return $-\mathbf{v}$ . Flip each column so that, say, its first entry is positive.
Real or complex. Depending on the NumPy version, real eigenvalues may come back as complex numbers such as 3+0j. When you know they are real, .real gives plain numbers.

For $S$ the columns are $(0.7071, 0.7071)$ and $(0.7071, -0.7071)$ : the eigenvectors $(1, 1)$ and $(1, -1)$ , scaled to length 1, since $\frac{1}{\sqrt{2}} \approx 0.7071$ . For $A$ they are $(0.5547, 0.8321)$ , which is $(2, 3)$ scaled, and $(0.7071, -0.7071)$ .

Dividing each column by its first entry gives $(1, 1.5)$ for $A$ , half of $(2, 3)$ . Lesson 3.6.2 puts these columns side by side to compute powers of $A$ , and case study 3.6.5 finds them for a covariance matrix.

A\mathbf{v} = \lambda\mathbf{v}

import numpy as np for M in (np.array([[2, 1], [1, 2]]), np.array([[1, 2], [3, 2]])): values, V = np.linalg.eig(M) values, V = values.real, V.real # every eigenvalue here is real order = np.argsort(values)[::-1] # largest first values, V = values[order], V[:, order] V = V * np.sign(V[0]) # first entry of each column positive print(np.round(values, 6) + 0.0) print(np.round(V, 4)) print(np.allclose(M @ V, V * values), np.round(V / V[0], 4))

⌘/Ctrl + Enter runs the cell

Common mistakes

The zero vector is never an eigenvector, but 0 can be an eigenvalue: $Z$ sends $(2, -1)$ to $\mathbf{0}$ .
Subtract $\lambda$ on the diagonal only. $A - \lambda I$ changes $a$ and $d$ , not $b$ and $c$ .
An eigenvector is a line. $(1, 1)$ , $(2, 2)$ and $(-0.7071, -0.7071)$ are the same answer for $S$ ; check by multiplying, not by matching numbers.
The diagonal entries are the eigenvalues only for a triangular matrix. $S$ has 2 and 2 on its diagonal, but eigenvalues 3 and 1.
A repeated eigenvalue may have only one line. The shear keeps just the $x_1$ -axis.
No real root does not mean no eigenvalues. A turn has a complex pair, $e^{\pm i\theta}$ , and no real eigenvector line.
eig returns eigenvectors as columns, in no fixed order and with either sign. Sort, fix the signs, and use .real when the values are real.

Eigenvalues and eigenvectors

0 of 12 correct

1.
$(1, 1)$ is an eigenvector of $\begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$ . What is its eigenvalue?
2.
A matrix sends $\mathbf{v} = (2, -1)$ to $(-6, 3)$ . What is the eigenvalue of $\mathbf{v}$ ?
3.
Write the characteristic polynomial $\det(A - \lambda I)$ of $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$ , multiplied out. Type x for $\lambda$
4.
Find the larger eigenvalue of $\begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$
5.
For $\lambda = 2$ , an eigenvector of $\begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$ is $(1, v_2)$ . Find $v_2$
6.
Find the smaller eigenvalue of $\begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$
7.
A $2 \times 2$ matrix has eigenvalues 6 and $-2$ . What is its determinant?
8.
A $2 \times 2$ matrix has trace 9, and one eigenvalue is 4. What is the other?
9.
Find the larger eigenvalue of $\begin{pmatrix} 3 & 5 \\ 0 & -2 \end{pmatrix}$
10.
$\begin{pmatrix} 2 & 4 \\ 1 & 2 \end{pmatrix}$ is singular. What is its other eigenvalue, besides 0?
11.
How many eigenvector lines does $\begin{pmatrix} 2 & 1 \\ 0 & 2 \end{pmatrix}$ have?
12.
Give the eigenvalue of $\begin{pmatrix} 1 & -2 \\ 2 & 1 \end{pmatrix}$ with a positive imaginary part, as $a + bi$

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

A

Eigenvalues and eigenvectors

Directions a matrix only stretches

Rule: Eigenvector and eigenvalue

Finding the eigenvalues

Rule: The eigenvalue equation

Finding the eigenvectors

Rule: Eigenvectors of a 2 × 2 matrix

Trace and determinant

Rule: Trace and determinant

Repeated and complex eigenvalues

Rule: Three cases for a 2 × 2 matrix

One eigenvalue, twice

No real direction: rotations

Eigenvalues in NumPy

Common mistakes

Eigenvalues and eigenvectors

Summary card

Free further reading