3.5.1 Span, independence and basis

The span of a set of vectors

A linear combination of vectors $\mathbf{v}_1, \ldots, \mathbf{v}_k$ is $c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k$ , for any numbers $c_1, \ldots, c_k$ (lesson 3.1.1). The span of the vectors is the set of all their linear combinations. Lesson 3.1.1 asked which points combinations of two vectors can reach: the span is the answer.

Rule: Span

\begin{gathered} \operatorname{span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \\[4pt] = \{c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k\} \end{gathered}

Every combination, with any numbers $c_1$ to $c_k$ . Setting every $c_i = 0$ gives $\mathbf{0}$ , so a span always contains the origin.

What the span looks like depends on how the vectors point:

One non-zero vector: its multiples, a line through the origin. The span of $(1, 2)$ holds $(-1, -2)$ , $(0.5, 1)$ and $(2, 4)$ .
Two vectors in the plane pointing different ways: the whole plane, $\mathbb{R}^2$ . With $\mathbf{u} = (3, 1)$ and $\mathbf{v} = (1, 2)$ , lesson 3.1.1 reached $(5, 5) = \mathbf{u} + 2\mathbf{v}$ and $(1, -3) = \mathbf{u} - 2\mathbf{v}$ .
Two vectors along one line, such as $(1, 2)$ and $(2, 4)$ : only that line. Lesson 3.3.2 drew this case: $(3, 6)$ is reached and $(3, 5)$ is not.
The zero vector alone: only the origin.

A plane in space

In $\mathbb{R}^3$ , take $\mathbf{a}_1 = (1, 0, 1)$ and $\mathbf{a}_2 = (0, 1, 1)$ . A combination is

s\,\mathbf{a}_1 + t\,\mathbf{a}_2 = \begin{pmatrix} s \\ t \\ s + t \end{pmatrix}

So the span is the set of points whose third component is the sum of the first two: a plane through the origin. $(1, 2, 3) = \mathbf{a}_1 + 2\mathbf{a}_2$ lies in it, and $(1, 2, 0)$ does not, since $1 + 2 \neq 0$ .

The span of $\mathbf{a}_1$ and $\mathbf{a}_2$ is a plane through the origin. Their sum stays in the plane and adds nothing new. $\mathbf{c} = (1, 1, 0)$ lies 2 below it, so adding $\mathbf{c}$ reaches all of space.

Lesson 3.3.2 gives a test that needs no picture. A vector $\mathbf{b}$ is in the span exactly when $A\mathbf{x} = \mathbf{b}$ has a solution, where $A$ has the vectors as its columns. So adding $\mathbf{b}$ as an extra column must not raise the rank.

s\mathbf{a}_1 + t\mathbf{a}_2

import numpy as np a1 = np.array([1, 0, 1]) a2 = np.array([0, 1, 1]) for s, t in [(1, 2), (2, -1), (0.5, 0.5)]: print(s, t, s * a1 + t * a2) rank = np.linalg.matrix_rank A = np.column_stack([a1, a2]) for b in [[1, 2, 3], [1, 2, 0]]: print(b, rank(A), rank(np.column_stack([A, b])))

⌘/Ctrl + Enter runs the cell

Linear independence

Vectors are linearly independent when none of them is a combination of the others. Otherwise they are dependent: at least one of them adds nothing to the span.

$(3, 1)$ and $(1, 2)$ are independent: neither is a multiple of the other.
$(1, 2)$ and $(2, 4)$ are dependent, since $(2, 4) = 2(1, 2)$ . Lesson 3.3.2 met them as the columns of a singular matrix.
$\mathbf{u}$ , $\mathbf{v}$ and $(5, 5)$ are dependent, since $(5, 5) = \mathbf{u} + 2\mathbf{v}$ . Any three vectors in $\mathbb{R}^2$ are.
$\mathbf{a}_1$ , $\mathbf{a}_2$ and $\mathbf{a}_1 + \mathbf{a}_2$ are dependent; $\mathbf{a}_1$ , $\mathbf{a}_2$ and $\mathbf{c} = (1, 1, 0)$ are independent. The figure shows why.
Any set that contains $\mathbf{0}$ is dependent: $\mathbf{0} = 0 \cdot \mathbf{u}$ is a combination of the others.

Dependence also shows as a combination that gives $\mathbf{0}$ without every coefficient being 0: $2(1, 2) - (2, 4) = \mathbf{0}$ . For independent vectors, the only combination that gives $\mathbf{0}$ is $0\mathbf{v}_1 + \cdots + 0\mathbf{v}_k$ .

Two tests

Put the $k$ vectors in the columns of a matrix $A$ . Elimination (lesson 3.3.2) finds a pivot in every column exactly when no column is a combination of the others.

Rule: Testing independence

\begin{gathered} \text{independent} \iff \operatorname{rank} A = k \\[4pt] \text{for } k = n\text{: } \det A \neq 0 \end{gathered}

$A$ has the $k$ vectors as columns, each with $n$ components. The determinant test (lesson 3.4.1) needs a square matrix: $n$ vectors in $\mathbb{R}^n$ . With more vectors than components, $k > n$ , the rank is at most $n$ , so they are always dependent.

vectors	rank	count	independent?
$(3, 1)$ , $(1, 2)$	2	2	yes, $\det = 5$
$(1, 2)$ , $(2, 4)$	1	2	no
$(3, 1)$ , $(1, 2)$ , $(5, 5)$	2	3	no
$\mathbf{a}_1$ , $\mathbf{a}_2$ , $\mathbf{a}_1 + \mathbf{a}_2$	2	3	no, $\det = 0$
$\mathbf{a}_1$ , $\mathbf{a}_2$ , $(1, 1, 0)$	3	3	yes, $\det = -2$
$(3, 1)$ , $\mathbf{0}$	1	2	no

Try it Each set becomes the columns of a matrix. The loop prints its rank beside the number of vectors: independent exactly when they match. The last loop prints the determinants of the three square cases. Adding 0.0 turns a -0.0 into 0.0, as in lesson 3.4.1.

import numpy as np rank = np.linalg.matrix_rank sets = { "u, v": [[3, 1], [1, 2]], "(1, 2), (2, 4)": [[1, 2], [2, 4]], "u, v, (5, 5)": [[3, 1], [1, 2], [5, 5]], "a1, a2, a1 + a2": [[1, 0, 1], [0, 1, 1], [1, 1, 2]], "a1, a2, c": [[1, 0, 1], [0, 1, 1], [1, 1, 0]], "u, 0": [[3, 1], [0, 0]], } for name, vectors in sets.items(): A = np.column_stack(vectors) print(name, "| rank", rank(A), "| count", A.shape[1]) for name in ["u, v", "a1, a2, c", "a1, a2, a1 + a2"]: A = np.column_stack(sets[name]) print(name, round(np.linalg.det(A), 10) + 0.0)

⌘/Ctrl + Enter runs the cell

Basis and dimension

A basis of a span is a set of independent vectors that spans it. It has no spare vectors: drop one and the span shrinks.

$\mathbf{e}_1 = (1, 0)$ and $\mathbf{e}_2 = (0, 1)$ are the standard basis of $\mathbb{R}^2$ ; $\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3$ are that of $\mathbb{R}^3$ .
$\mathbf{u} = (3, 1)$ and $\mathbf{v} = (1, 2)$ are another basis of $\mathbb{R}^2$ . Any two independent vectors in the plane are.
$\mathbf{a}_1$ and $\mathbf{a}_2$ are a basis of the plane in the figure. Adding $\mathbf{c}$ gives a basis of all of $\mathbb{R}^3$ .

From a dependent set, drop each vector that is a combination of the others until the rest are independent. $\mathbf{a}_1$ , $\mathbf{a}_2$ and $\mathbf{a}_1 + \mathbf{a}_2$ span the same plane as $\mathbf{a}_1$ and $\mathbf{a}_2$ alone.

Every basis of the same span has the same number of vectors, its dimension. $\mathbb{R}^n$ has dimension $n$ , a line through the origin dimension 1 and a plane dimension 2.

Rule: Dimension of a span

\dim \operatorname{span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} = \operatorname{rank} A

$A$ has the vectors as columns. The rank counts the independent columns, so it counts the vectors a basis needs. It is at most $k$ , and at most $n$ .

For example, $(1, 2, 3)$ , $(2, 4, 6)$ and $(0, 1, 1)$ have rank 2: the second is twice the first. So they span a plane, with $(1, 2, 3)$ and $(0, 1, 1)$ as a basis.

\mathbf{a}_1 + \mathbf{a}_2

import numpy as np rank = np.linalg.matrix_rank print(rank(np.column_stack([[1, 2, 3], [2, 4, 6], [0, 1, 1]]))) a1, a2 = np.array([1, 0, 1]), np.array([0, 1, 1]) print(rank(np.column_stack([a1, a2, a1 + a2])), rank(np.column_stack([a1, a2]))) print(rank(np.column_stack([a1, a2, [1, 1, 0]])), rank(np.eye(3)))

⌘/Ctrl + Enter runs the cell

Coordinates in a basis

A basis spans its space and has no spare vectors. So every vector in the space is exactly one combination of the basis vectors. The numbers in that combination are the vector's coordinates in that basis.

The components used so far are coordinates in the standard basis: $(4, -2) = 4\mathbf{e}_1 - 2\mathbf{e}_2$ . In the basis $\mathbf{u} = (3, 1)$ , $\mathbf{v} = (1, 2)$ , the same point is $2\mathbf{u} - 2\mathbf{v}$ , so its coordinates are $(2, -2)$ .

One point, two grids. In the standard basis $\mathbf{x} = 4\mathbf{e}_1 - 2\mathbf{e}_2$ ; in the basis $\mathbf{u}, \mathbf{v}$ it is $2\mathbf{u} - 2\mathbf{v}$ .

To find coordinates $c_1, c_2$ , solve $c_1\mathbf{u} + c_2\mathbf{v} = \mathbf{x}$ . That is a linear system $B\mathbf{c} = \mathbf{x}$ , where $B$ has the basis vectors as its columns (lesson 3.3.1):

\begin{aligned} 3c_1 + c_2 &= 4 \\ c_1 + 2c_2 &= -2 \end{aligned}

Twice the first minus the second gives $5c_1 = 10$ , so $c_1 = 2$ and then $c_2 = -2$ . Check: $2(3, 1) - 2(1, 2) = (4, -2)$ .

$\mathbf{x}$ in the standard basis	$(5, 5)$	$(7, 4)$	$(1, -3)$	$(4, -2)$	$(1, 0)$
coordinates in $\mathbf{u}, \mathbf{v}$	$(1, 2)$	$(2, 1)$	$(1, -2)$	$(2, -2)$	$(0.4, -0.2)$

Whole-number points can have fractional coordinates: $\mathbf{e}_1 = 0.4\mathbf{u} - 0.2\mathbf{v}$ . The order of the basis matters too: in the basis $\mathbf{v}, \mathbf{u}$ , the point $(4, -2)$ has coordinates $(-2, 2)$ .

Rule: Coordinates in a basis

\begin{gathered} \mathbf{x} = c_1\mathbf{b}_1 + \cdots + c_n\mathbf{b}_n \\[4pt] \mathbf{c} = B^{-1}\mathbf{x} \end{gathered}

$B$ has the basis vectors $\mathbf{b}_1, \ldots, \mathbf{b}_n$ as columns. Its columns are independent, so it has an inverse (lesson 3.4.1). Compute $\mathbf{c}$ with np.linalg.solve(B, x), as lesson 3.2.3 advises.

Only a basis gives coordinates that exist and are unique. With dependent vectors a point is reached in many ways, as $(3, 6)$ was in lesson 3.3.2. With too few vectors, some points are not reached at all.

An orthogonal basis

When the basis vectors are orthogonal (lesson 3.1.2), no system is needed. Take $\mathbf{b}_1 = (1, 1)$ and $\mathbf{b}_2 = (-1, 1)$ , whose dot product is 0. Each coordinate of $(4, -2)$ is a dot product divided by a squared length:

\begin{gathered} c_1 = \frac{(4, -2) \cdot (1, 1)}{2} = 1 \\[4pt] c_2 = \frac{(4, -2) \cdot (-1, 1)}{2} = -3 \end{gathered}

Check: $1(1, 1) - 3(-1, 1) = (4, -2)$ . Lesson 3.5.2 shows why this works: each coordinate measures a projection.

\mathbf{u}

import numpy as np B = np.column_stack([[3, 1], [1, 2]]) for x in [(5, 5), (7, 4), (1, -3), (4, -2), (1, 0)]: c = np.linalg.solve(B, x) print(x, np.round(c, 4), np.allclose(B @ c, x)) b1, b2 = np.array([1, 1]), np.array([-1, 1]) x = np.array([4, -2]) print(x @ b1 / (b1 @ b1), x @ b2 / (b2 @ b2))

⌘/Ctrl + Enter runs the cell

The column space and the null space

Two spans come with every matrix $A$ . The first is the column space: the span of the columns of $A$ . A product $A\mathbf{x}$ is a combination of the columns (lesson 3.2.1), so the column space holds every $A\mathbf{x}$ .

So $A\mathbf{x} = \mathbf{b}$ has a solution exactly when $\mathbf{b}$ is in the column space. That is the picture of lesson 3.3.2, where columns along one line reached only that line. The dimension of the column space is $\operatorname{rank} A$ .

Take the tall system of lesson 3.3.2:

T = \begin{pmatrix} 1 & 1 \\ 1 & -1 \\ 1 & 2 \end{pmatrix}

Its two columns are independent, so its column space is a plane in $\mathbb{R}^3$ . $\mathbf{b} = (3, 1, 4)$ lies in it, since $T(2, 1) = \mathbf{b}$ . $\mathbf{b} = (3, 1, 5)$ does not, so that system has no solution.

Elimination with letters says which $\mathbf{b}$ the plane holds. The steps of lesson 3.3.2 leave $b_3 - 1.5b_1 + 0.5b_2$ in the last row, so $\mathbf{b}$ is reached exactly when $3b_1 - b_2 - 2b_3 = 0$ .

For $(3, 1, 5)$ that number is $-2$ , not 0. Lesson 3.5.3 finds the point of the plane closest to it.

The null space

The null space of $A$ is the set of all $\mathbf{x}$ with $A\mathbf{x} = \mathbf{0}$ . It always contains $\mathbf{x} = \mathbf{0}$ , and it is a span too: a point, a line, a plane or more, through the origin.

Independent columns: only $\mathbf{x} = \mathbf{0}$ , since no other combination of the columns gives $\mathbf{0}$ . The null space of $T$ is just $\mathbf{0}$ .
$\begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix}$ : $x_1 + 2x_2 = 0$ , so the null space is the line of multiples of $(-2, 1)$ . Lesson 3.2.3's singular $Z = \begin{pmatrix} 1 & 2 \\ 0.5 & 1 \end{pmatrix}$ has the same one: it sends $(2, -1)$ to $\mathbf{0}$ .
$\begin{pmatrix} 1 & 2 & 1 \\ 2 & 4 & 3 \\ 3 & 6 & 4 \end{pmatrix}$ from lesson 3.3.2: the line of multiples of $(-2, 1, 0)$ .

That last direction is familiar. Lesson 3.3.2 found the solutions of this system with $\mathbf{b} = (4, 9, 13)$ as $(3, 0, 1) + t(-2, 1, 0)$ . Adding any vector of the null space to one solution gives another.

Rule: Column space and null space

\begin{gathered} \text{column space: every } A\mathbf{x} \\ \text{dimension } \operatorname{rank} A \\[8pt] \text{null space: every } \mathbf{x} \text{ with } A\mathbf{x} = \mathbf{0} \\ \text{dimension } n - \operatorname{rank} A \end{gathered}

$A$ has $n$ columns. $A\mathbf{x} = \mathbf{b}$ has a solution exactly when $\mathbf{b}$ is in the column space. The dimension of the null space is the number of free variables of lesson 3.3.2.

Lesson 3.6.1 finds eigenvectors as the null space of a matrix.

T

import numpy as np from scipy.linalg import null_space rank = np.linalg.matrix_rank T = np.array([[1, 1], [1, -1], [1, 2]]) for b in [[3, 1, 4], [3, 1, 5]]: print(b, rank(T), rank(np.column_stack([T, b])), 3 * b[0] - b[1] - 2 * b[2]) print(T @ np.array([2, 1])) Z = np.array([[1, 2], [0.5, 1]]) A = np.array([[1, 2, 1], [2, 4, 3], [3, 6, 4]]) print(Z @ np.array([2, -1]), A @ np.array([-2, 1, 0])) for t in [0, 1, 2]: print(A @ (np.array([3, 0, 1]) + t * np.array([-2, 1, 0]))) n = null_space(A)[:, 0] print(np.round(n / n[1], 4) + 0.0, 3 - rank(A))

⌘/Ctrl + Enter runs the cell

Common mistakes

A span always passes through the origin. The points $(x, 3)$ form a line, but not a span: $\mathbf{0}$ is not on it.
Independence belongs to the whole set. $(1, 0, 1)$ , $(0, 1, 1)$ and $(1, 1, 2)$ are dependent, though no two of them are multiples of each other.
The dimension is the rank, not the count. Three vectors can span a plane, a line or only the origin.
Too many or too few. More than $n$ vectors in $\mathbb{R}^n$ are always dependent; fewer than $n$ never span $\mathbb{R}^n$ .
Coordinates depend on the basis and its order. $(4, -2)$ is $(2, -2)$ in the basis $\mathbf{u}, \mathbf{v}$ and $(-2, 2)$ in $\mathbf{v}, \mathbf{u}$ .
The determinant test needs a square matrix. For $k$ vectors in $\mathbb{R}^n$ with $k \neq n$ , compare the rank with $k$ .
A null space is never empty. It always holds $\mathbf{0}$ ; "only $\mathbf{0}$ " means the columns are independent.

Span, independence and basis

0 of 12 correct

1.
$(4, 7) = a(1, 2) + b(1, 1)$ . Find $a$
2.
Is $(2, 5, 7)$ in the span of $(1, 0, 1)$ and $(0, 1, 1)$ ? Type 1 for yes, 0 for no
3.
For which $k$ are $(1, k)$ and $(2, 6)$ dependent?
4.
For which $k$ are $(1, 2, 0)$ , $(0, 1, 1)$ and $(1, k, 1)$ dependent?
5.
Five vectors lie in $\mathbb{R}^3$ . At most how many of them can be independent?
6.
What is the dimension of the span of $(1, 2, 3)$ , $(2, 4, 6)$ and $(0, 1, 1)$ ?
7.
Find the first coordinate of $(7, 4)$ in the basis $\mathbf{u} = (3, 1)$ , $\mathbf{v} = (1, 2)$
8.
Find the second coordinate of $(2, 9)$ in the basis $(1, 2)$ , $(3, 1)$
9.
$(1, 1)$ and $(1, -1)$ are orthogonal. Find the second coordinate of $(3, -1)$ in this basis
10.
Is $(2, 3, 4)$ in the column space of $\begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{pmatrix}$ ? Type 1 for yes, 0 for no
11.
$(x_1, 1)$ is in the null space of $\begin{pmatrix} 1 & 3 \\ 2 & 6 \end{pmatrix}$ . Find $x_1$
12.
A $3 \times 5$ matrix has rank 3. What is the dimension of its null space?

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

\mathbf{v}_1, \ldots, \mathbf{v}_k

Span, independence and basis

The span of a set of vectors

Rule: Span

A plane in space

Linear independence

Two tests

Rule: Testing independence

Basis and dimension

Rule: Dimension of a span

Coordinates in a basis

Rule: Coordinates in a basis

An orthogonal basis

The column space and the null space

The null space

Rule: Column space and null space

Common mistakes

Span, independence and basis

Summary card

Free further reading