3.5.4 Practice set: spaces and projections

Span, independence and basis

Span, independence and basis (lesson 3.5.1)

0 of 8 correct

1.
Is $(3, 4, 7)$ in the span of $(1, 0, 1)$ and $(0, 1, 1)$ ? Type 1 for yes, 0 for no
2.
$(5, 1) = a(1, 1) + b(2, -1)$ . Find $b$
3.
For which $k$ are $(1, 1, 0)$ , $(0, 1, 1)$ and $(1, 0, k)$ dependent?
4.
Find the second coordinate of $(1, 4)$ in the basis $(1, 1)$ , $(-1, 2)$
5.
$(2, 2)$ and $(-3, 3)$ are orthogonal. Find the first coordinate of $(4, 10)$ in this basis
6.
A $4 \times 6$ matrix has rank 4. What is the dimension of its null space?
7.
What is the dimension of the span of $(1, -1, 0)$ , $(0, 1, -1)$ and $(1, 0, -1)$ ?
8.
What is the dimension of the span of $(0, 0)$ and $(2, 1)$ ?

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

Orthogonal projection

Orthogonal projection (lesson 3.5.2)

0 of 8 correct

1.
Project $(4, 2)$ onto the line through $(1, 2)$ . What is the coefficient of $(1, 2)$ ?
2.
Project $(1, 1, 1)$ onto the line through $(1, 2, 2)$ . What is the first component of the projection?
3.
What is the distance from the point $(5, 0)$ to the line through $(3, 4)$ ?
4.
$\lVert \mathbf{v} \rVert = 13$ , and its projection onto a line has length 12. How far is $\mathbf{v}$ from the line?
5.
Project $(0, 0, 3)$ onto the plane spanned by $(1, 0, 1)$ and $(0, 1, 1)$ . How far is the point from the plane?
6.
$P$ projects $\mathbb{R}^3$ onto a plane through the origin. What is its rank?
7.
Projecting $(1, 2)$ onto the line through $(3, 1)$ gives $(1.5, 0.5)$ . What is the first component of the projection of $(3, 1)$ onto the line through $(1, 2)$ ?
8.
$Q = \begin{pmatrix} 0.6 & -0.8 \\ 0.8 & 0.6 \end{pmatrix}$ has orthonormal columns. What is the first component of $QQ^\top\mathbf{b}$ for $\mathbf{b} = (2, 1)$ ?

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

Least squares

Least squares (lesson 3.5.3)

0 of 8 correct

1.
Fit $y = c + mx$ to points with $x$ values 1, 2 and 3. What is the entry in row 1, column 2 of $A^\top A$ ?
2.
Fit $y = c + mx$ to $(1, 1)$ , $(2, 3)$ , $(3, 2)$ . Find $m$
3.
For the same points, find $c$
4.
For the line $y = 1 + 0.5x$ through those points, what is the sum of squared residuals?
5.
One unknown, three equations: $x = 1$ , $x = 2$ and $x = 6$ . What is the least-squares $\hat{x}$ ?
6.
Solve $x_1 = 1$ , $x_2 = 1$ , $x_1 + x_2 = 3$ by least squares. What is $\hat{x}_1$ ?
7.
A line fit with an intercept leaves four residuals. Three of them are $-0.4$ , 0.2 and 0.5. What is the fourth?
8.
All the points of a line fit have $x = 5$ . What rank does lstsq report for $A = [\mathbf{1}\ \mathbf{x}]$ ?

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

Coding exercise

Exercise

A projection, and a line that survives awkward data

Write two functions, with NumPy.

project(A, b) returns (p, r): the projection of b onto the column space of A and the residual b - p, as in lesson 3.5.2. A 1-D A is a single column. Raise ValueError if A and b have different numbers of rows, or if the columns of A are dependent.
fit_line(x, y) returns (c, m, ss): the intercept, the slope and the sum of squared residuals of the least-squares line $y = c + mx$ . Use the centred form of lesson 3.5.3, $m = \frac{\tilde{\mathbf{x}} \cdot \tilde{\mathbf{y}}}{\tilde{\mathbf{x}} \cdot \tilde{\mathbf{x}}}$ and $c = \bar{y} - m\bar{x}$ . Raise ValueError if the lengths differ, there are fewer than two points, or every $x$ is the same.

The tests use the examples of lessons 3.5.2 and 3.5.3 and the bad inputs above. One test has $x$ values near $10^8$ , such as times counted in seconds. There the normal equations, and even lstsq, return nonsense, while the centred form gets the slope exactly.

See the tests

import numpy as np

X5 = [0, 1, 2, 3, 4]
Y5 = [1, 2, 4, 4, 6]
PLANE = [[1, 0], [0, 1], [1, 1]]


def _raises_value_error(fn, *args):
    try:
        fn(*args)
    except ValueError:
        return True
    return False


def test_project_onto_a_line():
    """(1, 2) projected onto the line through (3, 1) is (1.5, 0.5), with residual (−0.5, 1.5)"""
    p, r = project([3, 1], [1, 2])
    expect_close(p, [1.5, 0.5], what="projection onto the line")
    expect_close(r, [-0.5, 1.5], what="residual")


def test_project_onto_a_plane():
    """(0, 1, 4) projected onto the plane of (1, 0, 1) and (0, 1, 1) is (1, 2, 3), with residual (−1, −1, 1)"""
    p, r = project(PLANE, [0, 1, 4])
    expect_close(p, [1, 2, 3], what="projection onto the plane")
    expect_close(r, [-1, -1, 1], what="residual")


def test_residual_is_orthogonal():
    """For a random 5 × 3 matrix, the residual is orthogonal to every column"""
    rng = np.random.default_rng(3)
    A = rng.normal(size=(5, 3))
    b = rng.normal(size=5)
    p, r = project(A, b)
    expect_close(A.T @ r, [0, 0, 0], abs_tol=1e-9, what="A.T @ r")
    expect_close(p + r, b, what="p + r")


def test_point_in_the_plane():
    """(2, 3, 5) is in the plane already: it is its own projection, with residual 0"""
    p, r = project(PLANE, [2, 3, 5])
    expect_close(p, [2, 3, 5], what="projection of a point in the plane")
    expect_close(r, [0, 0, 0], abs_tol=1e-9, what="residual")


def test_dependent_columns():
    """Columns (1, 2, 3) and (2, 4, 6) are dependent: ValueError"""
    if not _raises_value_error(project, [[1, 2], [2, 4], [3, 6]], [1, 0, 0]):
        raise AssertionError("expected ValueError for dependent columns")


def test_rows_differ():
    """A with 3 rows and b with 2 entries: ValueError"""
    if not _raises_value_error(project, PLANE, [1, 2]):
        raise AssertionError("expected ValueError when A and b have different numbers of rows")


def test_fit_five_points():
    """The five points of lesson 3.5.3 give y = 1 + 1.2x with a sum of squares of 0.8"""
    c, m, ss = fit_line(X5, Y5)
    expect_close([c, m, ss], [1.0, 1.2, 0.8], what="(c, m, ss)")


def test_fit_two_points():
    """Two points (1, 3) and (3, 7) give the line through them, y = 1 + 2x, with nothing left over"""
    c, m, ss = fit_line([1, 3], [3, 7])
    expect_close([c, m], [1.0, 2.0], what="(c, m)")
    expect_close(ss, 0.0, abs_tol=1e-12, what="sum of squares")


def test_noisy_line():
    """Lesson 3.5.3's 20 noisy points (seed 353) give c = 2.4162, m = 0.4571 and a sum of squares of 24.0997"""
    rng = np.random.default_rng(353)
    x = np.arange(1, 21, dtype=float)
    y = 2 + 0.5 * x + rng.normal(0, 1, 20)
    c, m, ss = fit_line(x, y)
    A = np.column_stack([np.ones(20), x])
    expected, residual, _, _ = np.linalg.lstsq(A, y, rcond=None)
    expect_close([c, m], expected, what="(c, m)")
    expect_close(ss, residual[0], what="sum of squares")


def test_large_x():
    """x near 10⁸ with y = 2 + 0.5(x − 10⁸): the slope is exactly 0.5 and the intercept 2 − 0.5 × 10⁸"""
    x = 1e8 + np.arange(5.0)
    y = 2 + 0.5 * np.arange(5.0)
    c, m, ss = fit_line(x, y)
    expect_close(m, 0.5, rel=1e-12, what="slope")
    expect_close(c, 2 - 0.5e8, rel=1e-12, what="intercept")


def test_all_x_equal():
    """Points (2, 1), (2, 3), (2, 5) all have x = 2: ValueError"""
    if not _raises_value_error(fit_line, [2, 2, 2], [1, 3, 5]):
        raise AssertionError("expected ValueError when every x is the same")


def test_lengths_differ():
    """Five x values and four y values: ValueError"""
    if not _raises_value_error(fit_line, X5, Y5[:4]):
        raise AssertionError("expected ValueError when the lengths differ")


def test_one_point():
    """A single point does not fix a line: ValueError"""
    if not _raises_value_error(fit_line, [1], [2]):
        raise AssertionError("expected ValueError for one point")

Span, independence and basis

Orthogonal projection

Least squares

Coding exercise

Free further reading