3.1.2 The dot product, length and angle

The dot product

The dot product of two vectors multiplies matching components and adds the results. It turns two vectors into one number, a scalar. Both vectors must have the same number of components.

For $\mathbf{u} = (3, 1)$ and $\mathbf{v} = (1, 2)$ , the dot product is $\mathbf{u} \cdot \mathbf{v} = 3 \times 1 + 1 \times 2 = 5$ .

Rule: Dot product

\begin{gathered} \mathbf{u} \cdot \mathbf{v} = u_1 v_1 + \cdots + u_n v_n \\[4pt] = \sum_{i=1}^{n} u_i v_i \end{gathered}

Multiply component $i$ of $\mathbf{u}$ by component $i$ of $\mathbf{v}$ , for every $i$ , and add. The $\Sigma$ is the sum notation of lesson 0.2.2. The answer is a number, not a vector.

A weighted sum

The dot product is a weighted sum: each component of one vector weights the matching component of the other. The shop of lesson 3.1.1 holds stock $\mathbf{s} = (12, 0, 7, 30)$ . Its four products sell at prices $\mathbf{p} = (2.50, 4, 10, 0.80)$ dollars each.

product	1	2	3	4	total
stock $s_i$	12	0	7	30
price $p_i$ ($)	2.50	4	10	0.80
$s_i p_i$ ($)	30	0	70	24	124

So $\mathbf{s} \cdot \mathbf{p} = 124$ : the stock is worth $124. The units multiply, items times dollars per item, giving dollars. Case study 3.1.4 uses the same weighted sum for a portfolio's return.

Its rules

The dot product follows rules much like ordinary multiplication. For vectors $\mathbf{u}$ , $\mathbf{v}$ , $\mathbf{w}$ of the same size and a scalar $c$ :

Order does not matter: $\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$ .
Scalars come out: $(c\mathbf{u}) \cdot \mathbf{v} = c\,(\mathbf{u} \cdot \mathbf{v})$ . So $(2\mathbf{u}) \cdot \mathbf{v} = 10$ .
It spreads over sums: $\mathbf{u} \cdot (\mathbf{v} + \mathbf{w}) = \mathbf{u} \cdot \mathbf{v} + \mathbf{u} \cdot \mathbf{w}$ .
A vector with itself: $\mathbf{u} \cdot \mathbf{u} = u_1^2 + \cdots + u_n^2$ , never negative. For $\mathbf{u} = (3, 1)$ it is 10.

Try it In NumPy the dot product is u @ v, or np.dot(u, v) . Both print 5. u * v is something else: the vector of products, [3 2] . Adding its components gives the dot product. The last line is the stock's value.

import numpy as np u = np.array([3, 1]) v = np.array([1, 2]) print(u @ v, np.dot(u, v), v @ u) print(u * v, (u * v).sum()) print((2 * u) @ v, u @ u) s = np.array([12, 0, 7, 30]) p = np.array([2.5, 4, 10, 0.8]) print(s @ p)

⌘/Ctrl + Enter runs the cell

Length

The length of a vector, written $\lVert \mathbf{u} \rVert$ , is the length of its arrow. In the plane the components are the legs of a right-angled triangle. So Pythagoras' theorem (lesson 1.7.1) gives $\lVert \mathbf{u} \rVert = \sqrt{u_1^2 + u_2^2}$ .

The sum of squares under the root is $\mathbf{u} \cdot \mathbf{u}$ . So the length of $(3, 4)$ is $\sqrt{9 + 16} = 5$ , and the length of $\mathbf{u} = (3, 1)$ is $\sqrt{10} \approx 3.1623$ . Lesson 1.7.2 measured the modulus of a complex number the same way.

In $\mathbb{R}^3$ Pythagoras applies twice and gives the same formula: $(2, 3, 6)$ has length $\sqrt{4 + 9 + 36} = 7$ . In $\mathbb{R}^n$ the formula is the definition of length.

Rule: Length

\begin{gathered} \lVert \mathbf{u} \rVert = \sqrt{\mathbf{u} \cdot \mathbf{u}} \\[4pt] = \sqrt{u_1^2 + \cdots + u_n^2} \end{gathered}

The length is in the same units as the components. It is never negative, and it is 0 only for the zero vector.

Squares remove signs, so $(-3, 4)$ also has length 5. Scaling by $c$ multiplies the length by $|c|$ , the size of $c$ (lesson 0.3.3). So $\lVert 2\mathbf{u} \rVert = 2\sqrt{10} \approx 6.3246$ , and $\lVert -3\mathbf{u} \rVert = 3\sqrt{10} \approx 9.4868$ .

Distance between two points

The arrow from the tip of $\mathbf{v}$ to the tip of $\mathbf{u}$ is $\mathbf{u} - \mathbf{v}$ (lesson 3.1.1). Its length is the distance between the two points. For $\mathbf{u} = (3, 1)$ and $\mathbf{v} = (1, 2)$ it is $\lVert (2, -1) \rVert = \sqrt{5} \approx 2.2361$ .

\mathbf{u}

import numpy as np u = np.array([3, 1]) v = np.array([1, 2]) print(round(np.linalg.norm(u), 4), round(np.sqrt(u @ u), 4)) for x in [[3, 4], [2, 3, 6], [-3, 4]]: print(x, np.linalg.norm(x)) print(round(np.linalg.norm(2 * u), 4), round(np.linalg.norm(-3 * u), 4)) print(round(np.linalg.norm(u - v), 4))

⌘/Ctrl + Enter runs the cell

Unit vectors

A unit vector has length 1. Dividing a non-zero vector by its length gives the unit vector pointing the same way; this is called normalising it.

$(3, 4)$ has length 5, so its unit vector is $(0.6, 0.8)$ . For $\mathbf{u} = (3, 1)$ it is $\frac{1}{\sqrt{10}}(3, 1) \approx (0.9487, 0.3162)$ .

The zero vector has length 0 and no direction, so it has no unit vector: the division would be by 0.

Rule: Unit vector

\frac{1}{\lVert \mathbf{u} \rVert}\,\mathbf{u} \quad \text{for } \mathbf{u} \neq \mathbf{0}

A vector of length 1 pointing the same way as $\mathbf{u}$ . In the plane every unit vector is $(\cos\alpha, \sin\alpha)$ for some angle $\alpha$ : a point on the unit circle of lesson 1.7.1.

The unit vectors along the axes are $\mathbf{e}_1 = (1, 0)$ and $\mathbf{e}_2 = (0, 1)$ . A dot product with them picks out a component: $\mathbf{u} \cdot \mathbf{e}_1 = u_1 = 3$ and $\mathbf{u} \cdot \mathbf{e}_2 = u_2 = 1$ .

(3, 4)

import numpy as np u = np.array([3, 1]) unit_u = u / np.linalg.norm(u) print(np.array([3, 4]) / 5, np.round(unit_u, 4), round(np.linalg.norm(unit_u), 4)) e1 = np.array([1, 0]) e2 = np.array([0, 1]) print(u @ e1, u @ e2)

⌘/Ctrl + Enter runs the cell

The angle between two vectors

The dot product measures how far two vectors point the same way. Take two unit vectors on the unit circle, at angles $\alpha$ and $\beta$ (alpha and beta) from the positive horizontal axis. Their dot product is the cosine of the angle between them, $\cos(\beta - \alpha)$ :

$\alpha$	$\beta$	dot product	$\cos(\beta - \alpha)$
0°	60°	0.5	0.5
30°	75°	0.707107	0.707107
20°	140°	$-0.5$	$-0.5$
50°	140°	0	0

Any non-zero vector is its length times a unit vector, and scalars come out of a dot product. So $\mathbf{u} \cdot \mathbf{v} = \lVert \mathbf{u} \rVert\,\lVert \mathbf{v} \rVert \cos\theta$ , where $\theta$ (theta) is the angle between $\mathbf{u}$ and $\mathbf{v}$ .

Rule: Angle between two vectors

\cos\theta = \frac{\mathbf{u} \cdot \mathbf{v}}{\lVert \mathbf{u} \rVert\,\lVert \mathbf{v} \rVert}

For non-zero $\mathbf{u}$ and $\mathbf{v}$ . The angle $\theta$ is taken between 0 and $\pi$ , that is 0° to 180°, which is what math.acos returns (lesson 1.7.1).

Example 1. For $\mathbf{u} = (3, 1)$ and $\mathbf{v} = (1, 2)$ , the dot product is 5 and the lengths are $\sqrt{10}$ and $\sqrt{5}$ :

\begin{aligned} \cos\theta &= \frac{5}{\sqrt{10}\,\sqrt{5}} = \frac{5}{\sqrt{50}} \\ &= \frac{1}{\sqrt{2}} \approx 0.7071 \end{aligned}

That is the cosine of 45°, so $\theta = 45°$ .

Example 2. In $\mathbb{R}^3$ , take $(1, 1, 0)$ and $(0, 1, 1)$ . The dot product is 1 and both lengths are $\sqrt{2}$ , so $\cos\theta = \frac{1}{2}$ and $\theta = 60°$ . With more than three components there is no picture, and the formula defines the angle.

The two extremes: $2\mathbf{u} = (6, 2)$ points the same way as $\mathbf{u}$ , with cosine 1 and angle 0°. $-\mathbf{u}$ points the opposite way, with cosine $-1$ and angle 180°. The angle is always taken the short way round, and it does not say which vector is clockwise from the other.

\alpha

import numpy as np for a, b in [(0, 60), (30, 75), (20, 140), (50, 140)]: ua = np.array([np.cos(np.radians(a)), np.sin(np.radians(a))]) ub = np.array([np.cos(np.radians(b)), np.sin(np.radians(b))]) print(a, b, round(ua @ ub, 6) + 0.0, round(np.cos(np.radians(b - a)), 6) + 0.0) def angle(u, v): cos = u @ v / (np.linalg.norm(u) * np.linalg.norm(v)) return round(np.degrees(np.arccos(np.clip(cos, -1, 1))), 4) u = np.array([3, 1]) v = np.array([1, 2]) print(round(u @ v / (np.linalg.norm(u) * np.linalg.norm(v)), 4)) print(angle(u, v), angle(np.array([1, 1, 0]), np.array([0, 1, 1]))) print(angle(u, 2 * u), angle(u, -u))

⌘/Ctrl + Enter runs the cell

The sign of the dot product

Lengths are positive, so $\mathbf{u} \cdot \mathbf{v}$ has the sign of $\cos\theta$ . The sign alone says whether the angle is below, at or above 90°.

Positive: acute, below 90°. $(3, 1) \cdot (1, 2) = 5$ , and the angle is 45°.
Zero: a right angle. $(3, 1) \cdot (-1, 3) = -3 + 3 = 0$ , and the angle is 90°.
Negative: obtuse, above 90°. $(3, 1) \cdot (-2, 1) = -6 + 1 = -5$ , and the angle is 135°.

$\mathbf{u} = (3, 1)$ against three vectors of the same length, $\sqrt{5}$ . The dot product is positive for an acute angle, zero for a right angle and negative for an obtuse one.

Drag the tip of $\mathbf{v}$ round $\mathbf{u}$ and watch the sign of $\mathbf{u} \cdot \mathbf{v}$ change as the angle passes 90°. Then make $\mathbf{v}$ a multiple of $\mathbf{u}$ : the angle is 0° or 180°, and $|\mathbf{u} \cdot \mathbf{v}|$ is as large as the two lengths allow. The dark arrow is the shadow of $\mathbf{v}$ , explained at the end of this lesson.

Explore

Drag the tips of u and v. Watch the sign of u · v as the angle passes 90°. The dark arrow is the projection of v on u: the shadow v casts on the line through u.

u₁ = 3

u₂ = 1

v₁ = 1

v₂ = 2

u: (3, 1), length 3.1623
v: (1, 2), length 2.2361
u · v: 3 × 1 + 1 × 2 = 5
Angle: cos θ = 0.7071, so θ = 45°
Projection of v on u: (1.5, 0.5), signed length u · v / |u| = 1.5811
So: The dot product is positive, so the angle is less than 90°.

Orthogonal vectors

Two vectors are orthogonal when their dot product is 0. For non-zero vectors that means they meet at a right angle; "perpendicular" is the everyday word.

$(3, 1)$ and $(-1, 3)$ are orthogonal. In general, $(a, b)$ turned a quarter turn is $(-b, a)$ (lesson 1.7.2), and $a(-b) + ba = 0$ .
In $\mathbb{R}^3$ , $(1, 2, 3) \cdot (3, 0, -1) = 3 + 0 - 3 = 0$ , so they are orthogonal too.
The axes are orthogonal: $\mathbf{e}_1 \cdot \mathbf{e}_2 = 0$ .

Rule: Orthogonal vectors

\mathbf{u} \cdot \mathbf{v} = 0

$\mathbf{u}$ and $\mathbf{v}$ are orthogonal. A dot product is far easier to check than an angle, in any number of dimensions. The zero vector is orthogonal to every vector, though it has no angle.

Module 3.5 builds projections and least squares on this one idea.

(3, 1)

import numpy as np u = np.array([3, 1]) quarter = np.array([-u[1], u[0]]) print(quarter, u @ quarter) print(np.array([1, 2, 3]) @ np.array([3, 0, -1])) print(np.array([1, 0]) @ np.array([0, 1]))

⌘/Ctrl + Enter runs the cell

The Cauchy–Schwarz inequality

In the plane, $\mathbf{u} \cdot \mathbf{v} = \lVert \mathbf{u} \rVert\,\lVert \mathbf{v} \rVert \cos\theta$ , and a cosine lies between $-1$ and 1. So the size of a dot product never exceeds the product of the two lengths. This is the Cauchy–Schwarz inequality; it holds in every $\mathbb{R}^n$ , and is stated here without proof.

Rule: Cauchy–Schwarz inequality

|\mathbf{u} \cdot \mathbf{v}| \le \lVert \mathbf{u} \rVert\,\lVert \mathbf{v} \rVert

Equality holds exactly when one vector is a multiple of the other. In $\mathbb{R}^n$ it is why the ratio in the angle rule always lies in $[-1, 1]$ , so every pair of non-zero vectors has an angle.

For $(3, 1)$ and $(1, 2)$ : $|5| \le \sqrt{10}\,\sqrt{5} \approx 7.0711$ . For $\mathbf{u}$ and $2\mathbf{u}$ the two sides are equal: $20 = \sqrt{10}\,\sqrt{40}$ .

\mathbb{R}^5

import numpy as np rng = np.random.default_rng(1) largest = 0 for _ in range(10_000): a = rng.normal(size=5) b = rng.normal(size=5) largest = max(largest, abs(a @ b) / (np.linalg.norm(a) * np.linalg.norm(b))) print(round(largest, 4)) u = np.array([3, 1]) print(abs(u @ np.array([1, 2])), round(np.linalg.norm(u) * np.linalg.norm([1, 2]), 4)) print(abs(u @ (2 * u)), round(np.linalg.norm(u) * np.linalg.norm(2 * u), 4))

⌘/Ctrl + Enter runs the cell

Two rounding traps

Computers round every decimal, and two things can go wrong. For vectors pointing the same way, the computed cosine can come out a hair above 1. Then math.acos stops with an error, and np.arccos returns nan, "not a number".

The fix is to clip the cosine into $[-1, 1]$ first, with np.clip(cos, -1, 1). The second trap is the reverse: a dot product that should be 0 can come out as a tiny number such as $1.4 \times 10^{-17}$ . Test orthogonality with a tolerance, not with == 0.

\mathbf{u} = (3, 1)

import math import numpy as np def dot(a, b): return sum(x * y for x, y in zip(a, b)) def length(a): return math.sqrt(dot(a, a)) u = [3.0, 1.0] v = [1.3 * x for x in u] cos = dot(u, v) / (length(u) * length(v)) print(cos) try: math.acos(cos) except ValueError as err: print("ValueError:", err) print(math.degrees(math.acos(np.clip(cos, -1, 1)))) d = dot([0.1, 0.2, 0.3], [0.5, 0.5, -0.5]) print(d, d == 0, abs(d) < 1e-12)

⌘/Ctrl + Enter runs the cell

The shadow of one vector on another

Shine a light straight down onto the line through $\mathbf{u}$ . The shadow of $\mathbf{v}$ on that line has signed length $\frac{\mathbf{u} \cdot \mathbf{v}}{\lVert \mathbf{u} \rVert}$ , which is $\lVert \mathbf{v} \rVert \cos\theta$ . It is positive when the shadow falls along $\mathbf{u}$ and negative when it falls behind the origin.

For $\mathbf{v} = (1, 2)$ it is $\frac{5}{\sqrt{10}} \approx 1.5811$ , the same as $\sqrt{5}\cos 45°$ . For $(-2, 1)$ , at 135° to $\mathbf{u}$ , it is $-1.5811$ . For a vector orthogonal to $\mathbf{u}$ the shadow is a single point, of length 0.

The shadow of a vector on the line through $\mathbf{u} = (3, 1)$ , cut off by a drop at right angles. Its signed length is $\frac{\mathbf{u} \cdot \mathbf{v}}{\lVert \mathbf{u} \rVert}$ : 1.58 for $(1, 2)$ , and $-1.58$ for $(-2, 1)$ , which points away from $\mathbf{u}$ .

The shadow is also called the projection of $\mathbf{v}$ on $\mathbf{u}$ . Lesson 3.5.2 develops it: the shadow as a vector, and the nearest point on a line or a plane.

\mathbf{u}

import numpy as np u = np.array([3, 1]) v = np.array([1, 2]) print(round(u @ v / np.linalg.norm(u), 4), round(np.linalg.norm(v) * np.cos(np.radians(45)), 4)) print(round(u @ np.array([-2, 1]) / np.linalg.norm(u), 4)) print(u @ np.array([-1, 3]) / np.linalg.norm(u))

⌘/Ctrl + Enter runs the cell

Common mistakes

The dot product is a number. In NumPy, u * v is the vector of products, [3 2]; the dot product is u @ v.
Length squares the components. $\lVert (3, 4) \rVert = 5$ , not $3 + 4 = 7$ . And $(-3, 4)$ has length 5 too: the squares remove the signs.
Scaling by a negative number does not give a negative length. $\lVert -3\mathbf{u} \rVert = 3\lVert \mathbf{u} \rVert$ .
The zero vector has no direction. It has no unit vector and no angle with anything.
Divide by both lengths. The cosine is $\frac{\mathbf{u} \cdot \mathbf{v}}{\lVert \mathbf{u} \rVert\,\lVert \mathbf{v} \rVert}$ ; the dot product 5 on its own is not a cosine.
np.arccos gives radians. Convert with np.degrees, and clip the cosine into $[-1, 1]$ first.
The shadow of $\mathbf{v}$ on $\mathbf{u}$ divides by $\lVert \mathbf{u} \rVert$ , the length of the vector it falls on, not $\lVert \mathbf{v} \rVert$ .
Test for 0 with a tolerance. A dot product that should be 0 may compute as about $10^{-17}$ .

The dot product, length and angle

0 of 12 correct

1.
Find $(2, -1) \cdot (4, 3)$
2.
Find the length of $(5, 12)$
3.
Find the length of $(2, -3, 6)$
4.
$\lVert \mathbf{u} \rVert = 13$ . What is $\lVert -3\mathbf{u} \rVert$ ?
5.
Normalise $(8, -6)$ . What is the second component of its unit vector?
6.
Find the angle, in degrees, between $(2, 0)$ and $(-1, 1)$
7.
Find $\cos\theta$ for $(1, 2, 2)$ and $(0, 3, 4)$
8.
Find $(3, -2) \cdot (1, 2)$ . Is the angle between the two vectors more or less than 90°?
9.
For which $k$ are $(k, 2)$ and $(3, -6)$ orthogonal?
10.
$\lVert \mathbf{u} \rVert = 3$ and $\lVert \mathbf{v} \rVert = 4$ . What is the largest value $\mathbf{u} \cdot \mathbf{v}$ can take?
11.
Find the signed length of the shadow of $(2, 1)$ on the line through $(3, 4)$
12.
Find the distance between the points $(1, 5)$ and $(4, 1)$

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

\mathbf{u} \cdot \mathbf{v} = u_1v_1 + \cdots + u_nv_n

The dot product, length and angle

The dot product

Rule: Dot product

A weighted sum

Its rules

Length

Rule: Length

Distance between two points

Unit vectors

Rule: Unit vector

The angle between two vectors

Rule: Angle between two vectors

The sign of the dot product

Orthogonal vectors

Rule: Orthogonal vectors

The Cauchy–Schwarz inequality

Rule: Cauchy–Schwarz inequality

Two rounding traps

The shadow of one vector on another

Common mistakes

The dot product, length and angle

Summary card

Free further reading