3.2.1 Matrices and matrix multiplication

What a matrix is

A matrix is a rectangular table of numbers. A matrix with $m$ rows and $n$ columns is an $m \times n$ matrix, read " $m$ by $n$ ": rows first, then columns. That pair of numbers is its shape.

Each number in the table is an entry. For a matrix $A$ , the entry in row $i$ and column $j$ is $a_{ij}$ : again, row first.

A workshop makes chairs and tables. A chair needs 4 legs, 8 screws and 1 board; a table needs 4 legs, 12 screws and 3 boards. Put the parts in rows and the products in columns:

P = \begin{pmatrix} 4 & 4 \\ 8 & 12 \\ 1 & 3 \end{pmatrix} \begin{matrix} \text{legs} \\ \text{screws} \\ \text{boards} \end{matrix}

$P$ is $3 \times 2$ : three parts, two products. Its first column lists what a chair needs, and its second what a table needs.

Write $p_{ij}$ for the entries of $P$ . Then $p_{22} = 12$ is screws per table, and $p_{31} = 1$ is boards per chair. Every entry has units: parts per item.

Rule: An m × n matrix

A = \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \end{pmatrix}

This $A$ is $2 \times 3$ : 2 rows, 3 columns, 6 entries. In $a_{ij}$ the first index is the row and the second the column.

A vector written as a column (lesson 3.1.1) is a matrix with one column: a vector with $n$ components is $n \times 1$ . A matrix with one row, $1 \times n$ , is a row vector. A matrix with as many rows as columns is square.

p_{22}

⌘/Ctrl + Enter runs the cell

A matrix times a vector

An order of 6 chairs and 2 tables is the vector $\mathbf{x} = (6, 2)$ , a column with one entry per product. The parts it needs make the vector $P\mathbf{x}$ , and there are two ways to work it out.

Row by row. Legs: 4 per chair times 6 chairs, plus 4 per table times 2 tables, so $4 \cdot 6 + 4 \cdot 2 = 32$ . That is the dot product of the legs row $(4, 4)$ with $\mathbf{x}$ : multiply matching components and add (lesson 3.1.2). Each entry of $P\mathbf{x}$ is one row of $P$ dotted with $\mathbf{x}$ .

Column by column. Six chairs need 6 copies of the chair column, and 2 tables need 2 copies of the table column. So $P\mathbf{x}$ is the linear combination $6\,(4, 8, 1) + 2\,(4, 12, 3)$ : vectors scaled by numbers and added, as in lesson 3.1.1. The entries of $\mathbf{x}$ are the weights.

part	row $\cdot\ \mathbf{x}$	6 × chair column	2 × table column	total
legs	$4 \cdot 6 + 4 \cdot 2$	24	8	32
screws	$8 \cdot 6 + 12 \cdot 2$	48	24	72
boards	$1 \cdot 6 + 3 \cdot 2$	6	6	12

Both ways add the same six products, in a different order, so they agree: $P\mathbf{x} = (32, 72, 12)$ . The units work out too: parts per item times items gives parts.

Rule: Matrix times vector

\begin{gathered} (A\mathbf{x})_i = a_{i1}x_1 + \cdots + a_{in}x_n \\[4pt] A\mathbf{x} = x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n \end{gathered}

Entry $i$ of $A\mathbf{x}$ is row $i$ of $A$ dotted with $\mathbf{x}$ . The whole of $A\mathbf{x}$ is a combination of the columns $\mathbf{a}_1, \ldots, \mathbf{a}_n$ of $A$ , weighted by the entries of $\mathbf{x}$ .

Try it @ is the matrix product. The second line dots each row with x; the third combines the two columns. A 1-D array such as x has no rows or columns of its own. On the right of @, NumPy reads it as a column.

import numpy as np P = np.array([[4, 4], [8, 12], [1, 3]]) x = np.array([6, 2]) print(P @ x) print(P[0] @ x, P[1] @ x, P[2] @ x) print(6 * P[:, 0] + 2 * P[:, 1])

⌘/Ctrl + Enter runs the cell

Shapes, and a picture in the plane

$\mathbf{x}$ needs one entry for each column of $A$ , because each entry weights a column. The answer has one entry for each row. So an $m \times n$ matrix turns vectors with $n$ components into vectors with $m$ . $P$ turns an order (2 numbers) into a parts list (3 numbers).

A $2 \times 2$ matrix turns arrows in the plane into arrows in the plane. Take

A = \begin{pmatrix} 2 & 1 \\ -1 & 3 \end{pmatrix}

with columns $\mathbf{a}_1 = (2, -1)$ and $\mathbf{a}_2 = (1, 3)$ . For $\mathbf{x} = (2, 1)$ the rows give $(2 \cdot 2 + 1 \cdot 1,\ -1 \cdot 2 + 3 \cdot 1) = (5, 1)$ . The columns give $2\mathbf{a}_1 + \mathbf{a}_2 = (4, -2) + (1, 3)$ , the same $(5, 1)$ .

A negative entry turns its column round. For $\mathbf{x} = (1, -1)$ , $A\mathbf{x} = \mathbf{a}_1 - \mathbf{a}_2 = (1, -4)$ ; row by row, $2 - 1 = 1$ and $-1 - 3 = -4$ .

$A\mathbf{x}$ as a combination of the columns of $A$ , drawn tip to tail. The entries of $\mathbf{x}$ are the weights: $(2, 1)$ gives $2\mathbf{a}_1 + \mathbf{a}_2$ , and $(1, -1)$ gives $\mathbf{a}_1 - \mathbf{a}_2$ .

\mathbf{x}

import numpy as np A = np.array([[2, 1], [-1, 3]]) for x in [np.array([2, 1]), np.array([1, -1])]: print(x, A @ x, x[0] * A[:, 0] + x[1] * A[:, 1]) P = np.array([[4, 4], [8, 12], [1, 3]]) y = np.array([1, 2, 3]) try: P @ y except ValueError: print("ValueError:", P.shape, "times", y.shape)

⌘/Ctrl + Enter runs the cell

A matrix times a matrix

Two shops send orders: shop 1 wants 6 chairs and 2 tables, shop 2 wants 10 chairs and 1 table. Put each order in a column of a matrix $B$ , with rows for the products:

B = \begin{pmatrix} 6 & 10 \\ 2 & 1 \end{pmatrix}

The parts for shop $j$ are $P$ times column $j$ of $B$ . Side by side, these columns make the product $PB$ :

PB = \begin{pmatrix} 32 & 44 \\ 72 & 92 \\ 12 & 13 \end{pmatrix}

The first column is the parts list found above. Shop 2 needs $4 \cdot 10 + 4 \cdot 1 = 44$ legs, $8 \cdot 10 + 12 \cdot 1 = 92$ screws and $10 + 3 = 13$ boards.

Rule: Matrix times matrix

(AB)_{ij} = a_{i1}b_{1j} + \cdots + a_{in}b_{nj}

Column $j$ of $AB$ is $A$ times column $j$ of $B$ . Entry $(i, j)$ of $AB$ is row $i$ of $A$ dotted with column $j$ of $B$ . An $m \times n$ matrix times an $n \times p$ matrix gives an $m \times p$ matrix.

The inner sizes must match: each row of $A$ must be as long as each column of $B$ . In the workshop the inner size counts the products.

$P$ is parts by products and $B$ is products by shops, so $PB$ is parts by shops. The sum in each entry runs over the products.

The picture shows the rule for $M = \begin{pmatrix} 1 & 2 & 0 \\ 3 & -1 & 4 \end{pmatrix}$ , which is $2 \times 3$ , times $N = \begin{pmatrix} 1 & 0 \\ 2 & 1 \\ 0 & 3 \end{pmatrix}$ , which is $3 \times 2$ .

Entry $(2, 1)$ of $MN$ comes from row 2 of $M$ and column 1 of $N$ . The 3 entries of the row pair off with the 3 entries of the column; that shared size drops out of the shape of the answer.

A second example, with two square matrices: $A$ from above and

C = \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix}

Each entry of $AC$ is a row of $A$ dotted with a column of $C$ :

entry	row of $A$	column of $C$	dot product
$(1, 1)$	$(2, 1)$	$(1, 2)$	$2 + 2 = 4$
$(1, 2)$	$(2, 1)$	$(0, 1)$	$0 + 1 = 1$
$(2, 1)$	$(-1, 3)$	$(1, 2)$	$-1 + 6 = 5$
$(2, 2)$	$(-1, 3)$	$(0, 1)$	$0 + 3 = 3$

AC = \begin{pmatrix} 4 & 1 \\ 5 & 3 \end{pmatrix}

A

import numpy as np P = np.array([[4, 4], [8, 12], [1, 3]]) B = np.array([[6, 10], [2, 1]]) print(P @ B) print(P @ B[:, 0], P @ B[:, 1]) A = np.array([[2, 1], [-1, 3]]) C = np.array([[1, 0], [2, 1]]) print(A @ C) for i in range(2): for j in range(2): print(i + 1, j + 1, A[i], C[:, j], A[i] @ C[:, j])

⌘/Ctrl + Enter runs the cell

Order matters, grouping does not

With numbers, $3 \times 5 = 5 \times 3$ . With matrices, swapping the order usually changes the answer, and can even make it meaningless. There are four cases to know.

Both orders exist, with different answers. $CA = \begin{pmatrix} 2 & 1 \\ 3 & 5 \end{pmatrix}$ , while $AC = \begin{pmatrix} 4 & 1 \\ 5 & 3 \end{pmatrix}$ .
Both orders exist, with different shapes. $M$ is $2 \times 3$ and $N$ is $3 \times 2$ , so $MN$ is $2 \times 2$ but $NM$ is $3 \times 3$ .
Only one order exists. $PB$ is $(3 \times 2)(2 \times 2)$ . $BP$ would be $(2 \times 2)(3 \times 2)$ : rows of length 2 against columns of length 3, so it is not defined.
Sometimes the order does not matter. Two matrices with zeros off the diagonal (the entries from top left to bottom right), such as $\begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$ and $\begin{pmatrix} 5 & 0 \\ 0 & -1 \end{pmatrix}$ , give the same product either way. That is the exception, so always check.

The extreme case is a row times a column. The row $(3\ \ 1)$ is $1 \times 2$ and the column $(1, 2)$ is $2 \times 1$ . Their product is $1 \times 1$ : the single number $3 + 2 = 5$ , their dot product.

The other order is $(2 \times 1)(1 \times 2)$ , a whole $2 \times 2$ table. Lesson 3.2.3 writes the row as a column turned on its side.

\begin{pmatrix} 1 \\ 2 \end{pmatrix} \begin{pmatrix} 3 & 1 \end{pmatrix} = \begin{pmatrix} 3 & 1 \\ 6 & 2 \end{pmatrix}

The grouping of a product, unlike its order, never matters. With $D = \begin{pmatrix} 0 & 1 \\ 1 & 1 \end{pmatrix}$ , multiplying $AC$ by $D$ gives the same as $A$ times $CD$ : both are $\begin{pmatrix} 1 & 5 \\ 3 & 8 \end{pmatrix}$ .

Rule: Order and grouping

\begin{gathered} AB \neq BA \text{ in general} \\[4pt] (AB)C = A(BC) \end{gathered}

Keep the matrices in their order. The brackets can go anywhere, so a product of three is written $ABC$ .

CA

import numpy as np A = np.array([[2, 1], [-1, 3]]) C = np.array([[1, 0], [2, 1]]) D = np.array([[0, 1], [1, 1]]) print(C @ A) print((A @ C) @ D) print(A @ (C @ D)) print(A * C) M = np.array([[1, 2, 0], [3, -1, 4]]) N = np.array([[1, 0], [2, 1], [0, 3]]) print(M @ N) print(N @ M) u = np.array([[3, 1]]) # a row: 1 x 2 v = np.array([[1], [2]]) # a column: 2 x 1 print(u @ v, (u @ v).shape) print(v @ u) S = np.array([[2, 0], [0, 3]]) T = np.array([[5, 0], [0, -1]]) print(np.array_equal(S @ T, T @ S))

⌘/Ctrl + Enter runs the cell

`@`, not `*`

In NumPy, A * C multiplies matching entries: $2 \cdot 1$ , $1 \cdot 0$ , $-1 \cdot 2$ and $3 \cdot 1$ , giving $\begin{pmatrix} 2 & 0 \\ -2 & 3 \end{pmatrix}$ . That is not the matrix product $AC$ . Use @ for the product.

Lesson 3.2.2 reads a matrix as a map that moves the plane, and lesson 3.3.1 runs the product backwards to solve $A\mathbf{x} = \mathbf{b}$ . Case study 3.2.5 writes the variance of a portfolio as a product of a row, a matrix and a column.

Common mistakes

Rows first. An $m \times n$ matrix has $m$ rows, and $a_{ij}$ is in row $i$ , column $j$ . In NumPy, counting from 0, $a_{12}$ is A[0, 1].
One entry of $\mathbf{x}$ per column. $A\mathbf{x}$ needs $\mathbf{x}$ as long as a row of $A$ , and has as many entries as $A$ has rows.
Check the inner sizes. $(m \times n)(n \times p)$ works and gives $m \times p$ . If the inner sizes differ, there is no product.
Keep the order. $AB$ and $BA$ usually differ, may have different shapes, and one of them may not exist.
A row times a column is a number; a column times a row is a table.
* is not the matrix product. In NumPy it multiplies entry by entry; the product is @.

Matrices and their products

0 of 13 correct

1.
A matrix has 4 rows and 3 columns. How many entries does it have?
2.
For $A = \begin{pmatrix} 2 & 7 & 1 \\ 5 & 0 & 3 \end{pmatrix}$ , what is $a_{21}$ ?
3.
With the workshop matrix $P$ , how many screws do 3 chairs and 5 tables need?
4.
For $A = \begin{pmatrix} 3 & 1 \\ 2 & 4 \end{pmatrix}$ and $\mathbf{x} = (2, 5)$ , find the first entry of $A\mathbf{x}$
5.
$A$ has columns $(1, 3)$ and $(2, 4)$ . Find the second entry of $A\mathbf{x}$ for $\mathbf{x} = (-1, 1)$
6.
$A$ is $3 \times 4$ . How many entries must $\mathbf{x}$ have for $A\mathbf{x}$ to exist?
7.
For $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ and $B = \begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}$ , find the entry in row 2, column 1 of $AB$
8.
$A$ is $2 \times 3$ and $B$ is $3 \times 4$ . How many entries does $AB$ have?
9.
$A$ is $2 \times 3$ , $B$ is $3 \times 3$ and $C$ is $3 \times 1$ . How many entries does $ABC$ have?
10.
For $A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$ and $B = \begin{pmatrix} 1 & 0 \\ 1 & 1 \end{pmatrix}$ , the top-left entry of $AB$ is 2. Find the top-left entry of $BA$
11.
A $3 \times 2$ matrix times a $2 \times 3$ matrix: how many rows does the product have?
12.
The row $(2\ \ {-1}\ \ 3)$ times the column $(1, 4, 2)$ is a single number. What is it?
13.
The column $(1, 2)$ times the row $(3\ \ 5)$ is a $2 \times 2$ matrix. Find its entry in row 2, column 2

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

m \times n

Matrices and matrix multiplication

What a matrix is

Rule: An m × n matrix

A matrix times a vector

Rule: Matrix times vector

Shapes, and a picture in the plane

A matrix times a matrix

Rule: Matrix times matrix

Order matters, grouping does not

Rule: Order and grouping

`@`, not `*`

Common mistakes

Matrices and their products

Summary card

Free further reading

What a matrix is

Rule: An m × n matrix

A matrix times a vector

Rule: Matrix times vector

Shapes, and a picture in the plane

A matrix times a matrix

Rule: Matrix times matrix

Order matters, grouping does not

Rule: Order and grouping

@, not *

Common mistakes

Summary card

Free further reading

`@`, not `*`