2.2.2 The derivative as a limit, and finite differences

A slope at every point

Lesson 2.2.1 found the slope of $x^2$ at 3 by letting chord slopes settle on 6. Nothing about 3 was special. Do the same at other points, with a gap of $h = 0.001$ on each side:

point $x$	from the right	from the left	slope
$-2$	$-3.999$	$-4.001$	$-4$
$-1$	$-1.999$	$-2.001$	$-2$
0	0.001	$-0.001$	0
1	2.001	1.999	2
2	4.001	3.999	4
3	6.001	5.999	6

Each slope is twice the point. Algebra shows why. At a general point $x$ , expand the top and cancel $h$ :

\frac{(x + h)^2 - x^2}{h} = \frac{2xh + h^2}{h} = 2x + h

As $h \to 0$ this tends to $2x$ . So the slope of $x^2$ at any input $x$ is $2x$ . That rule is a new function: the derivative.

Rule: Derivative

f'(x) = \lim_{h \to 0} \frac{f(x + h) - f(x)}{h}

The slope of $f$ at each input $x$ where this limit exists; there $f$ is differentiable. Read $f'(x)$ as "f prime of x". Finding $f'$ is called differentiating.

The derivative takes a point and returns the slope there. For $f(x) = x^2$ , $f'(3) = 6$ , as in lesson 2.2.1, and $f'(-1) = -2$ .

2x

def f(x): return x**2 h = 0.001 for x in [-2, -1, 0, 1, 2, 3]: right = (f(x + h) - f(x)) / h left = (f(x - h) - f(x)) / -h print(x, round(right, 6), round(left, 6), 2 * x)

⌘/Ctrl + Enter runs the cell

Reading the sign of the slope

The sign of $f'(x)$ says which way the graph is heading at $x$ .

$f'(x) > 0$ : the graph rises there. $x^2$ rises for every $x > 0$ .
$f'(x) < 0$ : the graph falls there. $x^2$ falls for every $x < 0$ .
$f'(x) = 0$ : the tangent is flat, as at the bottom of a valley. For $x^2$ that happens only at 0.

The sign says nothing about the height. At $x = -2$ , $x^2$ is 4, above zero, yet its slope is $-4$ : it is falling.

Slide the point along $x^2$ : the lower graph traces its slopes, the line $2x$ . Then switch to $x^3 - 3x$ . Its slope is zero at the hill ( $x = -1$ ) and the valley ( $x = 1$ ), and negative between them. The next section works out that slope, $3x^2 - 3$ , from the limit.

Explore

Slide the point along the curve. Its tangent shows the slope there, and the lower graph plots that slope for every x: the derivative.

The slope of y = x², which is 2x

Point at x = −2

Trace as you go

Point: (−2, 4)
Slope there: −4
The curve is: falling: the slope is negative, so the slope graph is below zero

Notation and units

If $y = f(x)$ , the derivative is also written $\frac{dy}{dx}$ , read "dee y by dee x". It recalls the chord slope $\frac{\Delta y}{\Delta x}$ of lesson 2.2.1, after the limit. It is one symbol, not a fraction of two numbers.

Rule: Two notations

f'(x) = \frac{dy}{dx}

Both mean the derivative of $y = f(x)$ . The slope at one point is written $f'(3)$ , or $\frac{dy}{dx}$ at $x = 3$ . The units are output units per input unit, as for any rate.

A square with side $s$ cm has area $A = s^2$ cm². By the first section, $\frac{dA}{ds} = 2s$ , in cm² per cm. At $s = 3$ the area grows at 6 cm² per cm of side.

So adding 0.1 cm to the side adds about $6 \times 0.1 = 0.6$ cm². The true gain is $3.1^2 - 9 = 0.61$ cm²: the tangent estimate of lesson 2.2.1.

Derivatives from the limit

Every derivative in this lesson comes from the same three steps.

Write the chord slope $\frac{f(x + h) - f(x)}{h}$ .
Simplify the top until $h$ cancels from the bottom.
Let $h \to 0$ .

$x^3$

Expanding gives $(x + h)^3 - x^3 = 3x^2h + 3xh^2 + h^3$ . Divide by $h$ to get $3x^2 + 3xh + h^2$ , which tends to $3x^2$ . At $x = 2$ that is 12, the slope lesson 2.2.1 found from chords.

For $x^3 - 3x$ , the extra $-3x$ adds $-3(x + h) + 3x = -3h$ to the top. The chord slope becomes $3x^2 + 3xh + h^2 - 3$ , so the derivative is $3x^2 - 3$ . It is zero at $x = -1$ and $x = 1$ , the hill and the valley in the slope tracer.

$\frac{1}{x}$

Put the two fractions over the common bottom $x(x + h)$ :

\begin{aligned} \frac{1}{x + h} - \frac{1}{x} &= \frac{x - (x + h)}{x(x + h)} \\ &= \frac{-h}{x(x + h)} \end{aligned}

Dividing by $h$ leaves $-\frac{1}{x(x + h)}$ , which tends to $-\frac{1}{x^2}$ . This holds for every $x \neq 0$ , and it is negative on both sides of 0: $\frac{1}{x}$ falls everywhere. At $x = 2$ the slope is $-0.25$ .

$\sqrt{x}$

Here no $h$ factors out of the top. Multiply the top and bottom by $\sqrt{x + h} + \sqrt{x}$ . The top becomes $(x + h) - x = h$ , by the trick $x - 9 = (\sqrt{x} - 3)(\sqrt{x} + 3)$ from lesson 2.1.1.

\begin{aligned} \frac{\sqrt{x + h} - \sqrt{x}}{h} &= \frac{(x + h) - x}{h\left(\sqrt{x + h} + \sqrt{x}\right)} \\ &= \frac{1}{\sqrt{x + h} + \sqrt{x}} \end{aligned}

As $h \to 0$ the bottom tends to $2\sqrt{x}$ , so the derivative is $\frac{1}{2\sqrt{x}}$ , for $x > 0$ . At $x = 4$ it is 0.25.

A pattern

Written as powers (lesson 1.4.1), $\frac{1}{x} = x^{-1}$ and $\sqrt{x} = x^{1/2}$ . In all four results the power comes down in front and drops by one: $2x$ , $3x^2$ , $-x^{-2}$ , $\frac{1}{2}x^{-1/2}$ . Lesson 2.3.1 turns that pattern into the power rule.

h \to 0

import math import sympy as sp x, h = sp.symbols("x h") for f in [x**2, x**3, x**3 - 3 * x, 1 / x, sp.sqrt(x)]: chord = (f.subs(x, x + h) - f) / h print(f, "|", sp.simplify(sp.expand(chord)), "|", sp.limit(chord, h, 0)) def slope(g, a, step): return (g(a + step) - g(a)) / step print(round(slope(lambda t: 1 / t, 2, 0.001), 6), round(slope(lambda t: 1 / t, 2, -0.001), 6)) print(round(slope(math.sqrt, 4, 0.001), 6), round(slope(math.sqrt, 4, -0.001), 6))

⌘/Ctrl + Enter runs the cell

Where a derivative does not exist

A derivative at $a$ needs a value $f(a)$ , and chord slopes that settle on one number from both sides. Three shapes break the second condition.

At a corner the two sides settle on different slopes. At a vertical tangent and at a jump the chord slopes grow without bound. In all three, $f$ has no derivative at 0.

A corner. For $|x|$ at 0, the chord slope is $\frac{|h|}{h}$ : 1 for every $h > 0$ and $-1$ for every $h < 0$ . The one-sided limits differ (lesson 2.1.2), so there is no derivative at 0.
A vertical tangent. For the cube root $\sqrt[3]{x}$ at 0, the chord slope is $\frac{\sqrt[3]{h}}{h}$ . It is 100 at $h = 0.001$ and 10,000 at $h = 0.000001$ , from either side. The tangent is vertical, and a vertical line has no slope (lesson 1.2.1).
A jump. Take the step $u(x)$ , which is 0 for $x < 0$ and 1 for $x \ge 0$ . From the right every chord slope is 0. From the left it is $\frac{0 - 1}{h}$ : 10 at $h = -0.1$ and 1,000 at $h = -0.001$ .

Away from 0, $|x|$ has slope $-1$ on the left and 1 on the right. That derivative is the function $\frac{|x|}{x}$ of lesson 2.1.2, with its jump at 0. Switch the slope tracer above to $|x|$ to watch it.

The ramp $\max(x, 0)$ of lesson 1.1.2 has a corner at 0 too: slope 0 on the left, 1 on the right. At 0, $\sqrt{x}$ has no left side, and from the right its chord slopes blow up like the cube root's.

Differentiable means continuous, not the reverse

Rule: Differentiable implies continuous

\begin{gathered} \text{differentiable at } a \\ \Downarrow \\ \text{continuous at } a \end{gathered}

If $f$ has a derivative at $a$ , its graph has no break at $a$ (lesson 2.1.2). The reverse fails: $|x|$ and $\sqrt[3]{x}$ are continuous at 0 but have no derivative there. A Proof Gym stage proves the first part.

So a jump, a hole or a blow-up always rules out a derivative. A graph without a break can still fail, at a corner or a vertical tangent.

-1

import math def chord(f, a, h): return (f(a + h) - f(a)) / h def step(x): return 1 if x >= 0 else 0 for h in [0.1, 0.001, 0.000001]: print("abs ", chord(abs, 0, h), chord(abs, 0, -h)) print("cbrt", round(chord(math.cbrt, 0, h), 4), round(chord(math.cbrt, 0, -h), 4)) print("step", chord(step, 0, h), chord(step, 0, -h))

⌘/Ctrl + Enter runs the cell

Finite differences

Sometimes you can compute $f(x)$ but cannot do the algebra: $f$ is a long program, or a price from a model. Then stop at a small $h$ instead of taking the limit. The result is a finite difference, an estimate of $f'(x)$ .

Rule: Finite differences

\begin{gathered} \text{forward: } \frac{f(x + h) - f(x)}{h} \\[4pt] \text{backward: } \frac{f(x) - f(x - h)}{h} \\[4pt] \text{central: } \frac{f(x + h) - f(x - h)}{2h} \end{gathered}

Forward uses a point to the right of $x$ , backward a point to the left. Central uses one on each side, $2h$ apart; it is the average of the other two.

Take $x^3$ at 1, where the true slope is $3 \times 1^2 = 3$ .

gap $h$	0.1	0.01	0.001
forward	3.31	3.0301	3.003001
backward	2.71	2.9701	2.997001
central	3.01	3.0001	3.000001

Algebra explains the pattern. Expanded, the forward difference is exactly $3 + 3h + h^2$ and the backward one $3 - 3h + h^2$ . The central one is $3 + h^2$ .

So the forward and backward errors are about $3h$ , with opposite signs: a gap 10 times smaller gives an error about 10 times smaller. The central error is $h^2$ : 10 times smaller $h$ , 100 times smaller error. For the same $h$ , central is far more accurate.

Case study 2.2.4 uses the central difference to measure how a bond price responds to its yield.

x^3

def f(x): return x**3 for h in [0.1, 0.01, 0.001]: forward = (f(1 + h) - f(1)) / h backward = (f(1) - f(1 - h)) / h central = (f(1 + h) - f(1 - h)) / (2 * h) print(h, round(forward, 6), round(backward, 6), round(central, 6))

⌘/Ctrl + Enter runs the cell

How small should $h$ be?

Smaller is better only down to a point. The computer stores about 16 significant digits (lesson 2.1.1). With a tiny $h$ , $f(x + h)$ and $f(x)$ agree in nearly all of them, and subtracting leaves mostly rounding error.

Dividing by $h$ then magnifies that error. A rounding error of about $10^{-16}$ divided by $h = 10^{-12}$ is about $10^{-4}$ . Here are the errors for $x^3$ at 1 as $h$ shrinks:

gap $h$	forward error	central error
$10^{-2}$	$3.0 \times 10^{-2}$	$1.0 \times 10^{-4}$
$10^{-5}$	$3.0 \times 10^{-5}$	$9.7 \times 10^{-11}$
$10^{-8}$	$4.0 \times 10^{-9}$	$7.1 \times 10^{-9}$
$10^{-11}$	$2.5 \times 10^{-7}$	$2.5 \times 10^{-7}$
$10^{-14}$	$2.4 \times 10^{-3}$	$2.4 \times 10^{-3}$

The errors of the forward and central differences of $x^3$ at 1, on log scales. Each falls as $h$ shrinks until rounding takes over; the central error falls faster and bottoms out at a larger $h$ .

For inputs of about size 1, the forward difference is best near $h = 10^{-8}$ and the central one near $h = 10^{-5}$ . Below that, every step smaller makes the answer worse. At $h = 10^{-20}$ , $1 + h$ is stored as exactly 1, and the forward difference of $x^2$ at 1 comes out as 0.

Lesson 2.8.1 explains the $3h$ and $h^2$ with Taylor series (lesson 2.7.1), and picks $h$ for any function.

3.0 \times 10^{-2}

def f(x): return x**3 for h in [1e-2, 1e-5, 1e-8, 1e-11, 1e-14]: forward = (f(1 + h) - f(1)) / h central = (f(1 + h) - f(1 - h)) / (2 * h) print(h, f"{abs(forward - 3):.1e}", f"{abs(central - 3):.1e}") print(((1 + 1e-20)**2 - 1) / 1e-20)

⌘/Ctrl + Enter runs the cell

Common mistakes

$f'(x)$ is a function; $f'(3)$ is a number. The derivative of $x^2$ is $2x$ , and its slope at 3 is 6.
The sign of $f'$ is about direction, not height. $x^2$ is positive at $-2$ but falling there.
Units are per input unit. $\frac{dA}{ds}$ for an area is in cm² per cm, not cm².
Mind the domain. $-\frac{1}{x^2}$ holds only for $x \neq 0$ , and $\frac{1}{2\sqrt{x}}$ only for $x > 0$ .
Check both sides. At a corner the slopes from the two sides differ. The slope of $|x|$ at 0 is not 0, nor the average of $-1$ and 1: it does not exist.
Continuous is not enough. $|x|$ and $\sqrt[3]{x}$ have no break at 0 and still have no derivative there.
A finite difference is an estimate. The forward one is off by about $3h$ for $x^3$ at 1; prefer the central one.
A smaller $h$ is not always better. Below about $10^{-8}$ , rounding swamps the answer.

The derivative

0 of 13 correct

1.
The derivative of $x^2$ is $2x$ . What is the slope of $x^2$ at $x = -3$ ?
2.
The chord slope of some $f$ simplifies to $4x + 2h$ . What is $f'(x)$ ?
3.
The slope of $x^3 - 3x$ is $3x^2 - 3$ . At which negative $x$ is its tangent flat?
4.
The derivative of $\frac{1}{x}$ is $-\frac{1}{x^2}$ . What is the slope at $x = \frac{1}{2}$ ?
5.
The derivative of $\sqrt{x}$ is $\frac{1}{2\sqrt{x}}$ . What is the slope at $x = 9$ ?
6.
A cube of side $s$ cm has volume $V = s^3$ cm³. How fast does $V$ grow, in cm³ per cm, when $s = 2$ ?
7.
Find the chord slope of $|x|$ at 0 with gap $h = -0.5$
8.
Find the chord slope of $\sqrt[3]{x}$ at 0 with gap $h = 0.000001$
9.
The step $u(x)$ is 0 for $x < 0$ and 1 for $x \ge 0$ . Find its chord slope at 0 with gap $h = -0.01$
10.
$f$ is differentiable at 2 and $f(2) = 7$ . What is $\displaystyle\lim_{x \to 2} f(x)$ ?
11.
Find the backward difference of $x^2$ at 3 with $h = 0.1$
12.
The central difference of $x^3$ at 1 is exactly $3 + h^2$ . How far off is it with $h = 0.01$ ?
13.
The computer stores $1 + 10^{-20}$ as exactly 1. What does the forward difference of $x^2$ at 1 give with $h = 10^{-20}$ ?

How to type answers

Powers with ^: x^7, 2^(1/3), a^-2.
Multiply with * or by writing side by side: 3x^2, 2(x+1).
ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
Your first answer to each question is saved: it counts towards your score and brings the question back for review.
Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

f'(x) = \lim_{h \to 0}\frac{f(x + h) - f(x)}{h}

The derivative as a limit, and finite differences

A slope at every point

Rule: Derivative

Reading the sign of the slope

Notation and units

Rule: Two notations

Derivatives from the limit

$x^3$

$\frac{1}{x}$

$\sqrt{x}$

A pattern

Where a derivative does not exist

Differentiable means continuous, not the reverse

Rule: Differentiable implies continuous

Finite differences

Rule: Finite differences

How small should $h$ be?

Common mistakes

The derivative

Summary card

Free further reading

A slope at every point

Rule: Derivative

Reading the sign of the slope

Notation and units

Rule: Two notations

Derivatives from the limit

x3x^3x3

1x\frac{1}{x}x1​

x\sqrt{x}x​

A pattern

Where a derivative does not exist

Differentiable means continuous, not the reverse

Rule: Differentiable implies continuous

Finite differences

Rule: Finite differences

How small should hhh be?

Common mistakes

Summary card

Free further reading

$x^3$

$\frac{1}{x}$

$\sqrt{x}$

How small should $h$ be?