A toolbox of derivatives
The definition from Part 3 works every time, but nobody takes a limit at breakfast. This part turns it into a kit: product and quotient rules derived from the definition, the power rule stated, and the familiar derivatives of sin, cos, exp and ln checked against numbers rather than asserted.
The product rule is a rectangle
Two quantities changing at once
Suppose two quantities $u(t)$ and $v(t)$ both change with time, and we want the rate of change of their product. Draw the product as a rectangle of width $u$ and height $v$, so its area is $uv$. When a small step of time passes, the width grows by $\Delta u$ and the height by $\Delta v$. The area gains three new pieces: a strip on the right of area $v\,\Delta u$, a strip on top of area $u\,\Delta v$, and a corner of area $\Delta u\,\Delta v$.
Divide by $\Delta t$ and let it shrink. The two strips are each proportional to a single small step, so they survive the limit; the corner is a product of two small steps, so it is second order and vanishes. What is left is the product rule.
The rectangle $uv$ (blue) plus the strip $v\,\Delta u$ (pink), the strip $u\,\Delta v$ (blue) and the corner $\Delta u\,\Delta v$ (dark). Drag the four sliders and watch the corner stay small compared with the strips.
The power rule
One family, one formula
State it first, derive the picture second: for any constant exponent $n$,
This is the difference quotient from Part 3 applied to $x^n$. The top panel is the curve with its tangent at the current point; the bottom panel samples the slope numerically and lays the formula $n\,x^{\,n-1}$ on top. Dial in any $n$ and slide $x$: the numeric derivative and the formula agree to the precision of the central difference.
Top: $f(x)=x^n$ with the tangent at the current $x$. Bottom: the numerically measured derivative (solid) against the formula $n\,x^{\,n-1}$ (dashed).
The reciprocal rule
One over a function
Before dividing two functions, handle one over a function. If $g(x)=1/v(x)$, then
The minus sign is the surprise: as $v$ grows, $1/v$ falls, and it falls fastest where $v$ is small. The curve below is $1/(1+ax^2)$ — a bump — and the tangent at the marker has slope given by that formula. Change the width parameter $a$ or slide $x$ and compare the numeric slope with the formula.
The reciprocal curve with its tangent. The steeper the underlying $v$ grows, the more negative the slope of $1/v$.
The quotient rule
Divide and differentiate
Now let $f = u/v$. The quotient rule is the product rule applied to the product $u \cdot (1/v)$, with the reciprocal rule supplying the second factor:
Take the concrete function $f(x)=x/(1+x^2)$. Here $u=x$ and $v=1+x^2$, so $u'v-uv' = 1\cdot(1+x^2) - x\cdot 2x = 1-x^2$, and the formula reads $(1-x^2)/(1+x^2)^2$. The bottom panel overlays the numerically measured derivative on that formula; the marker and the readout let you check any point.
Top: $f(x)=x/(1+x^2)$ with the tangent at the marker. Bottom: numeric derivative (solid) against the quotient-rule formula (dashed).
The derivatives worth memorising
A small table, checked against numbers
The rules above generate these, but they are faster recalled than re-derived. Pick a function and slide $x$: the solid curve is the numerical derivative of the chosen $f$, and the dashed curve is the remembered formula. They coincide wherever the function is smooth.
The function (faint, dotted), its numeric derivative (solid) and the known formula (dashed). Switch functions and the axes rescale.
Where this shows up
The grammar that composes the vocabulary
Every rule here handles one operation: multiply, divide, raise to a power. The toolbox is still missing the operation that matters most — applying one function to the output of another — and that is the subject of Part 5, The chain rule. The chain rule is the one rule that composes all the others: differentiate a complicated expression and the product, quotient and power rules fire inside it in the order the chain rule dictates.
That ordering is also what makes a network trainable. Backpropagation is the chain rule run backwards through a composition, so the rules on this page are the pieces it multiplies together — see Backpropagation once the chain rule is in hand.
Rules to carry forward
| Rule | Formula | The picture |
|---|---|---|
| Constant | (c)' = 0 | A flat line has no slope |
| Power | (xⁿ)' = n xⁿ⁻¹ | The difference quotient of xⁿ, Step 2 |
| Sum | (u + v)' = u' + v' | Slopes add; the graph just tilts by both amounts |
| Constant multiple | (cu)' = c u' | A vertical stretch scales every slope by the same c |
| Product | (uv)' = u'v + uv' | The growing rectangle: two strips, Step 1 |
| Reciprocal | (1/v)' = −v'/v² | The bump falls fastest where v is small, Step 3 |
| Quotient | (u/v)' = (u'v − uv')/v² | Product rule on u · (1/v), Step 4 |
| Chain | (f(g(x)))' = f'(g(x)) g'(x) | Gears in series: multiply the rates (Part 5) |
| sin | (sin x)' = cos x | Numeric slope lands on the cosine curve |
| cos | (cos x)' = −sin x | Numeric slope lands on the negated sine curve |
| exp | (eˣ)' = eˣ | The slope equals the height everywhere |
| ln | (ln x)' = 1/x | The slope decays like a reciprocal, Step 5 |
Further reading
- 3Blue1Brown, Visualizing the chain rule and product rule — the rectangle picture in motion, and the composition that follows.
- MIT 18.01, Single Variable Calculus — lecture notes and proofs for every rule on this page.
- Paul's Online Math Notes, Derivative proofs — the product and quotient rules derived carefully from the limit definition.