Differential equations

A rule for the velocity, plus one point the path must pass through

An ordinary differential equation (ODE) does not give you a function. It gives you the slope of a function at every place it could be. Written for a single unknown,

it says: at time $t$, if the state is $y$, move at velocity $f(t, y)$. That pins down a whole family of curves, one through every starting point. A physical problem selects one of them by naming where it starts, and the pair is an initial value problem (IVP):

The angle $\theta$ of a pendulum obeys $\theta'' = -(g/L)\sin\theta$; naming $\omega = \theta'$ turns that second-order equation into the first-order system $\theta' = \omega,\ \omega' = -(g/L)\sin\theta$. A system is just an ODE with a vector state, and everything below works the same way.

Part 1 drew streamlines of a vector field — curves tangent to the arrows at every point. Those streamlines are the solutions of $\dot p = F(p)$: every arrow a velocity, every curve a trajectory. For a scalar ODE the picture is the same with one component fixed — at each point draw a short tick whose slope is $f(t,y)$, and you have a slope field. Drop a point into it and integrate the rule; the curve that comes out is the unique solution through that point. Drag the point.

Building a stepper out of the Taylor series

Taylor's theorem — the series of Volume I, Part 7 — says that for a small step $h$,

Euler's method keeps the first two terms, throws the rest away, and replaces $y'(t)$ by the rule $f(t,y)$:

The discarded terms are $\tfrac{1}{2}h^2 y'' + \cdots$, so one step is wrong by about $h^2$. Those errors accumulate over $1/h$ steps, leaving a global error of order $h$: halve the step and the final error roughly halves. A method is of order $p$ when its global error is $O(h^p)$.

To do better, cancel the thrown-away terms instead of ignoring them. The midpoint rule samples the slope twice — at the point and halfway along a trial Euler step — and reaches order 2. Runge–Kutta 4 (RK4) samples four slopes: at the start, twice at the midpoint, once at the end, and averages them with weights $\tfrac16(1,2,2,1)$:

with $k_1 = f(t_n, y_n)$, $k_2 = f(t_n + \tfrac{h}{2},\, y_n + \tfrac{h}{2}k_1)$, and so on. Matching the Taylor expansion term by term cancels errors through $h^4$, so RK4 is order 4: halve $h$ and the error falls by a factor of sixteen.

All three run below on $y' = y - y^2$ — the logistic equation, a population growing to its carrying capacity — whose exact solution is known. Slide the step size and compare the curves; the second canvas is the punchline, a log–log plot where the slopes of the lines are the orders of accuracy.

A nonlinear ODE whose period is not $2\pi\sqrt{L/g}$

The pendulum is the model problem for everything that follows. Its exact equation of motion, with a damping term, is

and writing $\omega = \theta'$ makes it the system $\theta' = \omega,\ \omega' = -(g/L)\sin\theta - b\omega$. The state is now the pair $(\theta, \omega)$, so the solutions live in a plane. Plotting one coordinate against the other gives the phase portrait: closed loops mean oscillation, an inward spiral means damping is winning, and the equilibrium at the bottom is the fixed point $(\theta,\omega) = (0,0)$.

Multiplying by $\theta'$ and integrating turns the equation into a conserved quantity, the energy

which is constant when $b = 0$ and decreases when $b > 0$. It also sets the swing's size: the turning points are where $E = (g/L)(1-\cos\theta)$. For tiny angles $\sin\theta \approx \theta$ and the period is the familiar $2\pi\sqrt{L/g}$; pull the pendulum further out and $\sin\theta$ saturates below $\theta$, the restoring force is weaker than linear, and the period grows. Slide $\theta_0$ to see it, and $b$ to watch the energy drain away. Press play, then reset.

Where the state is a pose, not a number

The pose of a differential-drive robot is the triple $(x, y, \theta)$: where it is and which way it faces. With wheel speed $v$ and turn rate $\omega$, the kinematics are three coupled ODEs,

Notice the nonlinearity: the right-hand side depends on the state's own heading through $\cos\theta$ and $\sin\theta$. Integrate the rule and the robot's path falls out. Two facts are worth reading off the picture. First, if $\omega \neq 0$ the robot traces a circle of radius $R = v/\omega$, so driving forward while turning is how it reaches anywhere — this is the same non-holonomic constraint that makes car-like robots awkward to park. Second, the arc length of the path is the integral of speed, $\int |v|\,dt$, which for a constant $v$ is just $v$ times the elapsed time.

Set $v$ and $\omega$ and watch the path re-solve. The readout gives the pose, the path length and the turning radius.

The derivative as a stability test

An equilibrium is a state $z^\ast$ where the field vanishes: $f(z^\ast) = 0$. Park the system there and nothing moves. Whether it stays is a question about the derivative. Near $z^\ast$, Taylor expands the field as

so arbitrarily close to the equilibrium the nonlinear system behaves like the linear one $\delta' = J\delta$. Its solutions are sums of $e^{\lambda_j t}$, where the $\lambda_j$ are the eigenvalues of $J$. Therefore:

and when some $\operatorname{Re}\lambda_j = 0$ the linear test is inconclusive — the nonlinear terms decide. This is the payoff of the whole derivative story: a first-order local model answers a global question. The pendulum illustrates both cases: hanging down, its eigenvalues are a stable pair, while balanced upright it is a saddle whose unstable direction is the one it tips into.

Below is a gently nonlinear system chosen so its Jacobian at the origin has eigenvalues $\mu \pm i$. Slide $\mu$ through zero and watch the origin go stable, marginal, then unstable. The unstable growth does not run away, though — it saturates on a limit cycle of radius $\sqrt{\mu}$, drawn dashed. Drag the seed to trace a trajectory.

Every moving system on the site is an ODE

The unicycle above is the kinematic model behind most mobile robots, and the same three equations drive dead reckoning. Those equations come in forward and inverse forms, and integrating them is what turns the pose into an estimate that drifts. The step-by-step integrator is also the inner loop of a physics simulator and the prediction half of a Kalman filter.

When the state lives on something curved — a rotation, a sphere, a set of poses — a flat Euler step leaves the surface and the update has to be taken along it. That repair is Part 13: Calculus on manifolds. And the energy and path length read off above are definite integrals: the tools that evaluate them, exactly or numerically, are collected in Volume I: Integration techniques.

• 3Blue1Brown, Differential equations — the slope-field and phase-portrait pictures, animated.
• MIT 18.03SC, Differential Equations — the course that runs alongside this part, ODEs and linear systems in full.
• Wikipedia, Numerical methods for ordinary differential equations — the Euler, midpoint and Runge–Kutta families with their error orders.