Calculus of probability

Probability as area under a curve

A random variable $X$ has a probability density $p(x)$ if probabilities are given by integrals of it:

The density itself is not a probability: $p(x)$ can exceed $1$ (a narrow Gaussian does), because it is a probability per unit length. Only its integral is a probability. The total area is fixed at $1$ — that is the normalisation condition that makes the whole thing a distribution. The same integral defines every expectation you will meet later:

Below is a standard Gaussian. Drag the two handles (or the sliders) and watch the shaded area, computed by a fine trapezoid rule, report the probability of the interval. The readout also integrates the whole line to confirm the total is $1$.

Two integrals, then a shift and a stretch

The two numbers that summarise a distribution are themselves integrals:

Because the integral is linear, an affine map moves the mean and scales the spread in the obvious way. If $Z \sim \mathcal{N}(0,1)$ then $Y = \mu + \sigma Z$ is Gaussian with

The demo samples $Z$ once with a fixed seed, forms $Y = \mu + \sigma Z$, and histograms both. The analytic densities are overlaid; the sample mean and sample standard deviation track $\mu$ and $\sigma$. Note that the sample standard deviation is the $\sigma$ scaling made visible.

The density is divided by the derivative

An affine map is the easy case because its derivative is constant. For a general strictly monotone map $Y = g(Z)$, a small interval $dz$ is stretched into an interval of length $|g'(z)|\,dz$, so the probability $p_Z(z)\,dz$ must be packed into $p_Y(y)\,dy$. Setting the two probabilities equal gives the change-of-variables formula for densities:

The factor $|g'|$ is the one-dimensional Jacobian — exactly the local length scale from Part 5, where the two-dimensional version is $|\det J|$. Where the map stretches space, the density is thinned out; where it compresses space, the density piles up.

Pick a map, slide the probe, and read the three numbers that make the formula concrete: the density of $z$, the Jacobian factor $|g'(z)|$ at the probe, and their ratio, the density of $y$. The histogram is the empirical density of the transformed samples; the coloured curve is the analytic prediction.

Sample once, move the parameters

There is a way to read the affine rule that turns out to be the workhorse of modern generative modelling. Fix the noise: draw $z \sim \mathcal{N}(0,1)$ once, and write every Gaussian draw as a deterministic function of that noise,

Now $\mu$ and $\sigma$ are ordinary parameters sitting outside the random draw, so the sample $y$ depends on them smoothly and differentiably. This is the reparameterisation trick: a gradient can flow through $\mu$ and $\sigma$ into the samples. Drag the sliders and the whole cloud of fixed $z$'s slides and stretches rigidly — no resampling, no jitter, just an affine map of a fixed set of points.

An integral of a density ratio

To measure how far one distribution $q$ is from another $p$, integrate the log density ratio against $q$:

The two forms are the same statement: the integral is an expectation under $q$, so it can be estimated by sampling from $q$. For two Gaussians it has a closed form,

so we can check the Monte-Carlo estimate against an exact answer. The demo shades the integrand $q(x)\log\frac{q(x)}{p(x)}$ — green where it is positive, magenta where negative — and reports both numbers. The area under that curve is the KL.

This is the quantity inside the training objective. Cross-entropy splits as $H(q,p) = H(q) + D_{\mathrm{KL}}(q\|p)$, and the negative log-likelihood a model minimises is cross-entropy with $q$ the data distribution; when $q$ is a posterior over latents, the KL term is the regulariser that keeps it near a prior.

Every loss is an integral in disguise

Training a language model is this part applied at scale. The cross-entropy loss is $\mathbb{E}_{x\sim\text{data}}[-\log p_\theta(x)]$ — an expectation, hence an integral, hence the thing you differentiate; LLM Training builds its whole objective out of exactly that, and the KL regulariser in preference tuning is the divergence defined above. When the density lives in several variables the integral is a multiple integral and the Jacobian is a determinant — that is Part 6: Multiple integrals & coordinates. And when the density is constrained to a curved space, the Gaussian is written in the tangent plane and the change-of-variables factor becomes a metric — the beginning of Part 13: Calculus on manifolds.

• MIT 18.05, Introduction to Probability and Statistics — densities, expectations and transformations done carefully.
• Seeing Theory, Brown University — visual introductions to distributions, expectation and the central limit theorem.
• Gregory Gundersen, The reparameterization trick — why sampling can be made differentiable, with the VAE objective in view.