Conjugacy and Gaussian fusion

How should a belief and a measurement be combined?

Statistics usually hands you a sample and asks for a parameter. Bayesian inference hands you two distributions and asks for a third: start from a prior over an unknown, multiply by the likelihood of the data, normalise, and read off the posterior. For an arbitrary prior and an arbitrary likelihood that multiplication can be an intractable integral. But there is a special case where everything stays closed-form, and it happens to be the case that runs the machinery of robotics, tracking, and sensor fusion.

Take a single unknown number μ. Before seeing anything, you describe it with a Gaussian N(μ₀, σ₀²). An observation arrives as a number x, and the sensor's error is Gaussian with variance σ², so the likelihood of x given μ is a Gaussian centred at μ. Equivalently, the observation itself is a Gaussian centred at x. Bayes' rule multiplies the two curves point by point, then rescales so the area is one.

A Gaussian times a Gaussian is a Gaussian. The posterior is therefore again in the family you started from: a Gaussian. This self-reproducing property is called conjugacy, and it is what makes the arithmetic on this page possible without a single numerical integral.

Why the posterior stays Gaussian

Conjugacy is a promise about the shape of the answer. If your prior and your likelihood are both Gaussian, the posterior is Gaussian; you never leave the family. The prior contributes a curve that falls off like exp(−(μ−μ₀)²/2σ₀²), the likelihood contributes one that falls off like exp(−(x−μ)²/2σ²), and multiplying adds their exponents. The result is a quadratic in μ, and a quadratic exponent is precisely a Gaussian. Completing the square gives both the new centre and the new width at once:

Read the second expression first, because it is the clean one: the posterior precision is the sum of the prior precision and the observation precision. The first expression then says the posterior mean is the average of the two centres, weighted by those same precisions. There is no approximation here; this is the exact posterior under the stated model.

Conjugacy is a convenience, not a law of nature. Change the likelihood to something non-Gaussian, or put a hard constraint on the parameter, and the multiplication no longer closes; you are pushed toward the numerical methods of the MCMC part of this series. The Gaussian–Gaussian pair is the one case where the algebra is simple enough to reason about by eye, and it happens to be the case that matters most in practice. It is the same algebra that appears whenever independent Gaussian errors are added, as in the probability guide: adding variances there is adding reciprocals of precisions here.

The formula also makes clear what the prior is for. It is not a mystical extra ingredient; it is one more Gaussian observation, carrying its own precision. A vague prior carries little precision and is quickly overwhelmed. A confident prior carries a lot and resists a noisy measurement. Everything in the next section follows from that single reading.

Why the sharper source wins

Rewrite the update in terms of precision, τ = 1/σ². Variance is the error you carry; precision is the information you have. The update says the posteriors add their information, and the mean becomes a weighted average whose weights are the precisions themselves:

Two consequences follow immediately, and both are visible in the demo below. First, the answer is always sharper than either input: precisions are positive, so the sum exceeds each one, and the posterior variance is smaller than both σ₀² and σ². Fusing independent information can never make you less certain. Second, the blend leans toward whichever source has the larger precision. If the sensor is ten times more precise than your prior, it receives roughly ten elevenths of the weight.

Push the limits and the formula becomes a statement about humility. Let the observation variance go to infinity — a useless sensor — and τ → 0; the posterior collapses onto the prior. Let the prior variance go to infinity — you knew nothing — and τ₀ → 0; the posterior collapses onto the observation. A flat prior is the formal way of saying "I have no opinion", and with it the Gaussian update reduces to "believe the data".

Independence is doing real work in the word "add". If two readings share a calibration error, adding their precisions double-counts the same information and you will be overconfident. That failure, not the arithmetic, is what breaks sensor fusion in practice, and it is the reason an engineer spends more time on error models than on the update equation.

The signature interaction

The panel draws a prior (blue) and an observation (pink), both Gaussian. Drag either handle along the axis to move its mean; use the sliders to change each width. Press Merge and the posterior (dark) animates in, drawn from the same formula the text derived. Watch the posterior: it sits between the means, nearer the sharper curve, and it is always narrower than both.

Try dragging the observation far from the prior. If the observation is narrow, the posterior follows it almost completely; widen the observation and the prior pulls the answer back. Set the two widths equal and the posterior lands exactly halfway between the means, because equal variance means equal precision. Every movement recomputes the posterior from the precision formula; nothing is cached and nothing is approximated.

The readout also computes the same update the Kalman filter will use later in this series. In one dimension, with H = 1 and measurement noise R = σ², the filter forms the gain K = σ₀²/(σ₀²+σ²), then μ′ = μ₀ + K(x − μ₀) and σ′² = (1−K)σ₀². Substitute K, simplify once, and you get the precision formula exactly. The page runs the Kalman update on your current numbers, and the readout confirms it lands on the same mean and variance to machine precision. The scalar Kalman update is not an analogy for Gaussian fusion; it is the same equation rearranged.

From one fusion to a recursive estimator

One fusion is an event. The interesting behaviour starts when you repeat it. The posterior is a Gaussian, so it can play the role of the prior in the next fusion. Feed in a second observation and its precision is added to the posterior precision; a third adds again. After n independent observations of the same quantity, each with precision τ, the total is τ₀ + nτ and the variance falls like 1/(τ₀ + nτ). The mean drifts toward the data, more and more slowly, as the accumulated precision grows.

That is a recursive estimator: a running summary — here just a mean and a variance — that you update in place as each measurement arrives, without storing the measurements themselves. The prior is not a fixed thing; it is whatever you believed a moment ago. This is why the Gaussian pair is the natural language of tracking. The full Kalman filter adds a prediction step between updates: it lets the state move, which inflates the variance, and then the measurement update sharpens it again. Predict, update, predict, update — the Gaussian fusion on this page, run forever, with a motion model stretched between the steps.

Sequential and batch updating agree. Fuse observations one at a time or all at once and the final posterior is identical, because precision addition does not care about order. That equivalence is what lets an online filter produce exactly the answer a statistician would get by keeping every measurement and fitting at the end. It is also what makes the filter computationally cheap: each step costs the same, regardless of how long the system has been running. When the hidden state is a full vector with a covariance matrix, the scalar reciprocal becomes a matrix inverse and the precision sum becomes a matrix sum, but the shape of the idea — add information, reweight, sharpen — is unchanged.

One update, two worlds

The pattern is not particular to robots. Any pipeline that maintains a running estimate and folds in a stream of noisy measurements is doing Gaussian fusion somewhere: inertial navigation, satellite positioning, radar tracking, camera calibration, and the state estimate inside a visual odometry front end. The precision-weighted average, written once, is the whole idea.

Further reading

The references below treat conjugate updates as the analytic core of Bayesian inference rather than a special trick. If you take away one thing, take away the picture of two precision bars stacking into a taller one while the mean settles between them.

• Christopher Bishop, Pattern Recognition and Machine Learning, §2.3.3 — the Gaussian product derived by completing the square, exactly as above.
• Kevin Murphy, Machine Learning: A Probabilistic Perspective, §4.2.2 and §18.3 — conjugate Gaussian updates and their continuation into the Kalman filter.
• Andrew Gelman et al., Bayesian Data Analysis, chapter 2 — conjugate priors as the analytic starting point for posterior reasoning.
• 3Blue1Brown, "Bayes theorem, the geometry of changing beliefs" — the geometric reading of prior times likelihood that motivates the fusion picture.

Cheat sheet