Linear Algebra, Interactively
A step-by-step guide to the subject every other guide assumes, in the 3Blue1Brown tradition - geometry first, algebra second, and every idea attached to something you can drag. It starts at what a vector is and ends at attention matrices, Gauss-Newton and SO(3).
Parts build on each other, but each one stands alone. A single running example — a 2D point cloud that becomes, in turn, a set of vectors, a transformed grid, a least-squares fit, a covariance ellipse and a PCA basis — threads through the first three acts so you see the same object gain meaning. If you have seen the symbols and never seen the pictures, start at Part 1. If you are here for a specific tool, jump straight to it, and keep the glossary open for notation.
The parts
An arrow, a list of numbers, and an abstract object you can add and scale — and why they are the same thing.
When two vectors reach the whole plane, when they collapse to a line, and what a basis is for.
The one idea that makes matrices inevitable: a linear map is determined by where it sends the basis vectors.
Applying one transformation after another is a matrix product — which is why AB is not BA.
The signed area of the unit square after a transformation, and why det = 0 means information is destroyed.
Elimination as a sequence of row operations, and the geometric picture of three planes that meet at a point, a line, or nowhere.
Column space, null space, row space and left null space — the four subspaces that rank ties together.
Projection, cosine similarity, and the duality that a 1×n matrix is really a vector in disguise.
Turning any set of vectors into a perpendicular one, and why QR is what you actually compute.
When Ax = b has no solution, the best you can do is project b onto the column space.
The area of a parallelogram, the normal to a plane, and the matrix form of the cross product.
The same linear map looks different in every basis; P⁻¹AP is the change of costume.
Sweep a vector around a circle: almost all of them turn, except the special directions that only stretch.
Iterating x ← Ax, and why the eigenvalue magnitude decides whether a system settles, cycles or explodes.
Matrices you can rotate to face you: real eigenvalues, orthogonal eigenvectors, and xᵀAx as a shape.
Every matrix is a rotation, a stretch, and another rotation — the geometry behind everything in Act IV.
Keeping only the biggest singular values, and what PCA really does to a point cloud.
One widget for every shape of Ax = b: unique, least-squares, least-norm, and the big picture that unifies them.
Why a tiny change in b can move x a long way, and what each decomposition costs.
Rotations as matrices, angle-axis and the exponential map, gimbal lock, and why quaternions exist.
Jacobians, Hessians and the Gauss–Newton step, with the sparsity pattern that makes pose graphs tractable.
Attention as three matrices, and a low-rank update to a weight matrix — the linear algebra a transformer runs on.