Rank, null space, and the four subspaces

What survives the map, and what is lost

A matrix is a function, and every function does two things at once: it collapses some part of its input to a single output, and its image covers some region of the output space. For a linear map those two facts are subspaces. The inputs that land on zero form the null space. The outputs that can be hit form the column space, also called the image or range. The first is what the map destroys; the second is what it keeps.

The rank is the dimension of the column space — the number of independent output directions the matrix can produce. It is also the dimension of the row space, and the reason those two numbers agree is the deepest small fact in the subject. What rank does not count is the null space, and the exact relationship between the two is an accounting identity you can read off the columns of the matrix without solving anything.

There is a compact way to say all of it. Rank is the number of dimensions that survive the map. A matrix takes an input space of dimension n, and the directions it does not destroy form a subspace whose dimension equals the rank; the remaining n − r dimensions are exactly what it collapses. Nothing in the output can exceed the rank, because a linear combination of columns cannot invent a direction that is not already among them. Reachability and uniqueness are therefore two readings of the same number, taken from opposite ends of the map.

These ideas are not abstractions for their own sake. They are the language in which every later part is written: least squares asks whether b lies in the column space, the pseudoinverse is built from the four subspaces, and PCA is a statement about how much of a data cloud lives in a low-rank approximation. Getting the picture straight here pays for itself many times over.

Two arrows in space, and the inputs they miss

Take a three-by-two matrix. Its two columns are arrows in three-dimensional space, and the column space is everything their linear combinations can reach: normally a plane through the origin, because two arrows that point in genuinely different directions sweep out a flat sheet. Press the columns together until they point along the same line and the sheet collapses to that line. The dimension of the reachable set — the rank — drops from two to one, and you can see it happen.

There is a mechanical way to find the rank that matches the picture. Row-reduce the matrix to echelon form and count the pivots: each pivot marks a column that contributes a new independent direction. In a three-by-two matrix there can be at most two pivots, because two columns cannot span more than a two-dimensional sheet in three dimensions. That cap is a general rule in disguise: the rank can never exceed the smaller of the number of rows and the number of columns. When it hits that cap the columns are as spread out as their count allows; when it falls below, some column is a combination of the others, and the span loses a dimension exactly there.

The null space is the mirror image of the same count. Every free column — a column without a pivot — is one dimension of input the map cannot distinguish from the rest, and each gives one basis vector for the null space. This is why the null space appears at the precise moment the columns align: alignment is the geometric statement that the second column adds no new direction, so a pivot disappears and a free direction appears in the domain.

The same matrix acts on a two-dimensional input space, and the map has a kernel. When the two columns are independent, only the zero input lands on zero, so the null space is trivial. The instant the columns align, the map becomes many-to-one: some whole line of inputs is squashed onto the origin. The demo edits the matrix, tracks the columns and their span in three dimensions, and draws the null space in the two-dimensional domain beside it. Watch the null space appear at exactly the moment the rank drops.

The columns are conserved, not created

Every column of A is either a genuinely new direction or a combination of the columns before it. The pivot columns add to the rank; the free columns correspond to directions in the null space. Count both and you get back the number of columns, no more and no less:

The identity is a conservation law. Reducing a matrix to row echelon form does not change its rank, and it exposes the free variables directly: each free column contributes one vector to the null space, and the count of pivots is the rank. That is the whole proof, and the demo below shows the ledger staying balanced as you change the matrix in the previous section. Slide the columns apart and the rank block fills the bar. Push them together and the rank block shrinks while the nullity block grows to take its place.

The big picture, and where each one lives

Two subspaces are not the whole story. A matrix also has a row space, spanned by its rows and living in the input space, and a left null space, the inputs of the transpose that land on zero, living in the output space. That makes four: the row space and null space inside the domain, the column space and left null space inside the codomain. Between them they tile both rooms without overlap.

The map acts on this picture in a beautifully simple way. Restricted to the row space, A is one-to-one and onto: it sends the row space exactly onto the column space, dimension for dimension. Everything orthogonal to the rows is the null space, and the map sends all of it to zero. On the output side, the left null space is precisely the orthogonal complement of the column space — the directions no combination of the columns can produce. Choosing a subspace highlights it and lists the basis the current matrix actually has.

One number, every answer

Rank is the single number that decides the shape of the solution set of A x = b. If the rank equals the number of columns, the columns are independent and the map is one-to-one: any solution is unique, and the null space is just the zero vector. If the rank is smaller, the null space has positive dimension and every solution comes with a free family of others, because you can add any null vector without changing A x. Solvability depends on the other side: a solution exists exactly when b lies in the column space, which is the same as saying the rank of the augmented matrix [ A | b ] equals the rank of A.

The four subspaces also carry an orthogonality statement that turns the decomposition into a genuine coordinate system. The null space is orthogonal to the row space, and the left null space is orthogonal to the column space; each pair is an orthogonal complement, so the domain is a direct sum of its two pieces and the codomain is a direct sum of the other two. Take any vector in the input space, split it into its row-space part and its null-space part, and the map sends the first part faithfully onto the column space while discarding the second. That is the whole content of the fundamental theorem of linear algebra, and it is why a least-squares solution can be written so cleanly: you are separating the component of b the map can reach from the component it cannot.

Rank is also preserved by the operations you use most. Multiplying by invertible matrices on either side does not change it, which is why row reduction is a legitimate way to compute it, and why the rank of A and the rank of AᵀA agree in exact arithmetic. It is the quantity that low-rank methods deliberately reduce: truncating an SVD to its largest singular values chooses a smaller-rank matrix closest to the original, a construction taken up in the later parts on the SVD and PCA.

The four subspaces turn those statements into a coordinate system. The domain splits into the row space, where the map is faithful, and the null space, which it discards. The codomain splits into the column space, which is reachable, and the left null space, which is not. A square matrix is invertible exactly when both discarded pieces are trivial, which is exactly when the determinant from Part 5 is nonzero, and exactly when elimination from Part 6 produces a pivot in every column. These are not three facts; they are one fact in three costumes.

Sizes add a useful vocabulary. A matrix with more columns than rows is wide: it cannot be one-to-one, so it always has a nonzero null space and any solution it has comes in a family. A matrix with more rows than columns is tall: it can be one-to-one, but it cannot cover its whole codomain, so most right-hand sides are unreachable and the honest problem becomes least squares. The least-squares part takes the tall case and projects b onto the column space; the pseudoinverse part unifies every case into one formula.

One idea, two worlds

• Grant Sanderson, "Inverse matrices, column space and null space", Essence of Linear Algebra, 3Blue1Brown — the geometric picture of rank, reachable outputs and the null space.
• Gilbert Strang, 18.06 Linear Algebra, MIT OpenCourseWare — Lectures 5–10, where the four subspaces and the rank–nullity theorem are the centre of the course.
• Immersive Math, Chapter 4: Solving systems of linear equations and Chapter 10: Linear maps — rank and the image, interactively.
• Sheldon Axler, Linear Algebra Done Right, chapters 3 and 8 — the rank–nullity theorem and the fundamental subspaces, stated abstractly.