Linear transformations are matrices

A rule, not a table of numbers

Most introductions to matrices start with a rectangular block of numbers and a bizarre multiplication rule, leaving you to wonder who decided it. Turn it around. Suppose you want a rule that moves points around while staying compatible with vector addition and scaling. Compatibility forces the grid of integer lines to stay a grid: lines that were parallel stay parallel, evenly spaced lines stay evenly spaced, and the origin stays put. A map with that behaviour is called linear, and a linear map is completely described by two pieces of information in the plane — where the basis vector î = (1, 0) lands, and where ĵ = (0, 1) lands.

The reason two arrows are enough is the linear-combination idea from the previous parts. Every vector is a combination of the basis vectors,

Apply the transformation to both sides. Linearity lets you pull the map through the scalars and the sum, so the image of p is the same combination of the images of the basis vectors. If you know where the two basis vectors go, you know where everything goes. That is the entire content of the next four demos.

Write the image of î as a column and the image of ĵ as another column, and stack them side by side. The resulting array is the matrix of the transformation, and the rule above becomes matrix-vector multiplication. A matrix is not an arbitrary table; it is a compact record of where a linear map sends the basis, arranged so that evaluating the map is a single combination.

Spelled out in symbols, the whole procedure is one line. Suppose î goes to the column (a, c) and ĵ goes to the column (b, d). Then a general input (x, y), which is x î + y ĵ, must land on

Read the middle expression as an instruction and the rightmost as arithmetic. The instruction says: use x as a weight on the first column and y as a weight on the second, then add. The arithmetic says the same thing entry by entry. There is no mysterious rule being imposed from outside; the familiar row-times-column formula is simply the coordinate form of “combine the transformed basis vectors”.

It is worth pausing on why the origin is special. A linear map has no freedom to translate; if it moved the origin to some point q, then transforming the zero vector would give q, and transforming 0 · v would give 0 · T(v) = 0, a contradiction unless q is zero. Linearity alone forces the origin to stay put, which is why every matrix sends (0, 0) to (0, 0) and why the grid lines of the picture all pass through the same fixed point. Translations need homogeneous coordinates precisely because they are not linear; they are affine, and affine maps are linear maps followed by a shift. Keeping that distinction straight will matter when we meet camera projections and robot poses.

Drag the images of î and ĵ

Below, the two handles are the destinations of î and ĵ. Drag either one and the entire grid deforms with it, because the grid is just the images of all the integer-spaced lines, and each of those lines is dragged along by linearity. The straight solid arrows are the transformed basis vectors, and the faded square is the image of the unit square, whose sides are exactly those two arrows. The probe vector p, chosen by the sliders, is drawn dashed in its original position, and its image Mp is drawn solid: it is the combination x along the first column plus y along the second.

The live matrix on the right is the same information in numeric form. Its first column is the coordinates of the first handle and its second column is the coordinates of the second. Nothing is hidden: editing the geometry edits the numbers, and the readout reports where the probe actually lands. Watch what stays true no matter where you drag. The origin never moves. Straight lines through the origin stay straight. Parallel lines stay parallel. If the two handles line up, the grid collapses onto a single line and the image of the unit square flattens to zero area — the matrix has lost a dimension, which is the rank collapse from Part 2 reappearing inside a map.

A useful habit is to name the columns by what they do rather than by their entries. Column one is “where î goes” column two is “where ĵ goes”. When you later meet a matrix in someone else's code, the fastest way to understand it is to feed it (1, 0) and (0, 1) and look at the two outputs. Those outputs are the columns, and the columns are the transformation.

It is just as useful to notice what linearity rules out. A translation — sliding every point one unit to the right — is not linear, because it moves the origin and it does not preserve vector addition. A map that bends straight lines into curves is not linear either. The grid picture is a strict test, not a stylistic preference: if the origin moves, or if parallel evenly spaced lines lose their spacing, no matrix can represent the map. This is why so much of applied mathematics spends its effort approximating curved maps by linear ones near a point. The Jacobian of calculus is exactly that: the best linear transformation matching a nonlinear map locally, which is the bridge to the optimization parts of this series.

The two-dimensional case is the one you can draw, but the recipe does not care about dimension. In three dimensions a linear map is determined by where it sends the three basis vectors, and its matrix has three columns; in a thousand dimensions, by a thousand images. The picture stops being drawable but the rule “columns are the images of the basis” remains literally true, and it is the rule that makes matrix multiplication, determinants and eigenvalues meaningful in every dimension at once.

Once you trust that rule, a great deal can be read straight off the columns. Two columns that are scalar multiples of one another mean a direction has been lost, so the map crushes the plane onto a line and the determinant is zero. Two columns of equal length meeting at a right angle mean the map is a rotation or a reflection, and areas are preserved. A column of length greater than one means that direction is being stretched; less than one, compressed. None of these facts require any calculation beyond looking at the two arrows, and they are the same facts the numeric matrix reports. Learning to see them is the point of drawing the columns rather than the entries.

The same object, entered as numbers

This demo runs the same map as the one above, but the controls are four number inputs instead of two handles. Change any entry and the transformed grid, the columns and the magenta ellipse all update together. The ellipse is the image of the unit circle, and it is a faithful fingerprint of the map: its axes tell you the directions the transformation stretches most and least. Typing 2 in the top-left cell widens the first column, which stretches everything in the x direction; typing in the off-diagonal cells shears the grid sideways. The gap between “a matrix” and “a transformation” closes the moment you notice there is only one object here with two notations.

This is also why the standard basis is a convenience and not a law. A matrix stores a map relative to the basis you happened to use when you wrote down its columns. Change the basis — the subject of a later part — and the same transformation earns a different matrix, while the arrows it moves are unchanged. The numbers are a description; the map is the thing described.

Watch the determinant in the readout as you type. It is the signed area of the image of the unit square, and it changes continuously with the entries. Drive it to zero by making the two columns collinear and the ellipse degenerates into a line segment: the map has crushed the plane onto a lower-dimensional image, and the matrix is singular. The determinant is not a separate topic bolted on later; it is the area scaling you can read straight off this picture, and it is the next part's main character.

There is a second way to read the same multiplication, and it is the one textbooks usually teach first. Instead of combining columns, you take the dot product of the input with each row: the first output coordinate is the first row dotted with the input, and the second coordinate is the second row dotted with the input. The two views are equal because the arithmetic is associative, but they tell different stories. The row view is local — one number per output, a weighted average of the input's coordinates — and it is how a neural network layer is implemented in practice. The column view is global, and it explains what the transformation does to the space. Fluency means being able to switch between them without thinking.

Five matrices, five motions

Every matrix you meet is some combination of a handful of elementary moves, so it pays to recognize the classics on sight. The buttons below snap the shared matrix to a preset and, with it, snap the two handles of the first demo and the entries of the second. There is no animation: the point is to compare the numbers and the picture side by side.

Keep one question in mind as you click through them. Which directions survive untouched, and which are moved? The identity answers “both” and “none”. The shear keeps the x axis exactly where it is while tilting everything else, and the reflection keeps the same axis fixed while flipping the other side of the plane over it. The rotations and the scale keep no direction fixed at all except the trivial zero vector. That question — which nonzero vectors pass through the map unchanged in direction — is the eigenvector question, and it is answered by the same columns you are dragging here.

• Identity sends every vector to itself. The grid is untouched and the columns are (1, 0) and (0, 1).
• Rotate 90° turns the plane a quarter turn. The first basis vector goes to (0, 1) and the second to (−1, 0); lengths and areas are preserved, so the determinant is one.
• Shear slides each horizontal row sideways by an amount proportional to its height. It fixes the x axis, tilts the y axis, and keeps area unchanged.
• Scale x doubles the horizontal coordinate and leaves the vertical one alone. Area doubles, and the determinant reports it.
• Reflect flips the plane across the x axis. It reverses orientation, so the determinant is negative even though areas are unchanged in size.

Two of these presets are worth comparing directly. Rotation and reflection both preserve lengths and areas, but rotation keeps orientation while reflection reverses it. The determinant distinguishes them: rotation has determinant 1, reflection has determinant −1, and the sign is the orientation. Shear and scale-x both keep the x axis fixed, but shear keeps area while scale-x enlarges it, so their determinants differ even though their first columns agree. Reading the determinant as signed area makes all of these relationships visible at a glance.

Multiplying is combining

Here is the whole multiplication rule, stripped of notation. Take the input vector's coordinates. Use the first coordinate as a weight on the first column and the second coordinate as a weight on the second column. Add the two weighted columns. That is the output, and it is drawn below as a tip-to-tail walk in the image space: first a step of a along column one, then a step of b along column two, landing on Mu. The sliders set a and b; the readout highlights those two weights, because they are the only things that vary. The columns are fixed by the transformation; the input chooses how much of each to use.

The unit square makes the same point from the other direction. Its four corners are (0, 0), (1, 0), (1, 1) and (0, 1), so their images are the origin, the first column, the sum of the columns, and the second column. The image of the square is therefore built entirely out of the two columns, which is why it is a parallelogram whose sides are the columns. A second test vector, drawn in the darker ink, shows the same mechanism with different weights.

This is the bridge to everything that follows. Matrix multiplication is composition of transformations, determinants measure how the map scales area, and eigenvector hunting is the search for inputs whose two weights are a single number. All of it lives inside the columns. When a neural network computes y = Wx, it is not doing anything more exotic than the tip-to-tail walk drawn above: each output coordinate is a combination of the input's coordinates, weighted by a row of W.

Try it on a concrete pair of weights. With the matrix shown and a = 1, b = 1, the output is the sum of the two columns, which is where the corner (1, 1) of the unit square lands. Halve one weight and the output slides halfway back along that column; set a weight to zero and the corresponding column drops out of the combination entirely. Negative weights run along a column backwards, which is why the image can extend past the origin even when the input does not. Every one of these behaviours is visible in the demo as the two dashed steps rearrange themselves before the solid arrow lands.

The same bookkeeping will let us multiply two matrices. If a second transformation N acts after M, then the composite map sends the basis vector î to N applied to the first column of M, and similarly for the second column. Those two results become the columns of a new matrix, and that matrix is the product NM. The order matters because the transformations are applied in order; running the walk above and then running another map is not the same as doing it the other way round. The next part makes that composition rule explicit and shows why the usual row-times-column formula is exactly this statement written out.

One idea, two worlds

Further reading

• Grant Sanderson, “Linear transformations and matrices”, Essence of Linear Algebra, 3Blue1Brown — the chapter this part is in conversation with.
• Gilbert Strang, 18.06 Lecture 30: Linear Transformations and their Matrices, MIT OpenCourseWare — the formal treatment, including bases other than the standard one.
• Immersive Math, Chapter 6: Matrices — the same material in a draggable textbook.
• Sheldon Axler, Linear Algebra Done Right, chapter 3 — linear maps and their matrices, treated without coordinates first.