Span, independence and basis

From a fixed set of vectors, what can you build?

Fix two vectors v and w in the plane and allow yourself two moves: multiply either one by any number, and add the results. That single sentence defines a family of points,

and the collection of every point you can form this way is called the span of v and w. The question of this part is what that collection looks like for different choices of v and w, and what it means when it fails to fill the plane. That failure is not a defect; it is a message. It tells you the original set contained a vector that was redundant, and the size of the span — its dimension — is exactly the number of genuinely distinct directions you supplied.

Once you know how to find a set of vectors that spans everything and contains no redundancy, you have more than a spanning set. You have a basis: a minimal kit of building blocks, one for each independent direction. A basis is best understood as a coordinate system. The familiar x and y axes are just one particular basis, and the moment you allow a different one, you can describe the same point with different numbers without moving the point at all. That flexibility is the seed of change of basis, eigenvectors and every coordinate-free idea later in the series.

Stepping along two vectors

Read a linear combination as a walk. Start at the origin, step some distance along v, then step some distance along w from wherever you landed. The coefficient a says how far along v to go, and b says how far along w. Because a and b can be negative, each step also runs backwards, so the reachable set extends on both sides of every vector. The shaded region below is the set of all landings as a and b sweep over every real number.

When v and w point in genuinely different directions, that walk can land anywhere. From any point in the plane, there is a pair of coefficients that gets you there: step along v until you are on the line parallel to w through the target, then step along w to close the gap. The shading fills the plane, and we say v and w span it. The plane has dimension two, and it took two independent directions to cover it.

Now drag one handle until the two arrows line up. Nothing about the arithmetic changes — you may still use any coefficients you like — but the reachable set collapses onto a single line. Every step along w is now just a step along v in disguise, so the two moves are secretly one move, and no choice of a and b can leave that line. The span has dropped from dimension two to dimension one. Collapse is not gradual; it happens the instant the directions coincide.

Two quantities in the readout are worth naming now, because they return throughout the series. The first is the cross product v × w, which here is just a single number, v_xw_y − v_yw_x. It is exactly zero when the vectors are collinear, so it is a perfect algebraic test for the collapse you just saw. The second is its absolute value |v × w|, which measures the area of the parallelogram the two vectors sweep out. When that area shrinks to zero the two directions have merged; when it is large the directions are far apart and the pair is a strong, well-separated spanning set. Independence, dimension and area are all the same fact wearing three different names.

It is worth stating the definition in full, because the rest of the series leans on it. The span of a set of vectors is the collection of every linear combination of them. That collection is never an arbitrary blob: it always contains the origin, because you can take every coefficient to be zero, and it is closed under the two operations that built it. Adding any two reachable points is reachable, since the sum of two combinations is again a combination; scaling a reachable point is reachable, since scaling a combination just rescales its coefficients. A set with those two closure properties is called a subspace, and the crucial point is that a span is the smallest subspace containing the original vectors. Every point of the span was forced to be there by the operations; nothing else got dragged in.

That framing immediately classifies the possibilities in the plane. The span of the zero vector is the origin alone: the only combination available is zero, so the subspace has dimension zero. The span of a single nonzero vector is the entire line through the origin in that direction, because every multiple of the vector is reachable and nothing off the line is. Only when you supply two vectors that point differently does the span finally become two-dimensional and fill the plane. The dimension of the span therefore counts the number of independent directions you actually handed over, which is at most the number of vectors you gave and at most the dimension of the surrounding space.

One definition, three equivalent tests

The collapse in the demo has a precise cause, and naming it lets us predict it without a picture. A set of vectors is linearly dependent if at least one of them can be written as a linear combination of the others; otherwise the set is linearly independent. The phrase “the others” is doing the real work. If you can rebuild a member of the set from its companions, that member added no new direction, and the span is no larger than it would have been without it. Independence is the statement that no such shortcut exists — every vector in the set carries a direction the rest cannot fake.

In the plane with two vectors there are only two outcomes: they are independent, or they are collinear. Collinear means w is a multiple of v, so v alone already spans the line and w is redundant. That is the dependent case, and the cross product detects it. In higher dimensions the same idea scales up: a dependent set always contains a vector that lives in the span of the others, and a set of n vectors in n-dimensional space spans the whole space exactly when it is independent. Adding a redundant vector costs arithmetic and buys nothing.

There is a clean algebraic handle on all of this. Stack the vectors as the columns of a matrix

and ask whether A c = 0 has any solution besides the trivial c = 0. If a nonzero coefficient vector makes the combination vanish, the vectors are dependent; if the only solution is the all-zero one, they are independent. For a square matrix those two cases are separated by the determinant: det A = 0 exactly when the columns are dependent and the span collapses. Determinants earn their own part later, but this is the first place the idea appears — not as a symbolic formula, but as the area that dies when a direction is lost.

Useful equivalence to carry forward: for a square matrix, “columns are independent”, “columns span the space”, “det is nonzero” and “the only solution of Ac = 0 is c = 0” are four ways of saying the same thing. Any one of them implies the other three.

Two small examples make the tests concrete. Take v = (1, 2) and w = (2, 4). The second is exactly twice the first, so the pair is dependent, the cross product is 1·4 − 2·2 = 0, and the two vectors span only the line through (1, 2). Now replace the second by w = (2, 1). No multiple of (1, 2) can produce (2, 1), the cross product is 1·1 − 2·2 = −3, and the pair spans the whole plane. In both cases the arithmetic agrees with the picture, and the cross product is the single number that decides between the two worlds.

There is a constructive way to think about independence that scales to any dimension. Start with an empty collection and walk through your vectors one at a time. Keep a vector only if it is not already in the span of the vectors you have kept so far; discard it otherwise, because it added nothing new. When you are done, the vectors you kept are independent by construction — each one contributed a direction the earlier ones could not reach — and they span exactly the same set as the original collection. This greedy recipe always terminates and always produces a basis of whatever space you started with. It also proves that every basis of a given space has the same number of vectors, since each kept vector raises the dimension by exactly one and you cannot raise it past the dimension of the space.

That last observation is what makes dimension a well-defined notion rather than a choice. A plane is two-dimensional whether you build it from the standard axes, from two skewed arrows, or from any other independent pair, because every basis of it has two members. The specific vectors change; the count cannot. When a later part says that an n × n matrix has full rank, it is saying that its columns form a basis of n-dimensional space, which is another way of saying the map they define loses no directions.

A gauge, a target, and a solvable system

Here is the same question asked as an engineering problem. Suppose a target point T is fixed, and you want coefficients a and b with a v + b w = T. Written in coordinates, that is a two-equation linear system, and it is small enough to solve exactly. Move the two sliders to steer the ghost arrow a v + b w around and try to park it on T. With independent v and w you can always do it, and the readout shows the exact coefficients that finish the job — the solution the toolkit computes with a two-by-two solve. With a dependent pair the same system has either no solution or infinitely many, and the ghost arrow can only ever slide along one line.

Press the second button to make the pair collinear and watch the ghost arrow lose its second degree of freedom. Notice what did not change: the sliders still move, the arithmetic still runs, and the arrow still moves along the line. What disappeared is your ability to reach off that line. A rank-deficient pair has thrown away one direction of reachability, and no amount of coefficient tuning brings it back. This is the geometric content of a singular system, and it is precisely the situation that makes a robot lose a direction of motion.

Two cautionary details hide in this little demo. First, a system being square does not make it solvable; squareness says the equation count matches the unknown count, while the determinant decides whether the map is invertible. Second, when the pair is dependent but the target happens to sit on the span, the system has infinitely many solutions rather than none, and the readout says so. The distinction between “no solution” and “infinitely many” is not a technicality: it is the difference between a target you cannot reach and a target you can reach in more ways than one. Both are consequences of the columns failing to be independent.

Written out in coordinates, the target problem is about as explicit as linear algebra gets. If v = (v₁, v₂), w = (w₁, w₂) and T = (t₁, t₂), then a v + b w = T is the pair of scalar equations

Cramer's rule or plain elimination both give a formula whose denominator is v₁w₂ − v₂w₁, which is exactly the cross product from the first demo. The geometry and the algebra agree line for line: the pair is solvable for every target precisely when that denominator is nonzero, and the denominator is nonzero precisely when the two directions are independent. When it is zero the two equations are no longer independent constraints — one is a multiple of the other — so the system either contradicts itself or leaves a free parameter, which is the geometric statement that the target is off the line or on it.

This is not a rare pathology confined to hand-built examples. A robot arm at full stretch, a camera pointed at a scene that has become degenerate, a sensor array whose readings have drifted into a common mode — each produces a matrix whose columns have quietly become dependent. The solver still returns something, because software rarely refuses to answer, but the answer is no longer trustworthy: small changes in the target can produce enormous changes in the coefficients, and the readout's distance to the target stops going to zero. Rank and conditioning, the subjects of later parts, are the tools for detecting that situation before it causes trouble.

The same point, described differently

A basis of the plane is an independent set that spans it: enough directions to reach everywhere, and no direction too many. Given such a set {b₁, b₂}, every point P has exactly one representation as a combination c₁ b₁ + c₂ b₂, and the pair (c₁, c₂) are its coordinates in that basis. Existence comes from spanning, uniqueness from independence, and together they make the basis a coordinate system: a rule that assigns to each point a pair of numbers, and to each pair of numbers a point, with nothing ambiguous in between.

Drag the point P below and drag the two basis arrows. The readout shows two things at once: the standard coordinates of P, which are its coordinates in the default basis of the x and y axes, and its coordinates in the basis you have chosen. The same physical point keeps the same position; only its address changes. The dashed arrows draw the decomposition literally — first a step of c₁ along b₁, then a step of c₂ along b₂, tip to tail, arriving exactly at P. Solving the system B c = P is how the toolkit finds those coefficients, and when the basis is dependent the solve fails because the address is no longer unique.

Choosing a basis is choosing a language. The standard basis makes arithmetic easy and is the default for good reason, but it is not privileged. In a basis aligned with an object's own structure, a transformation that looked tangled in standard coordinates can reduce to a simple stretch along each axis — the whole point of eigenvectors later in the series. The point of this demo is that nothing about the point changed when you moved the basis arrows; your description of it did.

The vocabulary from this part is worth pinning down, because it will be used without explanation from here on.

The familiar axes are the standard basis, usually written e₁ = (1, 0) and e₂ = (0, 1). Every vector is already a combination of them, and its coefficients are just its ordinary coordinates, which is why the standard basis feels invisible. Its matrix is the identity, and solving B c = P in that basis returns P unchanged. The demo is more interesting when you move the arrows because it breaks that comfortable coincidence and shows what coordinates were doing all along: reporting how much of each basis direction you need. A basis is a measuring stick, and different sticks give different numbers for the same point.

This choice is not merely cosmetic. In later parts, a transformation that shears and stretches in standard coordinates can look like a plain scaling in the right basis, because that basis is aligned with the directions the map treats specially. Finding those directions is exactly the eigenvector problem. Everything rests on the guarantee this part established: as long as the basis vectors are independent, every point has one and only one address, so the coordinates are meaningful. Lose independence and the address becomes ambiguous or nonexistent, which is the same failure the span demo made visible.

One idea, two worlds

Further reading

• Grant Sanderson, “Linear combinations, span, and basis vectors”, Essence of Linear Algebra, 3Blue1Brown — the chapter this part is in conversation with.
• Gilbert Strang, 18.06 Lecture 9: Independence, Basis and Dimension, MIT OpenCourseWare — the formal treatment of the same three words.
• Immersive Math, Chapter 2: Vectors — the same material in a draggable textbook, including linear combinations.
• Sheldon Axler, Linear Algebra Done Right, chapter 2 — span, independence and bases proved carefully.