Groups, matrix groups and manifolds
The previous part ended with a claim: rotations and poses form a set that is both a group and a smooth manifold, and that combination is what makes the whole toolkit work. This part takes the two words one at a time. First comes the group, meaning what you are allowed to do with the elements. Then the matrix groups that robotics and vision actually use. Then the manifold, meaning what the set looks like up close. Every definition comes with a numerical test you can run.
Four rules make a group
Closure, associativity, identity, inverse
A group is a set $G$ with one operation $\circ$ that combines two elements into a third, obeying four rules:
Notice what is not on the list: commutativity. $X\circ Y = Y\circ X$ is allowed to fail, and for 3D rotations it does. Notice too that there is no addition and no scaling. A group has one operation only, and that is the whole reason a rotation cannot be "scaled by 0.5" or "added to" another. For rotations and poses, $\circ$ is matrix multiplication: "do $Y$, then $X$" (or "express $Y$ in $X$'s frame", section 5).
Pick a candidate set below and run the test. It draws random elements, checks each axiom numerically, and reports the worst violation it found. A handful of random samples cannot prove an axiom holds, but a single failure disproves it. That is exactly how you should sanity-check an implementation.
The matrix groups of robotics and vision
A group is known by what it preserves
Every group in this series is a set of invertible matrices under multiplication, a matrix group. The cleanest way to tell them apart is by what their elements leave invariant when they act on points:
| Group | Elements | Preserves | Dim. | Models |
|---|---|---|---|---|
| $SO(2)$ | $2\times2$, $R^\top R = I$, $\det R = 1$ | lengths, angles, handedness, origin | 1 | heading of a ground robot |
| $SO(3)$ | $3\times3$, $R^\top R = I$, $\det R = 1$ | lengths, angles, handedness, origin | 3 | attitude of a drone, camera or IMU |
| $SE(2)$ | $\begin{bmatrix}R & t\\ 0 & 1\end{bmatrix}$, $R\in SO(2)$ | lengths, angles, handedness | 3 | planar robot pose $(x, y, \theta)$ |
| $SE(3)$ | $\begin{bmatrix}R & t\\ 0 & 1\end{bmatrix}$, $R\in SO(3)$ | lengths, angles, handedness | 6 | camera or body pose |
| $Sim(3)$ | $\begin{bmatrix}sR & t\\ 0 & 1\end{bmatrix}$, $s>0$ | angles, ratios of lengths | 7 | monocular SLAM with unknown scale |
| $S^3 \cong SU(2)$ | unit quaternions | (double-covers $SO(3)$) | 3 | attitude in flight software |
| $GL(n)$ | all invertible $n\times n$ | only "is invertible" | $n^2$ | general linear maps, homographies |
The demo applies one element of the chosen group to a small shape. Only the sliders that belong to the group are enabled. The readout checks which properties survive: the length of each side, the angle at a corner, and whether the shape's orientation (clockwise or anticlockwise) flipped. Moving from $SO(2)$ to $SE(2)$ to $Sim(2)$ to $GL(2)$ gives up one invariant at a time and gains one freedom at a time.
Gray outline is the original shape. Blue is the shape after the group element acts on it.
Composing and inverting poses
The group operation, drawn
A planar pose is an element of $SE(2)$, written as a $3\times3$ homogeneous matrix
Read $T_A T_B$ as: start at $A$, then move by $B$ measured in $A$'s own frame. That is why the translation part is $R_A t_B + t_A$: the step $t_B$ gets rotated into the world before it is added. Planar rotations commute, yet $SE(2)$ does not, because rotating first and then stepping lands somewhere different from stepping first and then rotating. Drag pose $A$'s origin, set $B$ with the sliders, and compare the two orders.
Smooth and locally flat: the manifold
Zoom in until the curve is a line
A smooth manifold of dimension $n$ is a set that looks like $\mathbb{R}^n$ near every one of its points, with no corners, creases or edges. The circle is a 1-manifold, the sphere a 2-manifold, and $SO(3)$ a 3-manifold that happens to sit inside the 9-dimensional space of $3\times 3$ matrices. Its dimension is a count of freedoms: 9 entries, minus the 6 independent equations in $R^\top R = I$ (the matrix is symmetric, so only its upper triangle counts), leaves 3.
"Looks like $\mathbb{R}^n$" has a precise, measurable meaning. Walk a distance $s$ along the circle from a point, and the gap between where you are and where the tangent line says you would be grows like $s^2/2$. Halve the step and the error drops by a factor of four. The zoom below makes that visible: the more you zoom, the more the arc and the tangent line agree, relative to the window. That quadratic agreement is what linearization will cash in, in every optimizer from Part 10 onward.
Left: the unit circle, with the zoom window as a box. Right: the view inside the window, rescaled to fill it. Dark is the arc, dashed is the tangent line, and the magenta tick is the gap at the window edge.
Why SO(3) and not O(3)
A group can come in disconnected pieces
$O(3)$, all matrices with $R^\top R = I$, passed every axiom in section 1. Why do we bother with the extra condition $\det R = +1$? Because $O(3)$ is two separate islands. On one, $\det = +1$: the rotations. On the other, $\det = -1$: a rotation followed by a mirror. The determinant of an orthogonal matrix is always $\pm 1$ and varies continuously, so a smooth path cannot jump from one island to the other. A physical object moves along smooth paths, so it lives on the island containing the identity, $SO(3)$.
Try to get from $I$ to the mirror $F = \operatorname{diag}(1,1,-1)$ below by straight-line blending, $M(t) = (1-t)I + tF$. Halfway along, the matrix flattens space onto a plane, with $\det M = 0$, and it is orthogonal only at the two ends. No path through orthogonal matrices does better: every one of them would have to cross $\det = 0$, which no orthogonal matrix can do.
Left: the unit box pushed through $M(t)$. Right: $\det M(t)$ and the orthogonality error along the path, with the current $t$ marked.
Frames and a notation that checks itself
Subscripts that cancel
A pose is always a relation between two frames. Write $T_{AB}$ for the pose of frame $B$ expressed in frame $A$. Equivalently, it is the map that takes a point's coordinates in $B$, ${}^{B}p$, to its coordinates in $A$:
Adjacent inner subscripts cancel, like units: $T_{WR}T_{RC} = T_{WC}$ is legal, while $T_{WR}T_{WC}$ is a bug you can spot without running anything. Below, a robot $R$ carries a camera $C$ on an arm, and the camera sees a landmark at ${}^{C}p$. The chain world $\to$ robot $\to$ camera $\to$ point is a product of group elements, and moving any link moves everything downstream.
Drag the robot. Red and green arrows are each frame's x and y axes. The magenta dot is the landmark, known only in the camera frame.
Group plus manifold: a Lie group
The definition, and what it buys
A Lie group is a group that is also a smooth manifold, with multiplication and inversion that are smooth maps. Every group in the table of section 2 qualifies. Smoothness of the operation is the crucial extra ingredient. It means that composing with a fixed element, $X \mapsto A\,X$, carries a small neighbourhood of the identity onto a small neighbourhood of $A$ without tearing it. So the tangent space at the identity, studied once, describes the tangent space at every point. That one special tangent space is the Lie algebra, and the next part meets it in the simplest possible setting, the circle.