The Geometry of Normality
Stretching, turning, and polar decomposition: a geometric view of when a matrix is normal.
I’ve always found that when a linear algebra concept is either geometrically motivated or has geometrical implications, I’m able to understand it and see its significance more than if I’d only approached it algebraically. I don’t have much natural skill with geometric reasoning, but nonetheless, when the geometry of the problem is revealed to me, a much deeper understanding comes along with it.
One of the most satisfying examples was in graduate school reading Trefethen & Bau’s presentation of the Singular Value Decomposition in Numerical Linear Algebra [1]. I’d studied SVD in a couple of undergraduate courses and didn’t think there was much more juice to be squeezed, but Trefethen & Bau introduced SVD by pointing out that the image of the unit sphere under a real matrix with and full column rank, is a hyperellipse and the singular values of are the lengths of the principal semiaxes of .1 This fact seemed like it should’ve been obvious once it was pointed out to me, but it wasn’t. I hadn’t encountered this perspective in my previous linear algebra courses. Figure 1 illustrates this in two dimensions.
That revelation completely changed the way I viewed SVD, and in fact, all matrix decompositions. It is in that spirit of geometric understanding I want to discuss another important matrix decomposition: the polar decomposition. Polar decomposition gives a geometric way to understand normality by asking when stretching and turning can be performed in either order.
The Polar Decomposition
The polar decomposition is generalizable to rectangular matrices with , but for our purposes we will stick to square matrices. Any has a factorization , where is unitary and is Hermitian positive semidefinite. [2]
This decomposition is a natural extension of the polar representation of a complex number. In fact, as we shall see, functions like and like .
From here on, let be a real invertible square matrix. Its polar decomposition , with orthogonal and symmetric positive definite (SPD), is unique. Recall a matrix is SPD if and for every nonzero .
And since is orthogonal, this demonstrates an interesting fact. Any invertible linear transformation in space can be decomposed into two distinct steps: stretching of space along principal axes (application of ) followed by a rigid rotation (or reflection) with no distortion (application of ).
Stretching and turning
Geometrically, acts on a vector by stretching components along perpendicular axes by positive factors. If is a unit vector along one of those axes, then
The are also the singular values of . expands; contracts. So, when I say "stretch", I really mean "scale". In the plane, ’s image of the unit circle becomes an ellipse whose semiaxes lie along the and have lengths .
A symmetric stretch can still change a vector’s direction. For example, maps to . Its principal axes stay fixed, but the components aren’t scaled the same. So vectors not on the principal axes can get turned. Contrast this with a rigid rotation, which turns all vectors through the same angle in the plane.
But that seems to imply the amount of stretching depends on direction. If that is the case, wouldn’t rotating first change the result? So order matters here. Or, rather, order CAN matter here. But when does it not? That is, when do we have ?
When can we reverse the order?
First, let’s consider when . It’s easy to construct such cases. Consider a stretch that doubles horizontal components and leaves vertical components alone, followed by a counterclockwise quarter-turn:
Applied to the horizontal unit vector , the two orders give
The stretch treats horizontal and vertical directions differently. Rotating first changes which treatment this vector receives. The final vectors point in the same direction, but they aren’t the same length.
Now replace with the uniform stretch . Every direction gets the same factor, so a rotation can’t move a vector into a differently stretched direction. The two orders agree for every vector (Figure 2).
But does this mean we need uniform stretching for the operations to commute? Not necessarily. Consider our original , but this time replace the quarter-turn with a reflection across the horizontal axis, . Here the order doesn’t matter. The reflection reverses one of the principal directions, but it doesn’t swap the axes. The horizontal component is doubled, and the vertical component changes sign without changing magnitude, regardless of which operation acts first. So maybe the important thing isn’t whether the stretching is uniform, but whether the orthogonal operator preserves the stretching pattern.
A shear hides both operations
So far we’ve started with a stretch and a turn and asked what happens if we reverse their order. But what if we start with a transformation whose stretching and turning aren’t obvious in the first place? Consider a shear. A horizontal shear slides each horizontal line by an amount proportional to its height.
It leaves the horizontal axis fixed. There doesn’t appear to be a rotation involved. Nevertheless, its polar factors are
The matrix is a clockwise rotation through . The shear is not itself a rotation, but its rotation factor is exactly what undoes the stretch’s action on the horizontal axis.
The fixed horizontal vector makes the interaction clearer:
You can see the stretch tilts upward through without changing this vector’s length. But then the clockwise turn brings it back. So we can have a vector that is fixed in place by the complete transformation even though both stages move it (Figure 3).
What changes if we turn first? The same two factors now give
In particular,
The coordinated cancellation that left fixed previously disappears when we swap the factors. Figure 4 shows the change. The clockwise rotation doesn’t preserve the stretching pattern of , so these factors don’t commute.
Why this is normality
Turning first changes the stretch
So, we can’t generally switch the two factors. But can we always turn first then stretch? Yes, but with a different stretch factor, .
This is the left polar decomposition. Its stretch generally differs from the right stretch . The identity always holds with this adjusted stretch. It scales by the same amounts as , but along axes moved by .
But under what conditions can we turn first and still use the same stretch factor, ?
Where normality comes in
Recall a real, square matrix is called normal if . The two sides have a direct geometric interpretation. From , we get
describes stretching along the original principal axes. describes the same stretch magnitudes but along axes moved by .
We have is normal if and only if . And because of the uniqueness of positive-definite square roots, we have . Since , is the same statement as .
So we have the non-obvious (to me at least!) fact that normality is precisely the condition that the polar factors of commute [3, Chapter 8]:
This reminds me of what I found so satisfying about the sphere-to-ellipse picture of SVD I mentioned in the intro. I’d known the algebraic definition of normality for years, but viewing normality through this geometric construal lets me understand it from a different perspective.
Where the scalar analogy survives
I began by stating the polar decomposition is analogous to the polar form of a complex number, whose magnitude and phase factors, of course, always commute. For a normal matrix, we can recover that scalar picture one coordinate at a time.
We’ve just shown that normality makes and commute. Both factors are normal themselves, so the simultaneous diagonalization theorem gives us a shared orthonormal eigenbasis over [4, Theorem 2].2 If is the unitary matrix whose columns are those eigenvectors, then
Here is the conjugate transpose of . The are the eigenvalues of , hence the singular values of , and the eigenvalues of lie on the unit circle because is orthogonal.
Multiplying the two diagonal representations gives
So in this basis, applying just multiplies the th coordinate by , independently of all the others. We started with an analogy to ; now each coordinate has exactly that form. The requirement that stretching and turning commute has brought us back to ordinary complex multiplication. In particular,
where the singular values coincide with the magnitudes of the eigenvalues, ordered largest to smallest.
Our shear doesn’t fit this picture. Its two eigenvalues are both , but its singular values are
Some directions are amplified while others are contracted. And the eigenvalues alone conceal this stretching. For a nonnormal matrix, they don’t necessarily determine the singular values. This becomes noticeably important when we apply the transformation repeatedly.
What happens on the next application?
Applying the transformation twice gives
After the first application, a vector has been turned. The next stretch sees it in that new direction. When the factors commute, their effects can be collected:
Otherwise, this rearrangement isn’t generally valid. Once stretching depends on direction, the direction presented to the next stretch matters.
Polar decomposition makes that apparent. Normality identifies the case in which the orthogonal motion preserves the stretching pattern, so their order can be ignored. In another article I want to explore what happens under repeated application when that compatibility fails.
Appendix: Recovering the factors
For the figures, I computed and using scipy.linalg.polar. But it’s useful to know why the factors exist and are unique when is invertible. For nonsingular , let . It is positive definite, since for nonzero ,
Write its orthogonal spectral decomposition as
Then
These satisfy and
Now we can show they are unique. Any SPD square root of commutes with , and therefore preserves each eigenspace of . On the eigenspace for , is the only positive eigenvalue of the symmetric matrix , so it acts as on that space. Thus . This fixes , and then is fixed as well.
For numerical computation, an SVD avoids explicitly forming or a matrix inverse. If , then
The columns of are input singular directions; carries them to the output singular directions in . In the code for generating the figures in this article, I used scipy.linalg.polar to compute the right polar factors.
For a singular square matrix, the stretch is positive semidefinite and remains unique, but the orthogonal factor doesn’t have to be: a collapsed direction can’t determine its subsequent motion. For example, both and can serve as the orthogonal factor of .
Notes
Trefethen & Bau continue providing excellent geometric insight throughout the rest of the text as well. I highly recommend it for building intuition with linear algebra even if numerical computing isn’t your goal.
↩Even though is real, basis might need complex vectors. A quarter-turn in the plane doesn’t have real eigendirections. Over , its eigenvalues are and .
↩
References
- Lloyd N. Trefethen, David Bau III. Numerical Linear Algebra. Society for Industrial and Applied Mathematics (SIAM), 1997.↩1 ↩2
- Nick Higham. What Is the Polar Decomposition?. 2020. https://nhigham.com/2020/07/28/what-is-the-polar-decomposition/.↩
- Nicholas J. Higham. Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics (SIAM), 2008. doi:10.1137/1.9780898717778.↩
- Morris Newman. Two Classical Theorems on Commuting Matrices. Journal of Research of the National Bureau of Standards, Section B: Mathematics and Mathematical Physics, 71B(2–3), pp. 69-71, 1967. https://nvlpubs.nist.gov/nistpubs/jres/71b/jresv71bn2-3p69_a1b.pdf.↩