Matrix Decompositions

A matrix decomposition rewrites a matrix as a product of simpler matrices. The point is not cosmetic: the factors can expose rank, orthogonality, curvature, covariance axes, or a computational path that is more stable than operating on the original matrix directly.

Defining math

Common decompositions emphasize different structure:

$$A=QR,A=L{L}^{\top },A=Q\Lambda Q^{\top },A=U\Sigma V^{\top }.$$

QR uses orthonormal columns for least squares; Cholesky $A=L{L}^{\top }$ applies to symmetric positive definite matrices; eigendecomposition describes square maps with eigenvectors; singular value decomposition applies broadly and drives PCA and low-rank approximation.

Worked example

Cholesky factors a symmetric positive definite matrix as $A=L{L}^{\top }$ with $L$ lower-triangular. For $A=\left[\begin{array}{cc}4& 2\\ 2& 3\end{array}\right]$, write $L=\left[\begin{array}{cc}{\ell }_{11}& 0\\ {\ell }_{21}& {\ell }_{22}\end{array}\right]$ and match the entries of $L{L}^{\top }$ to $A$ one at a time:

$${\ell }_{11}^2=4\Rightarrow {\ell }_{11}=2,{\ell }_{21}{\ell }_{11}=2\Rightarrow {\ell }_{21}=1,{\ell }_{21}^2+{\ell }_{22}^2=3\Rightarrow {\ell }_{22}=\sqrt{2}.$$

So $L=\left[\begin{array}{cc}2& 0\\ 1& \sqrt{2}\end{array}\right]$, and indeed $L{L}^{\top }=\left[\begin{array}{cc}4& 2\\ 2& 3\end{array}\right]=A$. The positive diagonal $(2,\sqrt{2})$ certifies that $A$ is positive definite — the kind of property a decomposition makes easy to read off.

Caveats

Each decomposition has preconditions. Cholesky fails outside positive definite matrices; eigendecomposition can be ill-conditioned for non-normal matrices; forming $A^{\top }A$ may square the condition number, which is a numerical stability issue.

References

• MIT OpenCourseWare: 18.06 Linear Algebra

Nav

Section — Mathematical Foundations

← Laplacians and Graph Laplacians Singular Value Decomposition →