Rank

Rank is the number of independent directions in a matrix. It tells how many dimensions a linear map can preserve, how many independent columns a design matrix has, and how many factors a low-rank approximation is allowed to use.

Defining math

For $A\in {\mathbb{R}}^{m\times n}$,

$$\mathrm{rank}(A)=\dim (\mathrm{col}(A))=\dim (\mathrm{row}(A)).$$

Equivalently, rank is the number of nonzero singular values in the SVD:

$$\mathrm{rank}(A)=|\{i:{\sigma }_i(A)>0\}|.$$

Rank controls solvability and identifiability. If a regression design matrix lacks full column rank, several coefficient vectors can produce the same fitted values. In recommender latent-factor models, choosing factor dimension is choosing an explicit rank bottleneck.

Worked example

Take $A=\left[\begin{array}{ccc}1& 2& 3\\ 2& 4& 6\\ 1& 1& 1\end{array}\right]$. The second row is exactly twice the first, $R_2=2R_1$, so the rows span only two independent directions and $\mathrm{rank}(A)=2$. The same dependence shows up in the top-left $2\times 2$ block, whose determinant vanishes:

$$\det \left[\begin{array}{cc}1& 2\\ 2& 4\end{array}\right]=1\cdot 4-2\cdot 2=0.$$

Computed numerically, the singular values are $\sigma \approx (8.52,0.64,0)$. The single zero exposes the lost dimension even though the matrix has three rows and three columns.

Caveats

Numerical rank is thresholded. Floating-point noise can turn exact zeros into tiny nonzero singular values, and nearly collinear features can be full-rank but still unstable. Always interpret rank with the scale of the singular values and downstream sensitivity.

References

• MIT OpenCourseWare: 18.06 Linear Algebra

Nav

Section — Mathematical Foundations

← Determinants Orthogonality →