Classical SVD

Classical singular value decomposition factorizes a complete numeric matrix. In recommender systems it supplies the low-rank vocabulary used by matrix factorization, but it is not by itself a correct treatment of missing ratings.

The SVD factorization

For a dense matrix $A\in {\mathbb{R}}^{m\times n}$,

$$A=U\Sigma V^{\top },$$

where columns of $U$ and $V$ are orthonormal and $\Sigma$ contains nonnegative singular values. The rank-$k$ approximation keeps the largest $k$ singular values:

$$A_k=U_k{\Sigma }_k{V}_k^{\top }.$$

This connects to low-rank approximation. The recommender-specific problem is that utility matrices are usually sparse and missing entries mean unknown exposure, not zero dislike.

Worked example

This snippet computes the SVD of a dense rating matrix and forms a rank-2 reconstruction, reporting singular values, error, and one reconstructed row.

import numpy as np
A = np.array([[5., 4., 1.], [4., 4., 1.], [1., 1., 5.], [1., 0., 4.]])
U, s, Vt = np.linalg.svd(A, full_matrices=False)
A2 = (U[:, :2] * s[:2]) @ Vt[:2]
print("singular_values", np.round(s, 3).tolist())
print("rank2_error", round(float(np.linalg.norm(A - A2)), 3))
print("rank2_row0", np.round(A2[0], 2).tolist())

Observed output:

singular_values [9.304, 5.647, 0.739]
rank2_error 0.739
rank2_row0 [4.78, 4.24, 1.02]

The top two components almost reconstruct the dense matrix. Truncated SVD computes this approximation directly when only the leading components are needed.

Caveats

Do not turn sparse recommender data into a dense matrix by filling unknown cells with zeros unless that is the deliberate data-generating assumption. SVD versus matrix factorization is mainly about this distinction: SVD decomposes a given matrix, while recommender factorization learns from observed or weighted interactions.

References

• scikit-learn documentation: TruncatedSVD
• Koren, Bell, and Volinsky, 2009, Matrix Factorization Techniques for Recommender Systems

Nav

Section — Recommendation Systems and Personalization

← Latent Factor Models Truncated SVD →