SVD versus Matrix Factorization

Classical SVD decomposes a complete matrix. Recommender matrix factorization learns a predictive model from sparse observed or weighted interactions. They share low-rank geometry, but their objectives encode different assumptions about missing values.

Defining contrast

SVD solves

$$A_k=\arg \underset{\mathrm{rank}(B)\le k}{\min }\Vert A-B{\Vert }_F^2.$$

Recommender factorization usually solves

$$\underset{P,Q}{\min }\sum _{(u,i)\in \Omega }(r_{ui}-p_u^{\top }{q}_i{)}^2+\lambda (\Vert p_u{\Vert }^2+\Vert q_i{\Vert }^2).$$

The first formula requires a dense $A$; the second names the observed set $\Omega$. That single difference is why sparse utility matrices need special care.

Worked example

This snippet compares a rank-2 approximation after dense imputation with a rank-2 approximation after zero filling for the same missing rating.

import numpy as np
dense = np.array([[5., 4., 1.], [4., 4., 1.], [1., 1., 5.]])
U, s, Vt = np.linalg.svd(dense, full_matrices=False)
svd_rank2 = (U[:, :2] * s[:2]) @ Vt[:2]
mask = np.array([[1, 1, 0], [1, 1, 1], [0, 1, 1]], dtype=bool)
zero = dense * mask
U0, s0, Vt0 = np.linalg.svd(zero, full_matrices=False)
zero_rank2 = (U0[:, :2] * s0[:2]) @ Vt0[:2]
print("dense_missing_0_2_rank2", round(float(svd_rank2[0, 2]), 3))
print("zero_fill_missing_0_2_rank2", round(float(zero_rank2[0, 2]), 3))

Observed output:

dense_missing_0_2_rank2 0.994
zero_fill_missing_0_2_rank2 -0.036

The same held-out cell is near 1 in the dense matrix but near zero after pretending it was missing-and-zero. Funk SVD and ALS avoid this by optimizing over observed entries.

Caveats

SVD is excellent linear algebra; the mistake is applying it to a matrix whose entries do not mean what the algorithm assumes. If zeros are true negatives after complete exposure, a dense objective may be defensible. If zeros are mostly non-exposure, use observed-entry or confidence-weighted models and judge them with recommender evaluation.

References

• scikit-learn documentation: TruncatedSVD
• Koren, Bell, and Volinsky, 2009, Matrix Factorization Techniques for Recommender Systems

Nav

Section — Recommendation Systems and Personalization

← Sparse Utility Matrices and Ordinary SVD Funk SVD →

Learning path — Recommender systems

← Matrix Factorization for Recommender Systems Evaluation of Recommenders →