Funk SVD

Funk SVD is the informal name for SGD-trained recommender matrix factorization popularized during the Netflix Prize. Despite the name, it is not classical SVD: it does not decompose a complete matrix with orthogonal singular vectors. It learns user and item factors directly from observed ratings.

The Funk-SVD update

For one observed pair $(u,i)$, the prediction and squared-error update are

$${\hat{r}}_{ui}=p_u^{\top }{q}_i,e_{ui}=r_{ui}-{\hat{r}}_{ui},$$ $$p_u\leftarrow p_u+\eta (e_{ui}{q}_i-\lambda p_u),q_i\leftarrow q_i+\eta (e_{ui}{p}_u-\lambda q_i).$$

Bias terms can be added, but the mechanism is the same: move factors so observed pairs have higher dot products. Alternating least squares optimizes a similar objective with block solves instead of small stochastic steps.

Worked example

This snippet applies one gradient update to a Funk-SVD user and item vector, then compares squared error and the resulting dot-product prediction.

import numpy as np
rng = np.random.default_rng(9)
p = 0.1 * rng.normal(size=2); q = 0.1 * rng.normal(size=2)
r, lr, lam = 5., 0.05, 0.02
def se(): return (r - p @ q) ** 2
print("squared_error_before", round(float(se()), 3))
for _ in range(60):
    e = r - p @ q
    p0 = p.copy()
    p += lr * (e * q - lam * p)
    q += lr * (e * p0 - lam * q)
print("squared_error_after", round(float(se()), 3))
print("prediction", round(float(p @ q), 3))

Observed output:

squared_error_before 24.851
squared_error_after 0.0
prediction 4.98

The single observed rating pulls the factors until their dot product is near 5. Real systems update many pairs and hold out interactions for offline evaluation.

Caveats

The step size matters; too large can diverge, too small trains slowly. Popular items receive many more updates than rare items, so regularization and sampling shape the learned geometry. Funk SVD targets explicit ratings; Bayesian personalized ranking is usually a better fit when only positive implicit events are available.

References

• Koren, Bell, and Volinsky, 2009, Matrix Factorization Techniques for Recommender Systems

Nav

Section — Recommendation Systems and Personalization

← SVD versus Matrix Factorization Alternating Least Squares →