Linear Algebra

Linear algebra studies vector spaces and linear maps between them. In data science, it is the shared language for feature matrices, least-squares projections, embeddings, covariance geometry, neural layers, PCA, and recommendation factors.

Defining math

A linear map $T$ satisfies

$$T(\alpha x+\beta y)=\alpha T(x)+\beta T(y).$$

After choosing bases, every finite-dimensional linear map is represented by a matrix $A$, and applying it is matrix multiplication:

$$y=Ax.$$

The fundamental questions are geometric: what directions are stretched, which directions collapse to zero, how much volume is scaled, what subspace is reachable, and how far a vector is from a subspace. These questions become determinants, rank, orthogonality, projections, eigenvalues, and the singular value decomposition.

Worked example

Take $A=\left[\begin{array}{cc}1& 2\\ 3& 4\\ 5& 6\end{array}\right]$ and $x=(0.5,-1)$. Applying the map sends a two-coordinate input to three coordinates:

$$Ax=\left[\begin{array}{c}1(0.5)+2(-1)\\ 3(0.5)+4(-1)\\ 5(0.5)+6(-1)\end{array}\right]=\left[\begin{array}{c}-1.5\\ -2.5\\ -3.5\end{array}\right].$$

Its two columns $(1,3,5)$ and $(2,4,6)$ are independent, since neither is a scalar multiple of the other, so $A$ has rank $2$. That is why it can represent a two-dimensional plane inside ${\mathbb{R}}^3$, a useful picture for linear models and projections.

Caveats

Linear algebra gives exact identities over exact numbers, while machine computation uses floating point. Nearly dependent columns can make a matrix look full-rank algebraically but unstable numerically; check singular values or conditioning when estimates are sensitive.

References

• MIT OpenCourseWare: 18.06 Linear Algebra

Nav

Section — Mathematical Foundations

Vectors and Matrices →