History of AI and Machine Learning

This section is a sourced chronology map. Each page explains which technical bottleneck came first, which paper or system changed the path, and how that history connects to the wiki’s canonical concept pages.

Use the history pages to understand why today’s defaults look the way they do. Matrix factorization explains recommender practice, attention history explains transformer design, and self-supervised learning history explains why modern models can scale beyond hand-labeled datasets.

Knowledge map

Two long threads run through the history: the general classical-to-deep-learning line that branches into vision, language, and self-supervision, and the matrix-methods line behind recommenders.

flowchart TD
  Classical[Statistical Pattern Recognition] --> DL[Early Neural Networks to Deep Learning]
  DL --> Vision[Computer Vision Architectures]
  Vision --> Video[CNN Video to Video Transformers]
  DL --> LM[N-Gram to Transformers]
  LM --> Attn[Attention and Transformers]
  Attn --> SSL[Self-Supervised Learning]
  SSL --> Agents[RAG, Tool Use, World Models]
  Matrix[Matrix Decomposition] --> CF[Collaborative Filtering]

Reading path

Read roughly in chronological order, from statistical pattern recognition to modern agents and world models.

1. Statistical Pattern Recognition to Modern Machine Learning: the classical statistical roots.
2. Early Neural Networks to Deep Learning: perceptrons to backpropagation to deep nets.
3. Matrix Decomposition in Statistics and Recommenders: SVD’s path into recommendation.
4. Evolution of Collaborative Filtering: neighborhood methods to latent factors.
5. Evolution of Computer Vision Architectures: from hand-crafted features to CNNs and beyond.
6. From CNN Video Models to Video Transformers: adding time to vision models.
7. From N-Gram Language Models to Transformers: counting to neural language modeling.
8. Development of Attention and Transformers: the architecture that reshaped the field.
9. Self-Supervised Learning: scaling beyond labeled data.
10. Development of RAG: grounding generation in retrieval.
11. Development of Tool-Using Language Models and Agents: from text generation to action.
12. World Models and JEPA Background: predictive latent models of the world.

Connections

• Classical Machine Learning, Deep Learning, and Generative AI are the modern endpoints of these histories.