Natural Language Processing

Natural language processing covers models and systems that turn text into tokens, labels, spans, embeddings, structured records, or generated language. This section focuses on language-specific tasks and evaluation contracts; broader foundation-model training and agent workflows live in Generative AI and Agentic Systems, while retrieval systems live in Information Retrieval and Search.

The practical distinction is output shape. Text classification predicts document-level labels, sequence labelling predicts token or span annotations, semantic similarity compares meanings, and document understanding combines OCR, layout, text, and extraction.

Knowledge map

Representation comes first (preprocessing, tokenization, embeddings), then language models, then the task families they power, all judged by task-appropriate evaluation.

flowchart TD
  Prep[Text Preprocessing and Tokenization] --> Emb[Embeddings]
  Emb --> LM[Language Modelling]
  LM --> Enc[BERT-Style Encoders]
  LM --> Dec[Decoder-Only Transformers]
  Enc --> Tasks[Classification, Labelling, Extraction]
  Emb --> Sim[Semantic Similarity and Linking]
  Tasks --> Eval[Evaluation of NLP Systems]
  Sim --> Eval

Reading path

Read representation, then models, then the task families, ending on evaluation.

1. Text Preprocessing: normalization before anything else.
2. Tokenization: choosing the units a model consumes.
3. Embeddings: dense vector representations of tokens and text.
4. Language Modelling: predicting text and scoring likelihood.
5. BERT-Style Encoders: bidirectional encoders for understanding tasks.
6. Decoder-Only Transformers: causal models for generation.
7. Text Classification: document-level label prediction.
8. Topic Classification: assigning subject categories.
9. Urgency Classification: cost-sensitive triage of messages.
10. Sequence Labelling: token-level tagging.
11. Named Entity Recognition: extracting typed entity spans.
12. Information Extraction: turning text into structured records.
13. Entity Linking and Matching: resolving mentions to canonical entities.
14. Semantic Textual Similarity: comparing meanings across texts.
15. Summarization: extractive and abstractive condensation.
16. OCR and Handwritten Text Recognition: reading text from images.
17. Document Understanding: combining layout, text, and extraction.
18. Evaluation of NLP Systems: metrics and confidence intervals for language tasks.

Connections

• Deep Learning supplies the transformer architectures underneath.
• Generative AI and Information Retrieval build on these representations and tasks.

Nav

Learning path — Natural language processing

Tokenization →