Probability and Statistics

This section links the mathematical language of uncertainty to statistical inference and modelling. Start with the probability foundations, use the limit theorems to understand estimation error, then move to intervals, tests, process models, and experimental design.

Knowledge map

Probability foundations feed both the limit theorems (which explain why estimation works) and the stochastic-process models, and together they support inference and experiment design.

flowchart TD
  Spaces[Probability Spaces] --> RV[Random Variables]
  RV --> Dists[Common Distributions]
  RV --> EV[Expectation and Variance]
  RV --> Cond[Conditional Probability and Bayes]
  EV --> Limits[Law of Large Numbers and CLT]
  RV --> Proc[Markov Chains, Random Walks, Renewal]
  Limits --> Infer[Estimation and Inference]
  Cond --> Bayesian[Bayesian Statistics]
  Infer --> Design[Hypothesis Testing and Experimental Design]

Reading path

Read the probability foundations, then limits and processes, then inference and modelling.

1. Probability Spaces: outcomes, events, and probability measures.
2. Random Variables: turning outcomes into measurable quantities with distributions.
3. Common Distributions: reusable laws for counts, waiting times, errors, and positive quantities.
4. Expectation and Variance: averages, spread, and standard errors.
5. Conditional Probability: updating the reference population after evidence.
6. Bayes Theorem: turning priors and likelihoods into posteriors.
7. Covariance and Correlation: linear co-movement between variables.
8. Law of Large Numbers: why averages converge.
9. Central Limit Theorem: why standardized sample means become normal.
10. Markov Chains: memory-one state transitions.
11. Random Walks: accumulating random increments into paths.
12. Renewal Theory: repeated events separated by iid waiting times.
13. Markov Renewal Processes: state transitions with transition-specific holding times.
14. Statistical Estimation: estimands, estimators, estimates, and uncertainty.
15. Maximum Likelihood: fitting parameters by maximizing the observed-data likelihood.
16. Maximum A Posteriori Estimation: adding a prior and taking the posterior mode.
17. Bayesian Statistics: keeping the full posterior for inference and decisions.
18. Confidence Intervals: reporting uncertainty through repeated-sampling coverage.
19. Hypothesis Testing: comparing an observed statistic with a null model.
20. Statistical Modelling: specifying stochastic structure, parameters, and assumptions.
21. Experimental Design: planning assignment and measurement so inference targets the intended question.

Connections

• Mathematical Foundations supplies the calculus and linear algebra underneath these results.
• Classical Machine Learning and Experimentation and Evaluation turn estimation and testing into model fitting and decisions.

Nav

Learning path — Foundations

← Mathematical Foundations Classical Machine Learning →