Chapter 1: Statistical Paradigms and Core Concepts
Before we can simulate, estimate, or infer, we must establish what probability means and how to compute with it. This chapter weaves together the philosophical foundations that shape how we interpret probabilistic statements with the mathematical machinery—random variables, likelihood, and asymptotics—that supports everything that follows.
We begin with a fundamental question that has divided statisticians for over a century: What does probability represent? Kolmogorov’s axioms provide the mathematical rules everyone accepts, but interpretations diverge sharply. Frequentists see probability as long-run frequency—meaningful only for repeatable experiments. Bayesians see probability as degree of belief—applicable to any uncertain proposition. These aren’t merely philosophical positions; they lead to different inferential methods, different interpretations of results, and ultimately different answers to data science questions. We develop fluency in both perspectives.
From philosophy we turn to the mathematical machinery the course relies on: random variables, likelihood, convergence concepts, and the asymptotic results (law of large numbers, central limit theorem) that justify computational inference. The supporting reference material lives in the appendices—a complete catalogue of probability distributions (PMFs, PDFs, CDFs, quantile functions, and the families used throughout data science) and a practical guide to random generation in Python’s scientific stack (the random module, NumPy’s Generator, SciPy’s stats)—so this chapter stays focused on the conceptual foundations.
Learning Objectives: Upon completing this chapter, you will be able to:
Probability Foundations
State Kolmogorov’s three axioms and derive fundamental probability rules from them
Distinguish frequentist, Bayesian, and propensity interpretations of probability
Compare frequentist, Bayesian, and likelihood-based inference paradigms
Evaluate philosophical and practical trade-offs between inferential approaches
Mathematical Machinery
Compute with random variables through their PMF/PDF, CDF, and quantile representations, converting between them as needed
Distinguish the three modes of convergence (in probability, almost sure, in distribution) and the asymptotic notation used to compare rates
Explain how the Law of Large Numbers and Central Limit Theorem justify simulation-based inference and quantify its error