Chapter 1: Statistical Paradigms and Core Concepts

Before we can simulate, estimate, or infer, we must establish what probability means and how to compute with it. This chapter weaves together three essential threads: the philosophical foundations that shape how we interpret probabilistic statements, the mathematical machinery of random variables and distributions, and the computational tools that turn theory into practice.

We begin with a fundamental question that has divided statisticians for over a century: What does probability represent? Kolmogorov’s axioms provide the mathematical rules everyone accepts, but interpretations diverge sharply. Frequentists see probability as long-run frequency—meaningful only for repeatable experiments. Bayesians see probability as degree of belief—applicable to any uncertain proposition. These aren’t merely philosophical positions; they lead to different inferential methods, different interpretations of results, and ultimately different answers to data science questions. We develop fluency in both perspectives.

From philosophy we turn to the mathematical objects that make probability concrete. Random variables map outcomes to numbers; distributions describe how probability mass spreads across those numbers. We develop the full apparatus—PMFs, PDFs, CDFs, quantile functions—and catalogue the distributions that appear throughout data science: Bernoulli and Binomial for trials, Poisson for counts, Normal for measurements, Exponential and Gamma for durations, Beta for proportions, and the inference distributions (t, Chi-square, F) that arise in hypothesis testing. Finally, we bridge theory and practice through Python’s computational ecosystem: the random module for quick prototyping, NumPy’s Generator for high-performance simulation, and SciPy’s stats for the complete statistical toolkit.

Learning Objectives: Upon completing this chapter, you will be able to:

Probability Foundations

  • State Kolmogorov’s three axioms and derive fundamental probability rules from them

  • Distinguish frequentist, Bayesian, and propensity interpretations of probability

  • Compare frequentist, Bayesian, and likelihood-based inference paradigms

  • Evaluate philosophical and practical trade-offs between inferential approaches

Random Variables and Distributions

  • Define random variables and their associated PMFs, PDFs, and CDFs

  • Compute expectations, variances, and quantiles from distribution functions

  • Derive properties of major discrete distributions (Bernoulli, Binomial, Poisson, Geometric, Negative Binomial)

  • Derive properties of major continuous distributions (Uniform, Normal, Exponential, Gamma, Beta)

  • Apply inference distributions (Student’s t, Chi-square, F) in hypothesis testing contexts

  • Identify relationships between distributions (limits, sums, transformations)

Computational Implementation

  • Generate random samples from standard distributions using Python’s ecosystem

  • Select appropriate libraries (random, NumPy, SciPy) based on performance and functionality needs

  • Manage random seeds for reproducibility and parallel computation

  • Compute probabilities, quantiles, and moments using SciPy’s unified API

Sections