2. Chapter 3: Frequentist Statistical Inference

This chapter covers frequentist statistical inference, building on the probability foundations from Chapter 1 and the simulation methods from Chapter 2. We explore how to estimate parameters from data, quantify sampling variability, and make inferences about populations based on samples. The frequentist approach treats parameters as fixed but unknown quantities, with randomness arising from the sampling process.

We begin with sampling variability and the behavior of estimators across repeated samples, then develop systematic approaches for parameter estimation including plug-in methods and maximum likelihood. We explore exponential families—a powerful class of distributions with elegant mathematical properties—and their role in statistical modeling. The chapter culminates with linear models and their generalization to non-normal responses through generalized linear models (GLMs).

Learning Objectives: Upon completion of this chapter, students will be able to:

• Understand sampling variability and the distribution of estimators across repeated samples

• Analyze properties of estimators including bias, variance, consistency, and efficiency

• Apply plug-in (method of moments) estimators and understand when they are appropriate

• Implement maximum likelihood estimation for parametric models

• Recognize exponential family distributions and exploit their properties

• Derive maximum likelihood estimators analytically and numerically

• Construct and interpret linear regression models with least squares estimation

• Understand assumptions underlying linear models and diagnostic techniques

• Extend linear models to non-normal responses using generalized linear models

• Implement and interpret GLMs for binary, count, and other response types

Sections

• 2.1. Sampling Variability
  • 2.1.1. Introduction
  • 2.1.2. Key Concepts
  • 2.1.3. Mathematical Framework
  • 2.1.4. Python Implementation
  • 2.1.5. Examples
  • 2.1.6. Summary
• 2.2. Statistical Estimators
  • 2.2.1. Introduction
  • 2.2.2. Key Concepts
  • 2.2.3. Mathematical Framework
  • 2.2.4. Python Implementation
  • 2.2.5. Examples
  • 2.2.6. Summary
• 2.3. Plugin Methods
  • 2.3.1. Introduction
  • 2.3.2. Key Concepts
  • 2.3.3. Mathematical Framework
  • 2.3.4. Python Implementation
  • 2.3.5. Examples
  • 2.3.6. Summary
• 2.4. Parametric Inference
  • 2.4.1. Introduction
  • 2.4.2. Key Concepts
  • 2.4.3. Mathematical Framework
  • 2.4.4. Python Implementation
  • 2.4.5. Examples
  • 2.4.6. Summary
• 2.5. Exponential Families
  • 2.5.1. Introduction
  • 2.5.2. Key Concepts
  • 2.5.3. Mathematical Framework
  • 2.5.4. Python Implementation
  • 2.5.5. Examples
  • 2.5.6. Summary
• 2.6. Maximum Likelihood
  • 2.6.1. Introduction
  • 2.6.2. Key Concepts
  • 2.6.3. Mathematical Framework
  • 2.6.4. Python Implementation
  • 2.6.5. Examples
  • 2.6.6. Summary
• 2.7. Linear Models
  • 2.7.1. Introduction
  • 2.7.2. Key Concepts
  • 2.7.3. Mathematical Framework
  • 2.7.4. Python Implementation
  • 2.7.5. Examples
  • 2.7.6. Summary
• 2.8. Generalized Linear Models
  • 2.8.1. Introduction
  • 2.8.2. Key Concepts
  • 2.8.3. Mathematical Framework
  • 2.8.4. Python Implementation
  • 2.8.5. Examples
  • 2.8.6. Summary