STAT 350: Introduction to Statistics

Course Content

  • Part I: Foundations of Probability and Computation
    • Chapter 1: Statistical Paradigms and Core Concepts
      • Section 1.1 Paradigms of Probability and Statistical Inference
        • The Mathematical Foundation: Kolmogorov’s Axioms
        • Interpretations of Probability
        • Statistical Inference Paradigms
        • Historical and Philosophical Debates
        • Bringing It All Together
        • Looking Ahead: Our Course Focus
        • References and Further Reading
      • Section 1.2 Probability Distributions: Theory and Computation
        • From Abstract Foundations to Concrete Tools
        • The Python Ecosystem for Probability
        • Introduction: Why Probability Distributions Matter
        • The Python Ecosystem for Probability
        • Discrete Distributions
        • Continuous Distributions
        • Additional Important Distributions
        • Summary and Practical Guidelines
        • Conclusion
        • References and Further Reading
      • Section 1.3 Python Random Generation
        • From Mathematical Distributions to Computational Samples
        • The Python Ecosystem at a Glance
        • Understanding Pseudo-Random Number Generation
        • The Standard Library: random Module
        • NumPy: Fast Vectorized Random Sampling
        • SciPy Stats: The Complete Statistical Toolkit
        • Bringing It All Together: Library Selection Guide
        • Looking Ahead: From Random Numbers to Monte Carlo Methods
        • References and Further Reading
      • Section 1.4 Chapter 1 Summary: Foundations in Place
        • The Three Pillars of Chapter 1
        • How the Pillars Connect
        • What Lies Ahead: The Road to Simulation
        • Chapter 1 Exercises: Synthesis Problems
        • References and Further Reading
    • Chapter 2: Monte Carlo Simulation
      • Section 2.1 Monte Carlo Fundamentals
        • The Historical Development of Monte Carlo Methods
        • The Core Principle: Expectation as Integration
        • Theoretical Foundations
        • Variance Estimation and Confidence Intervals
        • Worked Examples
        • Comparison with Deterministic Methods
        • Sample Size Determination
        • Convergence Diagnostics and Monitoring
        • Practical Considerations
        • Chapter 2.1 Exercises: Monte Carlo Fundamentals Mastery
        • Bringing It All Together
        • Bringing It All Together
        • Transition to What Follows
        • References
      • Section 2.2 Uniform Random Variates
        • Why Uniform? The Universal Currency of Randomness
        • The Paradox of Computational Randomness
        • Chaotic Dynamical Systems: An Instructive Failure
        • Linear Congruential Generators
        • Shift-Register Generators
        • The KISS Generator: Combining Strategies
        • Modern Generators: Mersenne Twister and PCG
        • Statistical Testing of Random Number Generators
        • Practical Considerations
        • Chapter 2.2 Exercises: Uniform Random Variates Mastery
        • Bringing It All Together
        • Transition to What Follows
        • References
      • Section 2.3 Inverse CDF Method
        • Mathematical Foundations
        • Continuous Distributions with Closed-Form Inverses
        • Numerical Inversion
        • Discrete Distributions
        • Mixed Distributions
        • Practical Considerations
        • Chapter 2.3 Exercises: Inverse CDF Method Mastery
        • Bringing It All Together
        • Bringing It All Together
        • Transition to What Follows
        • References
      • Section 2.4 Transformation Methods
        • Why Transformation Methods?
        • The Box–Muller Transform
        • The Polar (Marsaglia) Method
        • Method Comparison: Box–Muller vs Polar vs Ziggurat
        • The Ziggurat Algorithm
        • The CLT Approximation (Historical)
        • Distributions Derived from the Normal
        • Multivariate Normal Generation
        • Implementation Guidance
        • Chapter 2.4 Exercises: Transformation Methods Mastery
        • Bringing It All Together
        • References
      • Section 2.5 Rejection Sampling
        • The Dartboard Intuition
        • The Accept-Reject Algorithm
        • Efficiency Analysis
        • Choosing the Proposal Distribution
        • Python Implementation
        • The Squeeze Principle
        • Geometric Example: Sampling from the Unit Disk
        • Worked Examples
        • Limitations and the Curse of Dimensionality
        • Connections to Other Methods
        • Practical Considerations
        • Chapter 2.5 Exercises: Rejection Sampling Mastery
        • Bringing It All Together
        • References
      • Section 2.6 Variance Reduction Methods
        • The Variance Reduction Paradigm
        • Importance Sampling
        • Control Variates
        • Antithetic Variates
        • Stratified Sampling
        • Common Random Numbers
        • Conditional Monte Carlo (Rao–Blackwellization)
        • Combining Variance Reduction Techniques
        • Practical Considerations
        • Bringing It All Together
        • Chapter 2.6 Exercises: Variance Reduction Mastery
        • References
      • Section 2.7 Chapter 2 Summary
        • The Complete Monte Carlo Workflow
        • Method Selection Guide
        • Quick Reference Tables
        • Common Pitfalls Checklist
        • Connections to Later Chapters
        • Learning Outcomes Checklist
        • Further Reading: Optimization and Missing Data
        • Final Perspective
        • References
  • Part II: Frequentist Inference
    • Chapter 3: Parametric Inference and Likelihood Methods
      • Section 3.1 Exponential Families
        • Historical Origins: From Scattered Results to Unified Theory
        • The Canonical Exponential Family
        • Converting Familiar Distributions
        • The Log-Partition Function: A Moment-Generating Machine
        • Sufficiency: Capturing All Parameter Information
        • Minimal Sufficiency and Completeness
        • Conjugate Priors and Bayesian Inference
        • Exponential Dispersion Models and GLMs
        • Python Implementation
        • Practical Considerations
        • Chapter 3.1 Exercises: Exponential Families Mastery
        • Bringing It All Together
        • References
      • Section 3.2 Maximum Likelihood Estimation
        • The Likelihood Function
        • The Score Function
        • Fisher Information
        • Closed-Form Maximum Likelihood Estimators
        • Numerical Optimization for MLE
        • Asymptotic Properties of MLEs
        • The Cramér-Rao Lower Bound
        • The Invariance Property
        • Likelihood-Based Hypothesis Testing
        • Confidence Intervals from Likelihood
        • Practical Considerations
        • Connection to Bayesian Inference
        • Chapter 3.2 Exercises: Maximum Likelihood Estimation Mastery
        • Bringing It All Together
        • References
      • Section 3.3 Sampling Variability and Variance Estimation
        • Statistical Estimators and Their Properties
        • Sampling Distributions
        • The Delta Method
        • The Plug-in Principle
        • Variance Estimation Methods
        • Applications and Worked Examples
        • Practical Considerations
        • Bringing It All Together
        • Exercises
        • References
      • Section 3.4 Linear Models
        • Matrix Calculus Foundations
        • The Linear Model
        • Ordinary Least Squares: The Calculus Approach
        • Ordinary Least Squares: The Geometric Approach
        • Properties of the OLS Estimator
        • The Gauss-Markov Theorem
        • Estimating the Error Variance
        • Distributional Results Under Normality
        • Diagnostics and Model Checking
        • Bringing It All Together
        • Numerical Stability: QR Decomposition
        • Model Selection and Information Criteria
        • Regularization: Ridge and LASSO
        • Chapter 3.4 Exercises: Linear Models Mastery
        • References
      • Section 3.5 Generalized Linear Models
        • Historical Context: Unification of Regression Methods
        • The GLM Framework: Three Components
        • Score Equations and Fisher Information
        • Iteratively Reweighted Least Squares
        • Logistic Regression: Binary Outcomes
        • Poisson Regression: Count Data
        • Gamma Regression: Positive Continuous Data
        • Inference in GLMs: The Testing Triad
        • Model Diagnostics
        • Model Comparison and Selection
        • Quasi-Likelihood and Robust Inference
        • Practical Considerations
        • Bringing It All Together
        • Further Reading
        • Chapter 3.5 Exercises: Generalized Linear Models Mastery
        • References
      • Section 3.6 Chapter 3 Summary
        • The Parametric Inference Pipeline
        • The Five Pillars of Chapter 3
        • How the Pillars Connect
        • Method Selection Guide
        • Quick Reference: Core Formulas
        • Connections to Future Material
        • Practical Guidance
        • Final Perspective
        • References
    • Chapter 4: Resampling Methods
      • Section 4.1 The Sampling Distribution Problem
        • The Fundamental Target: Sampling Distributions
        • Historical Development: The Quest for Sampling Distributions
        • Three Routes to the Sampling Distribution
        • When Asymptotics Fail: Motivating the Bootstrap
        • The Plug-In Principle: Theoretical Foundation
        • Computational Perspective: Bootstrap as Monte Carlo
        • Practical Considerations
        • Bringing It All Together
        • Chapter 4.1 Exercises
        • References
      • Section 4.2 The Empirical Distribution and Plug-in Principle
        • The Empirical Cumulative Distribution Function
        • Convergence of the Empirical CDF
        • Parameters as Statistical Functionals
        • The Plug-in Principle
        • When the Plug-in Principle Fails
        • The Bootstrap Idea in One Sentence
        • Computational Implementation
        • Bringing It All Together
        • Section 4.2 Exercises: ECDF and Plug-in Mastery
        • References
      • Section 4.3 The Nonparametric Bootstrap
        • The Bootstrap Principle
        • Bootstrap Standard Errors
        • Bootstrap Bias Estimation
        • Bootstrap Confidence Intervals
        • Bootstrap for Regression
        • Bootstrap Diagnostics
        • When Bootstrap Fails
        • Practical Considerations
        • Bringing It All Together
        • Exercises
        • References
      • Section 4.4: The Parametric Bootstrap
        • The Parametric Bootstrap Principle
        • Location-Scale Families
        • Parametric Bootstrap for Regression
        • Confidence Intervals
        • Model Checking and Validation
        • When Parametric Bootstrap Fails
        • Parametric vs. Nonparametric: A Decision Framework
        • Practical Considerations
        • Bringing It All Together
        • References
      • Section 4.5: Jackknife Methods
        • Historical Context and Motivation
        • The Delete-1 Jackknife
        • Jackknife Bias Estimation
        • The Delete-\(d\) Jackknife
        • Jackknife versus Bootstrap
        • The Infinitesimal Jackknife
        • Practical Considerations
        • Bringing It All Together
        • Exercises
        • References
      • Section 4.6 Bootstrap Hypothesis Testing and Permutation Tests
        • From Confidence Intervals to Hypothesis Tests
        • The Bootstrap Hypothesis Testing Framework
        • Permutation Tests: Exact Tests Under Exchangeability
        • Testing Equality of Distributions
        • Bootstrap Tests for Regression
        • Bootstrap vs Classical Tests
        • Permutation vs Bootstrap: Choosing the Right Approach
        • Multiple Testing with Bootstrap
        • Practical Considerations
        • Bringing It All Together
        • Exercises
        • References
      • Section 4.7 Bootstrap Confidence Intervals: Advanced Methods
        • Why Advanced Methods?
        • The Studentized (Bootstrap-t) Interval
        • Bias-Corrected (BC) Intervals
        • Bias-Corrected and Accelerated (BCa) Intervals
        • Choosing B and Assessing Monte Carlo Error
        • Diagnostics for Advanced Bootstrap Methods
        • Method Selection Guide
        • Bringing It All Together
        • Chapter 4.7 Exercises: Bootstrap Confidence Interval Mastery
        • References
      • Section 4.8: Cross-Validation Methods
        • Historical Context: From Jackknife to Cross-Validation
        • Leave-One-Out Cross-Validation
        • K-Fold Cross-Validation
        • Generalized Cross-Validation
        • Nested Cross-Validation
        • Bootstrap Prediction Error Estimation
        • Connection to Information Criteria
        • The Variance Estimation Problem
        • Cross-Validation for Structured Data
        • Computational Considerations
        • Practical Considerations
        • Bringing It All Together
        • Exercises
        • References
      • Section 4.9 Chapter 4 Summary
        • The Resampling Philosophy
        • The Complete Resampling Workflow
        • The Eight Pillars of Chapter 4
        • Method Selection Guide
        • Quick Reference Tables
        • Common Pitfalls Checklist
        • Connections to Other Chapters
        • Learning Outcomes Checklist
        • Practical Guidance
        • Further Reading: Advanced Resampling Topics
        • Final Perspective
        • References
  • Part III: Bayesian Inference
    • Chapter 5: Bayesian Inference
      • Section 5.1 Foundations of Bayesian Inference
        • Historical Development: From Bayes’ Essay to the MCMC Revolution
        • The Bayesian Workflow
        • Bayes’ Theorem for Parameters
        • Discrete Parameter Spaces
        • Continuous Parameters: Grid Approximation
        • The Posterior as Complete Inference
        • The Likelihood Principle Revisited
        • Practical Considerations
        • Bringing It All Together
        • Chapter 5.1 Exercises
        • References
      • Section 5.2 Prior Specification and Conjugate Analysis
        • The Role of the Prior
        • Beta-Binomial Conjugate Analysis
        • Normal-Normal Model: Known Variance
        • Normal-Inverse-Gamma: Unknown Mean and Variance
        • Poisson-Gamma Model
        • Multinomial-Dirichlet Model
        • Beyond Conjugacy: The Full Landscape of Prior Specification
        • Bayesian vs. Frequentist Synthesis and the Limits of Conjugacy
        • Practical Considerations
        • Bringing It All Together
        • Exercises
        • References
      • Section 5.3 Posterior Inference: Credible Intervals and Hypothesis Assessment
        • Credible Intervals: Probability Statements About Parameters
        • Posterior Probability and Directional Hypothesis Assessment
        • Posterior Predictive Intervals
        • Communicating Bayesian Results
        • Practical Considerations
        • Bringing It All Together
        • Exercises
        • References
      • Section 5.4 Markov Chains: The Mathematical Foundation of MCMC
        • From Grid Approximation to Markov Chains
        • Markov Chains
        • Stationary Distributions and Detailed Balance
        • The Ergodic Theorem: Why MCMC Averages Converge
        • The MCMC Estimator, Effective Sample Size, and \(\hat{R}\)
        • Python: Simulating Convergence and Diagnosing Chains
        • Mixing, Thinning, and Practical Considerations
        • Bringing It All Together
        • Exercises
        • References
      • Section 5.5 MCMC Algorithms: Metropolis-Hastings and Gibbs Sampling
        • The Metropolis-Hastings Algorithm
        • The Gibbs Sampler
        • Hamiltonian Monte Carlo: A Brief Introduction
        • Practical Workflow
        • Bringing It All Together
        • Exercises
        • References
      • Section 5.6 Probabilistic Programming with PyMC
        • How PyMC Works: From Model Spec to Gradient
        • The Diagnostic Toolkit
        • Full Worked Example 1: Logistic Regression
        • Full Worked Example 2: Poisson Regression
        • Scale Parameters: Half-Normal and Half-Cauchy Priors
        • Derived Quantities with pm.Deterministic
        • Practical Workflow Checklist
        • Bringing It All Together
        • Exercises
        • References
      • Section 5.7 Bayesian Model Comparison
        • The Predictive Accuracy Target
        • WAIC
        • PSIS-LOO Cross-Validation
        • Implementation: Comparing Two Models
        • Bayes Factors
        • Bringing It All Together
        • Exercises
        • References
      • Section 5.8 Hierarchical Models and Partial Pooling
        • The Pooling Problem
        • The Mathematical Structure
        • Worked Example 1: Eight Schools (Normal Likelihood)
        • Worked Example 2: Dirichlet-Multinomial (Categorical Likelihood)
        • LOO Model Comparison: Three Pooling Strategies
        • When to Use Hierarchical Models
        • Practical Considerations
        • Bringing It All Together
        • Exercises
        • References
      • Section 5.9 Chapter 5 Summary
        • The Bayesian Workflow
        • Section-by-Section Synthesis
        • Decision Guide: Which Bayesian Tool When
        • Quick Reference: Core Formulas
        • Common Pitfalls
        • Connections Across the Chapter
        • Connections to Earlier Chapters
  • Part IV: Large Language Models in Data Science
    • Chapter 6: LLMs in Data Science Workflows
STAT 350: Introduction to Statistics
  • Search

© Copyright .

Built with Sphinx using a theme provided by Read the Docs.