7.4. Discrete Random Variables and the CLT
We’ve explored how the Central Limit Theorem applies to continuous distributions, showing that the sampling distribution of the sample mean approaches a normal distribution as the sample size increases. But what about discrete random variables? Can we apply the CLT to distributions that count things rather than measure them?
In this chapter, we’ll examine how the CLT applies to discrete distributions like the binomial and Poisson, and explore when and how to use normal approximations for these discrete outcomes.
Road Map 🧭
Problem: How do we apply the CLT to discrete random variables that count successes or events?
Tool: Normal approximations to binomial and Poisson distributions with appropriate conditions and corrections
Pipeline: This extends our ability to use normal-based inference methods to discrete data contexts
7.4.1. The CLT for Discrete Random Variables
The Central Limit Theorem isn’t limited to continuous distributions. It also applies to discrete random variables like those from binomial and Poisson distributions, provided certain conditions are met.
When we use the CLT to approximate discrete distributions with the normal distribution, we’re using a continuous distribution to model discrete outcomes. This requires special consideration because discrete variables can only take specific values with gaps between them, while normal distributions are continuous across their range.
7.4.2. Binomial Distribution and the CLT
The binomial distribution B(n,p) counts the number of successes in n independent trials, each with probability of success p. A binomial random variable can be expressed as a sum of n independent Bernoulli random variables:
where each \(X_i\) is a Bernoulli random variable that equals 1 with probability p and 0 with probability (1-p).
Since the binomial random variable is a sum of independent and identically distributed random variables, the CLT applies as n increases. For sufficiently large n, the distribution of X can be approximated by:
The rule of thumb for when this approximation is valid includes two common criteria:
Both \(np ≥ 10\) and \(n(1-p) ≥ 10\), ensuring enough expected successes and failures
Alternatively, \(np(1-p) ≥ 10\), focusing on the variance of the distribution
These conditions ensure that the discrete binomial distribution is well-approximated by the continuous normal distribution.
7.4.3. Poisson Distribution and the CLT
The Poisson distribution with parameter λ counts the number of events occurring in a fixed interval, where events happen at a constant average rate and independently of each other.
An interesting property of the Poisson distribution is that the sum of independent Poisson random variables is also Poisson distributed. If X ~ Poisson(λ₁) and Y ~ Poisson(λ₂) are independent, then X + Y ~ Poisson(λ₁ + λ₂).
By extension, if X₁, X₂, …, Xₙ are independent Poisson random variables with the same parameter λ, then:
The CLT tells us that for large λ, a Poisson random variable can be approximated by a normal distribution:
This approximation works well when λ is sufficiently large (typically λ ≥ 10), as the Poisson distribution becomes more symmetric and bell-shaped.
7.4.4. Continuity Correction for Discrete Distributions
When using the normal distribution to approximate discrete distributions like the binomial or Poisson, we need to account for the difference between discrete and continuous variables. This is done through a continuity correction.
Continuity Correction for Binomial Distribution
Consider calculating a probability for a binomial random variable \(X ~ B(n,p)\) using the normal approximation \(X* ~ N(np, np(1-p))\). When computing \(P(X = k)\), we need to recognize that X can only take integer values, while \(X*\) is continuous.
The probability \(P(X = k)\) is approximated by the area under the normal curve between \(k-0.5\) and \(k+0.5\):
Similarly, for range probabilities:
For example, to calculate \(P(X = 48)\) for \(X ~ B(100, 0.5)\):
Without continuity correction: \(P(X* = 48) = 0\) (as the probability of any exact value in a continuous distribution is zero)
With continuity correction: \(P(X = 48) ≈ P(47.5 < X* < 48.5)\)
Continuity Correction for Poisson Distribution
Similar principles apply to the Poisson distribution. For a Poisson random variable \(X ~ Poisson(λ)\) approximated by \(X* ~ N(λ, √λ)\):
These corrections significantly improve the accuracy of normal approximations to discrete distributions, especially when calculating probabilities for specific values rather than ranges.
7.4.5. Applications of the CLT for Discrete Variables
The normal approximation to discrete distributions has numerous practical applications in statistics, making calculations more straightforward for large samples.
Quality Control and Manufacturing
In manufacturing processes, defective items often follow a binomial distribution. When large batches are produced, the CLT allows quality control engineers to efficiently calculate probabilities related to defect rates using the normal approximation.
For example, if the probability of a defect is 0.05 and 1000 items are produced, the normal approximation can be used to calculate the probability of observing more than 60 defective items:
With continuity correction:
Medical Research and Clinical Trials
In large clinical trials, the number of patients who respond to a treatment often follows a binomial distribution. The CLT enables researchers to efficiently calculate confidence intervals and perform hypothesis tests for treatment efficacy.
Network Traffic and Server Loads
The number of requests arriving at a server in a fixed time interval often follows a Poisson distribution. For high-traffic systems, the CLT allows network engineers to use the normal approximation to calculate probabilities related to server load and capacity planning.
Practical Example: Machine Calibration
Consider a manufacturing process producing components with a target diameter of 5mm. The diameters follow a normal distribution with mean 5mm and standard deviation 0.5mm. A sample of 64 components is taken to assess whether the machine needs recalibration.
If the sample mean is 4.85mm, we can use the CLT to determine if this deviation is statistically significant:
The probability of observing a deviation this large or larger is:
If the company policy requires recalibration when this probability is less than 0.05, then recalibration is necessary because 0.0164 < 0.05.
This application uses the normal distribution directly since the sample mean has a normal distribution by the CLT, even though the individual measurements follow a normal distribution as well.
7.4.6. Limitations and Considerations
While the normal approximation to discrete distributions is powerful, there are important limitations and considerations to keep in mind:
Sample Size Requirements
The required sample size or parameter value depends on the specific discrete distribution and its parameters:
For binomial distributions, both np and n(1-p) should be at least 10 for the approximation to be reliable
For Poisson distributions, λ should be at least 10
When p is very small or very large in the binomial distribution, larger sample sizes are needed
Alternative Methods
In some cases, other approaches may be more appropriate than using the normal approximation:
For small samples, exact methods using the probability mass functions of the discrete distributions are more accurate
For binomial distributions with small n, direct calculation using the binomial formula or binomial tables may be preferable
For Poisson distributions with small λ, direct calculation using the Poisson formula is more accurate
Statistical software often provides exact methods that don’t require the normal approximation
Importance of Continuity Correction
The continuity correction is crucial when calculating probabilities for specific values or narrow ranges. Without it, the normal approximation can produce significant errors, especially for:
Probabilities of exact values (e.g., :math:` P(X = k)`)
Probabilities near the tails of the distribution
Distributions with smaller parameter values that barely meet the criteria for normal approximation
Visual Assessment
Visually comparing the discrete distribution with its normal approximation can help assess the quality of the approximation:
For binomial distributions with \(p = 0.5\), the approximation is typically good even for moderate n
For binomial distributions with p far from 0.5 (e.g., p = 0.1 or p = 0.9), the approximation requires larger n
For Poisson distributions, as λ increases, the distribution becomes more symmetric and the normal approximation improves
7.4.7. Concluding the Chapter: The Universal Power of the Central Limit Theorem
Throughout these chapters, we’ve explored the Central Limit Theorem in various contexts - from continuous to discrete distributions, from measuring to counting, from theoretical foundations to practical applications.
The CLT stands as one of the most remarkable and powerful results in all of statistics, providing a mathematical bridge that connects diverse random phenomena through a common distributional form: the normal distribution.
What We’ve Learned
We began by examining how sample means from continuous distributions approach normality as sample size increases, regardless of the shape of the original population distribution. We saw through simulations and mathematical derivations that this convergence happens more quickly for some distributions than others, but the pattern remains universal for distributions with finite mean and variance.
We then extended these principles to discrete distributions, showing how the binomial and Poisson distributions can be approximated by the normal distribution when certain conditions are met. We learned about continuity corrections and how to handle the transition from discrete to continuous probability models.
Throughout, we’ve seen that the CLT has broad applications across numerous fields - from quality control and manufacturing to medical research, from environmental science to finance, and from network engineering to pharmaceutical development.
The Foundation for Statistical Inference
The CLT doesn’t just provide convenient computational shortcuts; it forms the theoretical foundation for many statistical inference procedures. By understanding how sample statistics behave across repeated samples, we can:
Construct confidence intervals that capture population parameters with specified reliability
Develop hypothesis tests that control error rates at predetermined levels
Make probability statements about sample statistics that quantify uncertainty
Compare different populations using samples of reasonable size
Without the CLT, statistical inference would be far more complex and likely require different procedures for different types of data. The theorem’s universality provides the common mathematical language that unifies statistical methodology.
Beyond the Sample Mean
While our focus has been primarily on the sample mean, we’ve noted that the CLT extends to other statistics as well - sample sums, weighted averages, linear combinations, and under certain conditions, to statistics like the sample variance. This broad applicability explains why the normal distribution appears so frequently in statistical analyses and why understanding its properties is essential for statistical practice.
As we move forward to topics in statistical inference in subsequent chapters, the CLT will serve as our theoretical bedrock, enabling us to make reliable inferences from sample data to population parameters across a wide range of scenarios.
The elegant simplicity of the theorem belies its profound implications: regardless of the complexity and diversity of random phenomena in our world, their averages tend toward a common, predictable pattern. This insight not only simplifies statistical computations but provides a deeper understanding of the underlying order in seemingly chaotic processes.
7.4.8. Bringing It All Together
The Central Limit Theorem extends beyond continuous distributions to discrete random variables, allowing us to use the normal distribution to approximate binomial and Poisson probabilities under appropriate conditions. By understanding when these approximations are valid and applying continuity corrections, we can leverage the mathematical convenience of the normal distribution while maintaining reasonable accuracy.
This capability is especially valuable when:
Manual calculations of exact probabilities would be cumbersome
Working with large sample sizes or parameter values
Creating confidence intervals or conducting hypothesis tests based on discrete data
The application of the CLT to discrete distributions further demonstrates its power and versatility as a foundational concept in statistical inference, building a bridge between discrete counting processes and continuous measurement models.
Key Takeaways 📝
The CLT applies to discrete distributions like the binomial and Poisson when certain conditions are met.
For binomial distributions, the normal approximation is valid when np ≥ 10 and n(1-p) ≥ 10 (or alternatively when np(1-p) ≥ 10).
For Poisson distributions, the normal approximation is valid when λ ≥ 10.
Continuity corrections are necessary when using a continuous distribution to approximate a discrete one.
The probability of a discrete outcome k is approximated by the area under the normal curve between k-0.5 and k+0.5.
Probabilities involving ranges (≤, ≥, <, >) require appropriate continuity corrections at the boundaries.
The quality of approximation depends on the sample size, parameter values, and the specific probability being calculated.
Exercises
Binomial Approximation: A fair coin is tossed 100 times.
Use the binomial formula to calculate the exact probability of obtaining exactly 55 heads
Use the normal approximation with continuity correction to approximate this probability
Calculate the percent error in the approximation
Poisson Approximation: A call center receives an average of 15 calls per hour.
What is the exact probability of receiving exactly 20 calls in an hour?
Approximate this probability using the normal distribution with continuity correction
What is the approximate probability of receiving between 10 and 20 calls (inclusive) in an hour?
Quality Control: A manufacturing process produces items with a 3% defect rate. In a batch of 500 items:
Use the normal approximation to calculate the probability of finding more than 20 defective items
Calculate the probability that the number of defective items is within 5 of the expected number
How large should the batch be to have a 90% probability that the number of defects is within 1% of the total batch size?
Binomial vs. Poisson: Under what circumstances can a binomial distribution be approximated by a Poisson distribution? When would you choose to use a normal approximation instead?
Comparing Approaches: For X ~ Binomial(20, 0.4):
Calculate P(X ≤ 5) using the exact binomial formula
Approximate the probability using the normal distribution with continuity correction
Would you recommend using the normal approximation in this case? Why or why not?
Simulation Exercise: Design a simulation to demonstrate how the quality of the normal approximation to the binomial distribution improves as n increases while p remains fixed at 0.3.
Comprehensive Review: For each of the following distributions, indicate whether the CLT can be applied, what conditions need to be met, and what the resulting normal approximation would be:
A population with unknown shape but finite mean μ = 100 and standard deviation σ = 20
A binomial distribution with n = 25 and p = 0.4
A Poisson distribution with λ = 8
A heavily right-skewed distribution with infinite variance
A binomial distribution with n = 1000 and p = 0.002
CLT Application Across Chapters: Reflect on how the CLT connects the material in this chapter to previous probability concepts and how it will serve as a foundation for upcoming inference methods. Provide specific examples of how the CLT bridges these topics.