Worksheet 9: The Normal Distribution

Learning Objectives 🎯

Understand the Normal (Gaussian) distribution and its properties
Apply the Empirical Rule (68-95-99.7 rule) for quick probability estimates
Master the standard Normal distribution and z-score transformations
Use Normal tables to calculate probabilities and percentiles
Solve forward problems (finding probabilities) and backward problems (finding values)
Apply Normal distribution to real-world scenarios

Introduction

We have previously discussed the general form of probability density functions and cumulative distribution functions, as well as two fundamental examples of named continuous distributions: the Uniform and Exponential distributions. Many real-world measurements, such as heights and measurement errors, naturally cluster around an average in a roughly symmetrical, bell-shaped pattern. The Gaussian distribution, also called the Normal distribution, is the most widely used model for describing this shape. Sums or averages of many independent non-Normal random variables often converge to a Normal distribution, which makes it a cornerstone of statistical inference and practical applications.

In this worksheet, we explore the key properties of the Normal distribution, show how the standard Normal serves as a reference distribution, and practice essential probability calculations for finding areas and percentiles under the bell-shaped curve.

Part 1: The Normal Distribution

A continuous Normal random variable \(X\) has support the entire real line \(\mathbb{R} = (-\infty, +\infty)\), and is defined by the following probability density function:

\[f_X(x) = \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2}\]

The distribution has the following properties: it is symmetrical, unimodal, and bell shaped (concave down at the center and then concave up starting about one standard deviation on each side). It is completely determined by two parameters, the mean \(\mu\) and standard deviation \(\sigma\). However, computing probabilities from this distribution directly is non-trivial since the cumulative distribution function has no closed-form solution. Instead, probabilities must be obtained through numerical methods, such as integration algorithms, statistical tables, or software.

We will begin with the traditional approach of using statistical tables, and in the next section when we discuss the Central Limit Theorem, we will incorporate R for these calculations. However, since there are infinitely many Normal distributions, one for each choice of mean \(\mu\) and standard deviation \(\sigma\), we first need a reference distribution, the standard Normal distribution, which allows us to standardize any Normal variable and use a common probability table for calculations.

The Standard Normal Distribution

A continuous Normal random variable \(Z\) with mean \(\mu = 0\) and standard deviation \(\sigma = 1\) is known as the standard Normal distribution and is defined by the following density function:

\[f_Z(z) = \frac{1}{\sqrt{2\pi}} e^{-\frac{z^2}{2}}\]

The standard Normal distribution will serve as a reference distribution for our probability calculations for Normal random variables.

Part 2: The Empirical Rule

However, let’s start by using the properties of the Normal distribution curve to answer some basic questions. One of the most useful properties of the Normal distribution is the Empirical Rule, also known as the 68-95-99.7 rule. This rule provides a quick way to estimate probabilities based on how data is distributed around the mean. Specifically, for any Normal distribution:

Approximately 68% of values fall within one standard deviation of the mean (\(\mu \pm \sigma\)):

\[P(\mu - \sigma < X < \mu + \sigma) = 0.68\]
Approximately 95% of values fall within two standard deviations of the mean (\(\mu \pm 2\sigma\)):

\[P(\mu - 2\sigma < X < \mu + 2\sigma) = 0.95\]
Approximately 99.7% of values fall within three standard deviations of the mean (\(\mu \pm 3\sigma\)):

\[P(\mu - 3\sigma < X < \mu + 3\sigma) = 0.997\]

The empirical rule allows us to quickly assess the likelihood of an observation occurring within a certain range without the need for complex calculations. Before using statistical tables for more precise probability computations, we will first apply this rule to answer some fundamental probability questions about the Normal distribution.

Question 1: A manufacturing company produces metal rods that follow a Normal distribution with a mean length of 50 cm and a standard deviation of 2 cm.

Using the Empirical Rule, determine the probability that a randomly selected rod has a length within the following ranges. Clearly write out the probability statement before computing each value.
1. Greater than 52 cm.
2. Less than 48 cm.
3. Between 44 cm and 54 cm.
4. Greater than 54 cm or less than 44 cm.
Suppose you randomly select a rod from a pile labeled “greater than 52 cm”. Using the Empirical Rule, determine the probability that the rod is also greater than 54 cm.
Using the Empirical Rule determine the 84th percentile of rod lengths.
A quality control engineer decides that rods must be between 46 cm and 54 cm to be considered within acceptable tolerance limits. A batch contains 10,000 rods, and rods are considered defective if their lengths fall outside the range 46 cm to 54 cm. How many rods would be expected to fail the quality check?

R Code for Empirical Rule Visualization:

# Parameters
mu <- 50      # mean length
sigma <- 2    # standard deviation

# Create a visualization of the Empirical Rule
x <- seq(mu - 4*sigma, mu + 4*sigma, 0.01)
y <- dnorm(x, mean = mu, sd = sigma)

plot(x, y, type = "l", main = "Normal Distribution with Empirical Rule",
     xlab = "Rod Length (cm)", ylab = "Density", lwd = 2)

# Add vertical lines for standard deviations
abline(v = mu, col = "black", lwd = 2)
abline(v = c(mu - sigma, mu + sigma), col = "blue", lty = 2)
abline(v = c(mu - 2*sigma, mu + 2*sigma), col = "red", lty = 2)
abline(v = c(mu - 3*sigma, mu + 3*sigma), col = "green", lty = 2)

# Add shaded regions
x1 <- seq(mu - sigma, mu + sigma, 0.01)
y1 <- dnorm(x1, mean = mu, sd = sigma)
polygon(c(x1, rev(x1)), c(y1, rep(0, length(y1))),
        col = rgb(0, 0, 1, 0.2), border = NA)

# Add labels
text(mu, max(y)*1.1, "μ", cex = 1.2)
text(mu + sigma, max(y)*0.9, "μ+σ", col = "blue")
text(mu - sigma, max(y)*0.9, "μ-σ", col = "blue")

# Add percentage labels
text(mu, max(y)*0.5, "68%", cex = 1.2, col = "blue")
text(mu, max(y)*0.2, "95%", cex = 1.2, col = "red")
text(mu, max(y)*0.05, "99.7%", cex = 1.2, col = "green")

Part 3: The Standard Normal Table

While the Empirical Rule provides a useful approximation, most real probability calculations do not fall neatly into the 68-95-99.7% intervals, requiring a more precise approach. The standard Normal table provides cumulative probabilities for the standard Normal distribution (\(\mu = 0, \sigma = 1\)), serving as a universal reference for probability calculations. Since the cumulative distribution function has no closed-form solution, the values in the table were obtained using numerical approximation methods. To use this table, we transform any Normal variable into standard Normal form through a process called the z-score transformation, which is analogous to u-substitution in integration, shifting and scaling the variable so that probabilities can be determined from a single distribution.

Even though we cannot obtain a closed-form solution for the cumulative distribution function (CDF), the standard Normal distribution is often referenced using the notation \(\Phi(z)\) to represent its CDF. That is,

\[\Phi(z) = P(Z \leq z)\]

where \(Z\) follows a standard Normal distribution, denoted as:

\[Z \sim N(\mu = 0, \sigma = 1)\]

The function \(\Phi(z)\) gives the probability that a standard Normal variable takes on a value less than or equal to \(z\) and is computed using numerical approximation methods. The standard Normal table provides precomputed values of \(\Phi(z)\), allowing us to determine probabilities without directly integrating the probability density function.

Question 2: Using the standard normal table found on Brightspace under “Extra Documents” answer the following questions. If you get stuck try drawing the normal curve and shading in the region you need to find before using the table.

Compute the following probabilities:
1. \(\Phi(2.34) =\)
2. \(P(Z > -0.12) =\)
3. \(P(-1.64 < Z < 1.64) =\)
Determine the following percentiles.
1. \(P(Z < z_p) = 0.9192\)
2. \(P(Z < z_p) = 0.95\)
Determine the follow upper percentiles.
1. \(P(Z > z_p) = 0.017\)
2. \(P(Z > z_p) = 0.9990\)
Determine two values a and b symmetric about zero such that \(P(a \leq Z \leq b) = 0.95\).

Part 4: Z-Score Transformation

Not all Normal distributions have a mean of 0 and a standard deviation of 1. In many cases, we deal with non-standard Normal distributions, which have different means (\(\mu\)) and standard deviations (\(\sigma\)). Since the standard Normal table only provides probabilities for the standard Normal distribution, we must first transform any Normal variable into standard form.

This is done using the z-score transformation, given by:

\[z = \frac{x - \mu}{\sigma}\]

This transformation expresses how many standard deviations a given value \(x\) from any Normal distribution \(N(\mu, \sigma)\) is from its mean. By converting to the standard Normal scale, we can use a single probability table for all Normal distributions.

There are two main types of problems involving the Normal distribution: forward problems and backward problems.

Forward problems involve computing probabilities for a given Normal value by first converting it to a z-score and then using the standard Normal table to determine the probability. Recommended steps to follow:

Draw the probability region of interest and identify what you need to calculate.
Standardize: Convert to z-score.
If necessary, change the probability statement so you can use the table.
Round z-score to 2 decimal places and look it up on the table.
Write your conclusion in the context of the problem.

Backward problems involve finding a specific value \(x\) when given a probability or percentile. In these cases, we look up the probability in the standard Normal table to find the corresponding z-score and then use the z-score transformation in reverse to solve for \(x\). Recommended steps to follow:

Draw the probability region of interest and determine the location you need to find.
Set up and modify your probability statements as needed
Find the z-score by looking up the probability in the main body of the normal table.
Convert the z-score to \(x\) using \(x = \mu + \sigma z\)
Write your conclusion in the context of the problem.

Question 3: Let \(X\) be a Normal random variable with mean \(\mu = 10\) and variance \(\sigma^2 = 25\). Answer the following questions.

Using the forward process determine the following probabilities.
1. \(P(X < 19.8) =\)
2. \(P(X > -10) \approx\)
3. \(P(0.2 < X < 19.8) =\)
Using the backward process determine the following percentiles in terms of the distribution of \(X\), in other words find \(x_p\).
1. \(P(X < x_p) = 0.7486\)
2. \(P(X < x_p) = 0.8\)
Using the backward process determine the following upper percentile \(P(X > x_p) = 0.99\).
Determine inter quartile range of the distribution. In other words find two values \(x_1\) and \(x_2\) centered around the mean such that \(P(x_1 \leq X \leq x_2) = 0.5\).

Question 4: An insurance company is analyzing policyholders who have all survived to age 60. Historical data suggests that their eventual ages at death can be approximated by a Normal distribution with mean of 85 years and a standard deviation of 4 years.

A newly turned 60-year-old policyholder asks, “What is the probability I will live past 90?” Compute the probability.
The insurance company wants to define an age cutoff above which only 5% of these policyholders will live. Find the value \(x_{0.05}\) the age threshold above which only top 5% of the 60+ insured population are expected to live.

R Code for Normal Distribution Calculations:

# Example: Normal distribution calculations in R
# (For when tables are not required)

# Question 3 parameters
mu <- 10
sigma <- 5  # sd = sqrt(25)

# Forward problems
# P(X < 19.8)
prob1 <- pnorm(19.8, mean = mu, sd = sigma)
cat("P(X < 19.8) =", prob1, "\n")

# Using z-score approach
z1 <- (19.8 - mu) / sigma
prob1_z <- pnorm(z1)
cat("Using z-score: z =", z1, ", P(Z < z) =", prob1_z, "\n")

# Backward problems
# Find x such that P(X < x) = 0.7486
x_p <- qnorm(0.7486, mean = mu, sd = sigma)
cat("\nPercentile: P(X <", x_p, ") = 0.7486\n")

# Using z-score approach
z_p <- qnorm(0.7486)
x_p_calc <- mu + sigma * z_p
cat("Using z-score: z =", z_p, ", x =", x_p_calc, "\n")

# Visualization
x <- seq(mu - 4*sigma, mu + 4*sigma, 0.1)
y <- dnorm(x, mean = mu, sd = sigma)

plot(x, y, type = "l", main = "Normal Distribution N(10, 25)",
     xlab = "x", ylab = "Density", lwd = 2)

# Shade area for P(X < 19.8)
x_shade <- seq(-Inf, 19.8, 0.1)
y_shade <- dnorm(x_shade, mean = mu, sd = sigma)
polygon(c(x_shade, rev(x_shade)), c(y_shade, rep(0, length(y_shade))),
        col = rgb(0, 0, 1, 0.3), border = NA)

abline(v = c(mu, 19.8), col = c("red", "blue"), lty = c(1, 2))
text(19.8, max(y)*0.8, paste0("P(X < 19.8) = ", round(prob1, 4)),
     pos = 4, col = "blue")

Key Takeaways

Summary 📝

The Normal distribution is bell-shaped, symmetric, and determined by mean \(\mu\) and SD \(\sigma\)
The Empirical Rule: approximately 68% within \(\pm 1\sigma\), 95% within \(\pm 2\sigma\), 99.7% within \(\pm 3\sigma\)
The standard Normal has \(\mu = 0, \sigma = 1\) and serves as reference distribution
Z-score transformation: \(z = \frac{x - \mu}{\sigma}\) standardizes any Normal variable
Forward problems: given value, find probability (standardize, then use table)
Backward problems: given probability, find value (use table to find z, then unstandardize)
The CDF is denoted \(\Phi(z) = P(Z \leq z)\) for standard Normal