Worksheet 13: Introduction to Confidence Intervals

Learning Objectives 🎯

Understand the concept of pivotal quantities and their role in constructing confidence intervals
Derive confidence interval formulas algebraically from probability statements
Calculate appropriate sample sizes to achieve desired precision levels
Construct both two-sided and one-sided confidence bounds for population parameters
Distinguish between confidence intervals when σ is known versus unknown
Apply the t-distribution for inference when population variance must be estimated
Implement R simulations to verify confidence interval properties and build intuition about coverage

Introduction

In this worksheet, we transition into statistical inference, building upon the foundations established earlier in the course. We have already examined important concepts such as sampling techniques, experimental design, the Central Limit Theorem (CLT), and the properties of estimators, particularly unbiasedness.

Up to now, our focus has primarily been on describing populations through parameters like the mean \(\mu\) and standard deviation \(\sigma\), and calculating probabilities when the population distribution is fully known.

Statistical inference allows us to reverse this approach. Instead of starting with a known population and predicting what sample results we might observe, we now use information from a single sample to estimate unknown population parameters. This transition is fundamental in statistics because, in practice, we rarely know the true characteristics of an entire population. Instead, we typically have only a single random sample, from which we aim to infer these unknown characteristics.

The Challenge of Uncertainty 🤔

Any estimate derived from a single sample carries uncertainty due to natural sampling variability. To quantify this uncertainty, we introduce confidence intervals (CIs).

Confidence intervals provide a range of plausible values for an unknown population parameter, constructed using the sample data we observe. The confidence interval is composed of two key parts:

A point estimate derived from our sample data
A margin of error that reflects the precision of this estimate

The general form of a confidence interval is:

\[\text{Point Estimate} \pm \text{Margin of Error}\]

The margin of error depends on two main components:

The confidence level (e.g., 95%)
The standard error of the estimator

The standard error tells us how much the estimator, like the sample mean, would fluctuate across repeated samples of the same size. The confidence level expresses the proportion of such intervals that would include the true population parameter if we could sample repeatedly and construct a new interval each time. A higher confidence level typically expands the interval, showing that we are more certain it covers the true parameter.

In this worksheet, we will specifically explore how to construct confidence intervals for estimating the population mean when the population standard deviation is known. This will lay the foundation for future discussions where the population standard deviation is not known.

Part 1: Pivotal Quantities and Deriving Confidence Intervals

A pivotal quantity is a fundamental tool for constructing confidence intervals. Understanding this concept will help you see where confidence interval formulas come from.

Definition: Pivotal Quantity 📐

A pivotal quantity is a statistic that:

Involves the unknown parameter of interest (here \(\mu\))
Has a known distribution that does not depend on any unknown parameters

Question 1: Suppose you have a random sample (SRS) \(X_1, X_2, \ldots, X_n\) from a population with unknown mean \(\mu\) and known standard deviation \(\sigma\).

Note

Recall from the Central Limit Theorem (CLT): If the sample size is sufficiently large, the sampling distribution of the sample mean \(\overline{X}\) is approximately Normal with:

Mean: \(\mu\)
Standard deviation: \(\sigma/\sqrt{n}\) (called the standard error)

If \(X_1, X_2, \ldots, X_n\) come from a Normal distribution, show clearly that the standardized sample mean

\[Z = \frac{\overline{X} - \mu}{\sigma/\sqrt{n}}\]

is a pivotal quantity.

Your work showing Z is pivotal:

\[Z \sim \text{____}\]

Because we know the exact distribution of the pivotal quantity \(Z\), we can form a probability statement about it:

\[P\left(-z_{\alpha/2} < \frac{\overline{X} - \mu}{\sigma/\sqrt{n}} < z_{\alpha/2}\right) = 1 - \alpha\]

where \(z_{\alpha/2}\) is the critical value from the standard Normal distribution.

Note

R Tip for Critical Values: Throughout this worksheet, we use lower.tail = FALSE when calculating critical values:

For two-sided intervals: qnorm(alpha/2, lower.tail = FALSE)
For one-sided bounds: qnorm(alpha, lower.tail = FALSE)

This is equivalent to qnorm(1 - alpha/2) but makes it explicit that we’re finding the upper tail critical value. The same applies to qt() for t-distributions.

i) Algebraic Derivation of the Confidence Interval

Use algebra to rearrange this probability statement into an explicit confidence interval form for the population mean \(\mu\).

Step 1: Start with the probability statement:

\[P\left(-z_{\alpha/2} < \frac{\overline{X} - \mu}{\sigma/\sqrt{n}} < z_{\alpha/2}\right) = 1 - \alpha\]

Step 2: Multiply all parts of the inequality by \(\sigma/\sqrt{n}\):

\[P\left(\text{____} < \overline{X} - \mu < \text{____}\right) = 1 - \alpha\]

Step 3: Isolate \(\mu\) in the middle of the inequality (subtract \(\overline{X}\) from all parts and multiply by -1):

Warning

Common mistake: When multiplying an inequality by -1, remember to flip the inequality signs!

\[P\left(\text{____} < \mu < \text{____}\right) = 1 - \alpha\]

Step 4: Express in standard confidence interval form:

\[\text{CI for } \mu: \quad \text{____________________}\]

ii) Interpretation of the Interval

Clearly identify the resulting \((1-\alpha)100\%\) confidence interval and briefly explain why its distribution is helpful in constructing confidence intervals for the unknown but non-random \(\mu\).

The \((1-\alpha)100\%\) confidence interval is:

Why this distribution is helpful:

iii) 🔍 Careful Interpretation of Confidence Level

Carefully explain what it means to say that this is a \((1-\alpha)100\%\) confidence interval. Use the concept of repeated sampling and interval construction to clearly illustrate your explanation.

Warning

Common Misconception: A 95% confidence interval does NOT mean “there is a 95% probability that μ is in this specific interval.” Once computed, the interval either contains μ or it doesn’t - μ is fixed, not random!

Your explanation using repeated sampling:

# Simulation to understand confidence interval interpretation
set.seed(350)

# True population parameters
true_mu <- 100
sigma <- 15
n <- 25
confidence_level <- 0.95
alpha <- 1 - confidence_level
z_crit <- qnorm(alpha/2, lower.tail = FALSE)

# Number of samples to generate
n_samples <- 100

# Initialize storage
ci_lower <- numeric(n_samples)
ci_upper <- numeric(n_samples)
contains_mu <- logical(n_samples)

# Generate many samples and construct CIs
for (i in 1:n_samples) {
  sample_data <- rnorm(n, mean = true_mu, sd = sigma)
  x_bar <- mean(sample_data)
  se <- sigma / sqrt(n)

  # Construct CI
  ci_lower[i] <- x_bar - z_crit * se
  ci_upper[i] <- x_bar + z_crit * se

  # Check if CI contains true mu
  contains_mu[i] <- (ci_lower[i] <= true_mu) & (true_mu <= ci_upper[i])
}

# Calculate coverage proportion
coverage <- mean(contains_mu)
cat("Proportion of CIs containing true μ:", coverage, "\n")
cat("Expected coverage (confidence level):", confidence_level, "\n")

# Visualization
library(ggplot2)
ci_df <- data.frame(
  sample = 1:n_samples,
  lower = ci_lower,
  upper = ci_upper,
  contains = contains_mu
)

# Plot first 50 intervals for clarity
ggplot(ci_df[1:50, ], aes(x = sample, color = contains)) +
  geom_errorbar(aes(ymin = lower, ymax = upper), width = 0.2) +
  geom_hline(yintercept = true_mu, color = "red",
             linetype = "dashed", size = 1.2) +
  coord_flip() +
  scale_color_manual(values = c("TRUE" = "darkgreen", "FALSE" = "red"),
                     labels = c("Misses μ", "Contains μ")) +
  labs(title = "95% Confidence Intervals from 50 Random Samples",
       subtitle = paste("Red dashed line = True μ =", true_mu,
                       "\nGreen intervals captured μ, Red intervals missed it"),
       x = "Sample Number", y = "Confidence Interval",
       color = "") +
  theme_minimal() +
  theme(legend.position = "top")

iv) Role of the Central Limit Theorem

Discuss what might change if the sample did not come from a Normal distribution. Include the role of the Central Limit Theorem for large sample sizes.

Interactive Exploration 🖥️

Confidence Interval Simulator

Explore the interactive confidence interval simulator to build intuition about what confidence levels really mean:

https://treese41528.github.io/STAT350/ShinyApps/CI_Simulator.html

How to use the simulator:

Start with defaults: Observe what happens when you generate 100 samples - Watch how some intervals contain μ (green) and some miss it (red) - Check the coverage rate at the bottom
Experiment with confidence level: - Try 90%, 95%, and 99% confidence levels - How does this affect the interval width? - How does it affect the coverage rate?
Explore sample size effects: - Start with n = 10, then try n = 25, n = 50, n = 100 - What happens to the interval widths as n increases? - Does the coverage rate change?
Test different population distributions: - Switch from Normal to Exponential or Uniform - Does the CLT work for these distributions? - How large does n need to be for good coverage with non-normal populations?

Key questions to answer:

If we construct 100 intervals at 95% confidence, approximately how many contain μ?
What’s the relationship between confidence level and interval width?
Does increasing sample size affect coverage rate or just precision?

Part 2: Sample Size Determination

In real-world scenarios, we often need to determine how many observations (sample size) to collect to ensure our confidence interval achieves a specific level of precision. The precision of our estimate is quantified by the margin of error.

Question 2: Recall that for estimating a population mean \(\mu\) when the standard deviation \(\sigma\) is known, the margin of error (ME) of a confidence interval is given by:

\[\text{Margin of Error} = z_{\alpha/2} \frac{\sigma}{\sqrt{n}}\]

Suppose we want our confidence interval to have a margin of error no larger than a specified value \(M\), while maintaining a confidence level of \((1-\alpha)100\%\).

i) Deriving the Sample Size Formula

Derive an explicit formula for the minimum required sample size \(n\) that ensures the margin of error does not exceed \(M\).

Step 1: Set up the inequality:

\[z_{\alpha/2} \frac{\sigma}{\sqrt{n}} \leq M\]

Step 2: Solve for \(n\):

\[\sqrt{n} \geq \text{____}\]

\[n \geq \text{____}\]

Final formula:

\[n = \text{____________________}\]

ii) Why Use the Ceiling Function?

The derived formula typically results in a non-integer value. Explain clearly why we must use the ceiling function \(\lceil \cdot \rceil\) to determine the sample size \(n\) and not the floor function \(\lfloor \cdot \rfloor\) or rounding to the closest integer.

Your explanation:

# Function to calculate required sample size
calculate_sample_size <- function(sigma, margin_error, confidence_level = 0.95) {
  alpha <- 1 - confidence_level
  z_crit <- qnorm(alpha/2, lower.tail = FALSE)

  # Calculate n
  n_exact <- (z_crit * sigma / margin_error)^2
  n_required <- ceiling(n_exact)

  cat("Exact calculation:", n_exact, "\n")
  cat("Required sample size:", n_required, "\n")

  # Verify the margin of error
  actual_ME <- z_crit * sigma / sqrt(n_required)
  cat("Achieved margin of error:", actual_ME, "\n")
  cat("Desired margin of error:", margin_error, "\n")

  return(n_required)
}

# Example: sigma = 10, want ME = 2, 95% confidence
calculate_sample_size(sigma = 10, margin_error = 2, confidence_level = 0.95)

Part 3: One-Sided Confidence Bounds

So far, we’ve focused on constructing two-sided confidence intervals, which provide both a lower and an upper bound for an unknown population parameter. However, there are practical situations where our interest lies only in placing an upper bound or only a lower bound on a population mean \(\mu\).

One-Sided Confidence Bounds 📊

Upper Confidence Bound: Gives a value that the population mean \(\mu\) is unlikely to exceed

Lower Confidence Bound: Provides a value that \(\mu\) is unlikely to fall below

Formulas when σ is known:

Upper bound at \((1-\alpha)100\%\) confidence:

\[\mu < \overline{x} + z_{\alpha} \frac{\sigma}{\sqrt{n}}\]

Lower bound at \((1-\alpha)100\%\) confidence:

\[\mu > \overline{x} - z_{\alpha} \frac{\sigma}{\sqrt{n}}\]

Note: For one-sided bounds, we use \(z_{\alpha}\) instead of \(z_{\alpha/2}\)

Question 3: An e-commerce company closely monitors the load time of its product webpage. Past experience and extensive historical monitoring have allowed the engineering team to reliably estimate the standard deviation of page load times to be \(\sigma = 1\) second.

Recently, however, the development team implemented updates to the website that might have impacted the page’s average load time \(\mu\). The team now wants to verify if these recent changes have influenced the average load time.

a) Sample Size Determination

Suppose the team wants to estimate the average load time with a margin of error of at most 0.2 seconds at a 95% level of confidence. Derive the minimum required sample size \(n\) needed to satisfy this precision requirement.

Given information:

\(\sigma =\) ____
\(M =\) ____
Confidence level = ____
\(\alpha =\) ____
\(z_{\alpha/2} =\) ____

Calculation:

\[n =\]

b) Constructing and Interpreting the Confidence Interval

After collecting the determined number of load time observations, the team calculates the sample mean load time as \(\overline{x} = 3.45\) seconds. Construct a 95% confidence interval for the true mean load time.

Confidence Interval Calculation:

\[\text{CI} = \overline{x} \pm z_{\alpha/2} \frac{\sigma}{\sqrt{n}} =\]

Your 95% confidence interval:

Interpretation in practical terms using repeated sampling:

c) Upper Confidence Bound for Performance Standard

The management team has established a performance standard, requiring that average load times do not exceed a specific threshold. To verify compliance with this standard, the team decides to collect a sample of \(n = 200\) load times from the updated site.

Use a one-sided upper confidence bound with a 99% confidence level to get an upper bound on the load times. The sample mean from the sample of 200 load times was found to be \(\overline{x} = 3.6\) seconds. Interpret your 99% upper confidence bound.

Given information:

\(n = 200\)
\(\overline{x} = 3.6\) seconds
\(\sigma = 1\) second
Confidence level = 99%
\(\alpha =\) ____
\(z_{\alpha} =\) ____ (Note: one-sided, so use \(z_{\alpha}\) not \(z_{\alpha/2}\))

Upper Confidence Bound Calculation:

\[\mu < \overline{x} + z_{\alpha} \frac{\sigma}{\sqrt{n}} =\]

Your interpretation:

Part 4: Confidence Intervals When σ is Unknown

Up to now, we have assumed that the population standard deviation \(\sigma\) is known. In many real-world situations, however, \(\sigma\) is not known in advance and must be estimated from the sample data.

The Challenge of Estimating σ 🎯

Estimating \(\sigma\) introduces an additional layer of uncertainty, because our estimator \(s\) (the sample standard deviation) varies from sample to sample.

This increased uncertainty changes the distribution of our pivotal statistic!

Previously, we relied on the standard Normal distribution (Z-distribution). Now, our pivotal quantity takes the form:

\[T = \frac{\overline{X} - \mu}{s/\sqrt{n}}\]

which follows Student’s t-distribution rather than the standard Normal distribution.

Properties of the t-Distribution 📐

The t-distribution is:

Similar in shape to the Normal distribution but has heavier tails
Parameterized by degrees of freedom (df)
For inference about a mean: \(\text{df} = n - 1\)
Approaches the standard Normal as \(n \to \infty\)

Degrees of freedom represent how many independent pieces of information remain in the sample data after estimating parameters. Smaller degrees of freedom lead to thicker tails and wider confidence intervals.

Formulas When σ is Unknown 📋

Confidence Interval for μ:

\[\overline{x} \pm t_{\alpha/2, n-1} \frac{s}{\sqrt{n}}\]

One-Sided Bounds:

Upper bound: \(\mu < \overline{x} + t_{\alpha, n-1} \frac{s}{\sqrt{n}}\)
Lower bound: \(\mu > \overline{x} - t_{\alpha, n-1} \frac{s}{\sqrt{n}}\)

Approximate Sample Size (using pilot estimate of σ):

\[n \approx \left\lceil \left(\frac{t_{\alpha/2, n'-1} \cdot s'}{M}\right)^2 \right\rceil\]

where \(s'\) is from a pilot study with \(n'\) observations

Note

In R, critical values are obtained using the qt function. Usage is similar to qnorm but requires specifying degrees of freedom:

qt(alpha/2, df = n - 1, lower.tail = FALSE) for two-sided intervals
qt(alpha, df = n - 1, lower.tail = FALSE) for one-sided bounds

Using lower.tail = FALSE is equivalent to 1 - alpha but is clearer and more explicit about finding upper tail probabilities.

Question 4: A software company has recently upgraded its cloud servers to improve the performance of its analytics application. Specifically, the company hopes to reduce the average computation time required to run complex statistical analyses.

The company does not yet know the new average computation time \(\mu\), nor the variability \(\sigma^2\) of computation times after the upgrade. To better understand the variability in performance, the engineering team first conducts a pilot study.

From this pilot study of \(n' = 20\) independent analyses, they observe a sample standard deviation of \(s' = 4.5\) seconds.

a) Sample Size Determination with Unknown σ

The company wishes to collect enough data to estimate the new mean computation time with a margin of error of no more than 1 second at a 95% confidence level. Derive the approximate minimum sample size required to obtain this level of precision.

Given information: - Pilot study: \(n' = 20\), \(s' = 4.5\) seconds - Desired margin of error: \(M = 1\) second - Confidence level: 95% - :math:`text{df} = n’ - 1 = ` ____ - :math:`t_{alpha/2, n’-1} = ` ____

Sample Size Calculation:

\[n \approx \left\lceil \left(\frac{t_{\alpha/2, n'-1} \cdot s'}{M}\right)^2 \right\rceil =\]

b) Constructing the Confidence Interval

After collecting the required number of observations determined above, the company finds: - Average computation time: \(\overline{x} = 28.7\) seconds - Sample standard deviation: \(s = 4.8\) seconds

Using the obtained statistics, construct a 95% confidence interval for the true mean computation time.

Confidence Interval Calculation:

\[\text{CI} = \overline{x} \pm t_{\alpha/2, n-1} \frac{s}{\sqrt{n}} =\]

Your 95% confidence interval:

Comparison: Z vs t Distribution 🔍

Explore how t-distribution compares to Normal distribution

library(ggplot2)

# Create data for comparison
x <- seq(-4, 4, length.out = 200)

df_values <- c(2, 5, 10, 30)
comparison_df <- data.frame(
  x = rep(x, length(df_values) + 1),
  density = c(dnorm(x),
             sapply(df_values, function(df) dt(x, df = df))),
  distribution = rep(c("Normal (Z)",
                      paste("t, df =", df_values)),
                    each = length(x))
)

ggplot(comparison_df, aes(x = x, y = density, color = distribution)) +
  geom_line(size = 1.2) +
  labs(title = "Comparison of Normal and t Distributions",
       subtitle = "Notice heavier tails for smaller df (more uncertainty when estimating σ)",
       x = "Value", y = "Density", color = "Distribution") +
  theme_minimal() +
  theme(legend.position = "right")

# Compare critical values
confidence_level <- 0.95
alpha <- 1 - confidence_level

cat("\nCritical values for 95% confidence:\n")
cat("Z-distribution:", qnorm(alpha/2, lower.tail = FALSE), "\n")
for (df in c(5, 10, 20, 30, 50)) {
  cat("t-distribution (df =", df, "):",
      qt(alpha/2, df = df, lower.tail = FALSE), "\n")
}

Key Takeaways

Summary 📝

Confidence intervals quantify uncertainty in parameter estimates by providing a range of plausible values
Pivotal quantities with known distributions enable us to construct confidence intervals through probability statements
Confidence level interpretation: In repeated sampling, \((1-\alpha)100\%\) of intervals constructed this way will contain the true parameter
Sample size formulas ensure we collect enough data to achieve desired precision (margin of error)
One-sided bounds are appropriate when interest lies in only an upper or lower limit
The t-distribution accounts for additional uncertainty when \(\sigma\) is unknown and must be estimated from sample data
Degrees of freedom in the t-distribution reflect the amount of information available after estimating parameters
R simulations help build intuition about coverage properties and the meaning of confidence levels