11.3. Comparing the Means of Two Independent Populations - Pooled Variance Estimator

In practice, we typically do not know the population standard deviation for one population, let alone two populations whose means we want to compare. The theoretical framework developed in the previous section, while mathematically elegant, requires the unrealistic assumption of known population variances. This section addresses the more realistic scenario where population standard deviations must be estimated from sample data, beginning with the case where we can reasonably assume both populations have equal variances.

Road Map 🧭

• Problem we will solve – How to compare two population means when standard deviations are unknown

but can be assumed equal across populations • Tools we’ll learn – Pooled variance estimation and t-procedures for independent samples under equal variance assumptions • How it fits – This bridges the gap between theoretical z-procedures and practical applications, introducing the complications that arise when parameters must be estimated

11.3.1. The Transition from Known to Unknown Variances

The assumption of known population standard deviations \(\sigma_A\) and \(\sigma_B\) served as a useful theoretical starting point, but practical statistical analysis requires methods that accommodate unknown parameters. When population variances must be estimated from sample data, we face several important changes:

Increased Uncertainty

Estimating variances from sample data introduces additional uncertainty beyond the sampling variability of the means themselves. This extra uncertainty must be properly accounted for in our inferential procedures.

Distributional Changes

Test statistics no longer follow standard normal distributions but instead follow t-distributions, requiring different critical values and probability calculations.

Estimation Strategy Considerations

When both populations have unknown variances, we must decide whether to estimate them separately or under some unifying assumption. This decision significantly affects both the complexity and performance of our procedures.

11.3.2. The Equal Variance Assumption: Mathematical Framework

The Fundamental Assumption

We now assume that while the population means \(\mu_A\) and \(\mu_B\) may differ (which is typically what we want to test), the population variances are equal:

\[\sigma^2_A = \sigma^2_B = \sigma^2\]

This assumption states that both populations have the same underlying variability, differing only in their central tendencies. While the common variance \(\sigma^2\) remains unknown, assuming equality allows us to pool information from both samples to estimate this single parameter.

Implications for Standard Error

Under the equal variance assumption, the standard error of our point estimator \(\bar{X}_A - \bar{X}_B\) simplifies from:

\[\sqrt{\frac{\sigma^2_A}{n_A} + \frac{\sigma^2_B}{n_B}}\]

to:

\[\sigma\sqrt{\frac{1}{n_A} + \frac{1}{n_B}}\]

This simplification occurs because \(\sigma^2_A = \sigma^2_B = \sigma^2\), allowing us to factor out the common variance.

11.3.3. The Pooled Variance Estimator: Construction and Properties

The Pooling Rationale

When both sample variances \(S^2_A\) and \(S^2_B\) are estimating the same underlying parameter \(\sigma^2\), we should not arbitrarily choose one and ignore the other. Instead, we can combine the information from both samples to create a more precise estimator of the common variance.

However, we cannot simply combine all observations into a single dataset because the populations have different means. The sample variance formula depends on deviations from the sample mean, and pooling observations with different means would distort our variance estimate.

The Weighted Average Approach

The pooled variance estimator combines the individual sample variances using a weighted average that accounts for the amount of information contributed by each sample:

\[S^2_p = \frac{(n_A - 1)S^2_A + (n_B - 1)S^2_B}{n_A + n_B - 2}\]

Understanding the Weights

The weights \((n_A - 1)\) and \((n_B - 1)\) represent the degrees of freedom associated with each sample variance. These weights ensure that:

1. Larger samples contribute more: If \(n_A > n_B\), then sample A receives higher weight in the pooled estimate

2. Proportional representation: The contribution of each sample is proportional to the precision of its variance estimate

3. Proper normalization: The weights sum to \(n_A + n_B - 2\), which becomes the denominator

Degrees of Freedom Interpretation

The denominator \(n_A + n_B - 2\) represents the total degrees of freedom for estimating the common variance:

• Total observations: \(n_A + n_B\)

• Parameters estimated: 2 (the means \(\mu_A\) and \(\mu_B\))

• Resulting degrees of freedom: \(n_A + n_B - 2\)

Unbiasedness of the Pooled Estimator

The pooled variance estimator is unbiased for the common variance \(\sigma^2\) when the equal variance assumption holds. To establish this:

\[E[S^2_p] = E\left[\frac{(n_A - 1)S^2_A + (n_B - 1)S^2_B}{n_A + n_B - 2}\right]\]

Since the weights are constants, expectation distributes:

\[= \frac{(n_A - 1)E[S^2_A] + (n_B - 1)E[S^2_B]}{n_A + n_B - 2}\]

From single-sample theory, we know that sample variances are unbiased: \(E[S^2_A] = \sigma^2_A\) and \(E[S^2_B] = \sigma^2_B\). Under our equal variance assumption, both equal \(\sigma^2\):

\[= \frac{(n_A - 1)\sigma^2 + (n_B - 1)\sigma^2}{n_A + n_B - 2} = \frac{\sigma^2[(n_A - 1) + (n_B - 1)]}{n_A + n_B - 2} = \frac{\sigma^2(n_A + n_B - 2)}{n_A + n_B - 2} = \sigma^2\]

Therefore, \(E[S^2_p] = \sigma^2\), confirming unbiasedness.

11.3.4. Standard Error Estimation with Pooled Variance

The Estimated Standard Error

Using the pooled variance estimator, we estimate the standard error of \(\bar{X}_A - \bar{X}_B\) as:

\[SE_p = S_p\sqrt{\frac{1}{n_A} + \frac{1}{n_B}}\]

where \(S_p = \sqrt{S^2_p}\) is the pooled standard deviation.

Comparison to the Known Variance Case

This estimated standard error replaces the theoretical standard error \(\sigma\sqrt{\frac{1}{n_A} + \frac{1}{n_B}}\) from the known variance case. The substitution of \(S_p\) for \(\sigma\) introduces the additional uncertainty that necessitates using t-distributions instead of the standard normal distribution.

Information Pooling Benefits

The pooled approach utilizes information from \(n_A + n_B - 2\) degrees of freedom to estimate the standard error, compared to \(n_A - 1\) or \(n_B - 1\) degrees of freedom if we used only one sample’s variance. This typically results in more precise estimation and improved statistical power.

11.3.5. Test Statistics and Distributional Theory

The t-Test Statistic

When using the pooled variance estimator, our test statistic becomes:

\[T_{TS} = \frac{(\bar{X}_A - \bar{X}_B) - \Delta_0}{S_p\sqrt{\frac{1}{n_A} + \frac{1}{n_B}}}\]

where \(\Delta_0\) is the hypothesized difference under the null hypothesis (typically 0).

Distributional Properties

Under the assumptions of:

1. Independent simple random sampling from each population

2. Normal distributions (or large enough samples for CLT)

3. Equal population variances: \(\sigma^2_A = \sigma^2_B = \sigma^2\)

4. The null hypothesis is true

the test statistic follows a t-distribution with \(df = n_A + n_B - 2\) degrees of freedom:

\[T_{TS} \sim t_{n_A + n_B - 2}\]

Why t-Distribution?

The t-distribution arises because we are standardizing with an estimated standard error rather than the true standard error. The additional uncertainty from estimating \(\sigma^2\) with \(S^2_p\) manifests as the heavier tails characteristic of t-distributions.

11.3.6. Confidence Intervals Using Pooled Procedures

The Pivotal Quantity

For confidence interval construction, we use the pivotal quantity:

\[T = \frac{(\bar{X}_A - \bar{X}_B) - (\mu_A - \mu_B)}{S_p\sqrt{\frac{1}{n_A} + \frac{1}{n_B}}}\]

This quantity follows a t-distribution with \(n_A + n_B - 2\) degrees of freedom and contains only one unknown parameter: \(\mu_A - \mu_B\).

Confidence Interval Formula

A \(100(1-\alpha)\%\) confidence interval for \(\mu_A - \mu_B\) is:

\[(\bar{x}_A - \bar{x}_B) \pm t_{\alpha/2, n_A + n_B - 2} \cdot S_p\sqrt{\frac{1}{n_A} + \frac{1}{n_B}}\]

Components Analysis

• Point estimator: \(\bar{x}_A - \bar{x}_B\)

• Critical value: \(t_{\alpha/2, n_A + n_B - 2}\) from the t-distribution

• Standard error: \(S_p\sqrt{\frac{1}{n_A} + \frac{1}{n_B}}\)

• Margin of error: The product of critical value and standard error

11.3.7. When to Use Pooled Procedures: Practical Considerations

Appropriate Scenarios

Pooled variance procedures are most appropriate when:

1. Equal variance assumption is reasonable: The populations have similar variability patterns

2. Process similarity: The data-generating mechanisms are similar except for location shifts

3. Small sample sizes: The efficiency gains from pooling are most pronounced with limited data

4. Experimental control: In designed experiments where conditions are controlled except for the treatment

Examples of Suitable Applications

• Manufacturing quality control: Comparing products from the same process under different settings

• Agricultural experiments: Testing fertilizer effects on crop yields using similar plots

• Medical trials: Comparing treatments when patient populations are homogeneous except for treatment assignment

Efficiency Considerations

When the equal variance assumption holds, pooled procedures offer several advantages:

1. Higher degrees of freedom: \(n_A + n_B - 2\) instead of separate estimations

2. More precise standard error estimates: Utilizing all available information

3. Increased statistical power: Better ability to detect true differences when they exist

Robustness Issues

The pooled procedure’s performance depends critically on the validity of the equal variance assumption. When variances are substantially unequal, pooled procedures can:

• Inflate Type I error rates: Lead to more false positives than the nominal \(\alpha\) level

• Reduce power: Decreased ability to detect true differences

• Produce misleading confidence intervals: Intervals that don’t achieve the stated coverage probability

11.3.8. The Alternative: Non-Pooled Procedures

When Equal Variance Assumption Fails

If we cannot reasonably assume equal variances, we must estimate \(\sigma^2_A\) and \(\sigma^2_B\) separately:

\[SE = \sqrt{\frac{S^2_A}{n_A} + \frac{S^2_B}{n_B}}\]

This approach avoids the equal variance assumption but introduces complications in determining appropriate degrees of freedom and critical values.

Preview of Coming Complications

The non-pooled approach leads to test statistics that don’t follow standard t-distributions, requiring approximation methods for degrees of freedom calculation. These procedures, while more flexible in their assumptions, are also more complex in their implementation and interpretation.

The Trade-off

The choice between pooled and non-pooled procedures represents a fundamental trade-off in statistical practice:

• Pooled procedures: More efficient when assumptions hold, but potentially misleading when they don’t

• Non-pooled procedures: More robust to assumption violations, but less efficient when pooling would be appropriate

11.3.9. Course Perspective on Pooled vs. Non-Pooled Methods

Practical Recommendation

As noted in the transcript, this course emphasizes non-pooled procedures for several practical reasons:

1. Assumption-free approach: Avoids the need to verify equal variance assumptions

2. General applicability: Works regardless of whether variances are equal or unequal

3. Conservative nature: Provides valid inference even when equal variance assumption fails

4. Modern statistical practice: Reflects current preferences in applied statistics

When Pooled Procedures Remain Valuable

Despite the general preference for non-pooled methods, pooled procedures remain important in specific contexts:

• Designed experiments: Where experimental control ensures similar variances

• Theoretical understanding: For comprehending the relationship between assumptions and procedures

• Historical context: For understanding classical statistical approaches

• Specific applications: Where domain knowledge strongly supports equal variance assumptions

Educational Value

Understanding both approaches provides insight into:

• The impact of assumptions: How different assumptions lead to different procedures

• Efficiency considerations: The statistical costs and benefits of making assumptions

• Methodological evolution: How statistical practice has developed over time

Key Takeaways 📝

1. Pooled variance procedures assume equal population variances (\(\sigma^2_A = \sigma^2_B = \sigma^2\)) while allowing means to differ.

2. The pooled variance estimator \(S^2_p = \frac{(n_A - 1)S^2_A + (n_B - 1)S^2_B}{n_A + n_B - 2}\) combines information from both samples using weights proportional to degrees of freedom.

3. Test statistics follow t-distributions with \(df = n_A + n_B - 2\) degrees of freedom when the equal variance assumption holds.

4. Pooled procedures offer efficiency gains when assumptions are satisfied, providing more precise estimates and higher statistical power.

5. The equal variance assumption is critical – violations can lead to incorrect Type I error rates and misleading confidence intervals.

6. Confidence intervals use the familiar format with t-critical values: \((\bar{x}_A - \bar{x}_B) \pm t_{\alpha/2, df} \cdot SE_p\).

7. Modern practice often favors non-pooled procedures due to their robustness to assumption violations, despite potential efficiency losses.

8. The choice between pooled and non-pooled approaches represents a fundamental trade-off between efficiency (when assumptions hold) and robustness (when they don’t).

Exercises

1. Pooled Variance Calculation: Two independent samples yield the following results:

   • Sample A: \(n_A = 12\), \(S^2_A = 25.6\)

   • Sample B: \(n_B = 15\), \(S^2_B = 31.2\)

   1. Calculate the pooled variance estimator \(S^2_p\)

   2. Find the pooled standard deviation \(S_p\)

   3. Determine the degrees of freedom for the pooled procedure

   4. Calculate the standard error \(SE_p\)

2. Assumption Analysis: For each scenario, discuss whether the equal variance assumption seems reasonable and justify your answer:

   1. Comparing customer satisfaction scores (1-10 scale) between two similar retail stores

   2. Comparing income levels between urban and rural populations

   3. Comparing reaction times before and after consuming caffeine (using different subjects)

   4. Comparing crop yields between two fertilizer treatments on similar plots

3. Degrees of Freedom Impact: Explain why the pooled procedure uses \(df = n_A + n_B - 2\) degrees of freedom instead of \((n_A - 1) + (n_B - 1)\). What parameters are being estimated, and how does this affect the degrees of freedom calculation?

4. Efficiency Comparison: Two researchers study the same phenomenon. Researcher A uses sample sizes \(n_A = n_B = 10\), while Researcher B uses \(n_A = 15, n_B = 5\) (same total sample size). Assuming equal population variances:

   1. Which design will have a smaller standard error? Explain why.

   2. How do the degrees of freedom compare between the two designs?

   3. What practical considerations might favor one design over the other?

5. Unbiasedness Verification: Suppose \(\sigma^2_A = 20\) and \(\sigma^2_B = 20\) (equal variances). If \(E[S^2_A] = 20\) and \(E[S^2_B] = 20\), verify algebraically that the pooled variance estimator with \(n_A = 8\) and \(n_B = 12\) is unbiased.

6. Critical Thinking: A colleague claims, “We should always use pooled procedures because they’re more efficient.” Provide a balanced response discussing both the benefits and potential risks of this approach. What factors should guide the choice between pooled and non-pooled procedures?