Slides

Download Chapter 8 slides (PPTX)

8.3. Basic Types of Experimental Design

Experimental designs provide structured templates for arranging experimental units, assigning treatments, and collecting data in ways that uphold the three principles of experimental design while addressing specific research challenges.

The three designs we will examine are among the most fundamental. Although more complex designs exist for specialized situations, these basic frameworks can address a wide range of research questions and form the foundation for understanding more advanced approaches.

Road Map 🧭

• Understand the structure of the three most fundamental types experimental design: Completely Randomized Design, Randomized Block Design, and Matched Pairs Design.

• Identify the advantages and limitations of each design, and select the most appropriate type for a given research setting.

8.3.1. Completely Randomized Design (CRD)

The Completely Randomized Design represents the most straightforward application of experimental principles. In a completely randomized design, each unit has an equal chance of being randomly assigned to a treatment group.

Implementation

Fig. 8.4 Completely Randomized Design

Let’s walk through implementing a CRD with a concrete example. Suppose we want to test the effects of four different medications on blood pressure of 40 healthy adults: one control (placebo) and three treatment groups with different dosage levels.

• Step 1: Identify experimental units and determine the treatment groups. In the current context, the experimental units would be the 40 human participants. The treatment groups are:

  • Group 1: Control (placebo)

  • Group 2: Low dose medication

  • Group 3: Medium dose medication

  • Group 4: High dose medication

• Step 2: Randomly assign each participant to one of the four groups. We can use a simple procedure like rolling a four-sided die or use computer-generated random numbers.

• Step 3: Apply Treatments.

• Step 4: Measure Responses. Record blood pressure measurements according to a consistent protocol for all groups.

• Step 5: Compare Results.

Advantages and Limitations of CRD

Advantages	Limitations
Works best when experimental units are relatively homogeneous or when we have enough units that randomization alone will likely create balanced groups. The most straightforward design to understand and implement.	If important characteristics vary substantially among experimental units, it may not distribute these characteristics evenly across groups, potentially confounding results.

8.3.2. Randomized Block Design (RBD): Controlling Known Variability

When we know that certain characteristics of our experimental units strongly influence the response variable, Randomized Block Design provides a method for controlling this variability while still maintaining the benefits of randomization.

Recall that blocking is a techinque of grouping experimental units into subgroups (blocks) based on shared characteristics. By blocking, we remove the effect of the extraneous variable from our comparison of treatments. By ensuring each treatment appears in each block, we can separate treatment effects from block effects.

Implementation

Fig. 8.5 Randomized Block Design

• Step 1: Identify the blocking variable. Choose a characteristic that is not the primary focus of the study but strongly influences the response variable.

• Step 2: Form blocks. Group experimental units into blocks based on the blocking variable. Each block should contain units that are similar with respect to this characteristic.

• Step 3: Randomize within blocks. This is essentially conducting a separate completely randomized design within each block.

• Step 4: Apply treatments and measure responses.

• Step 5: Analyze results. Compare treatments while accounting for block effects.

Example 💡: Testing Pain Medications 💊

Consider testing a new pain medication. If it is known that age significantly affects both baseline pain levels and response to medication, we can choose to create blocks based on age groups.

• Block 1: Ages 18-30 (young adults)

• Block 2: Ages 31-50 (middle-aged adults)

• Block 3: Ages 51-70 (older adults)

Within each age block, randomly assign participants to receive either the new medication or a placebo. This ensures that each treatment is tested across all age groups, while controlling for the known effect of age on pain response.

Advantages and Limitations of RBD

Advantages	Limitations
Detects treatment effects more easily by controlling for known sources of variability.	Requires thorough planning. Because the number of experimental units is limited, blocks cannot be created for all possible extraneous variables. Resources should be allocated strategically to control the few most important ones.

8.3.3. Matched Pairs Design: Maximum Control

Matched Pairs Design represents the extreme end of the blocking spectrum, where we create the smallest possible blocks—typically containing just two experimental units that are nearly identical with respect to important characteristics.

Type 1: Separate Units in Pairs

In this approach, we identify pairs of experimental units that are extremely similar and randomly assign different treatments to the units in each pair.

This design works when we can identify experimental units that are nearly identical with respect to characteristics that strongly influence the response.

Example 💡: Examples of Natural Pairs

• Identical twins in medical or behavioral research

• Litter mates in animal studies

• Adjacent plots in agricultural research

• Matched patients with very similar demographic and clinical characteristics

• Paired manufacturing units from the same production batch

Implementation of Type 1 Matched Pairs Design

Fig. 8.6 Type 1 Matched Pairs Design

• Step 1: Identify matching criteria based on variables that strongly affect the response.

• Step 2: Form pairs of units that are nearly identical on these criteria.

• Step 3: Randomly assign one member of each pair to one treatment, the other to another.

• Step 4: Apply treatments and measure responses.

• Step 5: Analyze pair differences.

Advantages and Limitations of Type 1 Matched Pairs Design

Advantages	Limitations
Maximum control over confounding variables	Difficult to find suitable pairs in many research contexts

Type 2: Self-Pairing

In self-pairing designs, each experimental unit serves as its own comparison by receiving two treatments in a randomized order, typically separated by a washout period.

Implementation of Type 2 Matched Pairs Design

Fig. 8.7 Type 2 Matched Pairs Design

Suppose a researcher wants to compare two pain medications.

• Step 1: Randomly assign either Medication A or B to each subject.

• Step 2: Each unit takes the first medication for two weeks with daily pain ratings.

• Step 3: Each unit enters a washout period of one week with no medication.

• Step 4: The second medication is taken for two weeks with daily pain ratings.

• Step 5: Compare results within each person.

Preventing the Carry-Over Effect

When the first treatment’s effects persist and influence the results of a subsequent treatment, this is called the carry-over effect. To minimize it:

• Allow a washout period—enough time for the first treatment’s effects to fully dissipate before the next treatment.

• Randomize treatment order so that not all units experience the same type of carry-over effects.

• Some treatments have permanent or very long-lasting effects that make self-pairing inappropriate. For example,

  • Surgical procedures cannot be “undone”.

  • Educational interventions create lasting learning.

  • Some medications have effects that persist long after discontinuation.

  • Behavioral interventions might create permanent attitude or skill changes.

  In these cases, Type 1 matched pairs is the only viable option.

Advantages and Limitaions of Type 2 Matched Pairs Design

Advantages	Limitations
Perfect matching eliminates individual differences High statistical power with relatively few subjects	Carry-over effects can confound results if treatments have lasting impacts. Time effects might influence responses differently in the two treatment periods. Participant dropout is more problematic since each participant contributes to both treatments.

8.3.4. Choosing the Right Approach

Selecting the appropriate experimental design requires balancing statistical considerations, practical constraints, and research objectives. Each design offers different advantages and faces different limitations.

Guidelines for Design Choice
Choose CRD when:	Experimental units are relatively homogeneouse Sample size is reasonably large No obvious blocking variables suggest themselves Simplicity is important for implementation or communication This is exploratory research to identify important factors
Choose RBD when:	A strong blocking variable is known and measurable Sample size is moderate (blocking can reduce required sample size) You want to control for known sources of variability Different subgroups (blocks) are important for generalization Cost or logistics make larger sample sizes difficult
Choose Matched Pairs Design when:	Individual differences are expected to be very large relative to treatment effects Sample size is necessarily small (matched pairs maximize power with few units) Natural pairs exist (twins, litter mates, adjacent plots) For crossover design: treatments are reversible and washout is feasible Maximum control is essential for credible causal inference

8.3.5. Bringing It All Together

Key Takeaways 📝

1. Three basic designs serve as building blocks (Completely Randomized Design (CRD), Randomized Block Design (RBD), and Matched Pairs Design) provide the foundation for most experimental approaches.

2. Design choice depends on context. The appropriate design depends on the homogeneity of experimental units, known sources of variability, sample size constraints, and research objectives.

3. CRD provides simplicity and flexibility but works best with homogeneous units and adequate sample sizes.

4. RBD controls known variability through blocking, increasing precision and efficiency when strong blocking variables exist.

5. Matched pairs maximize control but require either natural pairs or reversible treatments with adequate washout periods.

Exercises

1. Design Selection: For each research scenario, identify which of the three basic designs would be most appropriate and explain your reasoning:

   1. Testing whether a new fertilizer increases crop yield using 60 plots of land that vary substantially in soil quality.

   2. Comparing the effectiveness of three different teaching methods using 90 students of similar academic background.

   3. Evaluating whether a new sleep medication reduces insomnia using 40 patients with chronic sleep disorders.

   4. Testing whether background music affects concentration using 24 volunteers.

2. Blocking Variables: A researcher wants to test whether a new exercise program reduces blood pressure. Identify three potential blocking variables and explain why each might be important to control.

3. Matched Pairs Feasibility: For each scenario, determine whether Type 1 (separate pairs) or Type 2 (self-pairing) matched pairs design would be more appropriate, or if matched pairs design is not feasible:

   1. Testing two different surgical techniques for knee repair

   2. Comparing two methods for memorizing vocabulary words

   3. Evaluating two different crop varieties for yield

   4. Testing whether meditation reduces anxiety levels

4. Design from Scratch: Design a complete experiment to test whether background music affects concentration during studying using one of the three templates introduced in this section. Clearly identify all the components of an experiment and explain how the design addresses all three principles of experimental design.