8.3. Basic Types of Experimental Design
Understanding the three fundamental principles of experimental design provides the foundation, but translating these principles into practice requires concrete frameworks for organizing and implementing studies. Experimental designs are essentially templates that show how to arrange experimental units, assign treatments, and collect data in ways that satisfy our design principles while addressing specific research challenges.
The three designs we’ll explore represent the fundamental building blocks of experimental methodology. While more complex designs exist for specialized research situations, these basic frameworks can handle a wide range of research questions and serve as the foundation for understanding more advanced approaches.
Road Map 🧭
Problem: How do we organize subjects and treatments to implement good experimental principles in practice?
Tool: Standard experimental design templates that handle different research scenarios and constraints
Pipeline: Proper design choice affects both data collection procedures and statistical analysis methods
8.3.1. The Building Blocks of Experimental Design
Before examining specific designs, it’s important to understand that we’re focusing on the most fundamental approaches that serve as building blocks for more complex experimental structures. In advanced research settings, experimenters often combine or modify these basic designs to create sophisticated frameworks that address multiple research questions simultaneously or handle complex practical constraints.
Why Start with Basic Designs?
These three designs—Completely Randomized Design (CRD), Randomized Block Design (RBD), and Matched Pairs Design—form the conceptual foundation for experimental methodology because they each address different fundamental challenges:
CRD demonstrates pure randomization when no special constraints exist
RBD shows how to handle known sources of variability through blocking
Matched Pairs illustrates maximum control when variability is extremely high
Understanding these basic approaches provides the framework for recognizing when more complex designs might be needed and how they build upon these fundamental principles.
The Spectrum of Design Complexity
In practice, researchers might encounter situations requiring more sophisticated approaches such as:
Factorial designs that study multiple factors simultaneously and their interactions
Split-plot designs for situations where different factors are applied at different scales
Crossover designs for longitudinal studies with multiple treatment periods
Adaptive designs that allow modifications based on interim results
Cluster randomized trials for interventions applied to groups rather than individuals
However, these advanced designs all incorporate the same fundamental principles we’ve discussed, and many can be understood as extensions or combinations of the basic designs we’ll cover.
8.3.2. Completely Randomized Design (CRD): The Foundation
The Completely Randomized Design represents the most straightforward application of experimental principles. In CRD, we assume we have access to our experimental units and need to determine how to allocate them to treatment groups in the fairest, most unbiased way possible.
The Basic Structure
In a completely randomized design:
All experimental units are pooled together without regard to their characteristics
Random assignment distributes units among all treatment groups using chance alone
No special grouping or matching occurs before randomization
Each unit has an equal probability of being assigned to any treatment group
This approach works best when experimental units are relatively homogeneous or when we have enough units that randomization alone will likely create balanced groups.
Fig. 8.2 Completely Randomized Design: Simple random allocation to treatment groups
Implementation Process
Let’s walk through implementing a CRD with a concrete example. Suppose we want to test the effects of different medications on blood pressure, with four groups: one control (placebo) and three treatment groups with different dosage levels.
Step 1: Identify Experimental Units
We start with our sample of participants who have been recruited for the study. These participants represent our experimental units.
Step 2: Determine Treatment Groups
Group 1: Control (placebo)
Group 2: Low dose medication
Group 3: Medium dose medication
Group 4: High dose medication
Step 3: Random Assignment
Each participant gets randomly assigned to one of the four groups. We might use a simple procedure like drawing names from a hat and rolling a four-sided die, or use computer-generated random numbers.
Step 4: Apply Treatments
Each group receives their assigned treatment over the study period (perhaps several months).
Step 5: Measure Responses
We record blood pressure measurements according to our protocol, ensuring consistent measurement procedures across all groups.
Step 6: Compare Results
We analyze differences between groups, comparing each treatment group to the control and potentially to each other.
Advantages of CRD
Simplicity: CRD is the most straightforward design to understand and implement. There are no complex grouping procedures or matching requirements.
Flexibility: It works with any number of treatments and can easily accommodate unequal sample sizes if needed.
Wide Applicability: CRD is appropriate for many research situations, particularly when experimental units are reasonably similar.
Statistical Analysis: The analysis is relatively straightforward, using standard statistical methods for comparing means across groups.
Considerations and Limitations
Group Size Balance: Simple randomization might produce unequal group sizes by chance. While this doesn’t invalidate the design, balanced groups generally provide better statistical power.
Hidden Confounders: If important characteristics vary substantially among experimental units, randomization might not distribute these characteristics evenly across groups, potentially confounding results.
Efficiency: CRD might not be the most efficient design if we know about important sources of variability that could be controlled through blocking.
When to Use CRD
Completely randomized design works well when:
Experimental units are relatively homogeneous
We have a reasonably large sample size
No obvious grouping variables suggest themselves
Simplicity is important for implementation or understanding
We’re conducting preliminary research to identify important factors
8.3.3. Randomized Block Design (RBD): Controlling Known Variability
When we know that certain characteristics of our experimental units strongly influence the response variable, Randomized Block Design provides a method for controlling this variability while still maintaining the benefits of randomization.
The Blocking Concept
Blocking means grouping experimental units into homogeneous subgroups (blocks) based on characteristics we believe will affect the response. Within each block, units should be as similar as possible with respect to the blocking variable, while units in different blocks differ systematically on this characteristic.
The key insight is that we want to remove the effect of the blocking variable from our comparison of treatments. By ensuring each treatment appears in each block, we can separate treatment effects from block effects.
Implementation Process
Step 1: Identify the Blocking Variable
Choose a characteristic that:
Strongly influences the response variable
Can be measured before treatment assignment
Creates meaningful, distinct groups
Is not the primary focus of the study
Step 2: Form Blocks
Group experimental units into blocks based on the blocking variable. Each block should contain units that are similar with respect to this characteristic.
Step 3: Randomize Within Blocks
Within each block, randomly assign units to treatment groups. This is essentially conducting a separate completely randomized design within each block.
Step 4: Apply Treatments and Measure Responses
Proceed with the experiment, ensuring consistent procedures across blocks.
Step 5: Analyze Results
Compare treatments while accounting for block effects, enabling us to see treatment differences more clearly.
A Medical Example
Consider testing a new pain medication where we know that age significantly affects both baseline pain levels and response to medication. We might create blocks based on age groups:
Block 1: Ages 18-30 (young adults)
Block 2: Ages 31-50 (middle-aged adults)
Block 3: Ages 51-70 (older adults)
Within each age block, we randomly assign participants to receive either the new medication or a placebo. This ensures that each treatment is tested across all age groups, while controlling for the known effect of age on pain response.
Advantages of RBD
Increased Precision: By controlling for known sources of variability, we can detect treatment effects more easily.
Efficiency: RBD often requires fewer experimental units than CRD to achieve the same statistical power.
Additional Information: We learn about both treatment effects and the effect of the blocking variable.
Broader Applicability: Results can be generalized across the different levels of the blocking variable.
When Blocking is Most Beneficial
Blocking provides the greatest advantage when:
The blocking variable has a strong effect on the response
We have limited experimental units available
The cost of additional units is high
We want to ensure treatment comparisons across important subgroups
Designing Effective Blocks
Homogeneity Within Blocks: Units within each block should be as similar as possible with respect to characteristics that affect the response.
Sufficient Block Size: Each block should contain enough units to accommodate all treatments, preferably with some replication within blocks.
Meaningful Block Differences: Blocks should represent genuinely different conditions or characteristics, not arbitrary divisions.
Block Formation Before Randomization: Blocks must be formed based on pre-existing characteristics, not on responses or treatment assignments.
8.3.4. Matched Pairs Design: Maximum Control
Matched Pairs Design represents the extreme end of the blocking spectrum, where we create the smallest possible blocks—typically containing just two experimental units that are nearly identical with respect to important characteristics.
Two Types of Matched Pairs Design
Type 1: Separate Units in Pairs
In this approach, we identify pairs of experimental units that are extremely similar and randomly assign one member of each pair to treatment and the other to control.
Type 2: Self-Pairing (Crossover Design)
Here, each experimental unit serves as its own control by receiving both treatments in a randomized order, typically separated by a washout period.
Type 1 Matched Pairs: Separate Unit Pairs
This design works when we can identify experimental units that are nearly identical with respect to characteristics that strongly influence the response.
Examples of Natural Pairs:
Identical twins in medical or behavioral research
Litter mates in animal studies
Adjacent plots in agricultural research
Matched patients with very similar demographic and clinical characteristics
Paired manufacturing units from the same production batch
Implementation Process:
Identify matching criteria based on variables that strongly affect the response
Form pairs of units that are nearly identical on these criteria
Randomly assign one member of each pair to treatment, the other to control
Apply treatments and measure responses
Analyze pair differences to assess treatment effects
Advantages of Type 1 Matched Pairs:
Maximum control over confounding variables
High statistical power for detecting treatment effects
Clear causal inference due to strong control
Limitations:
Difficult to find suitable pairs in many research contexts
Expensive and time-consuming to identify and recruit matched units
Limited generalizability if pairs represent a narrow subset of the population
Type 2 Matched Pairs: Self-Pairing (Crossover Design)
In self-pairing designs, each experimental unit receives both treatments, serving as its own control. This provides the ultimate in matching since each unit is perfectly matched with itself.
Implementation Considerations:
Randomized Treatment Order: Half the units receive Treatment A first, then Treatment B; the other half receive Treatment B first, then Treatment A. This randomization controls for order effects.
Washout Period: Between treatments, there must be sufficient time for the effects of the first treatment to dissipate completely. The length depends on the nature of the treatment and response.
Carry-over Effects: We must carefully consider whether the first treatment might permanently alter the unit’s response to the second treatment.
Example: Testing Pain Medications
A researcher wants to compare two pain medications. Each participant:
Randomly assigned to receive either Medication A or B first
Takes the first medication for two weeks with daily pain ratings
Enters washout period of one week with no medication
Takes the second medication for two weeks with daily pain ratings
Results compared within each person to see which medication was more effective
Advantages of Self-Pairing:
Perfect matching eliminates individual differences
High statistical power with relatively few subjects
Cost efficiency as each unit provides two data points
Critical Limitations:
Carry-over effects can confound results if treatments have lasting impacts
Time effects might influence responses differently in the two treatment periods
Participant dropout is more problematic since each participant contributes to both treatments
Limited to reversible treatments where effects can wash out completely
When Washout is Insufficient
Some treatments have permanent or very long-lasting effects that make self-pairing inappropriate:
Surgical procedures cannot be “undone”
Educational interventions create lasting learning
Some medications have effects that persist long after discontinuation
Behavioral interventions might create permanent attitude or skill changes
In these cases, Type 1 matched pairs (using separate but similar units) is the only viable option.
8.3.5. Design Selection: Choosing the Right Approach
Selecting the appropriate experimental design requires balancing statistical considerations, practical constraints, and research objectives. Each design offers different advantages and faces different limitations.
Decision Framework
Start with Research Context:
What is the primary research question?
What treatments are being compared?
What response variables will be measured?
What population do we want to generalize to?
Assess Experimental Units:
How homogeneous are the available units?
What characteristics might strongly influence the response?
Can these characteristics be measured before treatment assignment?
Are there natural groupings or pairs?
Consider Practical Constraints:
How many experimental units are available?
What are the costs and logistics of different designs?
Are there ethical considerations that limit design options?
What level of design complexity can be managed effectively?
Guidelines for Design Choice
Choose CRD when:
Experimental units are relatively homogeneous
Sample size is reasonably large
No obvious blocking variables suggest themselves
Simplicity is important for implementation or communication
This is exploratory research to identify important factors
Choose RBD when:
A strong blocking variable is known and measurable
Sample size is moderate (blocking can reduce required sample size)
You want to control for known sources of variability
Different subgroups (blocks) are important for generalization
Cost or logistics make larger sample sizes difficult
Choose Matched Pairs when:
Individual differences are expected to be very large relative to treatment effects
Sample size is necessarily small (matched pairs maximize power with few units)
Natural pairs exist (twins, litter mates, adjacent plots)
For crossover design: treatments are reversible and washout is feasible
Maximum control is essential for credible causal inference
Hybrid and Complex Designs
In practice, research situations often require combinations or modifications of these basic designs. Some common extensions include:
Factorial Designs: Study multiple factors simultaneously, such as testing different medications at different dosage levels across different age groups.
Split-Plot Designs: Apply some factors to large units (like fields) and other factors to smaller units within them (like individual plants).
Repeated Measures Designs: Measure the same units multiple times to study changes over time.
Cluster Randomized Designs: Randomize groups (like schools or clinics) rather than individuals when interventions are applied at the group level.
These advanced designs often incorporate elements of CRD, RBD, and matched pairs, demonstrating how these basic approaches serve as building blocks for more complex experimental structures.
8.3.6. Connecting Design to Analysis
The experimental design directly determines which statistical analysis methods are appropriate and how the data should be interpreted.
Design Affects Analysis:
CRD typically uses analysis of variance (ANOVA) or t-tests comparing group means
RBD requires methods that account for block effects, such as two-way ANOVA
Matched pairs often uses paired t-tests or repeated measures analysis
Design Affects Interpretation:
CRD results generalize to the broader population from which units were sampled
RBD results can be interpreted both overall and within specific blocks
Matched pairs results may have limited generalizability but strong internal validity
Design Affects Sample Size:
Different designs require different sample sizes to achieve the same statistical power, with matched pairs typically requiring the fewest units and CRD potentially requiring the most.
8.3.7. Beyond Basic Designs: A Glimpse Ahead
While these three designs provide the foundation for experimental methodology, real-world research often requires more sophisticated approaches. Advanced designs build upon these basic frameworks but address additional challenges:
Factorial Designs allow researchers to study multiple factors simultaneously and examine how factors interact with each other. For example, studying both medication type and dosage level in the same experiment.
Crossover Designs extend the matched pairs concept to situations with multiple treatments and time periods, carefully controlling for period effects and carry-over.
Adaptive Designs allow researchers to modify the study based on interim results, potentially stopping early for efficacy or futility, or adjusting sample sizes.
Cluster Randomized Trials handle situations where treatments must be applied to groups rather than individuals, such as educational interventions applied to entire classrooms.
Sequential Designs plan for multiple stages of experimentation, using early results to inform later stages.
These advanced approaches all rely on the fundamental principles we’ve discussed and often can be understood as sophisticated combinations of the basic designs covered in this chapter.
8.3.8. Bringing It All Together
Understanding the basic experimental designs provides the foundation for conducting rigorous research that can establish causal relationships with confidence. Each design represents a different balance between control, efficiency, and practical feasibility, and selecting the right approach is crucial for producing reliable, interpretable results.
Key Takeaways 📝
Three basic designs serve as building blocks: Completely Randomized Design (CRD), Randomized Block Design (RBD), and Matched Pairs Design provide the foundation for most experimental approaches.
Design choice depends on context: The appropriate design depends on the homogeneity of experimental units, known sources of variability, sample size constraints, and research objectives.
CRD provides simplicity and flexibility but works best with homogeneous units and adequate sample sizes.
RBD controls known variability through blocking, increasing precision and efficiency when strong blocking variables exist.
Matched pairs maximize control but require either natural pairs or reversible treatments with adequate washout periods.
Design determines analysis: The experimental design directly affects which statistical methods are appropriate and how results should be interpreted.
Advanced designs build on basic frameworks: More complex experimental designs typically combine or extend these fundamental approaches.
Practical constraints matter: Real-world factors like cost, ethics, and logistics often influence design choice as much as statistical considerations.
As we continue exploring experimental methodology, we’ll see how these design principles apply to addressing common problems and challenges that arise in real research settings. The goal is not just to understand these designs in theory, but to develop the judgment needed to choose and implement appropriate designs for specific research questions and constraints.
Exercises
Design Selection: For each research scenario, identify which of the three basic designs would be most appropriate and explain your reasoning:
Testing whether a new fertilizer increases crop yield using 60 plots of land that vary substantially in soil quality.
Comparing the effectiveness of three different teaching methods using 90 students of similar academic background.
Evaluating whether a new sleep medication reduces insomnia using 40 patients with chronic sleep disorders.
Testing whether background music affects concentration using 24 volunteers.
Blocking Variables: A researcher wants to test whether a new exercise program reduces blood pressure. Identify three potential blocking variables and explain why each might be important to control.
Matched Pairs Feasibility: For each scenario, determine whether Type 1 (separate pairs) or Type 2 (self-pairing) matched pairs design would be more appropriate, or if matched pairs design is not feasible:
Testing two different surgical techniques for knee repair
Comparing two methods for memorizing vocabulary words
Evaluating two different crop varieties for yield
Testing whether meditation reduces anxiety levels
Sample Size and Design: Explain why a randomized block design might require fewer experimental units than a completely randomized design to detect the same treatment effect.
Crossover Design Challenges: A pharmaceutical company wants to test a new antidepressant using a crossover design where each patient receives both the new drug and a placebo in random order. Identify potential problems with this approach and suggest solutions.
Design Implementation: Design a complete experiment to test whether listening to classical music while studying improves test performance. Your design should specify: - Which experimental design you’ll use and why - How you’ll recruit and randomize participants - What control measures you’ll implement - How you’ll measure the response variable - What potential confounding variables you’ll address
Agricultural Experiment: A farmer wants to test four different fertilizer combinations on corn yield. The farm has 48 plots arranged in a 6×8 grid, with plots in the same row having similar soil drainage patterns. Design an appropriate experiment and explain your choice of design.
Complex Design Elements: Explain how you might combine elements from different basic designs to handle a situation where you want to test three teaching methods across both different grade levels (3rd, 4th, 5th) and different school districts (urban, suburban, rural).