Slides 📊
8.5. Examples of Experimental Design
The examples we will explore in this section demonstrate how researchers identify key features of their research context, match those features to appropriate design frameworks, and implement the designs while addressing practical constraints. Each example builds on the principles and design types we have studied, showing how they work together in realistic research scenarios.
8.5.1. Example 1: Instructional Methods Study
Educational researchers want to understand how different instructional methods affect student learning outcomes. The researchers identify three instructional approaches they want to compare:
Self-study from textbook: Students learn independently using written materials
Traditional classroom instruction: Students attend lectures and participate in class discussions
Online instruction: Students complete computer-based modules with interactive content
They recruit a sample of 90 students who will all be learning the same material. The students are randomly assigned to the three groups, with 30 students in each group. After a specified learning period, all students take the same comprehensive exam, and the researchers compare average scores across the three instructional methods.
Design Analysis
Fig. 8.8 Example 1: Instructional Methods
1. What type of experimental design is used?
This is a Completely Randomized Design (CRD). The key evidence is that students are simply randomly assigned to one of the three treatment groups without any prior grouping or blocking. No student characteristics are used to form blocks, and there’s no pairing or matching process.
2. What are the experimental units?
The 90 students serve as the experimental units. Since we’re dealing with human participants, they are more specifically called subjects.
3. What are the factors and their levels?
There is one factor: the type of instructional method. Its three levels are
Self-study from textbook
Traditional classroom instruction
Online instruction
Note that the traditional classroom instruction could be considered a control group since it represents the standard or conventional approach against which the other methods are being compared.
4. What is the response variable?
The response variable is the final exam score. This provides a quantitative measure of how well students learned the material under each instructional approach.
8.5.2. Example 2: Hand Cream Comparison
A cosmetics company wants to compare the effectiveness of two different hand cream formulations. Consumer satisfaction is highly subjective and varies greatly between individuals based on skin type, preferences, and expectations. This variability makes it difficult to detect treatment differences using simple comparisons.
The researchers recruit 40 women to participate in a hand cream comparison study. Half the women (20) use Hand Cream Type 1 for one month, while the other half (20) use Hand Cream Type 2 for one month. All participants stop using any hand cream for two weeks to allow the effects of the first cream to dissipate. The groups switch—those who used Type 1 now use Type 2 for one month, and those who used Type 2 now use Type 1 for one month. At the end of each phase, participants complete satisfaction surveys. The researchers then compare each participant’s satisfaction with Type 1 versus Type 2.
Design Analysis
Fig. 8.9 Example 2: Hand Cream Comparison
1. What type of experimental design is used?
This is a Matched Pairs Design, Type 2 (also called a self-paired design). The key identifying feature is that each participant serves as her own control by trying both treatments.
2. What are the experimental units?
The 40 women are the experimental units.
3. What are the factors and their levels?
There is one factor: the application of the two different hand cream formulations. Its levels are
Application of Hand Cream Type 1
Application of Hand Cream Type 2
4. What is the response variable?
The response variable is the satisfaction score. This is likely measured through a standardized survey that produces a numerical score representing each participant’s satisfaction with the hand cream they used during each phase.
8.5.3. Example 3: Dairy Cow Nutrition Study
Agricultural researchers want to investigate whether the type of grain mix affects milk production in dairy cows. The researchers plan to compare two different brands of grain mix to see which produces higher milk yields. They have access to 16 dairy cows from different farms, but these cows represent four different breeds. The breed of cow is known to significantly affect baseline milk production, with some breeds naturally producing much more milk than others.
The 16 cows are first separated into four blocks based on their breed, with 4 cows from each breed in each block. Within each breed (block), 2 cows are randomly assigned to receive Grain Mix Brand 1, and the other 2 cows receive Grain Mix Brand 2. After a specified feeding period, milk production is measured for each cow, and the researchers compare average production between the two grain mix brands while accounting for breed differences.
Design Analysis
Fig. 8.10 Example 3: Dairy Cow Nutrition Study
1. What type of experimental design is used?
This is a Randomized Block Design (RBD). The key identifying features are:
Experimental units are first grouped into blocks based on an important characteristic (breed).
Random assignment occurs within each block rather than across all units.
The blocking variable (breed) is not the primary interest but is controlled to improve precision.
2. What are the experimental units?
The 16 dairy cows serve as the experimental units.
3. What are the factors and their levels?
The factor of interest is the type of grain mix. This has two levels: Brand 1 and Brand 2.
4. What is the response variable?
The response variable is milk production, likely measured as the total volume of milk produced per cow over a specified time period (such as gallons per day or pounds per week).
8.5.4. Bringing It All Together
Exercises
Design Identification: For each scenario below, identify which experimental design would be most appropriate and explain your reasoning:
Testing whether three different fertilizers affect plant growth using 60 plants of the same species.
Comparing two pain medications using 40 patients with chronic pain, where patients try both medications separated by a washout period.
Studying the effect of teaching method on learning outcomes using students from three different grade levels.
Comparing the effectiveness of two workout routines using identical twins as participants.
Blocking Variable Selection: In the dairy cow study, breed was used as the blocking variable. Identify three other variables that might be good blocking variables for this study and explain why each would be appropriate.
Design Modification: Suppose the instructional methods study (Example 1) could be modified to use a randomized block design.
What variable would you use for blocking and why?
How would this change the implementation of the study?
What advantages and disadvantages would this create compared to the original CRD?
Sample Size and Design: Explain why the dairy cow study needed to use RBD rather than matched pairs, even though individual differences between cows are probably quite large.