Selecting an Experimental Design
Help Questions
AP Statistics › Selecting an Experimental Design
A researcher wants to compare four study strategies (flashcards, practice tests, rereading, and summarizing) on a vocabulary test. Students’ prior vocabulary level (low vs high, based on a pretest) is expected to strongly affect outcomes. The researcher has 80 students total and can assign each student to only one strategy. Which experimental design is most appropriate?
Completely randomized design assigning all 80 students to the four strategies without considering pretest level
Systematic assignment: alternate strategies in a fixed repeating order as students enroll
Matched-pairs design: pair students by teacher, then assign each pair to all four strategies
Latin square design using prior vocabulary level and study strategy as the two blocking variables
Randomized block design: block by prior vocabulary level (low/high), then randomly assign strategies within each block
Explanation
This educational research scenario presents a textbook case for randomized block design when a pre-existing characteristic strongly affects outcomes. Prior vocabulary level (low vs high) is expected to influence how well students respond to different study strategies, making it an ideal blocking variable. Option C correctly blocks the 80 students by their pretest vocabulary level, creating homogeneous groups, then randomly assigns the four strategies within each block. This ensures each strategy is tested on both low and high vocabulary students while controlling for initial ability differences. Option A ignores this crucial factor, while B misunderstands matched pairs by trying to assign multiple treatments to pairs. Options D and E either misapply Latin squares or abandon randomization. Blocking by ability level before randomizing treatments is standard practice in educational experiments to reduce variance.
A chemistry teacher wants to test whether three different lab instruction formats (video demo, written instructions, and live teacher demo) affect lab report scores. There are 3 lab sections meeting on different days (Monday, Wednesday, Friday), and the teacher suspects day-of-week differences (fatigue, scheduling) could affect scores. Each section must use only one format to avoid confusion during the lab. Which experimental design is most appropriate?
Matched-pairs design: match students across sections by prior chemistry grade, then assign formats within pairs
Cluster randomized design: randomly assign each entire lab section (day) to one of the three formats
Randomized block design blocking by student GPA, then assign formats to students across sections
Latin square design using day of week and instruction format as blocking variables, then assign students
Completely randomized design assigning individual students within each section to different formats during the same lab period
Explanation
This scenario requires cluster randomized design due to the practical constraint that each lab section must use only one instruction format to avoid confusion. The teacher cannot mix formats within a section, making each section (day) an indivisible cluster. Option C correctly identifies this by randomly assigning each entire lab section to one of the three formats. While the teacher suspects day-of-week effects, this becomes a secondary consideration to the primary constraint of needing uniform instruction within each section. Options A and D impossibly suggest mixing formats within sections, while B blocks by an irrelevant variable (GPA) and ignores the section constraint. Option E misapplies Latin squares. When practical constraints require treating groups as units, cluster randomization is necessary even if it means accepting some potential confounding with group characteristics.
An engineering team wants to test whether two materials (Material X vs Material Y) produce different average breaking strengths. The testing machine warms up over time, potentially affecting measured strength, so the team plans to test 20 samples total in one day. They can randomize the order of tests, and each sample can be tested only once (it breaks). Which experimental design is most appropriate?
Completely randomized design: test all X samples first, then all Y samples
Randomized block design: block by material, then randomly assign time slots to blocks
Cluster randomized design: randomly select 10 time slots for X and the remaining 10 for Y without pairing nearby slots
Observational study: record strengths for whatever material arrives from the supplier first
Matched-pairs design: for each time slot, test one sample of X and one of Y back-to-back in random order, then compare paired results
Explanation
This engineering scenario calls for a matched-pairs design to control for the systematic effect of machine warm-up over time. Since the testing machine's temperature affects measurements and changes throughout the day, testing materials in paired time slots controls for this temporal effect. Option A correctly pairs time slots, testing one sample of each material back-to-back in random order within each pair, then comparing the paired differences. This removes the time/temperature effect from the comparison since both materials in a pair experience similar machine conditions. Option B would confound material with time, C misunderstands blocking, D doesn't control for the warm-up effect, and E abandons experimentation. When a nuisance factor changes systematically over time and you're comparing two treatments, matched pairs with randomized order within pairs provides excellent control.
A medical researcher wants to compare the effect of three doses of a supplement (0 mg, 50 mg, 100 mg) on reaction time. Reaction time is also affected by time of day (morning, afternoon, evening). The researcher can run 9 sessions total and can recruit one participant per session (different participant each session). The researcher wants each dose tested once at each time of day to control for time-of-day effects without increasing the number of sessions. Which experimental design is most appropriate?
Latin square design so that each dose occurs once in each time-of-day category across the 9 sessions
Stratified sampling by time of day, then observe reaction times without assigning doses
Matched-pairs design where each participant receives all three doses in one session
Completely randomized design assigning doses to the 9 sessions, ignoring time of day
Randomized block design blocking by time of day, but assign the same dose to all sessions within a block
Explanation
This medical research scenario is perfectly suited for a Latin square design, which efficiently controls for two blocking factors simultaneously. The researcher wants to test three doses at three times of day with only 9 sessions total (3×3), and needs each dose tested once at each time to control for time-of-day effects. Option D correctly identifies the Latin square design, which creates a 3×3 grid where each dose appears exactly once in each row (time of day) and column, using different participants. This elegant design controls for time-of-day effects without increasing the number of sessions needed. Options A and B don't ensure balanced testing across times, C impossibly gives all doses to one participant, and E abandons the experimental nature. Latin squares are ideal when you have two blocking factors and the number of treatments equals the number of levels in each blocking factor.
A psychology teacher wants to test whether background music affects quiz performance. The teacher has two class periods of the same course (Period 2 and Period 5). Because the teacher cannot play music for some students and not others within the same room at the same time, the teacher must choose one condition per class period on quiz day. Which experimental design is most appropriate?
Completely randomized design assigning individual students within each class period to music or no music
Cluster randomized design: randomly assign each entire class period to either music or no music
Matched-pairs design pairing students across periods by prior grades, then assigning music within each pair
Latin square design using class period and seating row as blocking variables, then assign conditions
Randomized block design blocking by class period, then randomly assigning students within each period to conditions
Explanation
This scenario requires cluster randomized design due to a practical constraint: the teacher cannot apply different treatments (music vs no music) to individual students within the same classroom simultaneously. Option D correctly identifies that entire class periods must be assigned as clusters to either music or no music condition. This is different from blocking - we're not blocking by period to control variation, but rather treating each period as an indivisible unit due to the nature of the treatment. Options A and B impossibly suggest assigning individual students within a class to different conditions. Option C pairs students across periods unnecessarily, and E overcomplicates with Latin squares. Cluster randomization is necessary when treatments must be applied to groups rather than individuals, often due to practical constraints or to avoid treatment contamination.
An engineer wants to test whether two different machine settings (Setting X vs. Setting Y) affect the strength of a plastic part. Parts are produced on four different machines, and the engineer believes machines may differ slightly in calibration, affecting overall strength. Each machine can produce parts under both settings during the day, and the response is part strength. Which experimental design is most appropriate?
Matched-pairs design pairing machines and assigning one machine to X and the other to Y for the whole day
Factorial design adding a third factor for operator mood, requiring repeated measures
Randomized block design blocking by machine, then randomly assign Setting X or Y to parts produced on each machine
Systematic design alternating settings X and Y in a fixed pattern on all machines with no randomization
Completely randomized design assigning settings to parts without recording which machine produced them
Explanation
This AP Statistics task requires designing experiments to handle equipment variability, such as machine differences. The randomized block design blocking by machine, with random assignment of settings to parts produced on each, is best to control for calibration variations while testing setting effects on strength. It fits the ability of machines to use both settings daily and focuses on part-level responses. Distractors: B ignores machines; C pairs machines inefficiently; D lacks randomization; E adds irrelevant factors. Mini-lesson: Block on units like machines when they may introduce variability; randomize treatments within blocks for repeated measures, allowing precise estimation of treatment effects despite block differences.
A coffee shop tests whether two types of music (jazz vs. pop) affect average customer time spent in the shop. The manager knows Saturdays are much busier than Tuesdays and expects that day-of-week strongly influences time spent. The shop can play only one music type per day, and the study will run for 8 days total (4 Tuesdays and 4 Saturdays over a month). Which experimental design is most appropriate to compare the music types while accounting for day-of-week?
Matched-pairs design: play jazz in the morning and pop in the afternoon each day
Two-factor factorial design using music type and day-of-week as treatments applied to each customer
Randomized block design: block by day-of-week (Tuesday vs. Saturday) and randomly assign jazz/pop within each block
Observational study: record time spent on days when the manager happens to choose jazz or pop
Completely randomized design: randomly pick 4 of the 8 days for jazz and use pop on the others
Explanation
In AP Statistics, selecting an experimental design involves choosing methods to control for confounding variables, such as day-of-week effects here. The randomized block design blocking by day-of-week (Tuesday vs. Saturday) is best, as it accounts for expected differences in busyness by randomizing music types within each block, ensuring balanced comparison over the 8 days. This fits the constraint of one music type per day and allows causal inference about music's effect on time spent. Distractors include A, which ignores blocking and risks confounding; C alters the treatment by splitting days; D lacks randomization; and E misapplies factorial design to customers instead of days. Mini-lesson: Block on variables strongly related to the response, like day-of-week, to isolate the treatment effect; randomize within blocks to avoid bias, especially when units (days) are limited.
A school nutritionist wants to compare three breakfast options (A, B, C) on students’ morning alertness scores. Alertness is known to differ by grade level (9th–12th), and the nutritionist can only serve one breakfast to each student on the test day. Within each grade, there are enough volunteers to try all three breakfasts, and students will be tested during the same first-period class. Which experimental design is most appropriate to reduce the effect of grade level while fairly comparing the three breakfasts?
Randomized block design: block by breakfast option, then randomly assign students to grades
Completely randomized factorial design: assign students to all combinations of breakfast option and grade level
Randomized block design: block by grade level, then randomly assign students within each grade to A, B, or C
Matched-pairs design: have each student try all three breakfasts on the same day and compare within-student scores
Completely randomized design: randomly assign all students to A, B, or C with no blocking
Explanation
This question tests the skill of selecting an appropriate experimental design in AP Statistics, focusing on controlling for known sources of variation like grade level while comparing treatments. The randomized block design in choice B fits best because it blocks by grade level to reduce variability from this factor, then randomly assigns students within each grade to one of the three breakfast options, ensuring a fair comparison with one breakfast per student. Choice A ignores blocking, potentially confounding results with grade differences; C is impractical as students can't try all breakfasts on the same day given the constraint; D incorrectly blocks by breakfast instead of grade; and E introduces an unnecessary factorial element treating grade as a factor rather than a blocking variable. A key distractor is C, which might appeal if overlooking the one-breakfast limit, but it violates the setup. In a mini-lesson, match designs to constraints by using blocking when a known categorical variable affects the response, ensuring randomization within blocks to maintain validity while controlling extraneous variation.
An engineer wants to test the effect of temperature (low vs. high) and catalyst type (C1 vs. C2 vs. C3) on the time to complete a chemical reaction. Each batch of chemicals can be used for only one run, and the engineer can randomly assign runs to conditions. The engineer wants to know whether the best catalyst depends on temperature. Which experimental design is most appropriate?
Matched-pairs design: run the same batch at both temperatures with all three catalysts
Randomized block design: block by catalyst type and then assign temperature within each block, but do not allow comparison among catalysts
Completely randomized $2\times 3$ factorial design with six conditions (each temperature crossed with each catalyst)
Observational study: record reaction times from past runs at various temperatures and catalysts without random assignment
Completely randomized design with three groups: compare catalysts only and ignore temperature
Explanation
This AP Statistics question evaluates choosing a design to assess main effects and interactions of temperature and catalysts on reaction time. The completely randomized 2x3 factorial design in D, with six conditions, is most appropriate for randomizing batches to combinations, allowing evaluation of whether optimal catalyst varies by temperature under the one-run constraint. Choice A ignores temperature; B misblocks limiting comparisons; C can't reuse batches for pairs; E lacks randomization. Distractors like C confuse pairing with factorial needs, but batches are single-use. Mini-lesson: Utilize factorial designs for multiple factors and potential interactions; fully cross levels and randomize to conditions, ensuring causality when constraints like single-use units prevent repetitions.
A coach wants to compare two training programs (P1 vs. P2) on athletes’ sprint times. Sprint times differ by position group (sprinters vs. middle-distance), and the coach can assign athletes within each position group to a program. However, the coach wants each athlete to follow only one program for the season. Which experimental design is most appropriate to control for position group while comparing programs?
Add unnecessary complexity: use a $2\times 2\times 2$ factorial design including program, position group, and shoe brand, requiring athletes to switch shoes weekly
Randomized block design: block by position group, then randomly assign athletes within each block to P1 or P2
Cluster randomization: assign all sprinters to P1 and all middle-distance athletes to P2
Completely randomized design: randomly assign all athletes to P1 or P2 with no blocking
Matched-pairs design: have each athlete do both programs simultaneously and compare sprint times
Explanation
In AP Statistics, this skill entails selecting a design to control for position group differences in sprint times while comparing training programs. Choice A's randomized block design, blocking by position and randomizing programs within blocks, effectively reduces group variability with one program per athlete. Choice B lacks blocking; C can't do simultaneous programs; D clusters without randomization; E complicates with irrelevant factors. A common distractor is B, but blocking is needed for fairness. Mini-lesson: Apply blocking for inherent group differences; randomize within blocks to compare treatments reliably, especially under constraints limiting athletes to one treatment without crossover.