Introduction to Experimental Design - AP Statistics

1. A psychologist studies whether a 10-minute daily journaling routine reduces anxiety. She recruits 72 participants and measures anxiety score after 4 weeks. Participants are first grouped by baseline anxiety level (low, medium, high), and within each group, she randomly assigns half to journaling and half to no journaling. Which best describes why the psychologist grouped participants by baseline anxiety before random assignment?
   Answer: To block on a variable related to the response and reduce variability This AP Statistics question explores the purpose of blocking in experiments. The reason for grouping by baseline anxiety is to block on a related variable and reduce variability, as in choice C, improving treatment effect estimates. Choice A is a distractor, as blocking is not random sampling, which selects from a population. Mini-lesson: Blocking categorizes by a factor (anxiety level) then randomizes within blocks to control variability from that factor. This is useful when the blocking variable affects the response (anxiety score). It enhances precision without biasing assignment. The design supports causation within blocks.
2. A dermatologist is studying whether two different acne creams reduce the number of inflamed spots after 6 weeks. One hundred patients are recruited from a clinic. Patients are randomly assigned to one of three treatments: Cream A, Cream B, or a placebo cream with no active ingredient. The response variable is the number of inflamed spots after 6 weeks. Which design element best helps isolate the effect of the active ingredients from other factors?
   Answer: Including a placebo cream as a control treatment This question focuses on the role of control groups in isolating treatment effects. The correct answer is B because including a placebo cream creates a proper control group that experiences everything the treatment groups do (applying cream, clinic visits, expectation of improvement) except the active ingredients. This design isolates the effect of the active ingredients from placebo effects, natural healing, or regression to the mean. Option E is particularly problematic as it would destroy random assignment and introduce selection bias. Students often overlook that control groups don't just mean "no treatment"—they should match the treatment experience as closely as possible while lacking only the specific factor being tested. In experimental design, well-constructed control groups are essential for determining whether observed effects are due to the treatment itself or to other aspects of the experimental process.
3. A school district is studying whether using a new online homework system increases Algebra I test scores. The explanatory variable is homework system (new online vs. traditional paper), and the response variable is the end-of-unit test score. Four Algebra I classes (taught by the same teacher) participate; two classes are randomly assigned to use the online system and two are assigned to paper homework for one unit. Which design feature most directly reduces confounding and strengthens a cause-and-effect conclusion?
   Answer: Randomly assigning classes to the two homework systems This question focuses on identifying which design feature reduces confounding in an experiment. The correct answer is A because randomly assigning classes to treatments ensures that any pre-existing differences between classes (such as student ability, motivation, or class dynamics) are equally likely to appear in either treatment group. This random assignment balances potential confounding variables between the groups on average. Choice B (using four classes) provides replication but doesn't reduce confounding. Choice E would actually introduce confounding by allowing self-selection bias. The key principle here is that random assignment is the primary tool for reducing confounding in experiments, making it possible to attribute observed differences to the treatment rather than other factors.
4. A beverage company wants to test whether adding a citrus flavoring changes how sweet a drink is perceived. The explanatory variable is flavoring (citrus vs. none), and the response variable is the sweetness rating on a 1–9 scale. The company recruits 40 employees and randomly assigns 20 to taste the citrus version and 20 to taste the plain version. Each participant rates sweetness immediately after tasting. Which change would best reduce bias due to participants' expectations and improve the experimental design?
   Answer: Blind participants so they do not know which version they are tasting This question tests understanding of blinding as a method to reduce bias in experiments. The correct answer is C because blinding participants prevents their expectations about which version they're tasting from influencing their sweetness ratings. If participants know they're tasting the citrus version, they might expect it to taste different and rate it accordingly, introducing bias. Choice A (larger scale) might increase precision but doesn't address bias. Choice B (random sampling) would improve generalizability but not internal validity. Choice D would introduce selection bias, and Choice E would create order effects. Blinding is a crucial design feature that reduces psychological biases and ensures that measured differences reflect true treatment effects rather than participant expectations.
5. A nutrition researcher wants to test whether caffeine affects reaction time. The explanatory variable is drink type (caffeinated vs. decaf), and the response variable is reaction time on a computer task 30 minutes after drinking. Participants are randomly assigned to one of the two drinks, but the researcher who administers the reaction-time task knows which drink each participant received and gives extra encouragement to the caffeinated group. Which feature would best address this potential source of bias?
   Answer: Blind the person administering the reaction-time task to drink type This AP Statistics question examines bias reduction in experimental design, particularly experimenter bias. Blinding the person administering the reaction-time task to drink type prevents them from unconsciously influencing results, like giving extra encouragement to the caffeinated group. Choice C is a distractor as matched pairs control for individual differences, not this bias. Blinding is key in experiments to ensure objective measurement and avoid placebo or observer effects. Here, random assignment is already used, but without blinding, knowledge of treatment can skew administration. Adding blinding strengthens causal inference about caffeine's effect. This mini-lesson shows how design features like blinding enhance validity.
6. A city wants to test whether a new text-message reminder increases payment of parking tickets. The city has a list of 8,000 drivers who received a ticket last month. The explanatory variable is whether the driver receives the reminder text (yes/no), and the response variable is whether the ticket is paid within 14 days. Which plan uses random assignment (not random sampling) to support a cause-and-effect conclusion?
   Answer: Randomly assign 4,000 of the 8,000 drivers to receive the reminder text and the other 4,000 to receive no text This question tests the distinction between random assignment (for experiments) and random sampling (for surveys). The correct answer is C because it describes randomly assigning half the drivers to receive the reminder and half to receive no reminder, creating treatment and control groups necessary for causal inference. Option A uses random sampling to select who gets studied but gives everyone selected the same treatment, providing no comparison group. Option B allows self-selection, which would confound treatment with driver characteristics like conscientiousness. The key concept is that random assignment distributes drivers to different treatment conditions, while random sampling would select which drivers to study. For establishing causation about the reminder's effectiveness, we need random assignment to create comparable groups, not random sampling to ensure representativeness.
7. A principal wants to test whether a new tutoring program improves algebra test scores. One hundred 9th-grade students who are currently enrolled in algebra are available. The principal labels the students 1–100 and uses a random number generator to select 50 students to receive tutoring; the remaining 50 do not receive tutoring. After 6 weeks, all students take the same algebra test; the response variable is test score. Which statement best describes what the random number generator is used for in this study?
   Answer: It randomly assigns students to tutoring or no tutoring to reduce confounding This question directly asks about the purpose of using a random number generator in the experimental setup. The correct answer is A because the random number generator is used to randomly assign students to receive tutoring or not, which is the defining feature that makes this an experiment rather than an observational study. This random assignment ensures that factors like prior math ability, motivation, or study habits are balanced between groups on average, reducing confounding. Option B incorrectly describes random sampling from a population, which isn't what's happening—the 100 students are already identified. Options C and D make unrealistic claims about guarantees, while E is simply false. The key concept is recognizing that random number generators in experimental contexts are tools for implementing random assignment, creating the foundation for causal inference by ensuring treatment assignment is independent of participant characteristics.
8. An agricultural scientist compares three fertilizers on tomato yield. The explanatory variable is fertilizer type (F1, F2, F3), and the response variable is total tomato mass harvested per plant. The scientist has 30 similar plants in a greenhouse and assigns 10 plants at random to each fertilizer. Which feature of this design is an example of replication?
   Answer: Applying each fertilizer to multiple plants (10 per fertilizer) This AP Statistics question tests replication in experimental design. Applying each fertilizer to multiple plants (10 per type) exemplifies replication, reducing variability and increasing reliability of yield comparisons. Choice C is a distractor; random assignment balances groups, but replication provides multiple observations per treatment. Replication allows estimating treatment effects more precisely by averaging out random errors. Without it, results could be due to chance in single plants. The greenhouse controls conditions, aiding validity. This feature strengthens evidence for fertilizer differences.
9. A school district is comparing two reading programs for 3rd graders. The explanatory variable is the reading program (Program A vs. Program B), and the response variable is the score on a standardized reading test at the end of the semester. Within each of 12 schools, teachers list all 3rd graders and use a random number generator to assign half to Program A and half to Program B, so each school uses both programs. Which design principle is primarily being used to reduce the effect of differences among schools (such as resources or neighborhood factors) on the comparison of programs?
   Answer: Blocking by school before random assignment In AP Statistics, this question focuses on blocking in experimental design to control for variability. Blocking by school before random assignment reduces the effect of school differences, like resources, by ensuring both programs are tested within each school. A distractor like choice A mentions random sampling, which is for generalizability, not controlling variability within the experiment. Blocking is a design principle where subjects are grouped by a characteristic (here, schools) before randomizing treatments, minimizing its impact on the response. This allows a fairer comparison of reading programs. Without blocking, school differences could confound results. The large sample and standardized test help, but blocking directly addresses inter-school variability.
10. A public health team tests two methods for increasing daily steps: sending motivational texts or giving a wearable step-counter with daily reminders. The explanatory variable is intervention type (texts vs. wearable), and the response variable is the average number of steps per day over 1 month. Participants are randomly assigned to one intervention, but several in the wearable group stop wearing the device after the first week and are excluded from the analysis. Which issue is most likely introduced by excluding those participants?
    Answer: Attrition that can undermine the benefits of random assignment In AP Statistics, this question addresses attrition in experimental design and its impact on validity. Excluding participants who stopped using the wearable undermines random assignment benefits, as it may introduce bias if dropouts differ systematically. Choice C is a distractor, focusing on blinding, which isn't the issue here. Attrition can create non-comparable groups, losing randomization's balance. Intent-to-treat analysis might help, but exclusion risks bias. Random assignment initially balances, but dropouts disrupt it. This illustrates how real-world issues like noncompliance affect experimental integrity.
11. A professor wants to test whether allowing students to use a one-page notes sheet improves quiz performance. She has two sections of the same course taught back-to-back. She flips a coin to decide that the first section may use a notes sheet (treatment) and the second section may not (control). Both sections take the same quiz; the response variable is quiz score. Which factor is a potential confounder that could make it difficult to attribute differences in scores solely to the notes-sheet policy?
    Answer: Time of day of the class section (earlier vs later) could be related to performance This question tests understanding of confounding variables in experimental design. While the professor used randomization (coin flip) to assign treatments, the two class sections may differ systematically in ways that affect quiz performance. Time of day is a potential confounder - students in earlier sections might be more alert, or later sections might benefit from having more time to study. Other confounders could include different student populations (some majors might preferentially choose certain time slots) or the professor's energy level. These section-level differences could influence quiz scores independently of the notes-sheet policy, making it difficult to isolate the treatment effect. The coin flip for assignment doesn't eliminate these pre-existing differences between sections; it would be better to randomly assign individual students within sections or use both sections with a crossover design.
12. A company wants to test whether a new website layout increases the proportion of visitors who make a purchase. Over one week, every visitor to the site is randomly assigned by the server to see either the old layout (control) or the new layout (treatment). The response variable is whether the visitor makes a purchase during that visit. Which statement best describes what random assignment accomplishes in this experiment?
    Answer: It reduces the effects of confounding by balancing lurking variables between layouts on average This question examines what random assignment accomplishes in experimental design. Random assignment's primary benefit is reducing the effects of confounding by balancing lurking variables between treatment groups on average. When the server randomly assigns visitors to layouts, factors like visitor demographics, browsing habits, time of visit, and purchase intent are expected to be similarly distributed between groups. This balance means any difference in purchase rates can be more confidently attributed to the layout rather than these confounding factors. Random assignment does not guarantee equal outcomes, eliminate all bias, or make results generalizable to all customers (that would require random sampling). It creates comparable groups by distributing both known and unknown confounding variables evenly across treatments.
13. A nutrition scientist wants to test whether drinking a caffeinated beverage affects reaction time. Forty participants come to the lab twice, one week apart. Each participant is randomly assigned to drink caffeine on the first visit and decaf on the second visit, or decaf first and caffeine second. Reaction time is measured 30 minutes after the drink on each visit. Which design principle is primarily being used by having every participant receive both treatments in random order?
    Answer: Using a matched-pairs (within-subjects) design to control for person-to-person variability This question focuses on matched-pairs design as a method to control variability. The correct answer is C because having each participant receive both treatments (in random order) controls for person-to-person differences in baseline reaction time, caffeine sensitivity, and other individual factors. This within-subjects or matched-pairs design uses each person as their own control, dramatically reducing variability compared to a between-subjects design. The random ordering prevents order effects from systematically favoring one treatment. Option B about blinding is good practice but isn't what's being asked about regarding "every participant receiving both treatments." The key principle is that matched-pairs designs are powerful when individual differences are large relative to treatment effects, as each person's difference score (caffeine minus decaf) removes their baseline tendencies. This design is especially valuable in studies where recruiting many participants is difficult or expensive.
14. A researcher wants to estimate the average sleep time of all students at a large university and also test whether a new "sleep tips" email increases sleep. She randomly samples 200 students from the university roster, then randomly assigns half to receive the email for 3 weeks and half to receive no email. The response variable is average hours of sleep per night during the final week. Which statement is correct about what this study can conclude?
    Answer: It can generalize to all university students and establish causation because it uses random sampling and random assignment This AP Statistics question evaluates scope of inference in studies combining sampling and assignment. The correct statement is A: the study can generalize to all university students and establish causation due to random sampling and assignment. Choice B is a distractor, claiming no causation because of 'volunteers,' but the design uses random sampling from the roster, not volunteers. Mini-lesson: Random sampling allows generalization to the population, while random assignment enables causation by balancing groups. Here, sampling from the roster supports inference to all students, and assignment to email groups supports causal claims about the email's effect on sleep. Without random sampling, causation might hold but not generalizability (as in choice C). This design is ideal for both types of inference.
15. A botanist is testing whether three fertilizer types affect plant height after 8 weeks. She has 90 seedlings of the same species grown in identical pots. To reduce the impact of sunlight differences, she first groups the seedlings into 3 blocks based on which greenhouse bench they are on (Bench 1, Bench 2, Bench 3). Within each bench, she randomly assigns equal numbers of seedlings to Fertilizer 1, Fertilizer 2, or Fertilizer 3. The response variable is plant height after 8 weeks. Which feature of the design is specifically used to control for bench-to-bench sunlight variation?
    Answer: Blocking by greenhouse bench before assigning fertilizers This question specifically asks about blocking as a variance-reduction technique in experimental design. The correct answer is B because blocking by greenhouse bench before randomly assigning fertilizers within each block ensures that each fertilizer is equally represented on each bench, controlling for bench-to-bench sunlight differences. This is a randomized block design where the blocking factor (bench location) is identified as a source of variability that could mask treatment effects. Students might choose A thinking sample size helps, but that addresses precision, not the specific sunlight variation issue. The key principle is that blocking groups similar experimental units together before randomization, ensuring treatments are compared under similar conditions. This technique is particularly valuable when there's a known source of variation (like location effects) that researchers want to control for without making it a treatment factor.
16. A university wants to test whether background music affects quiz performance. The explanatory variable is music condition (music vs. no music), and the response variable is quiz score out of 20. Students volunteer, and the researcher flips a coin for each student to assign the music condition during the quiz. Afterward, the researcher concludes that music causes higher quiz scores for all college students. Which statement best describes what is justified from this study design?
    Answer: A cause-and-effect conclusion is justified for the volunteers, but generalizing to all college students may not be This AP Statistics question assesses causation and generalizability in experimental design. Random assignment justifies causation for volunteers, but lack of random sampling limits generalizing to all college students. Choice D is a distractor, overstating by claiming both are justified without random sampling. Experiments establish cause via random assignment, but generalizability requires representative samples. Coin flips ensure balanced groups for causation. Volunteers may differ from broader populations, so conclusions apply to similar groups. This distinguishes internal validity (causation) from external validity (generalization).
17. A company tests whether a new website layout increases the proportion of visitors who make a purchase. The explanatory variable is layout (current vs. new), and the response variable is whether a visitor purchases (yes/no). The company randomly assigns incoming visitors to see one of the two layouts and compares purchase rates. Which feature of this study most directly helps ensure that differences in purchase rates are due to the layout rather than preexisting differences between groups of visitors?
    Answer: Random assignment of visitors to layouts This AP Statistics question focuses on random assignment's role in experimental design. Randomly assigning visitors to layouts ensures differences in purchase rates are due to layouts, not preexisting visitor differences. Choice B is a distractor; large samples help precision, but random assignment directly balances groups. Without it, self-selection could confound results. This A/B testing is common in online experiments. Natural arrival aids realism, but randomization enables causation. It demonstrates how experiments isolate treatment effects.
18. A dermatologist tests whether a new acne cream reduces the number of pimples after 6 weeks. The explanatory variable is cream type (new cream vs. standard cream), and the response variable is the change in pimple count from baseline to 6 weeks. Each participant applies the new cream to the left side of the face and the standard cream to the right side, with the side assignments randomized for each person. Which feature of this design most directly helps control for person-to-person differences in acne severity?
    Answer: Having each participant receive both treatments (matched pairs) The question in AP Statistics explores matched-pairs design to control variability. Having each participant receive both treatments (new cream on one side, standard on the other) directly controls for person-to-person differences in acne severity by comparing within individuals. Choice B is a distractor; randomizing sides is good but secondary to the matched-pairs structure. Matched pairs reduce variability from individual factors, improving treatment effect detection. This design is like blocking on the individual level. Recruitment source affects generalizability, not control. Thus, it exemplifies how pairing minimizes confounding in experiments.
19. A doctor tests whether a new allergy medication reduces sneezing compared with a standard medication. She enrolls 100 patients and assigns 50 to the new medication and 50 to the standard medication using a random number generator. Patients report the number of sneezes in a 24-hour period after one week. Which design principle is used to help balance lurking variables between the two groups?
    Answer: Random assignment of patients to the two medications In AP Statistics, this question highlights principles for balancing variables in experiments. The design principle balancing lurking variables is random assignment to medications, as in choice A, creating equivalent groups on average. Choice B, using only allergy patients, is a distractor as it defines the population but does not balance variables without randomization. Mini-lesson: Random assignment distributes potential confounders evenly, allowing attribution of differences in response (sneezes) to the treatment (medication). Sample size (choice D) helps precision but not balancing without randomization. This enables causal conclusions about the new medication. Experiments without this risk confounding.
20. A teacher wants to test whether seating location affects quiz performance. The explanatory variable is seating (front half vs. back half of the room), and the response variable is quiz score. In one class period, the teacher asks students to choose any seat they want, then compares quiz scores of students who sat in the front half to those in the back half. Which feature is missing that prevents a cause-and-effect conclusion about seating and quiz performance?
    Answer: Random assignment of students to front or back seating locations This question identifies the absence of random assignment as the critical flaw preventing causal conclusions. The correct answer is B because when students choose their own seats, confounding variables like motivation, engagement, or vision problems systematically differ between front and back groups. More motivated students might choose front seats AND perform better on quizzes, making it impossible to determine if seating location itself causes the difference. Random assignment would balance these confounding factors between groups. Choice A is incorrect - the response variable is properly quantitative. Choice D affects generalizability, not internal validity. The fundamental principle is that without random assignment, observed associations cannot be interpreted as causal relationships due to potential confounding.

Start this quiz