Introduction to
Experimental Design
Learn the foundational principles of controlled experiments: randomization, replication, and control, and how they let us establish cause-and-effect relationships.
What Is an Experiment?
An experiment is a study in which the researcher deliberately imposes a treatment on individuals (called experimental units) in order to observe a response. Unlike observational studies, experiments allow us to draw conclusions about causation.
Core Vocabulary
Why Experiments Matter
Observational studies can reveal associations, but they cannot establish cause and effect. Lurking variables (confounders) may explain the relationship. Experiments solve this problem by using random assignment to balance out all other variables, isolating the treatment effect.
The Three Principles of Experimental Design
Why Randomization Is Essential
Random assignment is the mechanism that allows us to conclude causation. By assigning units to groups via chance, we ensure that every potential confounding variable is roughly evenly distributed across treatment groups, on average.
Experiment vs Observational Study
| Experiment | Observational Study | |
|---|---|---|
| Treatment | Imposed by researcher | Not imposed: researcher observes |
| Assignment | Random assignment to groups | Subjects self-select or pre-existing groups |
| Causation | Can establish cause-and-effect | Cannot establish cause-and-effect |
| Confounders | Balanced by randomization | May be present and uncontrolled |
| Example | Randomly assign a drug vs placebo | Survey smokers vs non-smokers |
Control Groups and the Placebo Effect
A control group provides a baseline for comparison. In medical and behavioral experiments, subjects may improve simply because they believe they are being treated: this is the placebo effect.
Completely Randomized Design (CRD)
The simplest experimental design: all experimental units are randomly assigned to treatment groups with no restrictions. Each unit has an equal chance of receiving any treatment.
Randomized Block Design
When a known variable (such as age or gender) is likely to affect the response, subjects are first grouped into blocks of similar individuals. Then, within each block, random assignment to treatments occurs separately.
Light defines the greenhouse blocks
The top shelf receives more light than the lower shelf. Place both fertilizer treatments on each shelf, control watering, soil and pot size, and measure height increase over four weeks.Randomize within each shelf
Matched Pairs Design
A special case of blocking where each block contains exactly two experimental units that are as similar as possible, or the same unit measured twice (before/after or crossover).
A listening study with fixed response cards and moving treatment badges
Measurements taken from the same person remain linked. Compare within-person differences instead of treating the two columns as independent samples.Keep the pairs linked
Suppose the same six IDs have differences 2, −1, 3, 0, 2 and 1, averaging 7/6. Under a sharp zero-effect null with randomized or exchangeable within-ID labels, 2⁶ = 64 sign assignments are allowed; 12 have absolute mean at least 7/6. Repeated measurement alone does not establish that exchangeability.How to Design an Experiment
Example 1: Fertilizer Study
A botanist wants to know if a new fertilizer increases tomato yield. She has 40 tomato plants of the same variety in a greenhouse.
Example 2: Blocking in a Drug Trial
A pharmaceutical company tests a new headache medication. They suspect that severity of headaches (mild, moderate, severe) will affect the response.
Example 3: Matched Pairs Taste Test
A coffee company wants to know if customers prefer a new blend over the original. They recruit 50 volunteers.
Common Mistakes
Experimental Designs Summary
| Design | How It Works | Best When |
|---|---|---|
| Completely Randomized | All units randomly assigned to treatments | Units are fairly homogeneous |
| Randomized Block | Group into blocks first, then randomize within blocks | A known variable affects the response |
| Matched Pairs | Blocks of size 2: matched subjects or same subject twice | Subjects can be closely paired or serve as own control |
Question 1 of 5
A researcher randomly assigns 60 volunteers to either take Vitamin C or a placebo daily for 3 months, then measures the number of colds each person gets. Which of the following is the response variable?
Question 2 of 5
A study finds that people who eat breakfast every day have lower rates of heart disease than people who skip breakfast. Can we conclude that eating breakfast causes lower rates of heart disease?
Question 3 of 5
In an experiment testing a new allergy medication, neither the patients nor the doctors measuring symptoms know who receives the drug and who receives the placebo. This describes a:
Question 4 of 5
A coach wants to test two training programs. She has athletes of varying skill levels (beginner, intermediate, advanced) and is worried this will affect results. She should use a:
Question 5 of 5
Which of the following is the primary purpose of random assignment in an experiment?