01 / 21

Introduction to
Planning a Study

Learn how to design observational studies, surveys, and experiments that produce valid, reliable statistical conclusions.

AP Statistics

Why Does Study Design Matter?

In statistics, how you collect data is just as important as how you analyze it. A poorly designed study can lead to biased results, invalid conclusions, and wasted resources: no amount of fancy analysis can fix bad data.

The Three Types of Statistical Studies

Observational Study
Researchers observe subjects without imposing any treatment. No variable is deliberately manipulated.
Survey
Data is collected by asking questions of a sample from the population. The tool: questionnaires or interviews.
Experiment
Researchers impose a treatment on subjects and measure the response. Allows causal conclusions.

Essential Vocabulary

1
Population: The entire group of individuals we want information about.
2
Sample: The subset of the population that we actually collect data from.
3
Explanatory variable: The variable we think explains or causes changes in another variable.
4
Response variable: The variable that measures the outcome we are interested in.
5
Confounding variable: A lurking variable associated with both the explanatory and response variables.

Choosing the Right Study Type

Can you impose a treatment? No Yes Observational Study Experiment How do you collect data? Ask Measure Survey Observe Requires: ★ Random assignment ★ Control group ★ Replication Result: Association only Result: Causation possible

Observational vs. Experiment

Observational Study Experiment
Treatment imposed? No Yes
Random assignment? No Yes
Can show causation? No: only association Yes: if well-designed
Confounders Always possible Controlled by randomization
Example Comparing test scores of students who chose to study vs. not Randomly assigning students to a study program vs. control

The Three Principles of Experimental Design

1. Control
Use a control group or hold other variables constant to isolate the effect of the treatment.
2. Randomization
Randomly assign subjects to treatment groups so confounding variables are evenly distributed.
3. Replication
Use enough subjects in each group to reduce the effects of chance variation on the results.

Random Selection vs. Random Assignment

Random Selection
How you choose your sample from the population. Ensures the sample is representative and allows generalization to the population.
Random Assignment
How you place subjects into treatment groups. Ensures groups are comparable and allows causal conclusions.

Scope of Inference

Random Assignment? Yes No Yes No Random Selection? Causation + Generalization Best possible design Causation only Can't generalize to larger population Generalization only Association, but no causal claim Neither No causation, no generalization This is the Scope of Inference framework: what conclusions your study supports.

How to Plan a Study: Step by Step

1
State the question: Clearly define what you want to learn. Identify the population and variables.
2
Choose the study type: Decide if an observational study, survey, or experiment is most appropriate and ethical.
3
Design data collection: Determine sampling method (random selection) and, for experiments, treatment assignment (random assignment).
4
Address bias and confounding: Identify potential sources of bias and plan controls (blinding, placebos, blocking).
5
Collect, analyze, and conclude: Gather data, perform appropriate analysis, and state conclusions within the scope of inference.

Common Sampling Methods

Simple Random Sample
Every individual has an equal chance of being selected. Use a random number generator or table.
Stratified Sample
Divide the population into strata (groups), then randomly sample from each stratum proportionally.
Cluster Sample
Divide the population into clusters, randomly select entire clusters, and sample everyone in them.
Systematic Sample
Select every \( k \)-th individual from a list after a random starting point.

Sources of Bias in Studies

Selection Bias
The sample systematically excludes part of the population. Example: surveying only people at a gym about exercise habits.
Response Bias
Subjects give inaccurate answers due to question wording, social desirability, or interviewer influence.
Nonresponse Bias
Those who don't respond differ systematically from those who do, skewing the results.

Example 1: Identifying Study Type

1
Treatment imposed? No: the researcher is not assigning anyone to drink coffee.
2
Data collection method? She asks questions (surveys 500 students).
3
Conclusion: This is an observational study (specifically a survey). She can find an association but cannot claim causation.

Example 2: Designing an Experiment

60 Students Random Assignment Group 1: New Method 30 students Group 2: Traditional 30 students (control) Compare test scores

Example 3: Blocking to Reduce Variability

Blocking groups subjects by a characteristic known to affect the response, then randomizes within each block. It's like a stratified sample for experiments.

60 Students Block 1 High GPA (30) Block 2 Low GPA (30) Random assign New (15) Traditional (15) Random assign New (15) Traditional (15) Compare

Light defines the greenhouse blocks

A researcher examines a seedling in a greenhouse with brighter upper shelving and shaded lower shelving.The top shelf receives more light than the lower shelf. Place both fertilizer treatments on each shelf, control watering, soil and pot size, and measure height increase over four weeks.

Randomize within each shelf

Thirty unique plant IDs: T01–T15 on the brighter top shelf and B01–B15 on the lower shelf. Blue indicates new fertilizer and white standard; each shelf contains both treatments.For 30 seedlings, form two 15-plant shelf blocks and randomize both fertilizers within each shelf. The shown allocation assigns 7 new and 8 standard on top, 8 new and 7 standard below. Compare treatment growth within shelves so the light difference stays in the analysis.Open full diagram

Common Mistakes

✕ Claiming Causation from Observation
Saying "coffee causes higher grades" from a survey. Only experiments with random assignment can support causal claims.
✕ Confusing Selection and Assignment
Random selection (who is in the sample) and random assignment (who gets the treatment) serve different purposes.
✕ Ignoring Confounding Variables
Failing to account for lurking variables that may explain the observed association between explanatory and response variables.

Question 1 of 5

A researcher records the blood pressure of 200 adults and notes how many hours per week they exercise. What type of study is this?

Question 2 of 5

Which of the following is the primary purpose of random assignment in an experiment?

Question 3 of 5

A survey is mailed to 2,000 randomly selected households, but only 300 respond. The opinions of non-respondents likely differ from respondents. This is an example of:

Question 4 of 5

In a study, participants are grouped by age (under 30, 30-50, over 50), and then within each group, randomly assigned to receive a drug or a placebo. This design is called a:

Question 5 of 5

A study uses random selection to choose participants from a city and random assignment to assign them to treatment groups. What conclusions can be drawn?