Understanding Surveys, Experiments, and Observational Studies

Help Questions

Statistics › Understanding Surveys, Experiments, and Observational Studies

Questions 1 - 10
1

A district wants to compare two tutoring formats (in-person vs online) for improving algebra test scores. They randomly select 10 algebra classes from the district and then randomly assign the selected classes (not individual students) to use in-person tutoring or online tutoring for 6 weeks. Which statement correctly distinguishes random sampling and random assignment in this study?

Random sampling supports causation, while random assignment supports generalizing to the whole district

Random sampling helps support generalizing to the district’s algebra classes, and random assignment of classes supports a causal comparison of tutoring formats

Only random assignment occurred, because selecting classes at random does not count as random sampling

Only random sampling occurred, because assigning tutoring formats is just observing what teachers already do

Explanation

In statistics, we distinguish between types of studies like surveys, observational studies, and experiments, and understand how randomization helps in making inferences about populations or causation. The key feature of an experiment is that researchers actively assign subjects to different treatments to observe their effects. Random sampling involves selecting subjects randomly from a population, which supports generalizing results from the sample to the broader population. Random assignment means randomly allocating subjects to treatment groups, which supports causal conclusions by balancing out confounding factors. In this scenario, the district randomly selects 10 classes (random sampling for generalization to all algebra classes) and then randomly assigns them to in-person or online tutoring (random assignment for causal comparison of formats on test scores). A common misconception is mixing up the roles, but random sampling supports generalization, while random assignment supports causation. To classify future studies, ask: 'Did researchers assign a treatment?' (yes) and 'Was the sample randomly selected?' (yes).

2

A student council wants to estimate the average number of hours of sleep students at their high school get on school nights. They use a random number generator to select 120 students from the school roster and email each selected student a short questionnaire asking, “How many hours did you sleep last night?” Which type of study is described?

Observational study, because the researchers observe students’ sleep without asking any questions

Experiment, because the researchers are studying sleep and can determine what causes more sleep

Sample survey, because a random sample of students is asked a question and no treatment is assigned

Experiment, because the random number generator randomly assigns students to different sleep amounts

Explanation

In statistics, we distinguish between types of studies like surveys, observational studies, and experiments, and understand how randomization helps in making inferences about populations or causation. The key feature of an experiment is that researchers actively assign subjects to different treatments to observe their effects. Random sampling involves selecting subjects randomly from a population, which supports generalizing results from the sample to the broader population. Random assignment means randomly allocating subjects to treatment groups, which supports causal conclusions by balancing out confounding factors. In this scenario, the student council randomly selects 120 students and asks them about their sleep without assigning any treatment, making it a sample survey aimed at estimating average sleep hours for the whole school. A common misconception is that random sampling is the same as random assignment, but here there's random sampling for generalization, not assignment for causation. To classify future studies, ask: 'Did researchers assign a treatment?' (no, so not an experiment) and 'Was the sample randomly selected?' (yes, so results can generalize).

3

To study whether background music affects study efficiency, a researcher recruits 60 volunteers from one high school. Each volunteer is randomly assigned to one of three conditions while completing the same 30-minute practice set: no music, instrumental music, or music with lyrics. Time to finish and number correct are recorded. Which type of study is described?

Observational study, because the researcher only records time and accuracy without changing anything

Sample survey, because random assignment makes the group representative of the whole school

Experiment, because the researcher imposes a music condition (treatment) through random assignment

Sample survey, because students volunteered and then answered questions about music preferences

Explanation

In statistics, we distinguish between types of studies like surveys, observational studies, and experiments, and understand how randomization helps in making inferences about populations or causation. The key feature of an experiment is that researchers actively assign subjects to different treatments to observe their effects. Random sampling involves selecting subjects randomly from a population, which supports generalizing results from the sample to the broader population. Random assignment means randomly allocating subjects to treatment groups, which supports causal conclusions by balancing out confounding factors. In this scenario, the researcher recruits volunteers and randomly assigns them to music conditions while they complete a practice set, making it an experiment to test music's effect on study efficiency. A common misconception is that volunteers create a random sample, but here random assignment supports causation, while the non-random sample limits generalization beyond volunteers. To classify future studies, ask: 'Did researchers assign a treatment?' (yes, music conditions) and 'Was the sample randomly selected?' (no, volunteers).

4

A teacher wants to know whether using a flashcard app improves weekly vocabulary quiz scores. All students in the class take a baseline quiz, then the teacher randomly assigns half the students to use the app for 10 minutes each night and the other half to study as they normally do. After 3 weeks, the teacher compares quiz scores between the two groups. What does the random assignment allow the teacher to conclude (assuming the study is well run)?

The results can be generalized to all students in the school because the class was randomly assigned

Any difference proves the app works for every individual student in the class

Any difference in average quiz scores between groups can be attributed to the app more confidently (a cause-and-effect conclusion)

The study is a sample survey, so it estimates the school’s average quiz score without bias

Explanation

In statistics, we distinguish between types of studies like surveys, observational studies, and experiments, and understand how randomization helps in making inferences about populations or causation. The key feature of an experiment is that researchers actively assign subjects to different treatments to observe their effects. Random sampling involves selecting subjects randomly from a population, which supports generalizing results from the sample to the broader population. Random assignment means randomly allocating subjects to treatment groups, which supports causal conclusions by balancing out confounding factors. In this scenario, the teacher randomly assigns students to use the flashcard app or study normally, allowing a more confident causal conclusion about the app's effect on quiz scores. A common misconception is that random sampling alone enables causation, but here random assignment (not sampling, since it's the whole class) is what supports attributing differences to the app. To classify future studies, ask: 'Did researchers assign a treatment?' (yes, so experiment) and 'Was the sample randomly selected?' (no, but that's for generalization, not causation here).

5

A principal wants to estimate what percent of students prefer block scheduling over a traditional schedule. A link to an online poll is posted on the school’s homepage, and students choose whether to respond. No incentives are provided. What is the best statement about what this design allows?

Because participation is voluntary and not a random sample, the results may be biased and may not represent the whole school

Because the poll is online, it is automatically a random sample of students

Because there is no random assignment, the poll can establish that block scheduling causes higher preference

Because it is a survey, the results can be generalized to all students at the school without concern

Explanation

In statistics, we distinguish between types of studies like surveys, observational studies, and experiments, and understand how randomization helps in making inferences about populations or causation. The key feature of an experiment is that researchers actively assign subjects to different treatments to observe their effects. Random sampling involves selecting subjects randomly from a population, which supports generalizing results from the sample to the broader population. Random assignment means randomly allocating subjects to treatment groups, which supports causal conclusions by balancing out confounding factors. In this scenario, the principal posts an online poll with voluntary participation, which is a survey but lacks random sampling, so results may be biased and not generalizable to all students. A common misconception is that online polls are random samples, but voluntary response can overrepresent strong opinions on block scheduling. To classify future studies, ask: 'Did researchers assign a treatment?' (no) and 'Was the sample randomly selected?' (no, so generalization is limited).

6

A math department wants to test whether giving students a “worked-example sheet” before homework increases homework accuracy. Two teachers agree to participate. One teacher’s class uses the worked-example sheet for a unit, and the other teacher’s class does not. The department then compares average homework accuracy between the two classes. No random assignment is used. Which statement is most accurate?

This design proves the worked-example sheet works for all math students because two classes were included

This is a sample survey because homework accuracy is measured for students in the population

This is an observational study because the department did not randomly assign the treatment, so confounding is possible

This is an experiment that supports a causal conclusion because there are two groups being compared

Explanation

In statistics, we distinguish between types of studies like surveys, observational studies, and experiments, and understand how randomization helps in making inferences about populations or causation. The key feature of an experiment is that researchers actively assign subjects to different treatments to observe their effects. Random sampling involves selecting subjects randomly from a population, which supports generalizing results from the sample to the broader population. Random assignment means randomly allocating subjects to treatment groups, which supports causal conclusions by balancing out confounding factors. In this scenario, the department compares homework accuracy between two classes where teachers chose whether to use the worked-example sheet, making it an observational study since no random assignment means confounding (like teacher style) could explain differences, not causation. A common misconception is that comparing groups implies an experiment, but without assignment, it's observational with possible bias. To classify future studies, ask: 'Did researchers assign a treatment?' (no, teachers chose) and 'Was the sample randomly selected?' (not specified, but focus on lack of assignment).

7

A school newspaper reports a link between students’ participation in after-school clubs and their GPA. A reporter collects data by looking up GPA in school records and recording each student’s number of clubs from the activities database. No one is told to join or not join clubs. Which type of study is described?

Experiment, because the reporter compares two groups (club vs no club)

Sample survey, because using a database is the same as asking a random sample of students questions

Observational study, because variables are measured from records and no treatment is assigned

Experiment, because GPA is the response variable and that means a treatment was applied

Explanation

In statistics, we distinguish between types of studies like surveys, observational studies, and experiments, and understand how randomization helps in making inferences about populations or causation. The key feature of an experiment is that researchers actively assign subjects to different treatments to observe their effects. Random sampling involves selecting subjects randomly from a population, which supports generalizing results from the sample to the broader population. Random assignment means randomly allocating subjects to treatment groups, which supports causal conclusions by balancing out confounding factors. In this scenario, the reporter observes GPA and club participation from records without assigning any treatment like joining clubs, making it an observational study that can show associations but not causation. A common misconception is that comparing groups means it's an experiment, but without random assignment, factors like motivation could confound the GPA-club link. To classify future studies, ask: 'Did researchers assign a treatment?' (no, so observational) and 'Was the sample randomly selected?' (not specified, but focus is on lack of treatment).

8

A counselor wants to estimate the proportion of seniors who have completed a college application. The counselor surveys the first 80 seniors who arrive at a morning assembly and asks whether they have submitted at least one application. What is the main issue with using this sample?

Because the sample size is 80, the counselor has proven the true proportion for the entire senior class

Because there is no random assignment, the counselor cannot measure the proportion of seniors

Because the counselor asked a question, the results automatically generalize to all seniors

Because the sample is not randomly selected, it may not represent all seniors (selection bias is possible)

Explanation

In statistics, we distinguish between types of studies like surveys, observational studies, and experiments, and understand how randomization helps in making inferences about populations or causation. The key feature of an experiment is that researchers actively assign subjects to different treatments to observe their effects. Random sampling involves selecting subjects randomly from a population, which supports generalizing results from the sample to the broader population. Random assignment means randomly allocating subjects to treatment groups, which supports causal conclusions by balancing out confounding factors. In this scenario, the counselor surveys the first 80 seniors at an assembly, which is a non-random convenience sample, so it may not represent all seniors due to selection bias, like early arrivers being more organized. A common misconception is that a large sample size ensures representation, but without random sampling, generalization to all seniors is questionable. To classify future studies, ask: 'Did researchers assign a treatment?' (no) and 'Was the sample randomly selected?' (no).

9

A coach wants to know whether athletes who drink water during practice report less fatigue afterward. The coach does not change anyone’s routine. After practice, the coach asks each athlete how many minutes they spent drinking water and to rate fatigue on a 1–10 scale. What does the lack of random assignment imply?

The coach can conclude drinking water causes lower fatigue because the athletes were all measured the same way

The coach used random sampling, so the results automatically generalize to all athletes everywhere

The coach can conclude any association may be due to other factors, so a cause-and-effect conclusion is not justified

This is a sample survey only if athletes were forced to drink water as a treatment

Explanation

In statistics, we distinguish between types of studies like surveys, observational studies, and experiments, and understand how randomization helps in making inferences about populations or causation. The key feature of an experiment is that researchers actively assign subjects to different treatments to observe their effects. Random sampling involves selecting subjects randomly from a population, which supports generalizing results from the sample to the broader population. Random assignment means randomly allocating subjects to treatment groups, which supports causal conclusions by balancing out confounding factors. In this scenario, the coach observes athletes' water drinking and fatigue without assigning any treatment, so it's an observational study where lack of random assignment means associations could be due to confounders like workout intensity, not causation. A common misconception is that measuring variables implies causation, but without assignment, cause-and-effect is not justified. To classify future studies, ask: 'Did researchers assign a treatment?' (no) and 'Was the sample randomly selected?' (not specified, but focus on lack of assignment).

10

A math department wants to test whether weekly practice quizzes improve final exam scores. Students choose whether to take the optional weekly practice quizzes throughout the semester. At the end, the department compares final exam scores for students who took at least 8 practice quizzes versus those who took fewer than 8. Which statement about conclusions is most appropriate?

Because the final exam is the same for everyone, random assignment is unnecessary to prove the quizzes work.

Because there are two groups, the study is an experiment and supports a causal claim about the quizzes.

Because the department used many students, the results automatically generalize to all schools in the state.

Because students self-selected into quiz-taking, the comparison is observational and does not justify a causal claim that practice quizzes increase scores.

Explanation

Identifying study types like observational studies versus experiments hinges on design, with randomization key for causation or generalization. Experiments involve assigning treatments, but here students self-chose quiz participation, making it observational. Random sampling from a population supports generalizing results. Random assignment to groups allows causal claims. In this observational study, self-selection prevents causal claims about quizzes improving scores, as confounders may differ between groups. People often confuse group comparison with random assignment, but without assignment, causation isn't supported—random sampling (absent) would generalize, not cause. For analysis, ask: 'Did researchers assign a treatment?' (no, observational) and 'Was the sample randomly selected?' (no).

Page 1 of 4