Math 3 Quiz: Observational Studies Vs Experiments

20 questions · exam conditions

0:00

Observational Studies Vs ExperimentsQuestion 1 of 20

A researcher studies whether meditation reduces stress by comparing stress hormone levels between people who have practiced meditation for at least 5 years versus those who have never meditated. She finds significantly lower stress hormones in the meditation group. A critic argues that people who choose to meditate long-term may have different baseline personalities or stress management skills. This criticism primarily targets which aspect of the study design?

The lack of random sampling from the population, which limits statistical generalizability of the results

The cross-sectional design rather than longitudinal follow-up, which cannot establish temporal relationships between variables

The use of observational rather than experimental methodology, which reduces measurement precision and reliability

The absence of random assignment to meditation groups, which allows for selection bias and confounding variables

← Back to quizzes

Math 3 Quiz

Math 3 Quiz: Observational Studies Vs Experiments

Practice Observational Studies Vs Experiments in Math 3 with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Observational Studies Vs Experiments, giving you a quick way to practice the rules, question types, and explanations that matter most for Math 3.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

The lack of random sampling from the population, which limits statistical generalizability of the results
The cross-sectional design rather than longitudinal follow-up, which cannot establish temporal relationships between variables
The use of observational rather than experimental methodology, which reduces measurement precision and reliability
The absence of random assignment to meditation groups, which allows for selection bias and confounding variables (correct answer)

Explanation: When evaluating research study criticisms, you need to identify which fundamental aspect of research design is being challenged. The critic's concern about people who "choose to meditate long-term" having different baseline characteristics points to a core issue in study methodology. The correct answer is D because the critic is highlighting selection bias - the fundamental problem that occurs when participants self-select into groups rather than being randomly assigned. When people choose whether to meditate, those who stick with meditation for 5+ years likely differ systematically from non-meditators in ways beyond just meditation practice (personality traits, health consciousness, stress management skills, etc.). Without random assignment, you can't determine whether the observed differences in stress hormones are due to meditation itself or these pre-existing differences between the groups. Option A is incorrect because this isn't about sampling from the population - it's about how participants were allocated to comparison groups. Option B misses the mark because while this is indeed a cross-sectional study, the critic isn't concerned about timing but about baseline differences between groups. Option C is wrong because the critic isn't questioning measurement precision or reliability of the stress hormone tests, but rather the validity of comparing self-selected groups. Remember this key distinction: when you see criticism about "people who choose" or "self-selection," think random assignment and confounding variables. This is one of the most fundamental threats to internal validity in research design, and recognizing it will help you identify similar issues across different study contexts.

Question 2

A software company wants to test whether a new user interface design increases productivity. They implement the new design for all employees in their West Coast offices (500 employees) and keep the old design for all employees in their East Coast offices (300 employees). After three months, they find that West Coast productivity increased by 15% while East Coast productivity remained unchanged.

What type of study is this, and what is the most serious limitation for establishing that the new interface caused the productivity increase?

This is a natural experiment with strong causal inference because the geographic separation eliminates contamination between treatment and control groups, and the large sample sizes ensure statistical reliability.
This is a quasi-experiment with strong causal inference because the company controlled the treatment assignment, but the limitation is the unequal sample sizes between groups affecting the statistical power of comparisons.
This is a controlled experiment with moderate causal inference, but the limitation is that employees knew which interface they were using, creating potential placebo effects that could influence productivity measurements.
This is an observational study with limited causal inference because the offices weren't randomly assigned to treatment conditions, so systematic differences between East and West Coast operations could explain the productivity difference. (correct answer)

Explanation: When evaluating research studies, you need to identify the study type and assess what threatens causal inference. The key distinction lies in whether researchers had control over who received the treatment. This study is observational because the company didn't randomly assign offices to treatment conditions - they simply chose to give West Coast offices the new interface and East Coast offices the old one. This lack of randomization is crucial because it means systematic differences between the regions could explain the productivity increase, rather than the interface itself. East and West Coast offices likely differ in many ways: employee demographics, local business culture, cost of living affecting motivation, types of clients served, or even seasonal work patterns. Looking at the wrong answers: Choice A incorrectly calls this a "natural experiment" - but natural experiments involve naturally occurring random assignment, not deliberate company decisions. Choice B labels it a "quasi-experiment," which requires some systematic treatment assignment with comparison groups, but the geographic division here is too arbitrary. The unequal sample sizes mentioned aren't the main concern. Choice C calls it a "controlled experiment," but true experiments require random assignment to treatment and control groups, which didn't happen here. The fundamental issue isn't sample size or employee awareness - it's that we can't separate the effect of the new interface from all the other differences between East and West Coast operations. Study tip: In research design questions, always ask first: "Was treatment randomly assigned?" If not, it's observational, and confounding variables become your biggest threat to causal inference.

Question 3

A researcher conducts a study where she randomly selects schools to implement either a new reading curriculum or continue with the existing curriculum. After one year, she compares reading scores between the two groups of schools. However, she discovers that schools assigned to the new curriculum had significantly more experienced teachers on average. How should this finding be interpreted regarding the study's experimental nature and causal conclusions?

The study remains experimental because schools were randomly assigned, and the teacher experience difference occurred by chance; this random imbalance doesn't affect the ability to draw causal conclusions about the curriculum's effectiveness.
The study becomes observational because the systematic difference in teacher experience indicates the randomization failed, eliminating the primary advantage of experimental design for establishing causation about curriculum effects.
The study remains experimental, but the teacher experience difference is a confounding variable that must be statistically controlled for in the analysis to maintain valid causal inferences about the curriculum.
The study remains experimental because random assignment was used, but the teacher experience imbalance weakens confidence in causal conclusions since it provides an alternative explanation for any observed differences in reading scores. (correct answer)

Explanation: This remains an experiment because schools were randomly assigned to treatments. However, the imbalance in teacher experience (which likely occurred by chance with the relatively small number of schools) creates ambiguity about whether curriculum or teacher experience caused any observed differences. While randomization doesn't guarantee perfect balance on all variables, this particular imbalance affects a key factor related to the outcome, weakening confidence in causal conclusions. Choice A dismisses a meaningful threat to causal inference. Choice B incorrectly suggests failed randomization makes it observational. Choice C suggests statistical control, but this doesn't fully address the fundamental ambiguity about what caused the effects.

Question 4

A researcher wants to test whether a new online learning platform improves student outcomes compared to traditional classroom instruction. Due to logistical constraints, she cannot randomly assign individual students to different teaching methods. Instead, she randomly assigns 20 schools to use either the online platform or continue with traditional instruction, then compares student performance across schools. What are the key advantages and limitations of this approach for establishing causation?

This cluster randomization approach maintains experimental design and can establish causation, but may have reduced statistical power due to correlation among students within schools and potential contamination effects between teaching methods. (correct answer)
This approach creates a natural experiment that provides stronger causal evidence than individual randomization because it reflects real-world implementation, but is limited by the smaller sample of schools compared to students.
This approach becomes observational rather than experimental because randomization occurred at the wrong level, eliminating the ability to establish causation but providing useful descriptive information about teaching method preferences.
This cluster randomization approach can establish causation through experimental design, but is primarily limited by ethical concerns about denying some schools access to potentially superior online learning technology.

Explanation: This is a cluster-randomized experiment (randomization at the school level) which maintains experimental design and can establish causation. However, it has important limitations: reduced statistical power because students within schools are more similar than students across schools (intracluster correlation), and potential for contamination if teachers or administrators share information between schools. The random assignment of schools still controls for school-level confounders and supports causal inference. Choice B incorrectly calls it a natural experiment and overstates its advantages. Choice C wrongly claims it becomes observational. Choice D focuses on ethics rather than the main statistical and design limitations.

Question 5

A psychologist studies the relationship between video game playing and aggressive behavior in teenagers. She recruits 200 teenagers and divides them into two groups based on their current gaming habits: heavy gamers (play more than 3 hours daily) and light gamers (play less than 1 hour daily). She then measures aggressive behavior using standardized tests and finds that heavy gamers score significantly higher on aggression measures.

Based on this study design, which statement best evaluates the researcher's ability to conclude that video game playing causes increased aggressive behavior?

The researcher can establish causation because she used standardized measures of aggression and clearly defined gaming categories, providing objective evidence that heavy gaming leads to more aggressive behavior in teenagers.
The researcher can establish causation because the large sample size and clear behavioral differences between groups provide sufficient statistical evidence that video game exposure directly increases aggressive tendencies in adolescents.
The researcher cannot establish causation because this observational study design cannot determine whether gaming causes aggression, aggression leads to more gaming, or other factors influence both gaming habits and aggressive behavior. (correct answer)
The researcher cannot establish causation because the study lacks a control group of teenagers who don't play video games at all, making it impossible to isolate the specific effects of gaming on aggressive behavior.

Explanation: When you encounter research design questions, the key is distinguishing between correlation and causation. Just because two variables are associated doesn't mean one causes the other. This study uses an observational design where the researcher compares existing groups of heavy and light gamers. While she found that heavy gamers show more aggressive behavior, this correlation alone cannot establish that gaming causes aggression. Three alternative explanations exist: gaming might cause aggression, aggressive tendencies might lead people to game more, or unknown third variables (like family environment, personality traits, or socioeconomic factors) might influence both gaming habits and aggression levels simultaneously. Answer A is wrong because having standardized measures and clear definitions improves measurement quality but doesn't address the fundamental causation problem. Good measurement tools don't transform correlational data into causal evidence. Answer B incorrectly assumes that large sample sizes and statistical significance establish causation. While these factors strengthen the reliability of the correlation found, they don't eliminate alternative explanations for the relationship. Answer D misidentifies the core issue. Adding a no-gaming control group would provide additional comparison data but wouldn't solve the causation problem. The fundamental issue isn't missing groups—it's the observational nature of the study design. To establish causation, researchers need experimental designs with random assignment to conditions, allowing them to control for confounding variables and isolate the effect of the independent variable. Study tip: Remember that correlation never equals causation in observational studies, regardless of sample size or measurement quality. Only controlled experiments can establish causal relationships.

Question 6

A nutrition researcher finds that people who eat organic food have lower rates of certain health problems. She has data from a large national survey where participants reported their eating habits and health status. The researcher concludes that eating organic food causes better health outcomes. Which factor most undermines this causal conclusion?

The study relies on self-reported data about eating habits, which may be inaccurate due to recall bias and social desirability bias, leading to unreliable measurements of organic food consumption.
People who choose to eat organic food may have higher incomes, better education, and healthier lifestyles overall, creating multiple confounding variables that could explain the observed health differences. (correct answer)
The study used a cross-sectional design that measured diet and health at the same time, making it impossible to determine whether diet changes preceded health improvements or vice versa.
The definition of 'organic food' varies across different regions and time periods, creating measurement inconsistency that could lead to misclassification of participants' actual dietary exposures.

Explanation: This observational study's main weakness for causal inference is confounding. People who buy organic food typically have higher socioeconomic status, which is associated with better healthcare access, education about health, exercise habits, and other factors that improve health outcomes. These confounding variables provide alternative explanations for the health differences that don't involve organic food causing better health. While choices A, C, and D identify real methodological concerns, choice B identifies the most fundamental threat to causal inference in observational studies - the inability to separate the effect of the exposure from other associated factors.

Question 7

A medical researcher studying a new treatment randomly assigns patients to receive either the new treatment or standard care. However, due to ethical concerns, patients who don't respond well to their assigned treatment after two weeks are allowed to switch to the other treatment. At the end of the study, the researcher compares outcomes based on the original random assignments (intention-to-treat analysis). Why is this still considered an experiment capable of establishing causation?

The study remains experimental because random assignment was maintained throughout, and switching treatments actually strengthens causal inference by showing the treatment works under real-world conditions where patients can change therapies.
The study remains experimental because the ethical treatment switching creates a more realistic clinical scenario, and the intention-to-treat analysis preserves randomization while providing stronger evidence for causation than per-protocol analysis would.
The study becomes observational once patients switch treatments because the researcher loses control over treatment assignment, but it can still establish causation through the natural experiment created by patient choices.
The study remains experimental because the initial random assignment eliminates selection bias and confounding, even though treatment switching may dilute the observed effect size and make the results more conservative. (correct answer)

Explanation: When you encounter questions about experimental design and causation, focus on what makes a study truly experimental: the initial random assignment of treatments, not what happens afterward. This study remains experimental because the crucial element—random assignment—occurred at the beginning and eliminates selection bias and confounding variables. The researcher randomly assigned patients to treatments, creating comparable groups where any differences in outcomes can be attributed to the treatment effect. When using intention-to-treat analysis, you analyze patients according to their original random assignments regardless of what actually happened during the study. While some patients switched treatments, this doesn't destroy the experimental nature. The treatment switching may dilute (weaken) the observed effect size because some control patients received the treatment and some treatment patients received standard care. This makes the results more conservative—you're less likely to detect a treatment effect even if one exists—but the causal inference remains valid. Choice A is wrong because switching treatments doesn't strengthen causal inference; it typically weakens the observable effect. Choice B incorrectly suggests that treatment switching provides stronger evidence than per-protocol analysis—actually, intention-to-treat analysis is more conservative. Choice C is incorrect because the study doesn't become observational; the original randomization preserves the experimental design regardless of subsequent treatment changes. Remember: A study's experimental status depends on initial random assignment, not on perfect adherence to assigned treatments. Treatment switching affects effect size estimation but doesn't eliminate the ability to establish causation through the original randomization.

Question 8

A city wants to evaluate whether installing speed cameras reduces traffic accidents. They compare accident rates before and after camera installation at 20 intersections where cameras were installed, and find a 30% reduction in accidents. A critic argues that this doesn't prove cameras cause the reduction.

Which statement best explains why this study design limits the ability to establish causation, and what would strengthen the causal inference?

The study is observational because there's no control group; adding a randomized control group of similar intersections without cameras would make it experimental and strengthen causal inference significantly. (correct answer)
The study is experimental but lacks sufficient statistical power; increasing the sample size to more intersections with cameras would provide stronger evidence for causation through better statistical significance.
The study is observational because it only examines one city; expanding to multiple cities with different traffic patterns would make it experimental and improve generalizability of causal claims.
The study is experimental but suffers from measurement bias; using more accurate accident reporting methods would strengthen the causal inference by reducing data quality issues.

Explanation: This is an observational study (specifically a before-after study) because there's no control group - we don't know what would have happened at these intersections without cameras. Other factors could explain the reduction (seasonal changes, other safety measures, regression to the mean). Adding a randomized control group of similar intersections without cameras would make it experimental and control for these confounding factors. Choice B incorrectly identifies it as experimental and focuses on statistical power rather than study design. Choice C incorrectly suggests that studying multiple cities would make it experimental. Choice D also incorrectly identifies it as experimental and focuses on measurement issues rather than the fundamental design flaw.

Question 9

A researcher wants to test whether a new study technique improves test scores. She randomly assigns 100 students to either use the new technique or continue with their usual study methods, then compares their test scores after one month. However, during the study, she discovers that students in the new technique group spent significantly more time studying overall. How does this affect the interpretation of her results?

This invalidates the experiment because the extra study time is a confounding variable that makes it impossible to determine whether the technique or the additional time caused any improvement in scores.
This strengthens the experiment because it shows the new technique motivates students to study more, which is part of the technique's overall causal effect on improving test scores through increased engagement. (correct answer)
This has no effect on the experiment's validity because random assignment ensures that any differences in study time between groups occurred by chance and don't affect causal conclusions.
This converts the experiment into an observational study because the researcher can no longer control for all variables, eliminating the ability to make causal inferences about the technique's effectiveness.

Explanation: This is still a valid experiment because random assignment was used to assign the treatment (study technique). The increased study time is likely a mechanism through which the new technique works - if the technique motivates students to study more, that's part of its causal effect. The researcher can still conclude that the technique caused the improvement, with increased study time being part of the causal pathway. Choice A incorrectly treats a mediating variable as a confounding variable. Choice C ignores that the systematic difference in study time is likely caused by the treatment itself. Choice D misunderstands what makes a study experimental versus observational.

Question 10

A pharmaceutical company reports that patients taking their new medication showed a 40% reduction in symptoms compared to patients not taking the medication. However, the study only included patients who voluntarily chose to take the medication versus those who chose not to take it. What is the most significant limitation of drawing causal conclusions from this study?

The sample size is too small to detect meaningful differences between the groups, leading to unreliable statistical conclusions about the medication's effectiveness.
Patients who chose to take the medication may differ systematically from those who chose not to, creating confounding variables that could explain the observed difference rather than the medication itself. (correct answer)
The 40% reduction is not large enough to be clinically significant, making any causal claims about the medication's benefits questionable regardless of study design.
The study lacks a proper control group since all participants had the condition, making it impossible to determine what would happen without any intervention at all.

Explanation: This describes an observational study where treatment assignment was not randomized. The key limitation is selection bias - patients who chose the medication may have been more motivated, healthier, had better access to healthcare, or differed in other ways that could affect outcomes. These confounding variables make it impossible to determine if the medication caused the improvement. Choice A focuses on sample size, which isn't the fundamental issue with observational studies. Choice C discusses clinical significance, not causal inference limitations. Choice D misunderstands what constitutes a control group in medical studies.

Question 11

A researcher studies whether a new meditation app improves sleep quality. She recruits volunteers through social media, randomly assigns half to use the app for 30 days while the other half receives no intervention, then compares sleep quality scores between groups. The app group shows significant improvement. However, all participants knew whether they were using the app or not. How does this knowledge affect the study's ability to establish causation?

It eliminates the study's ability to establish causation because participants' knowledge of their group assignment creates expectancy effects that could fully explain the observed improvements in sleep quality scores.
It strengthens the study's ability to establish causation because participants' awareness ensures they used the intervention as intended, providing clearer evidence of the app's direct effects on sleep quality.
It weakens but doesn't eliminate the study's ability to establish causation because while placebo effects may contribute to the results, the random assignment still controls for confounding variables and supports causal inference. (correct answer)
It has no effect on the study's ability to establish causation because random assignment eliminates all potential biases, and participant knowledge is irrelevant when treatment assignment was properly randomized.

Explanation: When evaluating causal claims from experiments, you need to consider both the strengths of the research design and potential threats to validity. This study uses randomized controlled trial methodology, which is the gold standard for establishing causation, but participant awareness introduces complications. The correct answer is C because this study demonstrates a nuanced situation where causal inference is weakened but not destroyed. Random assignment is powerful—it ensures that groups are equivalent on all variables except the intervention, controlling for confounding factors that could alternative explain differences. However, when participants know their group assignment, placebo effects and expectancy bias can inflate treatment effects. The observed improvements might partly reflect genuine app benefits and partly reflect participants' expectations of improvement. Option A is too extreme—while expectancy effects are real concerns, they don't necessarily "fully explain" all observed effects or completely eliminate causal inference when random assignment is present. Option B incorrectly suggests that participant awareness strengthens causation; awareness actually introduces bias rather than ensuring proper intervention use. Option D makes a fundamental error by claiming random assignment eliminates "all potential biases"—randomization controls for confounding variables but doesn't address measurement bias or expectancy effects that occur after assignment. Remember this pattern: randomized experiments provide strong causal evidence, but lack of blinding (when participants know their assignment) introduces expectancy effects that weaken but don't eliminate causal conclusions. Look for answer choices that acknowledge both the strengths and limitations of the design.

Question 12

An educational researcher notices that students who participate in after-school music programs have higher GPAs than students who don't participate. To investigate this relationship, she tracks 150 students over two years: 75 who joined music programs and 75 who didn't. She finds that music program participants maintained higher GPAs throughout the study period.

This study design has a critical limitation for establishing causation. Which statement best identifies this limitation and explains why it matters?

The study lacks random assignment of students to music programs, so students who choose music may already differ in motivation, family support, or academic ability, making these factors rather than music the likely cause of higher GPAs. (correct answer)
The study follows students for too short a period to detect meaningful changes in academic performance, since GPA improvements from music participation typically require at least three to four years to become statistically significant.
The study uses an unequal comparison because music programs require additional time commitment, so higher GPAs might simply reflect better time management skills rather than any specific benefits of musical training.
The study fails to account for different types of music programs, since classical music training may have different cognitive effects than popular music, creating heterogeneity that obscures true causal relationships.

Explanation: This is an observational study because students self-selected into music programs rather than being randomly assigned. The critical limitation is selection bias - students who choose to join music programs may already possess characteristics (higher motivation, more supportive families, better academic skills) that lead to higher GPAs regardless of music participation. This confounding makes it impossible to determine whether music causes the higher GPAs or whether the pre-existing differences do. Choice B incorrectly focuses on study duration. Choice C identifies a real factor but doesn't address the fundamental selection bias issue. Choice D focuses on program heterogeneity rather than the core causal inference problem.

Question 13

A public health researcher wants to study whether air pollution causes respiratory problems. She compares respiratory health in residents of City A (high pollution) versus City B (low pollution). She finds that City A residents have significantly more respiratory problems and concludes that air pollution causes these health issues.

Which statement best describes the primary weakness of this study design for establishing causation and suggests the most appropriate improvement?

The study is observational and subject to confounding since cities may differ in healthcare access, demographics, and other factors affecting respiratory health; a better approach would be a longitudinal study tracking individuals as pollution levels change over time. (correct answer)
The study is experimental but lacks proper controls since pollution exposure wasn't manipulated; a better approach would be to randomly assign participants to live in high versus low pollution areas for a controlled comparison, ensuring equal group sizes and standardized exposure measurements.
The study is observational and limited by small sample size since only two cities were compared; a better approach would be to include dozens of cities with varying pollution levels to increase statistical power and enable regression analysis controlling for potential confounders.
The study is experimental but suffers from selection bias since people choose where to live; a better approach would be to recruit participants randomly from the general population rather than from specific cities, ensuring representative sampling across all demographic groups.

Explanation: This is an observational study comparing existing conditions between cities. The primary weakness is confounding - cities with different pollution levels may also differ systematically in socioeconomic status, healthcare access, industrial activity, climate, and other factors affecting respiratory health. A longitudinal study following the same individuals over time as pollution changes would better control for these confounding factors. Choice B incorrectly suggests randomly assigning people to live in different cities, which is unethical and impractical. Choice C focuses on sample size rather than the fundamental confounding problem. Choice D incorrectly identifies this as experimental and misunderstands the selection bias issue.

Question 14

Dr. Martinez is studying the relationship between exercise and mental health. She randomly selects 200 adults and measures their current exercise habits and mental health scores. She finds that people who exercise more have better mental health scores and concludes that exercise improves mental health.

What type of study did Dr. Martinez conduct, and what is the primary limitation of her conclusion about causation?

She conducted an experiment, but her conclusion is limited because she didn't control for the duration of the study period or follow participants over time to observe changes.
She conducted an observational study, but her conclusion is limited because correlation doesn't imply causation - other factors could explain both exercise habits and mental health, or mental health could influence exercise habits. (correct answer)
She conducted a randomized controlled trial, but her conclusion is limited because 200 participants is too small a sample size to establish statistical significance for behavioral interventions.
She conducted a longitudinal study, but her conclusion is limited because measuring variables at a single point in time doesn't account for seasonal variations in both exercise and mental health.

Explanation: Dr. Martinez conducted an observational study because she measured existing behaviors and characteristics without manipulating any variables. The limitation is that observational studies cannot establish causation due to potential confounding variables (e.g., income, age, health status affecting both exercise and mental health) and the possibility of reverse causation (better mental health leading to more exercise). Choice A incorrectly identifies this as an experiment. Choice C wrongly calls it a randomized controlled trial. Choice D incorrectly identifies it as longitudinal when it's actually cross-sectional.

Question 15

A technology company wants to determine if flexible work arrangements improve employee satisfaction. They survey 1,000 employees across different departments: 600 work flexibly (chosen by their managers based on job requirements) and 400 work traditional schedules. The survey shows that employees with flexible arrangements report higher job satisfaction.

What is the most significant limitation of concluding that flexible work arrangements cause higher job satisfaction, and what aspect of the study design creates this limitation?

The limitation is measurement bias because job satisfaction surveys are subjective and employees with flexible schedules may overreport satisfaction; the study design creates this by not using objective performance measures instead of satisfaction surveys.
The limitation is sample size imbalance because 600 flexible versus 400 traditional workers creates unequal groups; the study design creates this by not ensuring equal numbers of employees in each work arrangement category.
The limitation is selection bias because managers chose which employees got flexible arrangements based on job requirements, meaning these employees may have had different types of work or characteristics that independently affect satisfaction levels. (correct answer)
The limitation is temporal ambiguity because the survey measured satisfaction at only one time point; the study design creates this by not tracking satisfaction changes before and after implementing flexible work arrangements.

Explanation: When evaluating research conclusions about cause and effect, you need to identify what could create spurious relationships—apparent connections that aren't actually causal. The key issue here is whether the groups being compared are truly equivalent except for the treatment being studied. The correct answer is C because this study has a fundamental selection bias problem. Since managers chose which employees received flexible arrangements "based on job requirements," the flexible and traditional groups likely differ in systematic ways beyond just their work arrangement. Employees selected for flexibility might have different job types, skill levels, seniority, or other characteristics that independently influence job satisfaction. This means any satisfaction difference could be due to these pre-existing differences rather than the flexible arrangement itself. Option A incorrectly focuses on measurement bias. While satisfaction surveys are subjective, this affects both groups equally and doesn't explain why we can't infer causation. Option B misidentifies the problem as sample size imbalance. Unequal group sizes (600 vs 400) don't prevent causal inference—the issue is how people were assigned to groups, not how many are in each. Option D suggests temporal ambiguity as the main limitation. While longitudinal data would be helpful, the primary threat to causal inference is the non-random assignment, not the timing of measurement. Remember: whenever you see research where participants weren't randomly assigned to treatment groups, immediately look for selection bias as a major limitation. The way people end up in different groups often determines what conclusions you can validly draw.

Question 16

A research team wants to investigate whether a new teaching method improves student performance in mathematics. They have access to 500 students across 10 different schools. The team is considering two approaches: (1) randomly assign half the students to receive the new teaching method while the other half continues with traditional methods, or (2) survey students who have already been exposed to the new method at schools that voluntarily adopted it and compare their performance to students at schools still using traditional methods.

Which statement best describes the key difference between these two approaches and their implications for establishing causation?

Approach 1 is an experiment that can establish causation because random assignment controls for confounding variables, while Approach 2 is an observational study that cannot definitively establish causation due to potential confounding factors. (correct answer)
Approach 1 is an observational study that provides stronger evidence because it uses a larger sample size, while Approach 2 is an experiment that may be biased due to voluntary participation of schools.
Both approaches are experiments since they compare two groups, but Approach 1 is more reliable because it involves direct manipulation of variables rather than relying on pre-existing conditions like Approach 2.
Approach 1 is a controlled study that shows correlation but not causation, while Approach 2 is a natural experiment that can establish causation because it reflects real-world implementation of the teaching method.

Explanation: Approach 1 is an experiment because researchers actively assign treatments (random assignment of teaching methods), which controls for confounding variables and allows for causal inference. Approach 2 is an observational study because researchers observe existing conditions without manipulating variables, making it susceptible to confounding factors (schools that chose the new method may differ systematically from those that didn't). Choice B incorrectly identifies the study types. Choice C wrongly calls both experiments. Choice D incorrectly suggests the observational study can establish causation better than the experiment.

Question 17

A researcher wants to determine whether a new study method improves test scores. She randomly assigns 100 students to either use the new method or continue with their usual study habits for 4 weeks, then compares their exam scores. However, during the study, she discovers that 15 students in the new method group also started attending tutoring sessions, while only 3 students in the control group attended tutoring. What is the most significant threat to the validity of her conclusions about causation?

The study is observational rather than experimental, so no causal conclusions can be drawn regardless of the tutoring issue
The confounding variable of tutoring attendance makes it difficult to isolate the effect of the study method alone (correct answer)
The sample size is too small to establish statistical significance for causal relationships in experimental studies
The random assignment was not properly implemented since students chose whether to attend tutoring sessions

Explanation: This is still an experiment because students were randomly assigned to treatment groups initially. However, the unequal tutoring attendance creates a confounding variable that threatens internal validity. The tutoring could be responsible for score differences rather than the study method. Choice A is wrong because this remains an experiment despite the confounding issue. Choice C is wrong because sample size affects statistical power, not the ability to draw causal conclusions from well-designed experiments. Choice D is wrong because random assignment refers to the initial treatment assignment, not controlling every subsequent behavior.

Question 18

A health researcher finds a strong positive correlation between coffee consumption and heart disease in a large dataset tracking 50,000 adults over 20 years. She notices that participants were not assigned coffee consumption levels but rather reported their natural drinking habits. A colleague argues this study can establish causation because of its large sample size and long duration. Which statement best evaluates this argument?

The argument is correct because large observational studies with long follow-up periods can establish causation when correlations are strong
The argument is incorrect because observational studies cannot control for confounding variables that might explain the coffee-heart disease relationship
The argument is incorrect because the study lacks random assignment, which is necessary for establishing causation regardless of other design features (correct answer)
The argument is correct because the 20-year duration allows researchers to observe the temporal sequence necessary for causal relationships

Explanation: Random assignment is the key feature that distinguishes experiments from observational studies and enables causal conclusions. Without random assignment, we cannot rule out confounding variables, selection bias, or other alternative explanations. Choice A is wrong because sample size and duration alone cannot establish causation in observational studies. Choice B identifies a real problem but is less precise than C about why this prevents causal conclusions. Choice D is wrong because temporal sequence is necessary but not sufficient for causation—confounding variables could still explain the relationship.

Question 19

A sports scientist wants to test whether a new training program improves athletic performance. She randomly assigns 60 athletes to either the new program or standard training for 12 weeks. However, athletes in the new program train at a state-of-the-art facility, while control group athletes use older equipment at a different location. At the end of the study, the new program group shows significantly better performance. What is the most appropriate conclusion?

The new training program causes improved performance, since random assignment controls for all potential confounding variables
No causal conclusion is possible because the study is observational rather than experimental in nature
The training program's effectiveness cannot be determined due to confounding between program type and facility quality (correct answer)
The results demonstrate correlation but not causation because the sample size is insufficient for causal inference

Explanation: While this is an experiment due to random assignment, the confounding of training program with facility quality makes it impossible to determine whether improved performance results from the program itself or the better facilities. This is a design flaw that undermines causal interpretation. Choice A is wrong because random assignment doesn't control for confounding variables introduced by the experimental design itself. Choice B is wrong because random assignment makes this an experiment. Choice D incorrectly focuses on sample size rather than the confounding issue.

Question 20

A university researcher studies academic performance by comparing students who live on campus versus those who commute. She finds that on-campus students have higher GPAs and concludes that campus living improves academic performance. However, she later discovers that students with higher family incomes are more likely to live on campus, and family income correlates with academic resources and support. What would be the best way to strengthen causal conclusions about campus living?

Increase the sample size to reduce sampling variability and improve statistical significance of the correlation
Use statistical controls for family income in the analysis while maintaining the observational study design
Conduct a longitudinal study following students over multiple semesters to establish temporal sequence
Randomly assign incoming students to live on campus or commute, controlling for family income as a potential confounder (correct answer)

Explanation: When you encounter questions about establishing causation versus correlation, remember that the gold standard for proving causal relationships is the randomized controlled experiment. The key issue here is that the researcher has identified a confounding variable—family income—that could explain the observed relationship between campus living and academic performance. Option D is correct because random assignment is the most powerful method for establishing causation. By randomly assigning students to live on campus or commute, you eliminate selection bias and ensure that any differences in GPA are likely due to the living arrangement itself, not pre-existing differences between the groups. Controlling for family income further strengthens this design by accounting for the identified confounder. Let's examine why the other options fall short: Option A misses the point entirely—increasing sample size won't eliminate the confounding variable problem, just make the biased results more precise. Option B uses statistical controls, which is better than nothing, but observational studies with statistical controls are still vulnerable to unmeasured confounders and selection bias. Option C addresses temporal sequence, but timing alone doesn't eliminate confounding—students with higher incomes will still systematically choose campus living regardless of when you measure them. Remember this pattern: when a question asks about strengthening causal conclusions, look for the option that involves random assignment or experimental manipulation. Observational studies, no matter how large or well-controlled statistically, cannot match the causal inference power of a well-designed experiment.