Math 3 Quiz: Random Sampling And Assignment
20 questions · exam conditions
0:00
Random Sampling And AssignmentQuestion 1 of 20

A pharmaceutical company tests a new blood pressure medication by randomly assigning 400 volunteers with hypertension to receive either the new drug or a placebo. After 12 weeks, the new drug group shows significantly lower blood pressure. However, all volunteers were recruited from cardiology clinics in urban areas. What can the company legitimately conclude?

The new medication causes blood pressure reduction in patients with hypertension, and these results apply to all hypertensive patients regardless of geographic location or treatment setting.
The new medication is associated with blood pressure reduction in the study sample, but causation cannot be established without proper randomization to treatment groups.
The new medication likely causes blood pressure reduction, but generalization is limited to hypertensive patients similar to those in urban cardiology clinics.
The new medication causes blood pressure reduction, and results can be generalized to all patients receiving treatment in urban cardiology clinic settings.
← Back to quizzes

Math 3 Quiz

Math 3 Quiz: Random Sampling And Assignment

Practice Random Sampling And Assignment in Math 3 with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Random Sampling And Assignment, giving you a quick way to practice the rules, question types, and explanations that matter most for Math 3.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

A pharmaceutical company tests a new blood pressure medication by randomly assigning 400 volunteers with hypertension to receive either the new drug or a placebo. After 12 weeks, the new drug group shows significantly lower blood pressure. However, all volunteers were recruited from cardiology clinics in urban areas. What can the company legitimately conclude?

  1. The new medication causes blood pressure reduction in patients with hypertension, and these results apply to all hypertensive patients regardless of geographic location or treatment setting.
  2. The new medication is associated with blood pressure reduction in the study sample, but causation cannot be established without proper randomization to treatment groups.
  3. The new medication likely causes blood pressure reduction, but generalization is limited to hypertensive patients similar to those in urban cardiology clinics. (correct answer)
  4. The new medication causes blood pressure reduction, and results can be generalized to all patients receiving treatment in urban cardiology clinic settings.
Explanation: Random assignment was used, allowing causal conclusions about the medication's effect. However, the sample was not randomly selected from all hypertensive patients - it came from a specific population (urban cardiology clinic patients) which may differ from rural patients, those receiving primary care, or those with different severity levels. Choice A overgeneralizes beyond the sampled population. Choice B incorrectly suggests randomization wasn't used. Choice D is too narrow - results could generalize to similar patients, not just those in urban clinics.

Question 2

An online retailer wants to estimate customer satisfaction across their entire customer base of 2 million people. They send email surveys to a random sample of 10,000 customers and receive 1,200 responses. The results show high satisfaction ratings. What is the most appropriate interpretation of these results?

  1. The high satisfaction ratings are representative of all 2 million customers because the initial sample was selected using proper random sampling methods from the complete customer list.
  2. The results may overestimate satisfaction for the entire customer base because customers who respond to surveys may be systematically different from non-respondents in their satisfaction levels. (correct answer)
  3. The sample size of 1,200 is too small relative to the population of 2 million customers to draw any meaningful conclusions about overall customer satisfaction levels.
  4. The results are valid for the 1,200 respondents but cannot be generalized because the company failed to use stratified sampling to ensure representation across customer segments.
Explanation: Even though the initial sample was properly randomized, only 12% responded (1,200/10,000). This creates nonresponse bias - customers who take time to respond to satisfaction surveys may be systematically different from those who don't, potentially being more engaged or satisfied. Choice A ignores the nonresponse issue. Choice C incorrectly focuses on absolute sample size rather than response rate and bias. Choice D is wrong because random sampling was used, and stratification isn't always necessary.

Question 3

A researcher studying the effectiveness of two different math curricula has access to 10 elementary schools. She randomly assigns 5 schools to use Curriculum A and 5 schools to use Curriculum B for one academic year. At the end of the year, she compares average test scores between the two groups. What potential confounding factor is most likely to threaten the validity of her causal conclusions?

  1. Teacher quality and experience may vary systematically between schools, creating differences in instruction effectiveness beyond the curriculum itself that could affect student outcomes.
  2. Students were not randomly assigned to schools, so pre-existing differences in student ability between school populations could account for observed differences in test scores. (correct answer)
  3. The sample size of 10 schools is too small to detect meaningful differences between curricula, leading to insufficient statistical power for valid conclusions.
  4. Seasonal variations in student motivation and attention may differentially affect schools using different curricula, depending on when each curriculum covers challenging topics.
Explanation: While schools were randomly assigned to curricula, students were not randomly assigned to schools. If schools serve different populations (different socioeconomic areas, academic backgrounds, etc.), pre-existing student differences could confound the curriculum comparison. Choice A identifies a real concern but teacher effects are somewhat controlled by random assignment of schools. Choice C addresses statistical power, not confounding. Choice D is less plausible since both curricula operate under the same seasonal conditions.

Question 4

A market research firm wants to estimate smartphone usage patterns among teenagers in a large metropolitan area. They obtain permission to survey students at 15 randomly selected high schools from the 200 schools in the area. At each selected school, they survey all students in 5 randomly chosen classes. This yields 2,250 student responses. What is the most significant limitation for generalizing these results?

  1. The cluster sampling design produces higher sampling variability than simple random sampling, reducing precision of population estimates.
  2. The sample size of 2,250 may be insufficient to detect important subgroup differences in smartphone usage across demographic segments.
  3. The multistage sampling approach introduces multiple sources of sampling error that compound to create less reliable population estimates.
  4. Students attending high school may not represent all teenagers, excluding those who are homeschooled, dropped out, or attend private schools. (correct answer)
Explanation: When evaluating sampling methods, you need to distinguish between technical sampling concerns and fundamental coverage issues that affect who can even be included in your sample. The correct answer is D because this study has a serious coverage problem. The sampling frame (high school students) systematically excludes entire groups of teenagers: those who are homeschooled, have dropped out, attend private schools, or aren't enrolled in traditional schools for other reasons. No matter how well the sampling is executed within high schools, these excluded teenagers might have very different smartphone usage patterns. This creates a fundamental bias that can't be fixed by better sampling techniques. Let's examine why the other options miss the mark: A is incorrect because while cluster sampling does increase sampling variability compared to simple random sampling, this is a manageable technical issue that affects precision, not the fundamental ability to generalize to all teenagers. B is wrong because 2,250 is actually a substantial sample size for most research purposes. Sample size adequacy depends on the research goals, but this size would typically provide sufficient power for detecting meaningful differences. C misunderstands multistage sampling. While it does introduce multiple sources of sampling error, this is a standard, well-understood statistical technique. The errors don't simply "compound" in a problematic way—they can be properly accounted for in the analysis. Study tip: Always distinguish between sampling precision issues (which affect how accurate your estimates are for your target population) and coverage issues (which affect whether you're even studying the right population).

Question 5

A university researcher obtains a complete list of all registered students and uses a random number generator to select 500 students for a study about sleep habits. However, due to budget constraints, she can only interview students who live on campus. Of the 500 selected students, 180 live on campus and complete the study. What type of sampling issue does this create?

  1. Systematic sampling bias occurs because the researcher used a non-random method to generate the initial list of 500 students from the population.
  2. Voluntary response bias occurs because students could choose whether or not to participate based on their interest in the sleep study topic.
  3. Undercoverage bias occurs because the final sample systematically excludes off-campus students, who may have different sleep patterns than on-campus students. (correct answer)
  4. Response bias occurs because students living on campus may not provide truthful answers about their sleep habits due to social desirability concerns.
Explanation: The initial random selection was valid, but the practical constraint of only interviewing on-campus students creates undercoverage bias. Off-campus students (who were part of the target population) are systematically excluded from the final sample, and they likely have different sleep patterns due to different living situations, commute times, etc. Choice A is wrong because the initial selection was properly random. Choice B is incorrect because students didn't self-select - they were excluded by the researcher's constraint. Choice D addresses data quality, not sampling methodology.

Question 6

A psychology professor wants to test whether background music affects test performance. She has two sections of the same course. Section A (morning class) takes their exam with classical music playing, while Section B (afternoon class) takes the exam in silence. Section A scores significantly higher. The professor concludes that classical music improves test performance. What is the most serious flaw in this conclusion?

  1. The study lacks random assignment to music conditions, so differences might be due to systematic differences between morning and afternoon students. (correct answer)
  2. The study uses a convenience sample from only two class sections rather than randomly sampling from the broader college population.
  3. The study fails to control for environmental factors such as room temperature, lighting, or noise levels between testing sessions.
  4. The study lacks a control condition where both sections take exams without music, making baseline performance comparisons impossible.
Explanation: Students self-selected into morning vs afternoon sections, creating systematic differences. Morning students might be more motivated, have different sleep patterns, be different personality types, or have different academic backgrounds than afternoon students. This confounding prevents causal attribution to music. Choice B addresses generalizability but not the internal validity problem. Choice C identifies potential confounders but they're less systematic than the time-of-day selection bias. Choice D misunderstands experimental design - one group serves as the control.

Question 7

A health researcher wants to determine if a new exercise program prevents heart disease. She identifies 1,000 healthy adults and randomly assigns 500 to follow the exercise program for 5 years while 500 continue their usual activities. After 5 years, she finds that 3% of the exercise group developed heart disease compared to 7% of the control group. What additional information would most strengthen the causal interpretation of these results?

  1. Evidence that participants in both groups had similar baseline characteristics after randomization, confirming the absence of confounding variables. (correct answer)
  2. Documentation that the exercise group consistently followed the prescribed program throughout the study period rather than abandoning it early.
  3. Confirmation that the 1,000 participants were randomly selected from a larger population to ensure generalizability of results.
  4. Data showing that both groups had equal healthcare access and similar medical monitoring rates throughout the study period.
Explanation: Random assignment should create balanced groups, but verifying that it actually worked strengthens causal inference by confirming no confounding variables differ between groups. This is especially important with only 1,000 participants. Choice B addresses compliance but the results already show a difference suggesting the program had some effect. Choice C addresses generalizability, not causation within this study. Choice D addresses a potential confounder but baseline balance is more fundamental to the randomization's success.

Question 8

A medical researcher studies whether a new rehabilitation program helps stroke patients recover motor function. She recruits 120 recent stroke patients from three hospitals. At Hospital A, patients receive the new program; at Hospital B, patients receive standard care; at Hospital C, patients receive both programs combined. After 6 months, Hospital A patients show the most improvement. What prevents a valid causal conclusion about the new program's effectiveness?

  1. The study lacks random assignment of patients to treatment conditions, as hospital location determines treatment rather than random allocation. (correct answer)
  2. The study uses a convenience sample from only three hospitals rather than random sampling from the broader stroke patient population.
  3. The study includes too many treatment conditions for valid comparison, preventing clear determination of which specific elements are effective.
  4. The study fails to establish a proper control group since all patients receive some rehabilitation treatment rather than no treatment.
Explanation: Patients weren't randomly assigned to treatments - their hospital determined their treatment. Hospitals likely differ in patient populations (socioeconomic status, stroke severity, other medical conditions), staff expertise, facilities, and care quality. These systematic differences between hospitals confound the treatment comparison. Choice B addresses generalizability, not internal validity for causal inference. Choice C is wrong because multiple treatment comparisons are valid research designs. Choice D misunderstands that comparing different treatments can establish relative causal effects.

Question 9

Dr. Martinez wants to test whether meditation reduces anxiety levels. She recruits volunteers from a wellness center, measures their baseline anxiety, then has all participants complete a 6-week meditation program. She finds that anxiety levels decreased significantly from baseline. Her colleague suggests the study design has limitations. What is the most significant limitation?

  1. The study lacks random assignment to treatment and control groups, preventing determination of whether meditation specifically caused the anxiety reduction. (correct answer)
  2. The study uses a convenience sample from a wellness center rather than random sampling, limiting generalizability to the broader population of anxious individuals.
  3. The study measures anxiety at only two time points, providing insufficient data to determine whether the meditation effects are sustained over time.
  4. The study fails to control for seasonal effects and other temporal factors that might naturally reduce anxiety levels during the study period.
Explanation: This is a single-group pre-post design without a control group or random assignment. All participants received meditation, so there's no way to determine if the improvement was due to meditation specifically versus other factors like attention, expectation, or natural fluctuation in anxiety. Choice B identifies a real limitation but generalizability is secondary to establishing whether the treatment works at all. Choices C and D identify minor design issues but miss the fundamental lack of experimental control.

Question 10

A researcher wants to determine if a new teaching method improves student performance compared to traditional methods. She randomly selects 200 students from all students in the district, then randomly assigns 100 to receive the new method and 100 to receive traditional instruction. After the study, she finds that students using the new method scored significantly higher. What conclusion is most appropriate?

  1. The new teaching method causes improved performance for all students in the district, and this conclusion can be generalized to other districts with similar demographics.
  2. The new teaching method likely causes improved performance, and results can be generalized to all students in the district from which the sample was drawn. (correct answer)
  3. There is an association between the new teaching method and improved performance, but causation cannot be established without random assignment to treatment groups.
  4. The new teaching method causes improved performance for the sample studied, but results cannot be generalized beyond this specific group of students.
Explanation: This study uses both random sampling (from the district population) and random assignment (to treatment groups). Random sampling allows generalization to the district population, while random assignment allows causal conclusions about the treatment effect. Choice A overgeneralizes beyond the sampled population. Choice C incorrectly states that random assignment wasn't used. Choice D fails to recognize that random sampling enables generalization.

Question 11

A school district wants to assess student satisfaction with the new lunch program. They randomly select 20 schools from their 100 schools, then survey all students in those selected schools. This approach yields responses from 8,000 students. What sampling method is being used, and what is its primary advantage over simple random sampling?

  1. Stratified sampling is being used, which ensures representation from different geographic areas and school types within the district compared to simple random sampling.
  2. Systematic sampling is being used, which reduces selection bias by following a predetermined pattern rather than relying purely on random chance like simple random sampling.
  3. Cluster sampling is being used, which is more practical and cost-effective than simple random sampling when the population is geographically dispersed across schools. (correct answer)
  4. Convenience sampling is being used, which allows for larger sample sizes and faster data collection compared to the time-intensive process of simple random sampling.
Explanation: This is cluster sampling - schools are the clusters, and entire clusters (all students in selected schools) are surveyed. The main advantage is practicality and cost savings since researchers can visit 20 schools instead of traveling to potentially 100 different schools to reach individually selected students. Choice A describes stratified sampling, which would divide schools into groups first. Choice B describes systematic sampling, which would use a pattern like every 5th student. Choice D is wrong because this uses random selection, not convenience.

Question 12

A social media company conducts a study by posting an online survey asking users to rate their satisfaction with a new feature. The survey receives 50,000 responses. Based on these responses, the company concludes that users generally love the new feature. What is the primary limitation of this approach for drawing valid conclusions?

  1. The sample size is too small to detect meaningful differences in user satisfaction across different demographic groups within the user base.
  2. The lack of random sampling creates selection bias, as only users motivated to respond participated, potentially overrepresenting satisfied users. (correct answer)
  3. The study lacks a control group of users who didn't receive the new feature, preventing comparison of satisfaction levels between groups.
  4. The company failed to use random assignment to determine which users received the new feature, limiting their ability to establish causation.
Explanation: This is a voluntary response sample, not a random sample. Users who choose to respond to surveys about new features are likely those with strong opinions (often positive for new features they chose to use). This creates selection bias that limits generalizability. Choice A is incorrect because 50,000 is a large sample. Choice C describes an experimental design issue but this is observational research about existing satisfaction. Choice D is irrelevant since this isn't testing causation but measuring current satisfaction levels.

Question 13

A city planning department wants to survey residents about a proposed park development. The city has 50 neighborhoods with varying income levels. Planners randomly select 10 high-income, 10 middle-income, and 10 low-income neighborhoods, then survey 20 randomly chosen residents from each selected neighborhood. What sampling design is being used and what is its main benefit?

  1. Multistage sampling combining stratified and cluster methods, ensuring representation across income levels while maintaining cost efficiency by concentrating data collection within selected neighborhoods. (correct answer)
  2. Systematic stratified sampling that reduces selection bias by using predetermined income categories and fixed sample sizes rather than relying on purely random selection methods.
  3. Proportional cluster sampling that maintains the natural income distribution of the city while achieving practical benefits of surveying complete neighborhood units.
  4. Complex random sampling that combines multiple randomization stages to eliminate both sampling bias and nonresponse bias across different socioeconomic population segments.
Explanation: This is multistage sampling: first stage stratifies by income and randomly selects neighborhoods within each stratum, second stage randomly selects residents within chosen neighborhoods. This ensures representation across income levels (stratification benefit) while being practical since surveyors only visit 30 neighborhoods instead of potentially all 50 (clustering benefit). Choice B incorrectly describes it as systematic. Choice C is wrong because equal sample sizes from each income stratum isn't proportional to city demographics. Choice D incorrectly claims it eliminates nonresponse bias.

Question 14

Dr. Kim wants to test whether a new therapy reduces depression scores. She has 80 volunteers with depression. She assigns the first 40 volunteers who signed up to the new therapy group and the remaining 40 to a control group receiving standard treatment. After 8 weeks, the new therapy group shows significantly lower depression scores. What threatens the validity of concluding that the new therapy caused the improvement?

  1. The study lacks sufficient statistical power due to small sample sizes in each group, making it difficult to detect true differences between the therapy approaches.
  2. The study lacks a true control group since both groups received treatment, preventing determination of whether either therapy is effective compared to no treatment.
  3. The study duration of 8 weeks is too short to determine whether the new therapy has lasting effects compared to standard treatment approaches.
  4. The assignment method may have created systematic differences between groups, as volunteers who signed up earlier might differ in motivation or other characteristics from later volunteers. (correct answer)
Explanation: When evaluating research studies, you need to examine whether the study design allows you to confidently attribute observed effects to the intended cause. The biggest threat here is how participants were assigned to groups. Dr. Kim's assignment method—giving the new therapy to the first 40 volunteers and standard treatment to the remaining 40—creates a serious confounding variable problem. People who sign up early for research studies often differ systematically from those who sign up later. Early volunteers might be more motivated, have different severity of symptoms, better access to healthcare, or different socioeconomic backgrounds. These pre-existing differences, not the therapy itself, could explain why the new therapy group improved more. Looking at the other options: Choice A is incorrect because 40 participants per group actually provides reasonable statistical power for detecting meaningful differences. Choice B misunderstands the study design—having a standard treatment control group is perfectly valid for comparing therapies; you don't need a "no treatment" group to test whether one therapy works better than another. Choice C focuses on follow-up duration, but the question asks about validity of the causal conclusion, not about long-term effectiveness. The correct answer is D because non-random assignment based on sign-up order threatens internal validity by potentially creating systematic baseline differences between groups. Study tip: In research design questions, always check the assignment method first. Random assignment is crucial for causal conclusions because it's the only way to ensure groups are equivalent at baseline.

Question 15

A national polling organization wants to estimate public opinion on a controversial policy issue. They use the following sampling procedure:

  1. Divide the country into 4 regions (Northeast, South, Midwest, West)

  2. Randomly select 25 counties from each region (100 counties total)

  3. Within each selected county, randomly select 50 registered voters

  4. Contact the 5,000 selected voters by phone

  5. Obtain responses from 1,800 voters (36% response rate)

Based on the sampling procedure described above, which statement about the validity of conclusions is most accurate?

  1. The stratified multistage design ensures representative results for registered voters nationally, despite the moderate response rate affecting precision.
  2. The complex sampling design introduces multiple error sources that compromise validity, making conclusions unreliable regardless of randomization.
  3. The random selection produces valid results for registered voters, but conclusions cannot be generalized to the broader adult population.
  4. The systematic selection process eliminates sampling bias, but the low response rate creates nonresponse bias affecting representativeness. (correct answer)
Explanation: When evaluating survey validity, you need to assess both sampling design quality and potential bias sources. This question tests your ability to identify how different aspects of data collection affect the reliability of conclusions. The sampling procedure described uses systematic random selection at multiple stages: regions are stratified, counties are randomly selected within regions, and voters are randomly selected within counties. This methodical approach effectively eliminates sampling bias because every registered voter has a known, non-zero probability of selection. However, the 36% response rate creates a significant concern—nonresponse bias occurs when people who respond differ systematically from those who don't, potentially skewing results away from the true population values. Looking at the wrong answers: Choice A incorrectly suggests the design ensures representativeness despite nonresponse—but low response rates can severely compromise representativeness regardless of good initial design. Choice B overstates the problem by claiming the sampling design itself compromises validity, when random selection actually strengthens it. Choice C correctly identifies the limitation to registered voters but misses the more critical issue of nonresponse bias affecting even that population. Choice D correctly identifies both strengths and weaknesses: the systematic random selection does eliminate sampling bias, but the low response rate creates nonresponse bias that threatens representativeness. Study tip: For survey methodology questions, evaluate sampling design and response rates separately. Good random sampling eliminates selection bias, but poor response rates can still undermine validity through nonresponse bias—both factors matter for drawing reliable conclusions.

Question 16

A researcher wants to estimate the average amount of time high school students spend on homework per night. She obtains a list of all students at Roosevelt High School, assigns each student a number, and uses a random number generator to select 150 students for her survey. However, she finds that students who participate in after-school sports are systematically underrepresented in her responses. What is the most likely explanation and primary concern?

  1. The sampling method was flawed because it didn't ensure equal representation of all student subgroups, limiting generalizability to the school population
  2. Random sampling was properly implemented, but nonresponse bias may affect the validity of conclusions about the entire school population (correct answer)
  3. The sample size is too small to make valid inferences, regardless of the sampling method used in the study
  4. Random assignment was not used, so causal relationships between sports participation and homework time cannot be established reliably
Explanation: The researcher used proper random sampling from the school population. The underrepresentation of student athletes is likely due to nonresponse bias (athletes may be less likely to respond), not sampling bias. This threatens the validity of conclusions about the entire school population, even though the sampling method was correct.

Question 17

Dr. Martinez conducts a study on the effects of background music on concentration. She recruits 120 college students who volunteer to participate. Upon arrival at the lab, she flips a coin for each student to determine whether they will complete concentration tasks in silence or with classical music playing.

Based on this study design, which limitation most significantly affects the types of conclusions Dr. Martinez can reasonably draw?

  1. The lack of random sampling prevents her from making causal claims about the effect of music on concentration
  2. The absence of random assignment limits her ability to generalize findings to the broader college student population
  3. Random sampling was not used, so generalization beyond the volunteer participants may not be valid or reliable (correct answer)
  4. The study lacks both random sampling and random assignment, preventing both causal inference and population generalization
Explanation: Dr. Martinez used random assignment (coin flip), which allows for causal conclusions about music's effect on concentration. However, she used a convenience sample of volunteers rather than random sampling from the population, which limits generalizability to the broader college student population. Random assignment enables causal inference, but random sampling is needed for generalization.

Question 18

A pharmaceutical company wants to test a new blood pressure medication. They randomly select 500 participants from a database of patients with hypertension across multiple medical centers. These participants are then randomly divided into two groups: 250 receive the new medication and 250 receive a placebo. Which statement best describes the roles of the two types of randomization used?

  1. Random sampling enables causal conclusions about drug effectiveness, while random assignment allows generalization to the hypertensive patient population
  2. Random assignment enables causal conclusions about drug effectiveness, while random sampling allows generalization to the hypertensive patient population (correct answer)
  3. Both random sampling and random assignment work together to enable causal conclusions, but neither supports population generalization
  4. Random sampling and random assignment serve identical purposes in this study design and can be used interchangeably
Explanation: Random assignment (randomly dividing participants into treatment and placebo groups) enables causal conclusions by ensuring groups are equivalent except for the treatment. Random sampling (selecting participants from the patient database) enables generalization to the broader population of hypertensive patients. These serve distinct and complementary purposes.

Question 19

A psychology professor wants to study whether taking notes by hand versus laptop affects exam performance. She teaches two sections of the same course. In Section A (morning class), all students are required to take notes by hand. In Section B (afternoon class), all students must use laptops for note-taking. She compares average exam scores between the two sections. What is the primary threat to drawing valid causal conclusions from this study design?

  1. Confounding variables associated with class timing and student characteristics may influence the observed differences in exam scores (correct answer)
  2. The lack of random sampling from a broader student population limits the generalizability of any causal findings
  3. The sample sizes of the two class sections are likely too small to detect meaningful differences in exam performance
  4. Random assignment was not implemented, but this does not affect causal conclusions when comparing intact groups
Explanation: When evaluating research designs for causal conclusions, you need to identify what could create alternative explanations for any observed differences between groups. The gold standard is random assignment, which helps ensure groups are equivalent except for the treatment. In this study, the professor assigned note-taking methods based on class sections rather than randomly assigning students to conditions. This creates a major problem: students who choose morning versus afternoon classes may differ systematically in ways that affect exam performance. Morning students might be more organized, have fewer work conflicts, or differ in motivation levels. Additionally, factors like alertness, hunger, or competing afternoon activities could influence performance regardless of note-taking method. These confounding variables make it impossible to determine whether any score differences result from the note-taking method or from pre-existing differences between the groups. Option B incorrectly focuses on external validity (generalizability) rather than internal validity (causal conclusions). While random sampling does affect generalizability, the question asks specifically about threats to causal conclusions. Option C assumes sample size is the issue, but even large samples can't fix confounding variables. Option D is completely wrong—random assignment is crucial for causal inference, and its absence is precisely the problem here. Study tip: When analyzing research designs, always ask "What else could explain the results?" If groups weren't randomly assigned, look for systematic differences between them that could serve as alternative explanations. Confounding variables are the primary threat to causal conclusions in quasi-experimental designs like this one.

Question 20

A marketing research company wants to estimate the percentage of adults nationwide who shop online at least once per week. They use random digit dialing to contact potential participants. However, they only conduct calls between 9 AM and 5 PM on weekdays. After collecting responses from 1,200 adults, they find that 78% shop online weekly.

Which aspect of this study design most likely affects the validity of the 78% estimate for the entire adult population?

  1. The random digit dialing method fails to achieve true random sampling of the target population
  2. The sample size of 1,200 adults is inadequate for making reliable population estimates with acceptable precision
  3. The lack of random assignment prevents establishing causal relationships about online shopping behavior patterns
  4. The calling schedule systematically excludes certain demographic groups, introducing coverage bias into the sample (correct answer)
Explanation: When evaluating survey methodology, you need to assess whether the sampling process gives every member of the target population a fair chance of being included. The key issue here isn't the sampling method itself, but rather systematic barriers that prevent certain groups from participating. The correct answer is D because calling only during 9 AM-5 PM weekdays creates a systematic bias. This schedule excludes people who work traditional daytime hours and can't answer calls during business hours. Since employment status, work schedules, and availability during weekday business hours correlate with demographic factors like age, income, and occupation, this timing systematically underrepresents entire population segments. This is coverage bias—when your sampling frame fails to adequately represent the target population. Option A is incorrect because random digit dialing actually does provide a reasonably random sampling method for reaching adults with phone access. Option B misses the mark—1,200 is typically an adequate sample size for population estimates with acceptable margins of error (usually around ±3%). Option C confuses study design types; this is an observational study estimating a population parameter, not an experiment trying to establish causation, so random assignment isn't relevant or expected. Remember: In survey methodology questions, always check for systematic exclusions in the data collection process. Coverage bias occurs when certain groups are systematically less likely to be included in your sample, and timing restrictions are a common source of this bias in phone surveys.