Math 3 Quiz: Sampling Bias
13 questions · exam conditions
0:00
Sampling BiasQuestion 1 of 13

A high school principal wants to assess student satisfaction with the cafeteria food. The school has 1,200 students across grades 9-12, with approximately 300 students per grade level.

The principal implements a sampling strategy where student council members from each grade distribute surveys to their friends and classmates until they collect 75 responses per grade. What is the primary source of bias in this approach, and what would be the most effective single modification to reduce it?

Convenience bias from student council distribution; modify by having teachers randomly distribute surveys in required classes
Volunteer response bias from self-selection; modify by requiring all students to complete the survey during lunch periods
Undercoverage bias from excluding certain groups; modify by increasing the sample size to 100 responses per grade
Measurement bias from unclear questions; modify by having multiple student council members review survey wording
← Back to quizzes

Math 3 Quiz

Math 3 Quiz: Sampling Bias

Practice Sampling Bias in Math 3 with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Sampling Bias, giving you a quick way to practice the rules, question types, and explanations that matter most for Math 3.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

A high school principal wants to assess student satisfaction with the cafeteria food. The school has 1,200 students across grades 9-12, with approximately 300 students per grade level.

The principal implements a sampling strategy where student council members from each grade distribute surveys to their friends and classmates until they collect 75 responses per grade. What is the primary source of bias in this approach, and what would be the most effective single modification to reduce it?

  1. Convenience bias from student council distribution; modify by having teachers randomly distribute surveys in required classes (correct answer)
  2. Volunteer response bias from self-selection; modify by requiring all students to complete the survey during lunch periods
  3. Undercoverage bias from excluding certain groups; modify by increasing the sample size to 100 responses per grade
  4. Measurement bias from unclear questions; modify by having multiple student council members review survey wording
Explanation: The primary bias is convenience bias because student council members will naturally survey people they know and can easily access, creating a non-representative sample that likely overrepresents more engaged, social students. Random distribution through required classes would ensure all types of students have equal chance of selection. Choice B misidentifies the bias type. Choice C incorrectly suggests sample size is the issue. Choice D addresses a different type of bias not mentioned in the scenario.

Question 2

A social media company analyzes user engagement by studying posts that receive comments. They find that 78% of posts with comments contain controversial topics, and conclude that users primarily engage with controversial content. What type of bias does this analysis demonstrate, and how does it affect the validity of their conclusion?

  1. Confirmation bias: The analysis was designed to support a predetermined conclusion about user preferences for controversial content
  2. Observation bias: The act of measuring engagement through comments changes how users interact with controversial posts
  3. Selection bias: The analysis excludes posts without comments, which may include non-controversial content that users engage with differently (correct answer)
  4. Survivorship bias: The analysis only considers posts that generated sustained engagement over time rather than initial reactions
Explanation: When analyzing data bias, you need to identify how the sampling or analysis method might systematically exclude relevant information or misrepresent the population being studied. This scenario demonstrates selection bias because the company only analyzed posts that received comments, creating a skewed sample. By excluding posts without comments, they're missing a huge portion of user behavior. Non-controversial content might generate engagement through likes, shares, or views rather than comments, but this analysis method makes that engagement invisible. The 78% figure becomes meaningless because it only represents a pre-filtered subset of all posts, not the complete picture of what users actually engage with. Choice A is incorrect because confirmation bias involves interpreting evidence to support preexisting beliefs, but there's no indication the company started with a predetermined conclusion. Choice B misidentifies observation bias, which occurs when the act of measurement changes behavior—but measuring comments doesn't alter how users interact with posts. Choice D suggests survivorship bias, but the issue isn't about posts surviving over time; it's about the fundamental exclusion of posts without comments from the very beginning of the analysis. The company's conclusion is invalid because their sample systematically excludes an entire category of content and engagement behavior. You can't make broad claims about user preferences when you've only looked at one slice of the data. Remember: When you see bias questions, always ask "What data is missing?" Selection bias often involves excluding relevant cases that could change the conclusion entirely.

Question 3

A university wants to evaluate the mental health services on campus. The counseling center director proposes surveying students who have used counseling services in the past year to assess satisfaction and identify areas for improvement.

While this approach would provide valuable feedback, what is the most significant limitation for understanding the overall adequacy of mental health services on campus, and what additional sampling strategy would best address this limitation?

  1. Sample size limitations from low service utilization; add focus groups with larger numbers of students to gather more comprehensive feedback
  2. Response bias from students wanting to appear grateful; add anonymous online surveys to encourage more honest feedback from service users
  3. Temporal bias from only recent service users; add surveys of students who used services in previous years to assess long-term outcomes
  4. Limited scope excludes students who need but haven't accessed services; add random sampling of all students to identify barriers to access (correct answer)
Explanation: When evaluating research design and sampling methods, you need to consider whether your sample can actually answer the research question being asked. The key issue here is identifying what population you're trying to understand and whether your sampling method captures the right people. The proposed survey only reaches students who have already accessed counseling services. While this tells us about service quality, it creates a major blind spot: we learn nothing about students who need mental health support but haven't sought it. To truly evaluate the "overall adequacy" of campus mental health services, you must understand both service quality AND accessibility barriers. Answer D correctly identifies this limitation and proposes random sampling of all students to capture those who need but haven't accessed services. Let's examine why the other options miss the mark: Answer A focuses on sample size, but having more feedback from the same limited population (service users) doesn't solve the fundamental scope problem. Answer B addresses response bias, which is a valid concern, but anonymous surveys still only reach existing users, not those who need services. Answer C suggests temporal expansion by including past users, but this still excludes the crucial population of students who have never accessed services at all. The core issue isn't about getting better feedback from current users—it's about reaching non-users who might need help. When you see research methodology questions, always ask: "Does this sampling strategy capture everyone relevant to the research question?" Look for gaps between the target population and the sampled population.

Question 4

A streaming service wants to understand viewer preferences for a new show. They send a survey to 10,000 randomly selected subscribers, but only 1,200 respond. Analysis reveals that 65% of respondents watched the entire first season, leading to a conclusion that the show is highly successful. What is the primary concern with this conclusion, and what information would be most valuable to validate it?

  1. Question wording may influence responses; most valuable information would be results from surveys with different question formats
  2. Sample size is too small for reliable conclusions; most valuable information would be responses from additional random subscribers
  3. Self-selection favors engaged viewers; most valuable information would be demographic comparison between respondents and non-respondents
  4. Low response rate suggests non-response bias; most valuable information would be viewing data from all 10,000 selected subscribers (correct answer)
Explanation: When evaluating survey research, you need to distinguish between different types of bias and identify which poses the greatest threat to valid conclusions. This question tests your understanding of non-response bias and how to address it effectively. The primary issue here is non-response bias caused by an extremely low response rate (12%). When only 1,200 out of 10,000 people respond, you can't assume the respondents represent the entire population. The 8,800 non-respondents likely have different viewing patterns than those who took time to complete a survey about the show. This creates a systematic bias where engaged viewers are overrepresented, making the 65% completion rate misleading for the general subscriber base. Answer D correctly identifies this non-response bias as the main concern and suggests the most valuable solution: actual viewing data from all 10,000 selected subscribers. This objective data would reveal true viewing patterns without bias. Answer A focuses on question wording bias, but there's no indication the questions were poorly constructed. Answer B incorrectly suggests the sample size is too small—1,200 responses can be statistically adequate if representative, but representation is the real problem here. Answer C identifies self-selection bias (which is related) but suggests demographic comparisons as the solution. While demographics might be interesting, they wouldn't directly address whether non-respondents actually watched the show. Remember: Low response rates are red flags for non-response bias. When evaluating survey conclusions, always consider whether non-respondents might systematically differ from respondents in ways that affect your key variable—in this case, viewing behavior.

Question 5

A fitness app company wants to study exercise habits among their users. They send a survey to users who have logged workouts in the past month, asking about their exercise frequency, motivation, and satisfaction with the app.

The company receives responses from 3,200 users and finds that 89% exercise at least 5 times per week. They conclude that their app effectively promotes regular exercise. What is the fundamental flaw in this reasoning, and what comparison would provide stronger evidence for their conclusion?

  1. Survival bias from studying only active users; compare with exercise habits of users before they started using the app (correct answer)
  2. Self-reporting bias from users overestimating their exercise; compare survey responses with objective data from fitness trackers integrated with the app
  3. Selection bias from studying only recent users; compare exercise habits between long-term users and new users of the app
  4. Confirmation bias from asking leading questions; compare results with surveys that use neutral wording about exercise frequency and habits
Explanation: The fundamental flaw is survival bias - they're only surveying users who remained active on the app, which naturally selects for people who exercise regularly. Users who don't exercise regularly likely stopped using the app and aren't in the sample. Comparing current users' habits with their pre-app habits would better demonstrate the app's effectiveness. Choice B addresses measurement accuracy but not the sampling bias. Choice C compares different user types but doesn't address the core selection issue. Choice D assumes question bias not evident in the scenario.

Question 6

A health department wants to estimate vaccination rates in a county. They have access to three potential data sources: electronic health records from major hospitals, a voluntary online survey, and door-to-door interviews with randomly selected households.

If the health department's goal is to minimize bias while maximizing the representativeness of their estimate, which approach should they prioritize and what is the main limitation they should address?

  1. Use electronic health records as they provide objective data; address undercoverage by including smaller clinics and pharmacies
  2. Conduct door-to-door interviews as they ensure random sampling; address response bias by training interviewers to ask neutral questions (correct answer)
  3. Implement the online survey as it reaches the broadest audience; address selection bias by weighting responses by demographic categories
  4. Combine all three sources to maximize coverage; address conflicting results by averaging the estimates from each method
Explanation: Door-to-door interviews with random selection provide the best foundation for representative sampling. The main concern is response bias, where people might give socially desirable answers about vaccination. Proper interviewer training can minimize leading questions and create a neutral environment. Choice A has severe undercoverage issues. Choice C has major self-selection problems that weighting can't fully address. Choice D incorrectly assumes equal validity of biased sources.

Question 7

A state education department wants to assess the effectiveness of a new teaching method by comparing test scores between schools using the new method and schools using traditional methods.

The department allows schools to voluntarily adopt the new teaching method rather than randomly assigning it. Two years later, they find that schools using the new method have significantly higher test scores. What is the most plausible alternative explanation for this result due to the sampling design?

  1. Hawthorne effect: Schools using the new method performed better because they knew they were being studied
  2. Measurement bias: Schools using the new method may have taught specifically to the test used for evaluation
  3. Survivor bias: Only schools that were successful with the new method continued using it throughout the study period
  4. Confounding bias: Schools that chose the new method may have been higher-performing initially due to better resources or leadership (correct answer)
Explanation: When evaluating research studies, you need to carefully consider how the study design might introduce bias that creates alternative explanations for the results. This question tests your ability to identify the most serious threat to validity when participants self-select into treatment groups. The correct answer is D because when schools voluntarily chose the new teaching method, this creates a fundamental confounding problem. Schools with better leadership, more resources, more motivated teachers, or higher-performing students to begin with would be more likely to voluntarily adopt an innovative teaching method. These same factors that led them to choose the new method could also be the real reasons for their higher test scores, not the teaching method itself. This is confounding bias - when a third variable is associated with both the treatment choice and the outcome. Let's examine why the other options are less plausible: A is incorrect because the Hawthorne effect occurs when people change their behavior simply because they know they're being observed, but both groups of schools knew they were being studied. B is wrong because teaching to the test could happen in both groups and doesn't specifically relate to the voluntary adoption design flaw. C misses the mark because survivor bias would require evidence that schools dropped out of using the new method, but the question doesn't suggest this happened. Remember: When you see studies where participants self-select into groups rather than being randomly assigned, always look for confounding bias as the primary threat. Random assignment is crucial because it helps ensure groups are comparable at baseline.

Question 8

A transportation authority wants to study public transit usage patterns. They install survey tablets at bus stops in the downtown area and collect responses from passengers waiting for buses during a one-week period.

This sampling method contains multiple sources of bias. If the transportation authority could make only one change to significantly improve the representativeness of their data, which modification would have the greatest impact?

  1. Extend the data collection period from one week to one month to capture more seasonal variation in ridership
  2. Survey passengers on buses during their trips rather than only those waiting at bus stops
  3. Include bus stops in suburban and residential areas rather than focusing only on the downtown area (correct answer)
  4. Add survey tablets at train stations and other transit hubs to capture multimodal transportation users
Explanation: When evaluating sampling methods, the most critical concern is whether your sample accurately represents the entire population you're trying to study. The biggest threat here is geographic bias — by only surveying downtown bus stops, the transportation authority is missing entire segments of the transit-using population. Choice C addresses this fundamental flaw by expanding beyond downtown to include suburban and residential areas. This single change would capture dramatically different ridership patterns, demographics, and usage needs. Downtown riders might be primarily commuters or tourists, while suburban riders could include students, elderly passengers, or low-income residents who rely on transit differently. Without this broader geographic representation, any conclusions about "public transit usage patterns" would be severely skewed. Let's examine why the other options fall short: Choice A (extending from one week to one month) improves temporal representation but doesn't fix the core geographic limitation — you'd still only understand downtown patterns. Choice B (surveying on buses vs. at stops) might capture slightly different rider perspectives but doesn't expand the geographic scope. Choice D (adding train stations) introduces multimodal data, but if those stations are also downtown-focused, you're still missing the geographic diversity that's most crucial. Study tip: In sampling bias questions, always identify which dimension of the population is most severely underrepresented. Geographic bias often creates the largest gaps because different locations can have fundamentally different characteristics, making it the highest-impact fix when resources are limited.

Question 9

A restaurant chain wants to evaluate customer satisfaction across its 200 locations nationwide. They decide to analyze online reviews from popular review websites rather than conducting their own survey.

What is the most significant bias limitation of using online reviews for this evaluation, and what additional data collection method would best complement this approach to improve validity?

  1. Recency bias from overweighting recent reviews; complement with historical sales data to identify long-term trends in satisfaction
  2. Extremity bias from overrepresenting very satisfied and very dissatisfied customers; complement with brief exit surveys of random customers (correct answer)
  3. Demographic bias from younger, tech-savvy reviewers; complement with phone surveys targeting older customer segments
  4. Geographic bias from urban locations having more reviews; complement with focus groups in rural areas with fewer online reviews
Explanation: Extremity bias is the most significant issue because people are motivated to write reviews primarily when they have very positive or very negative experiences, leaving out the 'average' customer experience that represents the majority. Brief exit surveys of random customers would capture the full spectrum of satisfaction levels, including neutral experiences. Choices A, C, and D identify real but secondary bias issues compared to the fundamental problem of missing moderate opinions.

Question 10

A university researcher wants to study sleep patterns among college students. She randomly selects 50 students from each dormitory and emails them a survey about their sleep habits. However, she receives responses from only 35% of the selected students. To address the potential bias from low response rate, which approach would be most statistically sound?

  1. Increase the initial sample size to 150 students per dormitory to compensate for the low response rate
  2. Compare demographic characteristics of respondents to the known dormitory populations and adjust results accordingly
  3. Follow up with non-respondents using different contact methods and offer incentives for participation (correct answer)
  4. Accept the 35% response rate as adequate since the initial selection was random and representative
Explanation: Following up with non-respondents addresses non-response bias directly by attempting to get responses from those who didn't initially participate, as non-respondents may systematically differ from respondents in their sleep patterns. This approach maintains the integrity of the random sampling design. Choice A doesn't address the bias, just increases sample size. Choice B attempts post-hoc adjustment but doesn't address the core issue. Choice D ignores the serious bias potential from 65% non-response.

Question 11

A market research company is hired to study consumer preferences for a new smartphone app. They conduct their survey by posting it on social media platforms and offering a $5 gift card incentive for completion.

After collecting 2,000 responses, the researchers discover that 78% of respondents are between ages 18-34, while census data shows this age group represents only 42% of smartphone users. Which combination of bias sources most likely contributed to this age skew?

  1. Self-selection bias and coverage bias, because social media users skew younger and people choose whether to respond (correct answer)
  2. Response bias and sampling bias, because younger people are more motivated by incentives and social media
  3. Voluntary response bias and measurement bias, because the survey platform appeals to certain demographics
  4. Selection bias and non-response bias, because older users are less likely to see or complete online surveys
Explanation: Self-selection bias occurs because respondents choose to participate, and coverage bias occurs because the sampling frame (social media users) doesn't represent the target population (all smartphone users). Social media platforms have younger user bases, creating systematic undercoverage of older demographics. Choice B incorrectly categorizes the bias types. Choice C misidentifies measurement bias. Choice D correctly identifies issues but uses less precise terminology for this scenario.

Question 12

A pharmaceutical company is testing a new medication and needs to recruit participants for a clinical trial. They post advertisements in medical clinics, online health forums, and local newspapers asking for volunteers who meet specific health criteria. Which aspect of this recruitment strategy poses the greatest threat to the generalizability of their results?

  1. The use of specific health criteria creates selection bias by excluding relevant patient populations from the study
  2. The volunteer recruitment method creates self-selection bias toward more health-conscious or motivated individuals (correct answer)
  3. The advertising in medical settings creates coverage bias by overrepresenting people who regularly seek medical care
  4. The use of multiple recruitment channels creates response bias by reaching people through different motivational contexts
Explanation: Self-selection bias from volunteer recruitment is the greatest threat because volunteers for medical studies may systematically differ from the general patient population in ways that affect treatment response - they may be more compliant, health-conscious, or motivated. This directly impacts generalizability to real-world patients. Choice A misunderstands that health criteria are necessary for safety and validity. Choice C is a concern but less fundamental than self-selection. Choice D incorrectly identifies multiple channels as problematic.

Question 13

A polling organization wants to estimate support for a local ballot measure. They randomly select 1,000 registered voters and achieve a 45% response rate. The results show 58% support, but the actual vote is only 51% in favor. If nonresponders were systematically different from responders, which scenario most likely explains this discrepancy?

  1. Supporters were more motivated to participate in the poll because they felt strongly about the issue and wanted their voices heard (correct answer)
  2. Opponents were more likely to respond because they wanted to express their dissatisfaction with current local government policies
  3. Undecided voters were more likely to participate because they wanted to learn more about the issue through the survey process
  4. Younger voters were underrepresented in responses and were more likely to oppose the measure than older voters
Explanation: Since the poll overestimated support (58% vs. 51% actual), supporters must have been overrepresented among responders. This occurs when people with strong positive feelings are more motivated to participate in surveys. The other options would either underestimate support (B, D) or wouldn't systematically bias results in either direction (C).