AP Statistics Quiz: Potential Problems With Sampling

What this quiz covers

This quiz focuses on Potential Problems With Sampling, giving you a quick way to practice the rules, question types, and explanations that matter most for AP Statistics.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

Question 1

1. Potential bias from a systematic sample if the ordering is related to support (for example, last name correlated with neighborhood or ethnicity) (correct answer)
2. Voluntary response bias because voters chose whether to be selected
3. Response bias because bond measures are too complex to ask about
4. Nonresponse bias because selecting every 20th name guarantees refusals
5. Sampling variability because systematic sampling always has zero variability

Explanation: Potential bias in systematic sampling is the focus of this AP Statistics question. The clerk sorted voters alphabetically by last name and selected every 20th, assuming representativeness for support of a bond measure among all registered voters. This threatens validity if the ordering correlates with opinions, like last names linked to ethnic or neighborhood groups that differ in support. Voluntary response bias (choice B) is a distractor, as selection was systematic, not self-chosen. Mini-lesson: when diagnosing sampling issues, check for patterns in the list that systematic methods might exploit; random selection avoids such biases by equalizing chances.

Question 2

1. Convenience sampling that overrepresents commuters who use that station at that time (correct answer)
2. Nonresponse bias because every 5th rider was required to answer
3. Measurement error because commute times should have been recorded with a stopwatch
4. Voluntary response bias because only people with strong opinions participated
5. Simple random sampling because every commuter had an equal chance to be selected

Explanation: In AP Statistics, this question assesses the ability to spot potential sampling issues like convenience sampling. The surveyors selected commuters at one specific subway station during a narrow morning window on a weekday, using a systematic approach of every 5th rider but limiting the frame to that location and time. This threatens validity because it overrepresents downtown subway users during rush hour, who may have different commute times than bus riders, evening commuters, or those using other stations. A distractor is voluntary response bias (choice D), which doesn't apply since participants were approached rather than self-selecting. A mini-lesson on diagnosing sampling problems: evaluate if the sampling process gives all population members an equal chance or if convenience leads to undercoverage of subgroups, as seen here where non-subway or off-peak commuters are excluded.

Question 3

1. Nonresponse bias because some early-arriving students might skip the survey
2. Undercoverage because students who do not attend early tutoring are not represented (correct answer)
3. Response bias because students might exaggerate their sleep to impress the principal
4. Sampling variability because a sample mean is always different from the population mean
5. Measurement error because sleep must be measured with a device, not a survey

Explanation: In AP Statistics, this question tests the ability to spot potential sampling issues, particularly when the sample doesn't cover the entire population of interest. The principal surveys only students arriving early for a 7:00 a.m. tutoring program, aiming to estimate average sleep for all high school students. This method introduces undercoverage because students not attending early tutoring—perhaps those who sleep later or have different schedules—are not included, potentially biasing the results toward students with specific habits. For instance, early risers might report more sleep, unrepresentative of the whole school. A distractor like nonresponse bias (choice A) might seem plausible, but the core problem is the incomplete coverage of the population, not just skipping the survey. A mini-lesson on diagnosing sampling problems: Evaluate if the sampling method systematically omits subgroups, and consider how that affects generalizability to the population parameter. Recognizing undercoverage promotes better sampling designs for accurate inferences.

Question 4

1. Nonresponse bias because some viewers did not own phones capable of texting
2. Voluntary response bias because people with strong opinions are more likely to text (correct answer)
3. Measurement error because YES/NO responses should be on a 1–5 scale
4. Confounding because tax support depends on income
5. Random assignment bias because viewers were not randomly assigned to watch the broadcast

Explanation: This question tests recognition of voluntary response bias in media polls. The news station wants to estimate support among ALL county adults but relies on viewers who choose to text during the broadcast. This creates strong voluntary response bias because people with passionate opinions (either strongly for or against the tax) are much more motivated to take action and text their response. Additionally, the sample is limited to people watching that particular news broadcast who also have texting capability and choose to participate. Even with 8,500 responses, this self-selected sample likely overrepresents extreme views and underrepresents moderate or indifferent adults. Media call-in or text-in polls are classic examples of voluntary response bias - they measure the opinions of motivated responders, not the general population. To diagnose this issue, ask whether participation requires extra effort that might correlate with the variable being measured.

Question 5

1. Undercoverage because adults who did not visit a county-run clinic are not in the sampling frame (correct answer)
2. Nonresponse bias because the department used phone calls instead of texts
3. Voluntary response bias because the sample was not randomly chosen
4. Response bias because vaccination status should be measured by a blood test
5. Randomization error because a simple random sample cannot be used for proportions

Explanation: Identifying undercoverage as a sampling problem is key in this AP Statistics scenario. The health department sampled randomly from adults who visited county clinics in the past year, then called them to ask about flu vaccines, reporting results for all county adults. This method's validity is threatened because non-clinic visitors are completely excluded from the sampling frame, and they might have lower vaccination rates due to less health engagement. A distractor is voluntary response bias (choice C), but the sample was randomly chosen, not self-selected. A mini-lesson: to diagnose undercoverage, ensure the sampling frame covers the entire population; here, restricting to clinic visitors misses large segments, leading to biased estimates.

Question 6

1. Voluntary response bias because the sample was drawn from a random list
2. Nonresponse bias because many selected students did not respond and may differ from responders (correct answer)
3. Undercoverage because students who eat off campus cannot be surveyed
4. Response bias because satisfaction cannot be measured with a survey
5. Sampling variability because a sample size of 620 is always too small

Explanation: This AP Statistics question focuses on potential problems with sampling, particularly nonresponse bias. The university randomly selected 3,000 undergraduates and emailed surveys, but only 620 responded despite reminders, and they used just those responses to estimate satisfaction for all undergrads. Validity is threatened because nonrespondents might differ systematically—perhaps less satisfied or busier students ignored the survey, biasing the results. Undercoverage (choice C) is a distractor, but the issue isn't the sampling frame excluding off-campus eaters; it's the low response rate. For diagnosing sampling problems, check response rates and consider if nonrespondents could have different views, as high nonresponse often indicates bias in who actually provides data.

Question 7

1. Response bias because employees may not answer honestly when managers can identify them (correct answer)
2. Undercoverage because only employees with managers were surveyed
3. Voluntary response bias because employees chose whether to be in the sample
4. Nonresponse bias because a simple random sample guarantees everyone responds
5. Sampling bias because the sample size of 200 is too large

Explanation: This question in AP Statistics examines response bias as a potential sampling issue. The company took a simple random sample of 200 employees but conducted surveys in manager-led meetings where responses weren't anonymous, potentially causing employees to inflate satisfaction scores out of fear. Validity is threatened because the lack of privacy may lead to dishonest answers, not reflecting true job satisfaction across all employees. Undercoverage (choice B) distracts by misapplying the concept, as all employees were in the frame regardless of having managers. Diagnosing sampling problems involves checking not just selection but also how data is collected; here, the non-anonymous setting introduces response bias by influencing honesty.

Question 8

1. Voluntary response bias because customers self-select into responding, possibly based on strong experiences (correct answer)
2. Undercoverage because every customer receives a receipt
3. Nonresponse bias because a random sample was selected but many refused
4. Measurement error because recommendations should be measured by tracking repeat visits
5. Sampling variability because incentives eliminate bias

Explanation: Voluntary response bias is the core sampling problem tested in this AP Statistics question. The restaurant invited all customers via receipt QR codes to take a survey for a gift card chance, then treated responses as representative of all customers' recommendation rates. This threatens validity because self-selecting participants, even with incentives, often have stronger opinions—dissatisfied customers might be more motivated to respond. Undercoverage (choice B) is a distractor since receipts reach everyone, but the issue is who chooses to participate. A mini-lesson on diagnosis: look for self-selection in sampling processes, as it biases results toward extremes, unlike random methods that aim for representativeness.

Question 9

1. Convenience sampling that likely overrepresents more frequent users who are at the gym on that day (correct answer)
2. Nonresponse bias because Saturday visitors are unlikely to answer questions
3. Voluntary response bias because the manager randomly selected members from a list
4. Response bias because the manager should have asked for exact dates of visits
5. Undercoverage because only members with membership cards can be surveyed

Explanation: Convenience sampling as a problem is central to this AP Statistics question. The gym manager surveyed only members present on a Saturday afternoon, using their reported visits to estimate the mean for all members. This method's validity is threatened because it overrepresents frequent or weekend users, who likely visit more often than average members. Voluntary response bias (choice C) distracts, but the manager approached people, not relying on self-selection. For diagnosing sampling problems, assess if accessibility drives selection; convenience often biases toward certain behaviors, as seen with gym attendance here.

Question 10

1. Undercoverage because parents without reliable internet access are less likely to be included
2. Nonresponse bias because some selected parents refused to answer after being contacted
3. Voluntary response bias because the sample consists of people who chose to participate (correct answer)
4. Response bias because the question wording must have been leading
5. Randomization error because the district did not use a larger sample size

Explanation: This question tests the skill of identifying potential problems with sampling in AP Statistics, specifically recognizing voluntary response bias. The school district used an online poll accessible via their homepage and social media, allowing parents to self-select into the sample by choosing to participate. This method threatens validity because those with strong opinions on the year-round calendar are more likely to respond, potentially skewing results away from the true proportion among all parents. A common distractor here is undercoverage (choice A), which might seem plausible if focusing on internet access, but the primary issue is the self-selection inherent in voluntary response. To diagnose sampling problems, always compare the sampling method to the population of interest and check if certain groups are systematically over- or underrepresented due to how the sample is chosen. In this case, the lack of random selection means the sample isn't representative, leading to biased estimates.

Question 11

1. Undercoverage because subscribers who did not contact support are excluded and may have different opinions (correct answer)
2. Voluntary response bias because the company forced support users to complete the survey
3. Measurement error because recommendation is not measurable with a yes/no question
4. Sampling variability because surveying support users reduces variability too much
5. Response bias because random sampling always creates biased answers

Explanation: This question tests your understanding of undercoverage from using a biased subset. The streaming service wants to estimate recommendation rates for ALL subscribers but only surveys those who contacted customer support. This creates undercoverage because support contacts likely represent dissatisfied customers with problems, while happy subscribers who never need support are completely excluded from the sampling frame. The correct answer is A because using only support contacts systematically excludes the (probably larger) group of subscribers with no issues. When identifying sampling problems, check whether the accessible subset used for sampling differs systematically from the full population on the variable of interest.

Question 12

1. Voluntary response bias because customers self-select into completing the survey for a chance at a prize (correct answer)
2. Undercoverage because the chain served too many customers to survey all of them
3. Measurement error because satisfaction cannot be measured with a survey
4. Nonresponse bias because the chain randomly selected customers but they did not respond
5. Sampling variability because a larger gift card would eliminate bias

Explanation: This question tests your ability to recognize voluntary response bias in customer feedback systems. The restaurant chain wants to estimate satisfaction for ALL customers, but only analyzes surveys from customers who voluntarily scan a QR code and complete a survey for a prize drawing. This creates voluntary response bias because satisfied or very dissatisfied customers are more likely to take the time to respond, while neutral customers typically won't bother. The correct answer is A because the sample consists entirely of self-selected respondents rather than a random sample. When evaluating customer feedback systems, recognize that opt-in surveys almost always overrepresent extreme opinions.

Question 13

1. Response bias caused by confusing wording in the survey questions
2. Undercoverage because families without internet access were not included in the population
3. Nonresponse bias because those who chose to respond may differ from those who did not (correct answer)
4. Voluntary response bias because parents were randomly selected to take the survey
5. Measurement error because the sample size is too large

Explanation: This question tests your ability to identify nonresponse bias in sampling. The school district emailed all parents but only 18% responded, and the results are based solely on these respondents. The key issue is that parents who chose to respond within 48 hours may systematically differ from those who didn't respond - perhaps they have stronger opinions about start times, more free time, or different priorities. This creates nonresponse bias because the 18% who responded are not representative of all parents. While the district did attempt to contact everyone (so there's no undercoverage), the low response rate means the sample is self-selected rather than random. When analyzing survey data, always consider whether non-respondents might differ systematically from respondents in ways that affect your conclusions.

Question 14

1. Nonresponse bias because some forum users did not click the poll
2. Voluntary response/selection bias because help-forum visitors are not representative of all subscribers (correct answer)
3. Random sampling error because a poll cannot be random
4. Confounding because satisfaction is mixed up with subscription price
5. A double-blind violation because users know which interface they saw

Explanation: This question tests understanding of voluntary response bias and selection bias. The streaming service wants to estimate satisfaction among ALL subscribers, but collects data from a poll on its help forum. Help forum visitors represent a highly selective, non-representative subset of subscribers - they're typically there because they're experiencing problems or seeking assistance. This creates voluntary response bias because only motivated users (likely those with complaints) choose to visit the forum and vote, combined with selection bias from the biased sampling location. Even getting 2,000 responses doesn't fix this fundamental flaw - a large biased sample is still biased. When evaluating sampling methods, consider whether the location or method of data collection attracts certain types of respondents more than others. Polls on help forums, complaint hotlines, or review sites typically overrepresent dissatisfied users.

Question 15

1. Sampling bias because manager-nominated employees may not represent all employees' stress levels (correct answer)
2. Random sampling error because the sample size per department is fixed at 5
3. Measurement error because stress cannot be measured with a categorical response
4. Nonresponse bias because nominated employees are required to complete the survey
5. Undercoverage because employees are sampled from every department

Explanation: In AP Statistics, this question tests spotting sampling flaws when estimating the proportion of all employees with high stress via manager-nominated surveys. Managers pick 5 employees per department for 'thoughtful feedback,' surveying only them. Sampling bias is the main issue, as nominees may not represent average stress levels—perhaps managers choose less stressed workers, skewing the proportion. Undercoverage (choice E) distracts, but sampling from every department doesn't fix the biased selection within them. Diagnosing requires checking for nonrandom selection that favors certain traits. Opt for anonymous random sampling to minimize bias. Understanding this ensures valid workplace estimates.

Question 16

1. Voluntary response bias because people with strong opinions are more likely to respond (correct answer)
2. Simple random sampling because the newspaper reaches many households
3. Measurement error because mail-in surveys cannot measure opinions accurately
4. Sampling variability because the sample proportion always equals the population proportion
5. Undercoverage because each household can only respond once

Explanation: This AP Statistics question focuses on detecting sampling flaws in estimating household opinions via a newspaper survey for all town households. The council prints the survey and relies on mailed or online responses, summarizing only those returned. Voluntary response bias threatens validity most, as people with strong opinions (pro or con) are more likely to respond, potentially overstating extremes and not reflecting the average household. Simple random sampling (choice B) is a distractor, but the method is self-selected, not random. To diagnose, assess if participation is voluntary and consider how that might amplify certain views. A mini-lesson: Use random selection over convenience methods to avoid bias in proportion estimates. This ensures reported percentages accurately represent the population.

Question 17

1. Voluntary response/nonresponse bias because respondents may differ systematically from nonrespondents (correct answer)
2. Convenience sampling because the survey used email instead of in-person interviews
3. Undercoverage because sending to all undergraduates ensures everyone is included
4. Measurement error because support cannot be measured with a yes/no question
5. Random sampling error because a 12% response rate is always too small to be accurate

Explanation: This AP Statistics question evaluates identifying sampling problems, especially in surveys with low participation rates when estimating proportions for all undergraduates. The process sends an email survey to every undergraduate, but only 12% respond, using those results as the estimate for tuition support. The primary threat is voluntary response or nonresponse bias, as responders may differ systematically from nonresponders—perhaps more opinionated students reply, skewing the proportion. Undercoverage (choice C) is a distractor because the initial frame includes everyone, but the low response creates self-selection bias. To diagnose such issues, examine response rates and consider if nonrespondents might hold different views, which could invalidate generalizations. This highlights the importance of follow-up methods to minimize bias in large-scale surveys. Understanding these biases ensures more reliable statistical conclusions.

Question 18

1. Undercoverage because only customers with phone numbers were selected
2. Nonresponse bias because customers who do not answer may have different satisfaction levels (correct answer)
3. Response bias because a 1–10 scale is too subjective to measure satisfaction
4. Sampling variability because the sample size is not exactly 500
5. Voluntary response because customers chose whether to buy in the first place

Explanation: Assessing potential sampling problems in AP Statistics, this question examines a phone survey of randomly selected customers to estimate average satisfaction for all recent purchasers. Out of 800 called, only 430 respond, and the mean is based solely on them. Nonresponse bias is the biggest threat, as those who don't answer might have lower satisfaction (e.g., unhappy customers avoiding calls), making the estimate unrepresentative. A distractor is undercoverage (choice A), but the random selection from the database covers the population; the issue arises from nonresponse. In diagnosing sampling problems, check if nonrespondents could differ systematically and explore ways to boost response rates, like incentives. This approach helps in creating valid estimates of population means. Recognizing nonresponse bias is key to trustworthy survey results.

Question 19

1. Undercoverage because fish that live in deeper water may be less likely to be sampled (correct answer)
2. Nonresponse bias because some fish avoid the nets
3. Response bias because fish might behave differently when they see the biologists
4. Measurement error because species identification is impossible in the field
5. Voluntary response because fish choose whether to swim into the nets

Explanation: This AP Statistics question examines potential sampling problems in estimating the proportion of protected fish species in a lake by netting only near the shoreline. Undercoverage is the primary threat, as deeper-water fish are less likely to be caught, potentially underestimating or overestimating the proportion if species differ by habitat. Voluntary response (choice E) is a distractor, but fish don't 'choose'—the issue is the limited sampling area excluding parts of the population. To diagnose, evaluate if the method misses subgroups based on location or behavior. A mini-lesson: Ensure sampling covers the full environment to avoid bias in ecological estimates. This promotes accurate population proportion inferences.

Question 20

1. Convenience sampling because the locations and customers are chosen based on ease rather than randomness (correct answer)
2. Nonresponse bias because some customers might not keep their receipts
3. Response bias because customers might change what they order when watched
4. Sampling variability because 40 customers per location is too consistent
5. Measurement error because receipts do not measure spending accurately

Explanation: Evaluating sampling issues in AP Statistics, this question involves estimating average spending for all nationwide restaurant customers by having managers collect receipts from the next 40 customers at 15 locations. Convenience sampling threatens validity, as selections based on ease (whoever is next) may not represent all customers, like varying by time or location. Nonresponse bias (choice B) distracts, but the core flaw is the nonrandom, convenience-based method. In diagnosis, look for reliance on accessible subjects over random ones, which can introduce bias. Use systematic or random sampling across diverse times/locations for better representation. Recognizing convenience sampling aids in reliable mean calculations.