Potential Problems with Sampling - AP Statistics

1. A town wants to estimate the proportion of all registered voters in the town who support a proposed bond measure. The town clerk takes the voter registration list, sorts it alphabetically by last name, and selects every 20th name starting with the first name on the list. The clerk assumes this produces a representative sample. Which issue most threatens the validity of this sampling method?
   Answer: Potential bias from a systematic sample if the ordering is related to support (for example, last name correlated with neighborhood or ethnicity) Potential bias in systematic sampling is the focus of this AP Statistics question. The clerk sorted voters alphabetically by last name and selected every 20th, assuming representativeness for support of a bond measure among all registered voters. This threatens validity if the ordering correlates with opinions, like last names linked to ethnic or neighborhood groups that differ in support. Voluntary response bias (choice B) is a distractor, as selection was systematic, not self-chosen. Mini-lesson: when diagnosing sampling issues, check for patterns in the list that systematic methods might exploit; random selection avoids such biases by equalizing chances.
2. A restaurant chain wants to estimate the average amount spent per visit by all customers nationwide. Managers at 15 locations are asked to "collect receipts from the next 40 customers you see," and they send those receipts to headquarters. Headquarters computes the mean spending from the collected receipts. Which issue most threatens the validity of this sampling plan?
   Answer: Convenience sampling because the locations and customers are chosen based on ease rather than randomness Evaluating sampling issues in AP Statistics, this question involves estimating average spending for all nationwide restaurant customers by having managers collect receipts from the next 40 customers at 15 locations. Convenience sampling threatens validity, as selections based on ease (whoever is next) may not represent all customers, like varying by time or location. Nonresponse bias (choice B) distracts, but the core flaw is the nonrandom, convenience-based method. In diagnosis, look for reliance on accessible subjects over random ones, which can introduce bias. Use systematic or random sampling across diverse times/locations for better representation. Recognizing convenience sampling aids in reliable mean calculations.
3. A university wants to estimate the proportion of all undergraduate students who use the campus tutoring center at least once per semester. Researchers sample 300 students by standing outside the tutoring center for two days and randomly selecting students as they enter. Which issue most threatens the validity of generalizing the results to all undergraduates?
   Answer: Undercoverage because students who never visit the tutoring center cannot be sampled with this method This question tests understanding of severe undercoverage in location-based sampling. The university wants to estimate tutoring center usage among ALL undergraduates but samples only students entering the tutoring center. This creates extreme undercoverage because students who never use the tutoring center have zero probability of selection - you cannot estimate the proportion of all students who use the center by only sampling those who are already using it! This is like estimating the proportion of people who own cars by only sampling in a parking lot. The sampling frame (students entering the tutoring center) is a subset of users, not representative of all undergraduates. Even with random selection at the entrance, this method will drastically overestimate usage rates. When diagnosing sampling problems, check if your sampling location or method automatically excludes the very people you're trying to measure - here, non-users cannot possibly be sampled.
4. A streaming service wants to estimate the proportion of all its subscribers who would recommend the service to a friend. It surveys only subscribers who contacted customer support in the last 30 days, because their email addresses are easy to access, and calculates the recommendation rate from those responses. Which issue most threatens the validity of using this sample to represent all subscribers?
   Answer: Undercoverage because subscribers who did not contact support are excluded and may have different opinions This question tests your understanding of undercoverage from using a biased subset. The streaming service wants to estimate recommendation rates for ALL subscribers but only surveys those who contacted customer support. This creates undercoverage because support contacts likely represent dissatisfied customers with problems, while happy subscribers who never need support are completely excluded from the sampling frame. The correct answer is A because using only support contacts systematically excludes the (probably larger) group of subscribers with no issues. When identifying sampling problems, check whether the accessible subset used for sampling differs systematically from the full population on the variable of interest.
5. A teacher wants to estimate the proportion of all students in a large high school who get at least 8 hours of sleep on school nights. The teacher surveys students in her first-period class because they are easy to access, then reports the result as representing the whole school. Which issue most threatens the validity of the teacher's estimate?
   Answer: Undercoverage because students not in the first-period class have no chance of selection and may differ in sleep habits This AP Statistics question tests recognition of undercoverage in sampling. The teacher surveyed only her first-period class for sleep habits, then generalized to all students in the large high school. Validity is threatened because students not in that class are excluded and might have different sleep patterns, like later-period students sleeping less. Voluntary response bias (choice C) is a distractor, as the teacher required responses, not allowing self-selection. A mini-lesson: diagnose by ensuring the sample frame covers the population; convenience in choosing one class leads to undercoverage of others, biasing schoolwide estimates.
6. A state health department wants to estimate the proportion of all adults in the state who received a flu shot this season. Surveyors call phone numbers selected from a list of landline telephone subscribers in the state and interview one adult in each household reached. Which issue most threatens the validity of the estimate?
   Answer: Undercoverage because adults without landlines are less likely to be included This question tests your ability to identify undercoverage due to sampling frame limitations. The health department wants to estimate flu vaccination rates for ALL adults in the state but only calls landline subscribers. This creates undercoverage because many adults, particularly younger ones and those in lower income brackets, rely exclusively on cell phones and have no landline. The correct answer is B because the sampling frame (landline subscriber list) systematically excludes a significant portion of the target population. When evaluating sampling methods, always check whether the list or method used to contact people actually includes everyone in the population of interest.
7. A state agency wants to estimate the mean number of hours per week that all high school students in the state spend on homework. The agency randomly selects 30 high schools and then surveys only students enrolled in AP classes at those schools. The agency reports the result as the statewide average for all high school students. Which issue most threatens the validity of the estimate?
   Answer: Undercoverage because many types of students (non-AP) have zero chance of being selected This AP Statistics question highlights undercoverage in sampling designs. The agency randomly picked 30 high schools but only surveyed AP students there, extrapolating homework hours to all high school students statewide. Validity is compromised because non-AP students are entirely excluded and likely spend fewer hours on homework, leading to an overestimate. Voluntary response bias (choice B) distracts, but students were required to participate, not self-selecting. To diagnose such problems, verify if the sampling process includes all population subgroups; here, limiting to AP classes creates undercoverage and non-representative results.
8. A company wants to estimate the proportion of all its employees who would prefer a 4-day workweek. The company surveys 200 employees chosen by a manager who selects "reliable team members who give thoughtful feedback" from across departments. Which issue most threatens the validity of using this sample to represent all employees?
   Answer: Sampling bias due to a nonrandom, judgment-based selection that may exclude certain viewpoints This question tests recognition of sampling bias from non-random, judgment-based selection. The company wants to estimate preferences of ALL employees, but the manager hand-picks "reliable team members who give thoughtful feedback." This judgment-based selection creates systematic bias because the manager's criteria for selection (reliability, thoughtfulness) might correlate with work preferences - perhaps these selected employees are more invested in the company or have different work-life balance needs than average employees. This is neither simple random sampling nor any form of probability sampling; it's convenience sampling based on subjective criteria. The manager might unconsciously select employees who share similar views or exclude those seen as complainers or troublemakers. To identify this bias, look for subjective selection criteria that could correlate with the variable being measured. True random sampling gives every employee an equal chance regardless of perceived qualities.
9. A statewide university system wants to estimate the proportion of all enrolled undergraduates who support a proposed tuition increase. An email survey link is sent to every undergraduate, but only about 12% of students respond after two reminder emails. The results are reported as the systemwide estimate. Which issue most threatens the validity of the estimate?
   Answer: Voluntary response/nonresponse bias because respondents may differ systematically from nonrespondents This AP Statistics question evaluates identifying sampling problems, especially in surveys with low participation rates when estimating proportions for all undergraduates. The process sends an email survey to every undergraduate, but only 12% respond, using those results as the estimate for tuition support. The primary threat is voluntary response or nonresponse bias, as responders may differ systematically from nonresponders—perhaps more opinionated students reply, skewing the proportion. Undercoverage (choice C) is a distractor because the initial frame includes everyone, but the low response creates self-selection bias. To diagnose such issues, examine response rates and consider if nonrespondents might hold different views, which could invalidate generalizations. This highlights the importance of follow-up methods to minimize bias in large-scale surveys. Understanding these biases ensures more reliable statistical conclusions.
10. A company wants to estimate the proportion of all employees who feel workplace stress is "high." Human resources selects a sample by asking each department manager to nominate 5 employees who "will provide thoughtful feedback," then surveys only those nominated employees. Which issue most threatens the validity of the resulting estimate?
    Answer: Sampling bias because manager-nominated employees may not represent all employees' stress levels In AP Statistics, this question tests spotting sampling flaws when estimating the proportion of all employees with high stress via manager-nominated surveys. Managers pick 5 employees per department for 'thoughtful feedback,' surveying only them. Sampling bias is the main issue, as nominees may not represent average stress levels—perhaps managers choose less stressed workers, skewing the proportion. Undercoverage (choice E) distracts, but sampling from every department doesn't fix the biased selection within them. Diagnosing requires checking for nonrandom selection that favors certain traits. Opt for anonymous random sampling to minimize bias. Understanding this ensures valid workplace estimates.
11. A principal wants to estimate the average number of hours of sleep per night for all students at the high school. She selects a sample by surveying students who arrive early to school to attend a 7:00 a.m. tutoring program, recording their reported sleep from the previous night. Which issue most threatens the validity of this sampling method?
    Answer: Undercoverage because students who do not attend early tutoring are not represented In AP Statistics, this question tests the ability to spot potential sampling issues, particularly when the sample doesn't cover the entire population of interest. The principal surveys only students arriving early for a 7:00 a.m. tutoring program, aiming to estimate average sleep for all high school students. This method introduces undercoverage because students not attending early tutoring—perhaps those who sleep later or have different schedules—are not included, potentially biasing the results toward students with specific habits. For instance, early risers might report more sleep, unrepresentative of the whole school. A distractor like nonresponse bias (choice A) might seem plausible, but the core problem is the incomplete coverage of the population, not just skipping the survey. A mini-lesson on diagnosing sampling problems: Evaluate if the sampling method systematically omits subgroups, and consider how that affects generalizability to the population parameter. Recognizing undercoverage promotes better sampling designs for accurate inferences.
12. A streaming service wants to estimate the proportion of all subscribers who are satisfied with a new interface. It posts a poll on its help forum and reports the results from the first 2,000 votes. Users who visit the help forum are often there because they have problems. Which issue most threatens the validity of the estimate?
    Answer: Voluntary response/selection bias because help-forum visitors are not representative of all subscribers This question tests understanding of voluntary response bias and selection bias. The streaming service wants to estimate satisfaction among ALL subscribers, but collects data from a poll on its help forum. Help forum visitors represent a highly selective, non-representative subset of subscribers - they're typically there because they're experiencing problems or seeking assistance. This creates voluntary response bias because only motivated users (likely those with complaints) choose to visit the forum and vote, combined with selection bias from the biased sampling location. Even getting 2,000 responses doesn't fix this fundamental flaw - a large biased sample is still biased. When evaluating sampling methods, consider whether the location or method of data collection attracts certain types of respondents more than others. Polls on help forums, complaint hotlines, or review sites typically overrepresent dissatisfied users.
13. A school district wants to estimate the proportion of all parents of students in the district who support switching to a year-round calendar. The district posts a link to an online poll on its homepage and social media accounts and leaves it open for 48 hours. A total of 1,842 responses are submitted, and the district reports the poll result as representing districtwide parent opinion. Which issue most threatens the validity of this sampling method?
    Answer: Voluntary response bias because the sample consists of people who chose to participate This question tests the skill of identifying potential problems with sampling in AP Statistics, specifically recognizing voluntary response bias. The school district used an online poll accessible via their homepage and social media, allowing parents to self-select into the sample by choosing to participate. This method threatens validity because those with strong opinions on the year-round calendar are more likely to respond, potentially skewing results away from the true proportion among all parents. A common distractor here is undercoverage (choice A), which might seem plausible if focusing on internet access, but the primary issue is the self-selection inherent in voluntary response. To diagnose sampling problems, always compare the sampling method to the population of interest and check if certain groups are systematically over- or underrepresented due to how the sample is chosen. In this case, the lack of random selection means the sample isn't representative, leading to biased estimates.
14. A restaurant chain wants to estimate the proportion of all customers who are satisfied with a new menu. It prints a survey code on receipts, and customers can complete the survey online for a chance to win a $50 gift card. The chain summarizes results from the customers who chose to participate. Which issue most threatens the validity of using these results to represent all customers?
    Answer: Sampling bias due to voluntary response, since customers with unusually positive or negative experiences may be more likely to respond This question tests understanding of voluntary response bias in customer feedback systems. The restaurant wants to estimate satisfaction among ALL customers but relies on those who choose to complete an optional online survey. This creates voluntary response bias because customers with extreme experiences (very satisfied or very dissatisfied) are more motivated to provide feedback than those with average experiences. The gift card incentive might attract some neutral customers, but generally, optional feedback systems overrepresent the extremes. Additionally, the online format might exclude less tech-savvy customers or those without internet access. Even if thousands respond, this self-selected sample doesn't represent the silent majority of customers with moderate satisfaction levels. To diagnose voluntary response bias, look for optional participation where the likelihood of responding correlates with the variable being measured - here, satisfaction level influences the decision to complete the survey.
15. A company wants to estimate the average satisfaction rating (1–10) among all customers who made a purchase in the last month. They randomly select 800 customers from their database and call them. Of those selected, 430 answer the phone and complete the survey; the company reports the mean rating from the 430 responses. Which issue most threatens the validity of generalizing to all customers?
    Answer: Nonresponse bias because customers who do not answer may have different satisfaction levels Assessing potential sampling problems in AP Statistics, this question examines a phone survey of randomly selected customers to estimate average satisfaction for all recent purchasers. Out of 800 called, only 430 respond, and the mean is based solely on them. Nonresponse bias is the biggest threat, as those who don't answer might have lower satisfaction (e.g., unhappy customers avoiding calls), making the estimate unrepresentative. A distractor is undercoverage (choice A), but the random selection from the database covers the population; the issue arises from nonresponse. In diagnosing sampling problems, check if nonrespondents could differ systematically and explore ways to boost response rates, like incentives. This approach helps in creating valid estimates of population means. Recognizing nonresponse bias is key to trustworthy survey results.
16. A fitness app wants to estimate the mean number of steps per day for all adults in a country. The company analyzes step counts from a random sample of 20,000 current app users and reports the result as the countrywide mean. Which issue most threatens the validity of this conclusion?
    Answer: Undercoverage because adults who do not use the app cannot be included and may differ in activity This question tests your understanding of undercoverage from self-selected populations. The fitness app wants to estimate steps for all adults in the country but only samples from current app users. This creates severe undercoverage because adults who don't use fitness apps are completely excluded. Non-app users likely differ dramatically in activity levels - they may be less health-conscious, older, less tech-savvy, or simply prefer other exercise forms. Even with 20,000 users, the sample only represents the app-using population, not all adults. The automatic measurement isn't the problem; the issue is generalizing from a self-selected, health-conscious subgroup to the entire adult population. To estimate population parameters, your sampling frame must include all types of people in that population, not just those who opt into a particular service.
17. A town wants to estimate the proportion of all registered voters in the town who favor a proposed property-tax increase. A polling firm selects a simple random sample of 500 registered voters and mails each a questionnaire. The firm reports results based only on the 190 voters who return the questionnaire. Which issue most threatens the validity of the reported estimate?
    Answer: Nonresponse bias because those who return the questionnaire may differ from those who do not This question tests your understanding of nonresponse bias in mail surveys. The polling firm starts correctly with a simple random sample of 500 voters, but only 190 return questionnaires (38% response rate). This creates nonresponse bias because voters with strong opinions about taxes (either for or against) are more likely to return the survey than those who are indifferent. The correct answer is A because nonresponse bias occurs when survey respondents differ systematically from non-respondents on the variable being measured. When conducting surveys, high nonresponse rates threaten validity even when the initial sample was selected randomly.
18. A school district wants to estimate the proportion of all parents of students in the district who support changing the start time. The district emailed a survey link to every parent on file and reported results based only on the 18% who completed the survey within 48 hours. Which issue most threatens the validity of using this sample to generalize to all parents in the district?
    Answer: Nonresponse bias because those who chose to respond may differ from those who did not This question tests your ability to identify nonresponse bias in sampling. The school district emailed all parents but only 18% responded, and the results are based solely on these respondents. The key issue is that parents who chose to respond within 48 hours may systematically differ from those who didn't respond - perhaps they have stronger opinions about start times, more free time, or different priorities. This creates nonresponse bias because the 18% who responded are not representative of all parents. While the district did attempt to contact everyone (so there's no undercoverage), the low response rate means the sample is self-selected rather than random. When analyzing survey data, always consider whether non-respondents might differ systematically from respondents in ways that affect your conclusions.
19. A restaurant chain wants to estimate the proportion of all customers who would recommend the restaurant. Receipts include a QR code inviting customers to complete a short survey for a chance to win a $50 gift card. The chain summarizes the survey responses and reports them as representative of all customers. Which issue most threatens the validity of this approach?
    Answer: Voluntary response bias because customers self-select into responding, possibly based on strong experiences Voluntary response bias is the core sampling problem tested in this AP Statistics question. The restaurant invited all customers via receipt QR codes to take a survey for a gift card chance, then treated responses as representative of all customers' recommendation rates. This threatens validity because self-selecting participants, even with incentives, often have stronger opinions—dissatisfied customers might be more motivated to respond. Undercoverage (choice B) is a distractor since receipts reach everyone, but the issue is who chooses to participate. A mini-lesson on diagnosis: look for self-selection in sampling processes, as it biases results toward extremes, unlike random methods that aim for representativeness.
20. A city wants to estimate the mean number of minutes all residents spend commuting to work each day. A researcher stands at a downtown bus terminal from 7:00–8:00 a.m. on Tuesday and randomly selects every 10th person who enters to answer a commuting-time question. Which issue most threatens the validity of generalizing the results to all residents?
    Answer: Undercoverage because residents who do not commute by bus or at that time are less likely to be included This question tests recognition of undercoverage in convenience sampling. The city wants to estimate commuting time for ALL residents, but sampling only at a downtown bus terminal from 7-8 AM systematically excludes large portions of the population. Residents who drive, walk, bike, work from home, work night shifts, or commute at different times have zero chance of being selected. Even though the researcher uses systematic random sampling (every 10th person), this doesn't fix the fundamental problem that the sampling frame (people at the bus terminal during that hour) doesn't represent the target population (all residents). This is classic undercoverage - when the sampling method makes it impossible or unlikely for certain groups to be included. To diagnose undercoverage, ask: "Does everyone in my target population have a chance of being selected?" If entire groups are excluded by the sampling location, time, or method, you have undercoverage.

Start this quiz