Measuring Public Opinion - AP US Government and Politics

1. A news site posts a "click to vote" poll and reports 78% support. Which poll type is this?
   Answer: Voluntary response (online opt-in) poll, because participants choose to respond, creating strong self-selection bias. This question tests the skill of identifying types of polls in measuring public opinion for AP US Government and Politics. The news site's 'click to vote' poll allows self-selection, resulting in 78% support, which is a voluntary response or online opt-in poll prone to strong self-selection bias as only motivated individuals participate. Unlike random sampling, this method doesn't give everyone an equal chance, leading to unrepresentative results. The explanation highlights how voluntary polls often attract extreme views, making them unreliable for gauging broader opinion. A key distractor is choice A, claiming it's a random-sample survey due to large numbers, but size doesn't fix self-selection issues. Effective polling strategy emphasizes random sampling to ensure representativeness, whereas opt-in methods amplify biases. Recognizing poll types helps evaluate their credibility in political contexts.
2. A poll uses random-digit dialing and reports $\pm 3\%$ error; what concept explains this uncertainty?
   Answer: Margin of error from sampling error, meaning repeated random samples would vary and the estimate likely falls within a stated range. This question tests understanding of margin of error in public opinion polling. The ±3% error reported by the poll refers to the margin of error, which quantifies uncertainty due to sampling error. When pollsters use random sampling methods like random-digit dialing, they're only surveying a subset of the population, not everyone. This sampling process introduces variability—if we repeated the poll with different random samples, we'd get slightly different results each time. The margin of error tells us the likely range within which the true population value falls, typically with 95% confidence. Option C correctly identifies this as margin of error from sampling error, while the distractors represent other polling concepts that don't explain the ±3% uncertainty range.
3. A pollster calls only landlines to survey "all adults." Which methodological issue is most likely?
   Answer: Coverage error from an incomplete sampling frame, because adults without landlines have zero chance of selection and may differ systematically. This question examines coverage error in sampling frames. By calling only landlines, the pollster excludes all adults who rely solely on cell phones - a growing demographic that skews younger and more diverse. This creates coverage error where certain population segments have zero probability of selection, potentially biasing results if cell-only adults differ politically from landline users. The sampling frame (landline numbers) doesn't cover the target population (all adults). Option A correctly identifies this coverage error problem. Option B wrongly claims this ensures representation; C incorrectly states this reduces error; D confuses phone type with question wording; and E misidentifies the polling method.
4. Poll of 1,000 adults with a 3.1% margin of error reports 49% approve, 46% oppose. Best conclusion?
   Answer: The result is effectively tied The 3-point gap between 49% and 46% is smaller than the 3.1% margin of error, so the difference is not statistically meaningful. You cannot say approval is truly ahead. The tempting mistake is thinking 49% beats 46% by 3 points, but that lead does not exceed the margin, so the result is effectively tied.
5. A pollster weights responses to match census demographics after under-sampling young adults. What technique is being used?
   Answer: Post-stratification weighting, adjusting results so sample demographics align with known population parameters, attempting to reduce representation errors. This question addresses post-stratification weighting in polling. When a poll under-samples certain demographic groups (like young adults), pollsters can apply weights to adjust the results. Post-stratification weighting gives more weight to underrepresented groups and less to overrepresented ones, attempting to make the sample demographics match known population parameters from census data. This technique tries to reduce bias from unrepresentative samples, though it can't completely fix all sampling problems. Option B correctly identifies this as post-stratification weighting. This isn't push polling (which uses leading questions), doesn't create a census or eliminate margin of error—it's specifically about adjusting for demographic imbalances.
6. A poll of 1,000 likely voters claims a $\%$ margin of error. What is the most accurate critique?
   Answer: It is likely overstated precision, because typical $n=1000$ probability samples have about $\%$ MOE, and weighting/design can increase it. This question critiques unrealistic precision claims in polling. A typical probability sample of 1,000 respondents produces a margin of error around ±3.1% at 95% confidence. Claims of ±1% margin of error with n=1,000 suggest either misreporting, misunderstanding of statistics, or use of a non-standard confidence level. Additionally, design effects from weighting and clustering typically increase margins of error beyond simple random sampling calculations. Option B correctly identifies this as likely overstated precision. Option A wrongly claims automatic 1% margins; C incorrectly requires a census; D confuses margin calculation; and E misunderstands that margins don't guarantee prediction accuracy.
7. A statewide poll randomly samples 1,200 registered voters; results report $.8\%$ margin of error. What concept is illustrated?
   Answer: The margin of error from probability sampling, meaning the estimate likely falls within a stated range due to random sampling variation. This question tests understanding of margin of error in probability sampling. When a poll randomly samples 1,200 registered voters and reports a margin of error, it's illustrating how random sampling produces estimates that likely fall within a stated range of the true population value. The margin of error (typically around 3% for n=1,200) reflects sampling variation - the fact that different random samples would produce slightly different results. Option C correctly identifies this fundamental concept. Option A is incorrect because nothing suggests persuasion; B is wrong as 1,200 voters isn't everyone in a state; D misunderstands margin of error as relating to nonresponse; and E confuses random sampling with convenience sampling.
8. A campaign polls only people attending its rallies; which polling limitation is most clearly illustrated by this method?
   Answer: Selection bias from a nonrepresentative sample, since rally attendees differ systematically from the broader voting population. This question examines selection bias in polling methodology. Polling only rally attendees violates the fundamental principle of representative sampling because people who attend political rallies are systematically different from the general voting population - they tend to be more politically engaged, partisan, and supportive of that candidate. The correct answer C identifies this as selection bias from a nonrepresentative sample. Answer A incorrectly suggests random sampling error, which applies to properly conducted polls, not convenience samples. Answer B mentions question-order effects which are unrelated to sampling methods, D describes bandwagon effects which concern response behavior not sampling, and E misidentifies this as a push poll when it's simply poor sampling. The critical insight is that where and how you sample determines whether results can generalize to the broader population.
9. A poll's results shift after weighting to match census demographics for age, race, and education. What practice is illustrated?
   Answer: Post-stratification weighting, adjusting sample responses to better reflect population demographics and reduce bias from differential response and coverage. This question illustrates post-stratification weighting in polling. When poll results are adjusted to match known population demographics from census data for characteristics like age, race, and education, this is weighting or post-stratification (Option B). This practice compensates for differential response rates and coverage issues - for example, if young adults are underrepresented in the raw sample, their responses are given more weight. This isn't random assignment (A) which is an experimental technique, push poll correction (C), placebo effect (D), or census adjustment (E). Weighting is a standard practice in scientific polling to improve representativeness when certain groups are over or undersampled.
10. A news poll reports Candidate X at 51% and Y at 49% with $\pm 4\%$; what is the best interpretation?
    Answer: The race is too close to call, because the difference is within the margin of error and could reflect sampling variation. This question tests interpretation of polling results within margin of error. With Candidate X at 51% and Y at 49% with a ±4% margin, the true values could range from 47-55% for X and 45-53% for Y, meaning their ranges overlap substantially. The correct answer B properly concludes the race is too close to call because the 2-point difference falls within the margin of error. Answer A wrongly claims any lead over 50% is definitive, ignoring sampling uncertainty. Answer C incorrectly suggests margins favor underdogs, D wrongly invalidates polls with margins over 2%, and E makes an absurd claim about turnout. The key principle is that when candidate differences are smaller than the margin of error, we cannot conclude with statistical confidence that one candidate truly leads - the observed difference could be due to sampling variation alone.
11. Two surveys ask about the same policy, but one uses "welfare" and the other "assistance to the poor"; what concept is shown?
    Answer: Question wording (framing) effects, where different terms trigger different associations and can shift reported support levels. This question illustrates question wording or framing effects in public opinion measurement. Research consistently shows that using "welfare" versus "assistance to the poor" can produce dramatically different response patterns, even though both refer to similar government programs. The correct answer A identifies this as a framing effect where different terms trigger different associations and attitudes. Answer B incorrectly connects wording to phone number selection, C wrongly links it to sampling coverage, D misunderstands margin of error as related to wording rather than sampling, and E incorrectly limits this to exit polling. The key principle is that how questions are worded can substantially influence responses because different terms carry different connotations - "welfare" often triggers negative associations while "assistance to the poor" emphasizes helping those in need.
12. A poll deliberately samples equal numbers of rural, suburban, and urban residents, then weights to population shares; what principle is illustrated?
    Answer: Stratified sampling with weighting, designed to ensure subgroup representation and then match the sample to known population proportions. This question illustrates stratified sampling with post-stratification weighting. By deliberately sampling equal numbers from rural, suburban, and urban areas (stratification) and then weighting responses to match known population proportions, pollsters ensure adequate representation of all groups while adjusting for oversampling. The correct answer A accurately describes this two-step process. Answer B incorrectly claims this is simple random sampling, which wouldn't involve deliberate subgroup quotas. Answer C confuses quantitative polling with qualitative focus groups, D wrongly limits weighting to exit polls, and E misunderstands weighting as reducing margin of error rather than correcting representation. The key principle is that stratified sampling ensures sufficient data from all important subgroups, while weighting adjusts the final results to reflect actual population proportions.
13. A campaign emails a link to supporters asking if taxes are "too high," then publicizes the results. What limitation is shown?
    Answer: A voluntary response sample with selection bias, because people choose to participate and the sample likely overrepresents highly motivated supporters. This question illustrates voluntary response bias in polling. When a campaign emails supporters asking about taxes being "too high," this creates a self-selected sample where only motivated individuals choose to participate. This voluntary response sample (Option A) systematically overrepresents people with strong opinions, particularly supporters who received the email. The results cannot be generalized to the broader population because the sample wasn't randomly selected. Options B, C, and D describe legitimate scientific polling methods, while E incorrectly suggests a census approach. The key limitation is selection bias from voluntary participation, not the loaded question wording alone.
14. A firm calls only landlines to survey adults about politics. What limitation is most directly illustrated?
    Answer: Coverage error, because excluding cell-phone-only households can systematically miss groups, making the sample less representative of all adults. This question examines coverage error in telephone polling. Calling only landlines systematically excludes households that rely exclusively on cell phones, which now represent a significant portion of the population. Cell-phone-only households tend to be younger, more mobile, and potentially different in political views, creating coverage error where the sampling frame (landline users) doesn't match the target population (all adults). Option A correctly identifies this as coverage error, explaining how excluding cell-phone-only households makes the sample less representative. The distractors misapply concepts: bandwagon effects involve opinion change after hearing results, not phone type (B), using landlines doesn't make something a push poll (C), restricting to landlines creates bias rather than eliminating error (D), and random assignment relates to experiments not survey sampling (E). This highlights how technological changes in communication require polling methods to adapt to maintain representative samples.
15. Two polls sample 1,000 adults each; one has 8% response rate. What concern is highlighted?
    Answer: Nonresponse bias, because those who do not participate may differ from respondents, potentially skewing results beyond the stated margin of error. This question tests understanding of nonresponse bias in polling. When a poll has only an 8% response rate, it means 92% of contacted individuals did not participate, raising serious concerns about whether respondents differ systematically from non-respondents. Those who choose to participate might be more politically engaged, have stronger opinions, or differ demographically from those who refuse or cannot be reached. Option B correctly identifies this as nonresponse bias, noting it can skew results beyond the stated margin of error (which only accounts for random sampling error). The distractors misapply concepts: bandwagon effects involve opinion change after hearing results, not response rates (A), push polls aim to persuade rather than measure (C), response rate doesn't directly affect margin of error calculation for a fixed sample size (D), and a census attempts to survey everyone in a population, not just achieve 8% response (E). The key insight is that low response rates threaten representativeness even with probability sampling.
16. A pollster divides voters by region, then randomly samples within each region proportionally. What method is this?
    Answer: Stratified sampling, because the population is split into strata (regions) and random samples are drawn within each to improve representativeness. This question tests knowledge of stratified sampling methodology. When a pollster divides the population into regions (strata) and then randomly samples within each region proportionally, they're using stratified sampling. This technique ensures that each region is represented in the final sample according to its share of the population, which can improve the sample's representativeness compared to simple random sampling. Option B correctly identifies this method. This differs from cluster sampling (which would survey everyone in selected regions), convenience sampling (which doesn't use random selection), push polling (which aims to persuade), or a census (which surveys everyone).
17. A poll of 2,000 has $\pm2\%$ MOE; a poll of 500 has $\pm4\%$. What principle explains this?
    Answer: Larger samples generally reduce sampling error, so the margin of error shrinks as sample size increases, assuming probability sampling methods. This question addresses the relationship between sample size and margin of error. In probability sampling, margin of error decreases as sample size increases because larger samples provide more information about the population, reducing sampling variability. The mathematical relationship shows diminishing returns - quadrupling the sample size (from 500 to 2000) only halves the margin of error (from ±4% to ±2%). Option A correctly explains this principle, noting it assumes probability sampling methods where each person has a known chance of selection. The distractors misunderstand margin of error: response rate affects bias not margin of error calculation (B), question wording affects measurement not sampling error (C), larger samples don't create push polls (D), and even 2000 respondents remain a sample not a census of millions (E). This fundamental principle helps explain why national polls often use 1000-1500 respondents as a balance between precision and cost.
18. Two polls ask about the same policy, but one uses "assistance" and another says "welfare," producing different results. What is illustrated?
    Answer: Question wording and framing effects, where emotionally loaded or connotative terms alter responses even when the underlying policy is similar. This question demonstrates how question wording affects poll responses. When identical policies are described using different terms - "assistance" versus "welfare" - respondents react differently due to the connotations and emotional associations of these words. "Welfare" often carries negative connotations while "assistance" sounds more positive, leading to different support levels for the same policy. Option C correctly identifies this as question wording and framing effects. This isn't about sampling issues (A), bandwagon effects (B), margin of error (D), or push polling (E). The key insight is that word choice in survey questions can significantly influence responses, highlighting the importance of neutral wording in scientific polling.
19. A poll gets responses from only 8% of those contacted; which concern about accuracy is most relevant here?
    Answer: A low response rate may create nonresponse bias if respondents differ systematically from nonrespondents on the measured issue. This question addresses nonresponse bias in polling. When only 8% of contacted individuals respond, there's serious concern that respondents may differ systematically from non-respondents on the issues being measured. The correct answer A accurately identifies that low response rates can create nonresponse bias if those who participate differ from those who don't. Answer B incorrectly claims low response rates increase sample size, when they actually reduce it. Answer C wrongly suggests margin of error corrects for nonresponse bias, when it only addresses random sampling error. Answer D incorrectly limits the concern to push polls, and E confuses nonresponse with question-order effects. The critical understanding is that nonresponse bias threatens validity when those willing to participate have different opinions than those who refuse, which becomes more likely with very low response rates.
20. A survey asks, "Do you support the reckless plan that hurts families?" What polling practice is illustrated?
    Answer: Push polling/leading wording, using loaded language to influence responses rather than to measure opinion objectively and accurately. This question demonstrates push polling or leading question wording. The phrase "reckless plan that hurts families" uses emotionally charged, negative language designed to influence how respondents answer rather than neutrally measuring their opinions. This is a classic example of a leading question that pushes respondents toward a particular answer. True public opinion polls strive for neutral wording to accurately measure views, not shape them. Option B correctly identifies this as push polling/leading wording. This isn't neutral question design (which would avoid loaded language), stratified sampling (about sample selection), or related to random sampling error or census methods.

Start this quiz