- A voter receives an automated phone call that says: 'Hello, we are conducting a political survey. Would you be more or less likely to vote for Councilwoman Miller if you knew she voted to double her own salary during a city-wide budget crisis?' After the voter responds, the call immediately ends. This activity is best characterized as:
Answer: A push poll intended to disseminate negative information under the guise of research.
A push poll is not a legitimate survey. It is a political persuasion tactic disguised as a poll. Its purpose is to 'push' voters away from a candidate by asking a question that contains negative, often false or misleading, information. The lack of demographic questions or further survey items is a key indicator.
- A critic argues the poll is biased. Which of the following statements most accurately diagnoses the primary sources of error?
Answer: The methodology is flawed by both coverage bias from excluding cell-phone-only households and non-response bias from certain groups being less likely to answer.
The methodology has two main flaws. First, using only landlines creates coverage bias because it excludes the significant portion of the population that only uses cell phones (who are often younger and more urban). Second, even among those called, older individuals being overrepresented suggests that younger people were less likely to answer, which is non-response bias.
- In post-election surveys, the proportion of respondents who claim to have voted is consistently higher than the actual official turnout rate. Analysis shows this gap persists even when polls are perfectly weighted to match population demographics. This persistent discrepancy is a classic illustration of:
Answer: Social desirability bias, where respondents provide the answer they believe is more socially respectable.
Social desirability bias occurs when respondents answer questions in a way that will be viewed favorably by others. Since voting is widely seen as a civic duty, some people who did not vote will report that they did to avoid embarrassment or to appear as good citizens. While non-response bias (B) also contributes to turnout overestimation, the act of misreporting one's own behavior is specifically social desirability bias.
- Which of the following describes a potential source of coverage bias that is inherent in this research design?
Answer: Using property tax rolls as the sampling frame in the second stage may systematically exclude tenant farmers who do not own their land.
When evaluating survey methodology, you need to distinguish between different types of bias. Coverage bias occurs when your sampling frame systematically excludes certain members of your target population, creating a gap between who you want to study and who you can actually reach.
In this agricultural survey, the researchers want to estimate farmer support broadly, but their second-stage sampling frame—property tax rolls—creates a fundamental coverage problem. Property tax rolls only list landowners, which means tenant farmers who rent or lease their land will never appear in the sampling frame and thus can never be selected for the survey. This systematically excludes an entire category of farmers from potential participation, regardless of their attitudes toward the subsidy. Since tenant farmers may have different economic circumstances and policy preferences than landowners, this exclusion could significantly bias the results.
Looking at the other options: (A) describes sampling variability, not coverage bias—random selection could theoretically include any agricultural region, even if some are missed by chance. (B) identifies nonresponse bias, where selected participants choose not to respond, rather than coverage bias where people can't be selected at all. (C) describes clustering effects that increase sampling error but don't prevent any farmers from being included in the sampling frame.
Remember that coverage bias is about who can't even be reached by your sampling design, while other biases involve problems with people who could theoretically be included. Always check whether the sampling frame matches the target population.
- A political advocacy group commissions a survey that includes the following question: 'In light of the constant threat of foreign cyberattacks and the need to protect national security, do you support or oppose government programs that monitor internet communications?' This question is most likely to produce biased results due to its use of:
Answer: A leading premise and loaded terminology that encourages an affirmative response.
The question's primary flaw is the introductory clause, 'In light of the constant threat...and the need to protect national security'. This is a leading premise that frames the issue in a way that makes support seem like the only reasonable option. Terms like 'constant threat' are loaded and designed to elicit an emotional response, biasing the results toward agreement.
- A researcher studying political trust in a large, geographically diverse state divides the state into its 58 counties. The researcher then randomly selects 10 of these counties and attempts to survey every adult within those 10 selected counties. Which of the following is the most significant methodological weakness of this approach for generating a precise statewide estimate?
Answer: It is susceptible to high sampling error because individuals within counties may be more politically homogeneous than the state as a whole.
The method described is one-stage cluster sampling. Its primary weakness is that clusters (counties) are often internally homogeneous but different from other clusters. By selecting only a few clusters, the sample may not capture the full diversity of the state, leading to higher sampling error compared to a simple random sample of the same size. The estimate's precision is therefore reduced.
- A researcher conducts a split-ballot experiment. Half the respondents are asked Question A ('Overall, do you approve or disapprove of the job Congress is doing?') and then Question B ('Do you approve or disapprove of the job your own representative in Congress is doing?'). The other half are asked the questions in reverse order (B then A). The researcher finds that approval for 'your own representative' is significantly higher when asked before the general question about Congress. This research design is explicitly measuring the impact of:
Answer: Question order effects, where the context of prior questions influences subsequent answers.
This is a classic experimental design to test for question order effects. By randomly assigning respondents to different question orders and observing a systematic difference in responses, the researcher can isolate the causal impact of the question sequence. In this case, asking about the unpopular Congress first 'primes' respondents to be more critical, and some of that negativity carries over to their evaluation of their own representative.
- A student researcher studying political engagement on campus stands in front of the main library on a weekday and surveys students who pass by. The researcher's goal is to survey 150 students, ensuring that the sample includes 50 freshmen, 40 sophomores, 30 juniors, and 30 seniors. This sampling method is best described as:
Answer: Quota sampling, because participants were selected non-randomly until pre-defined categories were filled.
This is a non-probability sampling method. While it involves creating categories (strata) like stratified sampling, the selection of individuals within those categories is not random. The researcher surveys conveniently available students until the quota for each class year is met. This combination of convenience selection with demographic targets is the definition of quota sampling.
- A polling firm aims to predict the winner of a Republican primary election. The firm obtains a list of all registered Republicans from the local board of elections and draws a simple random sample from it. Which potential source of bias is most directly related to this specific choice of sampling frame?
Answer: Coverage bias, as the frame of all registered Republicans is not identical to the population of actual primary voters.
Coverage bias occurs when the sampling frame (the list from which the sample is drawn) does not perfectly match the target population. In this case, the target population is 'Republican primary voters.' The sampling frame is 'registered Republicans.' These are not the same; many registered Republicans will not vote in the primary, and in some states, independents or new registrants may be eligible to vote. This mismatch is a form of coverage bias.
- A university surveys students about their experiences with illegal substance use on campus. It considers two methodologies: a live, in-person interview conducted by a trained research assistant, or a self-administered, anonymous online questionnaire. The anonymous online questionnaire is likely to yield more accurate data on this topic primarily because it reduces:
Answer: Response bias stemming from social desirability, as respondents are more willing to admit to sensitive behaviors anonymously.
The key factor here is the sensitive nature of the topic (illegal substance use). In a face-to-face or live interview setting, respondents are more likely to feel judged or worried about consequences, leading them to underreport sensitive behaviors. This is a form of response bias driven by social desirability. An anonymous, self-administered format removes the interviewer from the equation, making respondents feel safer and more likely to provide truthful answers.
- A researcher is studying the political attitudes of a minority group that constitutes 4% of a state's population. To ensure the group's views are captured for meaningful subgroup analysis, the researcher designs a sample of 1,200 people that includes 300 members of the minority group (25% of the sample) and 900 from the rest of the population. To produce an accurate estimate of opinion for the entire state's population from this sample, what is the most critical next step?
Answer: The data must be statistically weighted to down-weight the oversampled minority group to its true 4% proportion in the population.
When you encounter a question about survey sampling with oversampling, you're dealing with the fundamental challenge of making valid population inferences from non-representative samples. Oversampling minorities is a common technique to ensure adequate sample sizes for subgroup analysis, but it creates a critical problem for population estimates.
The correct approach is A - the data must be weighted to reflect the true population proportions. Since the minority group represents 25% of your sample but only 4% of the actual population, you need to statistically down-weight their responses when calculating statewide estimates. This weighting process restores representativeness and allows valid population inference.
B is incorrect because oversampling doesn't make this a non-probability sample - it's still probability-based sampling with known selection probabilities that can be corrected through weighting. C is wrong because the groups can absolutely be combined once proper weights are applied; reporting separately would actually prevent you from achieving your goal of statewide estimates. D misunderstands the solution - while disproportionate sampling does affect precision, simply increasing the margin of error doesn't address the fundamental bias problem of overrepresentation.
Remember this pattern: oversampling creates bias (systematic error), not just increased variance (random error). Weighting fixes bias by mathematically restoring proper population proportions. When you see oversampling questions, immediately think "weighting" rather than "larger margins of error" - the latter only addresses random sampling variation, not systematic overrepresentation.
- Compared to a standard pre-election telephone poll that surveys 'likely voters,' what is the principal methodological advantage of an exit poll that interviews voters as they leave polling places?
Answer: It samples directly from the population of actual voters, thus eliminating the error and assumptions inherent in turnout models.
When evaluating polling methodologies, the key distinction is between measuring intentions versus actual behavior. Pre-election polls face the fundamental challenge of predicting who will actually vote, while exit polls capture reality as it happens.
Answer A correctly identifies the principal advantage: exit polls sample directly from actual voters, eliminating turnout model errors. Pre-election polls must guess which respondents will actually vote using complex "likely voter" models based on past voting history, enthusiasm levels, and demographic factors. These models introduce significant uncertainty because voting behavior can be unpredictable. Exit polls bypass this entirely by interviewing people who have demonstrably voted.
Answer B is incorrect because social desirability bias doesn't disappear after voting. Voters may still feel pressure to give socially acceptable responses about their choices, and the privacy of their actual vote doesn't necessarily make them more honest with pollsters.
Answer C is wrong because exit polls typically have smaller, more geographically concentrated samples than major pre-election polls. They're limited to specific polling locations and can miss certain demographic groups, like early voters or mail-in ballot users.
Answer D mischaracterizes exit polls as non-probability sampling, when they actually use systematic sampling methods at selected precincts. While potentially faster to execute on election day, the primary advantage isn't speed or cost but accuracy in identifying the actual voting population.
Remember: the fundamental polling challenge is predicting behavior versus measuring it. Exit polls' strength lies in capturing actual voting behavior rather than predicting it.
- Two polling firms, A and B, survey the same population about an upcoming election. Poll A surveys 400 likely voters and reports a margin of error of ±5.0%. Poll B, using an identical methodology and confidence level, surveys 1,600 likely voters. Assuming all other factors are equal, what is the most likely margin of error for Poll B?
Answer: ±2.50%
The margin of error is inversely proportional to the square root of the sample size (n). To get from Poll A's sample size to Poll B's, we multiply by 4 (400 * 4 = 1600). Therefore, the margin of error for Poll B will be the margin of error for Poll A divided by the square root of 4, which is 2. So, 5.0% / 2 = 2.5%.
- A major news website features a daily online poll on its homepage asking visitors to rate the president's performance. The poll consistently receives over 100,000 responses each day. Despite the very large sample size, these results are not considered scientifically valid primarily because:
Answer: The sample is self-selected, making it unrepresentative of the general population.
This is an opt-in or self-selected poll. The participants are not chosen through a random process; they choose to participate themselves. This introduces massive selection bias. The sample will be composed of people who visit that specific website and who feel strongly enough to participate, rather than a representative cross-section of the population. No sample size, however large, can overcome this fundamental flaw. Weighting (D) cannot fix a sample this biased.
- A survey on government priorities asks respondents the following question: 'Do you agree or disagree that the federal government should reduce taxes on small businesses and decrease regulations on the energy sector?' What is the most significant flaw in this survey question's design?
Answer: It is a double-barreled question, compelling a single response to two separate policy issues.
This is a classic double-barreled question. It asks about two distinct issues: 1) reducing taxes on small businesses, and 2) decreasing regulations on the energy sector. A respondent might agree with one part but disagree with the other, but the question forces them into a single 'agree' or 'disagree' answer, making the results impossible to interpret accurately.
- During the final month of a presidential campaign, a polling organization interviews 300 likely voters each night. To report a daily trend in candidate support, they use a 4-day rolling average, combining the most recent 1,200 interviews for each day's report. What is the primary analytical trade-off involved in using this rolling average method?
Answer: It reduces the poll's margin of error for the daily estimate but is less sensitive to very sudden shifts in opinion.
The main benefit of a rolling average is that it uses a larger sample size (1,200 instead of 300), which reduces the random sampling error (margin of error) and creates a more stable, less volatile trend line. The trade-off is that because it includes data from previous days, it will react more slowly to a sudden, major event that changes public opinion overnight. It sacrifices some responsiveness for greater statistical stability.
- This discrepancy, where a minority candidate's support is inflated in pre-election polls, is most consistent with which polling phenomenon?
Answer: The Bradley Effect, a form of social desirability bias affecting polls with minority candidates.
The Bradley Effect (or Wilder Effect) is the theory describing a phenomenon where polls overstate the support for a minority candidate. It is attributed to a form of social desirability bias, where some white voters tell pollsters they are undecided or will vote for the minority candidate because they feel it is the socially acceptable answer, but then vote for the white candidate in the privacy of the voting booth.
- Which of the following represents the most methodologically sound interpretation of this poll?
Answer: The race is a statistical tie because the confidence intervals for the two candidates' support levels overlap.
The margin of error applies to each candidate's percentage. Candidate A's true support is likely between 44.9% and 51.1%. Candidate B's true support is likely between 41.9% and 48.1%. Because these two ranges overlap, it is statistically possible that Candidate B is actually ahead of Candidate A. Therefore, the lead is not statistically significant, and the race is considered a statistical tie.
- A poll reports that 45% of citizens approve of a new law, with a margin of error of ±4% at a 95% confidence level. What is the correct interpretation of the '95% confidence level'?
Answer: If the same survey were conducted 100 times, the resulting confidence interval would contain the true population value in approximately 95 of the trials.
The confidence level refers to the long-run performance of the method, not the probability associated with a single, specific interval. The correct interpretation is that if we were to repeat the sampling process many times, 95% of the confidence intervals we construct would capture the true, unknown population parameter. Answer A is a common but incorrect interpretation from a frequentist statistical perspective.
- What is the primary purpose of this statistical adjustment?
Answer: To apply a weight to responses so that the sample's gender composition matches the known population parameter.
This process is called weighting. The responses of the underrepresented group (men) are given slightly more weight, and the responses of the overrepresented group (women) are given slightly less weight. The goal is to make the final, adjusted sample accurately reflect the demographic composition of the population it is intended to represent (in this case, the actual electorate).
