College Political Science Quiz: Validity And Reliability

A study uses an automated text analysis program to code the sentiment of tweets about a political candidate. To check the quality of this measure, the researchers manually code a random subsample of 1,000 tweets and find that the computer's sentiment score for a given tweet is highly correlated with the human-coded score. This procedure is primarily designed to establish the measure's:

A political science study finds that a get-out-the-vote (GOTV) intervention conducted by highly charismatic and trained student volunteers significantly increases voter turnout. Several campaigns attempt to replicate the intervention using their regular, less-trained volunteers but see no effect. The failure to replicate the original finding most likely stems from an issue with the original study's:

A researcher is studying the impact of campaign spending on election outcomes using observational data from 50 states over 20 years. The researcher uses a sophisticated statistical model to control for dozens of potentially confounding variables. A reviewer raises a concern that, despite the controls, the states that spend more on campaigns may differ from those that spend less in unmeasured ways, such as the underlying political culture. This concern fundamentally questions the study's:

To measure 'support for democratic norms,' a professor creates a survey consisting of ten questions about economic policy preferences, such as views on taxation and regulation. The survey produces consistent results when administered multiple times. The primary weakness of this measure is its lack of:

To measure 'support for democratic norms,' a professor creates a survey consisting of ten questions about economic policy preferences, such as views on taxation and regulation. The survey produces consistent results when administered multiple times. The primary weakness of this measure is its lack of:

A researcher develops a new scale to measure populist attitudes. To assess its validity, the researcher finds that scores on the scale are strongly correlated with support for parties widely considered populist, and negatively correlated with measures of support for global institutions. This process is an example of establishing:

A researcher conducts a longitudinal study on the effects of a civics education program on high school students' political engagement. A pre-test is administered in September, the program is implemented throughout the school year, and a post-test is administered in May. Between the pre-test and post-test, a highly contentious and mobilizing presidential election occurs. The researcher finds a large increase in political engagement. The election represents a significant threat to the study's internal validity known as:

A researcher hypothesizes that a new online deliberation platform increases participants' political efficacy. To test this, the researcher needs to ensure that the study has high internal validity. Which of the following design choices is most critical for achieving this goal?

A study uses an automated text analysis program to code the sentiment of tweets about a political candidate. To check the quality of this measure, the researchers manually code a random subsample of 1,000 tweets and find that the computer's sentiment score for a given tweet is highly correlated with the human-coded score. This procedure is primarily designed to establish the measure's:

A political scientist wants to generalize the findings of their experiment on campaign messaging to the entire voting population of the United States. Which of the following is most essential for ensuring high external validity of the findings?

A researcher is concerned that in her experiment on political persuasion, participants in the control group might become aware of the treatment being received by the other group and feel resentful, potentially altering their behavior on the outcome measure. This phenomenon, where the control group's behavior is altered due to their awareness of the treatment group, is a threat to internal validity known as:

An experiment is designed to test whether watching a negative political advertisement increases voter apathy. The researchers measure apathy before and after the ad is shown. They are concerned that the pre-test measurement itself might make participants more aware of their own feelings of apathy, causing them to respond differently to the post-test, regardless of the ad's content. This potential issue is a threat to internal validity known as:

A researcher conducts a longitudinal study on the effects of a civics education program on high school students' political engagement. A pre-test is administered in September, the program is implemented throughout the school year, and a post-test is administered in May. Between the pre-test and post-test, a highly contentious and mobilizing presidential election occurs. The researcher finds a large increase in political engagement. The election represents a significant threat to the study's internal validity known as:

Two graduate students are tasked with coding the tone of newspaper editorials regarding a new environmental policy. Student 1 codes 75% of the editorials as 'negative,' while Student 2, using the same coding scheme, codes only 45% as 'negative.' This discrepancy indicates a significant problem with which of the following?

A study compared voting rates between two groups of citizens. Group A received a series of mailers encouraging them to vote, while Group B did not. The researchers did not randomly assign citizens to the groups; instead, Group A consisted of individuals who had opted in to receive political communications. The study found that Group A had a higher turnout rate. Why is the study's internal validity compromised?

A research firm is hired to evaluate public support for a proposed tax increase. They conduct a phone survey using random digit dialing. However, the survey is conducted only between 9 a.m. and 5 p.m. on weekdays. The results show that 60% of the population opposes the tax. Critics argue the results are flawed.

The methodology described in the passage introduces a significant threat to the survey's external validity because:

A team of researchers is studying the effect of exposure to partisan news on political polarization. They recruit participants from a national, representative sample. Participants are randomly assigned to watch either a partisan news broadcast or a neutral news broadcast for 30 minutes. The researchers find that the group watching partisan news reports significantly higher levels of polarization. However, a critic notes that the study took place in a university laboratory, a highly controlled and artificial environment. This criticism most directly challenges the study's:

Two graduate students are tasked with coding the tone of newspaper editorials regarding a new environmental policy. Student 1 codes 75% of the editorials as 'negative,' while Student 2, using the same coding scheme, codes only 45% as 'negative.' This discrepancy indicates a significant problem with which of the following?

A political science study finds that a get-out-the-vote (GOTV) intervention conducted by highly charismatic and trained student volunteers significantly increases voter turnout. Several campaigns attempt to replicate the intervention using their regular, less-trained volunteers but see no effect. The failure to replicate the original finding most likely stems from an issue with the original study's:

Which of the following statements accurately describes the logical relationship between reliability and validity in social science measurement?