College Political Science Quiz: Experimental Designs
20 questions · exam conditions
0:00
Experimental DesignsQuestion 1 of 20

A researcher investigates the impact of local television news on political polarization. They exploit the fact that some counties in a state are assigned to a media market dominated by an out-of-state, highly partisan news station due to historical broadcast signal patterns. This assignment is unrelated to the counties' political leanings. This research design is best categorized as a:

laboratory experiment, because the treatment (news exposure) is clearly defined.
natural experiment, because a process outside the researcher's control created as-if random assignment.
correlational study, because the researcher is only observing an association.
regression discontinuity design, because it uses a geographic cutoff point.
← Back to quizzes

College Political Science Quiz

College Political Science Quiz: Experimental Designs

Practice Experimental Designs in College Political Science with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Experimental Designs, giving you a quick way to practice the rules, question types, and explanations that matter most for College Political Science.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

A researcher investigates the impact of local television news on political polarization. They exploit the fact that some counties in a state are assigned to a media market dominated by an out-of-state, highly partisan news station due to historical broadcast signal patterns. This assignment is unrelated to the counties' political leanings. This research design is best categorized as a:

  1. laboratory experiment, because the treatment (news exposure) is clearly defined.
  2. natural experiment, because a process outside the researcher's control created as-if random assignment. (correct answer)
  3. correlational study, because the researcher is only observing an association.
  4. regression discontinuity design, because it uses a geographic cutoff point.
Explanation: This is a classic example of a natural experiment. The assignment of counties to media markets was determined by an external process (broadcast technology and history) that is plausibly random with respect to the outcome of interest (political polarization). The researcher did not manipulate the treatment but is leveraging this 'natural' assignment to make a causal claim. It is stronger than a simple correlational study (C) due to this as-if randomization.

Question 2

A state legislature is considering a bill to provide a tax credit to small businesses. The legislature is split 50-50, and the Lieutenant Governor casts the tie-breaking vote in favor of the bill. A researcher compares the economic outcomes of small businesses in this state to those in a neighboring state where a similar bill failed by a wide margin. This research design is best described as:

  1. a field experiment, because it studies the effect of a real-world policy.
  2. a natural experiment, because the tie-breaking vote created as-if random assignment of the policy. (correct answer)
  3. a difference-in-differences design, because it compares two states over time.
  4. a correlational study, because there was no explicit randomization by the researcher.
Explanation: This scenario represents a natural experiment that uses a regression discontinuity logic. The tie-breaking vote creates a situation where the policy was passed by the narrowest possible margin, making its passage 'as-if' random. By comparing the outcome in this state to a state where the bill failed, the researcher can leverage this chance event. While it has elements of a difference-in-differences setup (C), the key feature that strengthens the causal claim is the exogenous shock provided by the tie-breaking vote, making 'natural experiment' the most accurate description.

Question 3

In an experiment designed to test the persuasive effects of different campaign messages, participants in the treatment group are brought into a university media lab and shown a political ad. The researchers observe a significant change in their stated policy preferences. However, the effect may be partly explained by the participants' awareness that they are being studied, causing them to be more thoughtful or agreeable than they would be otherwise. This phenomenon is known as:

  1. attrition bias.
  2. selection bias.
  3. the Hawthorne effect. (correct answer)
  4. the placebo effect.
Explanation: The Hawthorne effect describes a change in subjects' behavior resulting from their awareness of being observed. In this case, the artificiality of the lab setting and the knowledge of being part of a study could be influencing their responses, separate from the content of the political ad. The placebo effect (D) is related but typically refers to a beneficial effect produced by a subject's belief in a treatment, which is less applicable here than the broader concept of observation effects.

Question 4

A researcher wants to assess an online workshop designed to improve citizens' ability to detect misinformation. Participants take a pre-test measuring this skill, complete the workshop, and then take the same test again as a post-test. A significant improvement in scores is observed. A colleague worries that the improvement might be due to the pre-test itself, rather than the workshop. This potential confound is an example of:

  1. a history effect.
  2. an instrumentation effect.
  3. a testing effect. (correct answer)
  4. a maturation effect.
Explanation: A testing effect occurs when the act of taking a test (the pre-test) influences the scores on a subsequent test (the post-test). For example, the pre-test may have clued participants into what the workshop would be about or made them more familiar with the question format, leading to improvement independent of the workshop's content. To mitigate this, a researcher could use a Solomon four-group design, which includes control groups that do not receive the pre-test.

Question 5

A political psychology experiment conducted on a university campus finds that a brief, empathy-inducing video about immigrants significantly reduces prejudice scores among undergraduate student participants. Critics question whether this effect would be observed among the general adult population in a non-academic setting. This criticism is primarily focused on the study's:

  1. internal validity.
  2. construct validity.
  3. statistical power.
  4. external validity. (correct answer)
Explanation: External validity refers to the extent to which the results of a study can be generalized to other populations, settings, or times. The criticism here is precisely that the findings from a specific population (college students) in a specific setting (a university lab) may not apply to the broader population. Internal validity (A), by contrast, refers to the degree of confidence that the causal relationship being tested is trustworthy and not influenced by other factors within the experiment itself.

Question 6

A researcher compares two groups of citizens to understand the effects of jury duty on attitudes toward the justice system. The treatment group consists of people who chose to serve on a jury after being summoned, while the control group consists of people who were summoned but used a legal exemption to avoid service. Why does this design fail as a true experiment?

  1. The sample size is likely to be too small for statistical significance.
  2. The treatment and control groups are not comparable due to self-selection bias. (correct answer)
  3. Attitudes toward the justice system are too subjective to be measured reliably.
  4. The study lacks a pre-test to measure attitudes before the treatment.
Explanation: This is a quasi-experimental design, not a true experiment, because it lacks random assignment. The core problem is self-selection: individuals who choose to serve on a jury may already have different attitudes, levels of civic duty, or other characteristics compared to those who choose to be excused. These pre-existing differences, rather than the experience of jury duty itself, could account for any observed differences in attitudes, making it impossible to draw a valid causal conclusion.

Question 7

A research team partners with a large city to study the effect of participatory budgeting on residents' trust in local government. The city randomly selects 50 neighborhoods to implement participatory budgeting, while another 50 demographically matched neighborhoods serve as a control group. Researchers survey residents in all 100 neighborhoods before and after the program's implementation.

The research design described in the passage is best classified as a:

  1. natural experiment.
  2. field experiment. (correct answer)
  3. laboratory experiment.
  4. difference-in-differences study.
Explanation: This is a field experiment. It has the defining feature of a true experiment—random assignment of the treatment (participatory budgeting)—but it takes place in a real-world setting rather than a controlled laboratory. While it uses pre- and post-treatment data like a difference-in-differences study (D), the randomization makes it a field experiment, which is a stronger design. It is not a natural experiment (A) because the researchers controlled the random assignment.

Question 8

A researcher hypothesizes that watching satirical news shows (e.g., The Daily Show) increases political cynicism. Which of the following designs would provide the LEAST convincing evidence of a causal relationship?

  1. A field experiment randomly encouraging one group to watch such shows and another group to watch traditional news.
  2. A laboratory experiment where participants are randomly assigned to watch clips from either a satirical show or a sitcom.
  3. A natural experiment exploiting the sudden cancellation of a popular satirical show in some media markets but not others.
  4. A quasi-experimental study comparing survey data from self-reported viewers and non-viewers of satirical news. (correct answer)
Explanation: The quasi-experimental study comparing existing viewers and non-viewers provides the weakest causal evidence due to a severe self-selection problem. People who are already cynical may be more likely to seek out satirical news. The other three designs—field experiment (A), lab experiment (B), and natural experiment (C)—all use some form of random or as-if random assignment to create more comparable groups, thus providing stronger bases for causal inference.

Question 9

In a get-out-the-vote experiment, 2,000 people are randomly assigned to the treatment group (to receive a canvassing visit) and 2,000 to the control group. Due to logistical issues, canvassers only succeed in talking to 800 of the 2,000 people in the treatment group. An 'intention-to-treat' (ITT) analysis would preserve the benefits of randomization by comparing the turnout of which groups?

  1. The 800 people who were contacted versus the 2,000 people in the control group.
  2. The 800 people who were contacted versus the 1,200 people who were not contacted within the treatment group.
  3. All 2,000 people assigned to the treatment group versus all 2,000 people assigned to the control group. (correct answer)
  4. The 800 people who were contacted versus 800 people randomly chosen from the control group.
Explanation: The intention-to-treat (ITT) principle states that subjects should be analyzed in the groups to which they were originally randomized, regardless of whether they actually received the treatment. This approach preserves the baseline comparability created by randomization. Comparing only those who received the treatment (A, B, D) would introduce selection bias, as the 800 people who were successfully contacted might be systematically different from those who were not (e.g., more likely to be home, more willing to talk).

Question 10

A state implements a law allowing for automatic voter registration (AVR). To evaluate its impact, a researcher gathers data on voter turnout in that state for the election cycle before and after the law's passage. They also gather the same data from a neighboring state without AVR. For this difference-in-differences design to yield a valid causal estimate, which of the following assumptions is most critical?

  1. The two states must have had identical voter turnout rates before the law was passed.
  2. In the absence of the AVR law, the change in voter turnout in the treatment state would have been identical to the change in the control state. (correct answer)
  3. The AVR law must have been the only major electoral reform passed in the treatment state during the study period.
  4. Voter enthusiasm must have remained constant in both states over the two election cycles.
Explanation: This is the 'parallel trends' assumption, which is the cornerstone of the difference-in-differences (DiD) design. The method assumes that any difference in the change (or trend) in the outcome variable between the two groups is attributable to the treatment. For this to be true, we must assume that their trends would have been parallel if the treatment had not occurred. The other options are either too strong (A), related to confounding events (C), or unlikely to be true but not a core assumption of the method itself (D).

Question 11

A political scientist wants to study the effect of a new, intensive civics curriculum on high school students' political knowledge. The researcher convinces one school district to adopt the new curriculum, while a neighboring, demographically similar district continues with the standard curriculum. After one year, the researcher finds students in the first district have significantly higher political knowledge scores. This research design is best described as a quasi-experiment because:

  1. it takes place in a real-world setting rather than a laboratory.
  2. the treatment was assigned to school districts, not to individual students.
  3. the researcher could not control for all possible confounding variables.
  4. students and schools were not randomly assigned to the curriculum conditions. (correct answer)
Explanation: A quasi-experiment resembles a true experiment but lacks the key element of random assignment. In this scenario, the school districts were not randomly assigned to receive the new curriculum. This creates a major challenge for causal inference, as the districts might have differed in other ways (e.g., funding, teacher quality, student motivation) that could also explain the difference in outcomes.

Question 12

To assess the impact of winning a close election on a legislator's subsequent roll-call voting record, a researcher compares legislators who won their election by a margin of less than 1% to those who lost by a margin of less than 1%. This strategy of comparing subjects just above and below a specific threshold is the defining feature of which research design?

  1. A difference-in-differences design.
  2. A field experiment.
  3. A regression discontinuity design. (correct answer)
  4. An instrumental variable design.
Explanation: This is a regression discontinuity design (RDD). The logic is that, at the threshold (the 50% vote share needed to win), it is essentially random whether a candidate barely wins or barely loses. By comparing those just on either side of this cutoff, the researcher can isolate the causal effect of winning the election, as these candidates are likely very similar in all other respects.

Question 13

A political psychology experiment conducted on a university campus finds that a brief, empathy-inducing video about immigrants significantly reduces prejudice scores among undergraduate student participants. Critics question whether this effect would be observed among the general adult population in a non-academic setting. This criticism is primarily focused on the study's:

  1. internal validity.
  2. construct validity.
  3. statistical power.
  4. external validity. (correct answer)
Explanation: External validity refers to the extent to which the results of a study can be generalized to other populations, settings, or times. The criticism here is precisely that the findings from a specific population (college students) in a specific setting (a university lab) may not apply to the broader population. Internal validity (A), by contrast, refers to the degree of confidence that the causal relationship being tested is trustworthy and not influenced by other factors within the experiment itself.

Question 14

A political scientist uses a lottery for admission to an oversubscribed charter school to study its effect on students' later civic engagement. The lottery randomly determines which applicants are admitted. In this design, the lottery itself, which is perfectly randomized, is best understood as a(n):

  1. confounding variable.
  2. dependent variable.
  3. instrumental variable. (correct answer)
  4. moderating variable.
Explanation: This scenario describes the use of an instrumental variable (IV). The lottery (the instrument) is randomly assigned and influences the treatment (attending the charter school), but it does not directly affect the outcome (civic engagement) through any other path. Researchers use IVs to estimate the causal effect of a treatment when the treatment itself is not perfectly randomized (e.g., some who win the lottery may not attend).

Question 15

A political scientist investigates the effect of serving in student government on future political ambition. The researcher surveys alumni from a specific university, comparing the self-reported political ambition of those who served in student government with those who did not. The study finds a strong, positive association between serving and later ambition.

The study's causal conclusion that serving in student government increases political ambition is most seriously weakened by which of the following potential issues?

  1. Alumni may not accurately recall their level of ambition from years ago.
  2. The sample from a single university may not be representative of all young adults.
  3. The survey questions used to measure 'ambition' may lack construct validity.
  4. Students with high pre-existing political ambition are more likely to self-select into student government. (correct answer)
Explanation: This is a classic self-selection problem, a major threat to internal validity in quasi-experimental and observational research. It is highly plausible that individuals who already possess high levels of political ambition are the ones who choose to run for and serve in student government. Thus, the association found by the researcher may not be a causal effect of the experience, but rather a reflection of these pre-existing differences. The other options are valid methodological concerns, but self-selection is the most direct challenge to the specific causal claim.

Question 16

A researcher is studying the effect of incumbency on election outcomes. They compare the vote share of politicians who won their previous election with 50.1% of the vote to the vote share of those who lost with 49.9% of the vote. Which of the following is a necessary assumption for this regression discontinuity design to be valid?

  1. Voters must be unaware of the exact vote margin in the previous election.
  2. The politicians on either side of the 50% threshold must be, on average, otherwise identical in terms of quality and ideology. (correct answer)
  3. The outcomes of elections must be determined by a truly random process.
  4. The effect of incumbency must be constant across all levels of electoral competitiveness.
Explanation: The central assumption of an RDD is that subjects (in this case, politicians) just on either side of the cutoff are comparable on all other relevant characteristics. The design assumes that it is essentially random who falls just above versus just below the threshold. If, for example, more skilled politicians were systematically able to secure 50.1% of the vote while less skilled ones ended up with 49.9%, the assumption would be violated, and the design would be biased.

Question 17

A political scientist wants to study the effect of a new, intensive civics curriculum on high school students' political knowledge. The researcher convinces one school district to adopt the new curriculum, while a neighboring, demographically similar district continues with the standard curriculum. After one year, the researcher finds students in the first district have significantly higher political knowledge scores. This research design is best described as a quasi-experiment because:

  1. it takes place in a real-world setting rather than a laboratory.
  2. the treatment was assigned to school districts, not to individual students.
  3. the researcher could not control for all possible confounding variables.
  4. students and schools were not randomly assigned to the curriculum conditions. (correct answer)
Explanation: A quasi-experiment resembles a true experiment but lacks the key element of random assignment. In this scenario, the school districts were not randomly assigned to receive the new curriculum. This creates a major challenge for causal inference, as the districts might have differed in other ways (e.g., funding, teacher quality, student motivation) that could also explain the difference in outcomes.

Question 18

A study finds that cities with female mayors have, on average, smaller gender wage gaps in municipal employment. The researcher argues this is because female leaders implement more equitable pay policies. What is the most significant threat to the internal validity of this quasi-experimental study?

  1. Measurement error in calculating the gender wage gap.
  2. The Hawthorne effect, as municipal workers may change their behavior.
  3. Selection bias, as the types of cities that elect female mayors may already have more equitable values. (correct answer)
  4. Lack of external validity to private sector employment.
Explanation: The primary threat to internal validity in this design is selection bias. The cities that elect female mayors might already be more politically liberal or have stronger pre-existing commitments to gender equity. These underlying characteristics, rather than the mayor's gender itself, could be the true cause of the smaller wage gap. The other options describe different types of methodological issues, but selection bias is the most direct challenge to the causal claim.

Question 19

A research team partners with a large city to study the effect of participatory budgeting on residents' trust in local government. The city randomly selects 50 neighborhoods to implement participatory budgeting, while another 50 demographically matched neighborhoods serve as a control group. Researchers survey residents in all 100 neighborhoods before and after the program's implementation.

The research design described in the passage is best classified as a:

  1. natural experiment.
  2. field experiment. (correct answer)
  3. laboratory experiment.
  4. difference-in-differences study.
Explanation: This is a field experiment. It has the defining feature of a true experiment—random assignment of the treatment (participatory budgeting)—but it takes place in a real-world setting rather than a controlled laboratory. While it uses pre- and post-treatment data like a difference-in-differences study (D), the randomization makes it a field experiment, which is a stronger design. It is not a natural experiment (A) because the researchers controlled the random assignment.

Question 20

A political scientist uses a lottery for admission to an oversubscribed charter school to study its effect on students' later civic engagement. The lottery randomly determines which applicants are admitted. In this design, the lottery itself, which is perfectly randomized, is best understood as a(n):

  1. confounding variable.
  2. dependent variable.
  3. instrumental variable. (correct answer)
  4. moderating variable.
Explanation: This scenario describes the use of an instrumental variable (IV). The lottery (the instrument) is randomly assigned and influences the treatment (attending the charter school), but it does not directly affect the outcome (civic engagement) through any other path. Researchers use IVs to estimate the causal effect of a treatment when the treatment itself is not perfectly randomized (e.g., some who win the lottery may not attend).