All questions
Question 1
A school district wants to measure teacher effectiveness across different schools. They are considering four survey questions. Which question would produce the most reliable and unbiased data for comparison purposes?
- How would you rate your teacher's ability to help you learn on a scale of 1-5, considering your personal learning style?
- Compared to other teachers you've had, is this teacher above average, average, or below average in effectiveness?
- Rate how often your teacher clearly explains new concepts: Never, Rarely, Sometimes, Often, Always (correct answer)
- Do you think your teacher is effective at teaching, and what specific improvements would you suggest for better learning?
Explanation: Option C is correct because it measures a specific, observable behavior with a clear ordinal scale that allows for consistent interpretation across responses. Option A introduces bias by mentioning 'personal learning style.' Option B requires students to make comparisons they may not be qualified to make. Option D combines multiple questions and includes open-ended elements that are difficult to quantify consistently.
Question 2
A researcher studying teenage social media usage wants to minimize response bias in their survey. They are concerned that teens might not answer honestly about their screen time. Which approach would be most effective in reducing this bias?
- Use anonymous online surveys and ask teens to estimate their daily social media use in broad ranges (0-1 hours, 1-3 hours, etc.)
- Conduct face-to-face interviews with teens and their parents present to ensure honest responses about screen time
- Ask teens to track their usage using built-in phone apps for one week, then report those objective measurements anonymously (correct answer)
- Include multiple questions about social media habits throughout the survey to check for consistency in responses
Explanation: Option C is correct because it uses objective measurement (phone apps) rather than subjective estimates, reducing recall bias and social desirability bias, while maintaining anonymity. Option A still relies on estimates and social desirability bias. Option B would increase social desirability bias with parents present. Option D checks consistency but doesn't address the underlying bias in self-reporting screen time.
Question 3
A health researcher is designing a study to measure the effectiveness of a new exercise program on cardiovascular health. Which combination of variables and measurement timing would provide the strongest evidence for causation?
- Measure heart rate and blood pressure before and after 8 weeks, comparing participants to a control group doing regular activities (correct answer)
- Survey participants about their perceived fitness improvement and energy levels after completing the 8-week program
- Compare cardiovascular health metrics of program participants to national averages for their age and gender groups
- Track heart rate during exercise sessions and correlate with participants' self-reported effort levels throughout the program
Explanation: Option A is correct because it uses objective, quantifiable measures (heart rate, blood pressure), includes pre/post measurements to establish change, and uses a control group for comparison - all essential for establishing causation. Option B relies on subjective measures. Option C lacks a proper control group and baseline measurements. Option D measures correlation during exercise rather than health improvements from the program.
Question 4
A researcher studying the effectiveness of online learning wants to compare student performance between online and in-person classes. Which study design choice would best control for selection bias while maintaining practical feasibility?
- Compare volunteer online students with volunteer in-person students, matching them by GPA and major
- Randomly assign students to online or in-person sections of the same course taught by the same instructor (correct answer)
- Use existing data from students who chose online versus in-person options, controlling statistically for demographics and prior achievement
- Survey students in both formats about their learning preferences and self-assessed performance improvements
Explanation: Option B is correct because random assignment eliminates selection bias by ensuring that personal preferences and characteristics don't determine group membership, while keeping other factors constant (same course, same instructor). Option A still allows self-selection despite matching. Option C uses statistical controls but can't eliminate all unmeasured confounders from self-selection. Option D relies on subjective self-assessment rather than objective performance measures.
Question 5
A company wants to measure employee job satisfaction across five different departments with varying sizes (Department A: 200 employees, B: 50, C: 150, D: 75, E: 25). Which sampling approach would best ensure each department's concerns are adequately represented in the results?
- Simple random sample of 100 employees from the entire company, ensuring proportional representation by department size
- Stratified random sample taking 20 employees from each department, then weight responses by department size in analysis (correct answer)
- Survey all employees in the three smallest departments (B, D, E) and randomly sample 100 from the two largest (A, C)
- Cluster sampling by selecting two departments randomly and surveying all employees within those selected departments
Explanation: Option B is correct because stratified sampling ensures adequate representation from each department (especially smaller ones), while weighting by department size in analysis maintains the overall company perspective. Option A might miss smaller departments entirely. Option C creates inconsistent sampling methods across departments. Option D risks excluding entire departments and may not be representative of company-wide satisfaction.
Question 6
A marketing team wants to study customer satisfaction with their new product launch. They have a database of 10,000 customers who purchased the product. Which sampling strategy would best ensure their survey results are generalizable to all customers?
- Survey the first 500 customers who purchased the product, as they represent early adopters with strong opinions
- Post the survey on social media and company website, allowing any customer to respond voluntarily
- Randomly select 400 customers from the database and follow up with non-respondents using multiple contact methods (correct answer)
- Survey 100 customers from each of four different geographic regions to ensure balanced regional representation
Explanation: Option C is correct because random sampling from the full customer database ensures representativeness, and following up with non-respondents reduces non-response bias. Option A creates temporal bias by only including early adopters. Option B suffers from self-selection bias. Option D uses stratified sampling but doesn't ensure the geographic regions are representative of the customer base or that the sample size is adequate.
Question 7
A researcher wants to study the relationship between student sleep habits and academic performance. Which combination of study design and measurement approach would best minimize confounding variables while maintaining ethical standards?
- Experimental design where students are randomly assigned different sleep schedules, measuring GPA changes over one semester
- Observational study tracking self-reported sleep hours and existing GPA data, controlling for study time and course difficulty (correct answer)
- Survey asking students to estimate their average sleep and rank their academic satisfaction on a scale of 1-10
- Longitudinal study following students for four years, measuring sleep with wearable devices and tracking cumulative GPA
Explanation: Option B is correct because it uses an observational approach (ethical for sleep studies) while controlling for major confounding variables like study time and course difficulty. Option A is unethical as you cannot randomly assign sleep deprivation. Option C relies on subjective rankings rather than objective academic measures. Option D, while thorough, doesn't address confounding variables and would be extremely resource-intensive.
Question 8
A researcher wants to study the relationship between homework time and test scores. They plan to survey students about both variables. Which potential confounding variable should be addressed in the study design to strengthen the validity of conclusions?
- Student motivation levels, which could affect both time spent on homework and test performance independently (correct answer)
- The difficulty of specific test questions, which varies from test to test throughout the school year
- Teacher grading consistency, which might affect how test scores are assigned across different classes
- Student attendance rates, which could impact their familiarity with homework assignments and test material
Explanation: Option A is correct because student motivation is a true confounding variable - it causally affects both the exposure variable (homework time) and outcome variable (test scores), potentially creating a spurious association. Option B affects only the outcome variable. Option C affects only measurement of the outcome. Option D is more of a mediating variable in the causal pathway rather than a confounder.
Question 9
A pollster wants to estimate voting preferences in a city with three distinct neighborhoods that differ significantly in demographics and political leanings. The neighborhoods have populations of 15,000, 8,000, and 2,000 residents respectively. Which sampling strategy would produce the most accurate citywide estimate?
- Proportional stratified sampling with sample sizes of 150, 80, and 20 from each neighborhood respectively (correct answer)
- Simple random sampling of 250 residents from the entire city voter registration database
- Equal allocation stratified sampling with 83-84 residents sampled from each neighborhood
- Cluster sampling by randomly selecting one neighborhood and surveying 250 residents within it
Explanation: Option A is correct because proportional stratified sampling ensures each neighborhood is represented according to its actual size in the population while capturing the distinct political leanings of each stratum. Option B might miss smaller neighborhoods or not adequately represent diversity. Option C over-represents smaller neighborhoods relative to their population size. Option D completely ignores two neighborhoods and cannot provide citywide estimates.
Question 10
A survey researcher is designing questions to measure community support for a new public transportation system. Which question format would be most likely to produce valid, unbiased responses?
- Given the environmental benefits and reduced traffic congestion, how much do you support the new public transit system?
- Do you support or oppose the proposed public transportation system that will cost taxpayers $50 million annually?
- Considering both the benefits and costs, would you vote yes or no on the public transportation initiative?
- Rate your level of support for the proposed public transportation system: Strongly oppose, Oppose, Neutral, Support, Strongly support (correct answer)
Explanation: When evaluating survey questions, you need to identify potential sources of bias that could influence respondents' answers and look for formats that allow genuine opinion measurement.
Option D is correct because it provides a balanced, neutral format that doesn't lead respondents toward any particular answer. The five-point Likert scale from "Strongly oppose" to "Strongly support" captures the full spectrum of opinions without suggesting what the "right" answer should be. The question stem is factual and doesn't include persuasive language or selective information.
Option A contains leading language that primes respondents to think positively by emphasizing only benefits ("environmental benefits and reduced traffic congestion") while ignoring potential drawbacks. This creates response bias by suggesting support is the expected answer.
Option B commits the opposite error by front-loading negative information ("$50 million annually" cost) without mentioning any benefits. This pushes respondents toward opposition and creates the same bias problem as A, just in the opposite direction.
Option C appears balanced by mentioning "benefits and costs," but it forces respondents into a binary yes/no choice. This eliminates nuanced opinions and doesn't capture the range of support levels that exist in real populations. People might have mild preferences that get lost in forced binary choices.
For survey design questions, remember that valid measurement requires neutrality in both language and response options. Watch for questions that include only positive or negative framing, use loaded terminology, or force artificial binary choices when opinions naturally exist on a spectrum.