Psychology Quiz: Reliability And Validity

A researcher is assessing the split-half reliability of a new 40-item survey. She correlates the sum of scores on the 20 odd-numbered items with the sum of scores on the 20 even-numbered items, yielding a correlation coefficient of r = .70. What is the most appropriate next step and conclusion?

A high school physics teacher creates a cumulative final exam. The course curriculum was divided equally among four topics: mechanics, thermodynamics, electromagnetism, and optics. However, over 80% of the exam questions are about mechanics. A review by the department head would most likely identify a primary weakness in the exam's:

A researcher conducts a six-month study to assess the impact of a mindfulness meditation program on employee stress. Three months into the study, the company undergoes a major restructuring, leading to widespread job uncertainty and layoffs. The researcher notes that stress levels, as measured by a self-report questionnaire, are significantly higher at the end of the study than at the beginning. Attributing this increase in stress solely to the failure of the program would ignore which threat to internal validity?

A social psychologist publishes a study concluding that people are less likely to help a stranger in need when other bystanders are present. The study was conducted on a public plaza using only American college students as participants. Which of the following critiques most directly addresses the study's external validity?

Two clinical psychologists independently watch a video of a client's therapy session and rate the client's 'expression of affect' on a 1-to-7 scale for 20 distinct one-minute intervals. A comparison of their ratings yields a very low correlation (r = .23). Which of the following is the most direct threat to the validity of the study's conclusions about the client's affect?

A cognitive psychologist studies the impact of a new 'brain-training' game on fluid intelligence. An experimental group plays the game for 30 minutes daily for four weeks, while a control group does not. Both groups are given an IQ test before and after the four-week period. The experimental group shows a significantly greater increase in IQ scores. A critic argues that the improvement may be due to the experimental group becoming more comfortable with the testing situation itself. This criticism refers to which potential confound?

A school district implements a new anti-bullying program in its middle schools. To identify the schools most in need, they select the 10 schools with the highest reported incidents of bullying in the previous year to receive the program. At the end of the school year, these 10 schools show a marked decrease in bullying incidents. The superintendent claims the program was a success. A methodologist might temper this conclusion by citing which threat to internal validity?

To establish the construct validity of a new self-report measure of 'emotional intelligence,' a researcher correlates scores on the new measure with scores from three existing scales: a well-validated emotional intelligence test, a measure of social desirability, and a test of general cognitive ability (IQ). Which pattern of correlations would provide the strongest evidence for the new measure's construct validity?

To validate a new leadership aptitude test for corporate managers, a company administers the test to its entire current management team. The researchers then gather the most recent annual performance evaluation scores for each manager and find a strong positive correlation between the test scores and the performance ratings. This method of validation primarily gathers evidence for which specific type of validity?

A psychologist develops a 30-item questionnaire to measure trait conscientiousness. An analysis reveals that the items on the first half of the test are not strongly correlated with the items on the second half. Further analysis shows that several individual items have weak or negative correlations with the total score of the other 29 items. These findings present the most direct challenge to the questionnaire's:

To avoid socially desirable responding on a new test for hiring police officers, a psychologist includes many subtle, indirect items such as preferences for certain geometric shapes and colors. The test proves to be a very strong predictor of which recruits will receive commendations for bravery two years after being hired. This test most likely has:

A high school physics teacher creates a cumulative final exam. The course curriculum was divided equally among four topics: mechanics, thermodynamics, electromagnetism, and optics. However, over 80% of the exam questions are about mechanics. A review by the department head would most likely identify a primary weakness in the exam's:

A psychologist develops a 30-item questionnaire to measure trait conscientiousness. An analysis reveals that the items on the first half of the test are not strongly correlated with the items on the second half. Further analysis shows that several individual items have weak or negative correlations with the total score of the other 29 items. These findings present the most direct challenge to the questionnaire's:

A research team develops a new scale to measure 'scholarly grit.' The scale is administered to a group of graduate students twice, one month apart, and the scores show a high correlation (r = .89). However, scores on the scale do not correlate with existing measures of perseverance, conscientiousness, or long-term goal attainment. Which statement best describes the psychometric properties of this new scale?

A school district implements a new anti-bullying program in its middle schools. To identify the schools most in need, they select the 10 schools with the highest reported incidents of bullying in the previous year to receive the program. At the end of the school year, these 10 schools show a marked decrease in bullying incidents. The superintendent claims the program was a success. A methodologist might temper this conclusion by citing which threat to internal validity?

A researcher conducts a six-month study to assess the impact of a mindfulness meditation program on employee stress. Three months into the study, the company undergoes a major restructuring, leading to widespread job uncertainty and layoffs. The researcher notes that stress levels, as measured by a self-report questionnaire, are significantly higher at the end of the study than at the beginning. Attributing this increase in stress solely to the failure of the program would ignore which threat to internal validity?

A researcher is assessing the split-half reliability of a new 40-item survey. She correlates the sum of scores on the 20 odd-numbered items with the sum of scores on the 20 even-numbered items, yielding a correlation coefficient of r = .70. What is the most appropriate next step and conclusion?

A researcher investigating the effect of caffeine on motor skills tells the experimental group, 'You are receiving a high dose of caffeine that should improve your performance.' He tells the control group, 'You are receiving a placebo.' The researcher is not blinded to the group assignments. He observes that the experimental group performs better. The outcome is potentially compromised by which two confounds?

Two clinical psychologists independently watch a video of a client's therapy session and rate the client's 'expression of affect' on a 1-to-7 scale for 20 distinct one-minute intervals. A comparison of their ratings yields a very low correlation (r = .23). Which of the following is the most direct threat to the validity of the study's conclusions about the client's affect?

A cognitive psychologist studies the impact of a new 'brain-training' game on fluid intelligence. An experimental group plays the game for 30 minutes daily for four weeks, while a control group does not. Both groups are given an IQ test before and after the four-week period. The experimental group shows a significantly greater increase in IQ scores. A critic argues that the improvement may be due to the experimental group becoming more comfortable with the testing situation itself. This criticism refers to which potential confound?