Test Construction - Eppp Knowledge

1. A negative discrimination index most strongly suggests what item problem?
   Answer: The item is measuring a different construct This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of a negative discrimination index is key, as it ensures detection of items that may measure unintended constructs. The correct answer works because it accurately identifies the item measuring a different construct and its importance in test construction. A common distractor fails because it incorrectly links negative discrimination to appropriate difficulty, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
2. Which practice best exemplifies applying item analysis to revise a test?
   Answer: Removing items with negative discrimination after a pilot study This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of item analysis in revision is key, as it ensures removal of poorly performing items. The correct answer works because it accurately identifies removing negative discrimination items and its importance in test construction. A common distractor fails because it alters norms without data, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
3. A normative sample overrepresents high-income students. A low-income student's percentile will likely appear
   Answer: Lower than it should be Because the norm group contains too many high-income students, their higher scores push the percentile benchmarks up. A low-income student's score is then compared against an inflated standard, so their percentile looks lower than it would be with a representative sample. The tempting error is thinking overrepresentation inflates the student's percentile, but it actually raises the comparison group, not the student's score.
4. A test development team creates norms based on a sample recruited through online panels, resulting in overrepresentation of individuals with internet access and technology familiarity. What statistical approach would best address this sampling limitation?
   Answer: Apply post-stratification weighting to match known population characteristics on relevant demographics Post-stratification weighting adjusts the sample to match known population characteristics (age, education, income, geographic distribution, etc.) that correlate with internet access and technology familiarity. This statistical correction helps address the bias introduced by the recruitment method without requiring complete re-norming. A: Incorrect - reliability coefficients don't address norm bias. C: Incorrect - larger samples with the same bias don't solve the representativeness problem. D: Incorrect - alternative norms aren't necessary when weighting can address the bias.
5. In developing a cognitive assessment, researchers find that Item 15 has a difficulty index (p-value) of 0.95. What implication does this have for the item's contribution to the overall test?
   Answer: The item is too easy and likely contributes minimal information about individual differences in ability A difficulty index (p-value) of 0.95 means 95% of examinees answered correctly, making the item extremely easy. Very easy items provide little discrimination between examinees of different ability levels and contribute minimal information about individual differences. Such items typically have low discrimination indices and don't help differentiate among examinees. A: Incorrect - very easy items actually provide poor discrimination. C: Incorrect - extremely easy items are generally not optimal. D: Incorrect - difficulty level doesn't directly indicate criterion validity.
6. Which situation most clearly indicates norms may be outdated?
   Answer: Population performance shifts due to cohort (Flynn) effects This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of outdated norms is key, as it ensures recognition of performance shifts like Flynn effects. The correct answer works because it accurately identifies population shifts due to cohort effects and its importance in test construction. A common distractor fails because it links positive discrimination to outdated norms, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
7. A test has 90% sensitivity and 95% specificity; prevalence is 2%. What is the approximate positive predictive value?
   Answer: 27% In 10,000 people, 200 have the disease and 180 test positive; 9,800 do not, and 5% of them, or 490, test false positive. So positive tests total 670, and 180/670 is about 27%. The tempting 95% mistakes specificity for predictive value, ignoring that false positives overwhelm true positives when prevalence is low.
8. Which example best illustrates standardization in practice?
   Answer: Providing identical prompts and timing for all examinees This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of standardization in practice is key, as it ensures identical procedures for all. The correct answer works because it accurately identifies providing identical prompts and timing and its importance in test construction. A common distractor fails because it focuses on item selection instead of administration, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
9. During item analysis, which statistic reflects the proportion answering correctly?
   Answer: Item difficulty index This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of item difficulty index is key, as it ensures the proportion of correct responses is quantified for test refinement. The correct answer works because it accurately identifies the item difficulty index and its importance in test construction. A common distractor fails because it confuses item analysis with reliability measures like test-retest, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
10. Which item analysis approach uses high-low group comparisons?
    Answer: Upper-lower discrimination method This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of high-low group comparisons is key, as it ensures effective evaluation of item discrimination. The correct answer works because it accurately identifies the upper-lower discrimination method and its importance in test construction. A common distractor fails because it confuses this method with reliability estimation like split-half, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
11. In establishing norms for a new personality inventory, researchers stratify their sample based on age, education, and geographic region, then recruit participants within each stratum. What type of sampling strategy does this represent?
    Answer: Stratified random sampling designed to ensure proportional representation across key demographic variables Stratified random sampling involves dividing the population into subgroups (strata) based on important characteristics (age, education, geographic region), then randomly sampling within each stratum. This ensures representation across key demographic variables for norming purposes. A: Incorrect - systematic sampling uses predetermined intervals, not stratification. C: Incorrect - cluster sampling uses naturally occurring groups, not artificial strata. D: Incorrect - the description indicates systematic stratification, not convenience sampling with quotas.
12. A psychological test manual reports coefficient alpha of 0.91, test-retest reliability of 0.85, and standard error of measurement of 3.2 points. What do these statistics collectively suggest about the test's measurement precision?
    Answer: The test exhibits strong reliability characteristics with good precision, though some measurement error remains All three statistics indicate good reliability: coefficient alpha of 0.91 shows excellent internal consistency, test-retest of 0.85 shows good temporal stability, and SEM of 3.2 indicates reasonable precision (exact interpretation depends on scale, but these values collectively suggest strong psychometric properties with acknowledgment that measurement error exists). A: Incorrect - 0.85 test-retest is good, not concerning. C: Incorrect - these reliability coefficients are strong. D: Incorrect - no test has minimal measurement error; some error always exists.
13. Which statement best distinguishes norming from standardization?
    Answer: Norming provides comparison distributions; standardization ensures uniform procedures This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of distinguishing norming from standardization is key, as it ensures clarity in test development processes. The correct answer works because it accurately identifies norming for comparisons and standardization for procedures and its importance in test construction. A common distractor fails because it reverses their roles, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
14. An item analysis reveals that 25% of examinees in the high-scoring group and 65% of examinees in the low-scoring group answered Item 23 correctly. What is the discrimination index for this item?
    Answer: $D = -0.40$, indicating that this item negatively discriminates between ability levels The discrimination index D = (proportion correct in high group) - (proportion correct in low group) = 0.25 - 0.65 = -0.40. The negative value indicates that more low-scoring examinees got the item correct than high-scoring examinees, suggesting the item is functioning in reverse of what's expected and likely has problems (e.g., incorrect keying, ambiguous wording). B: Incorrect - fails to account for which group performed better. C: Incorrect calculation and wrong interpretation. D: Incorrect calculation that doesn't reflect the actual discrimination pattern.
15. How does norming contribute to fairness in a psychology exam?
    Answer: It allows interpretation relative to an appropriate comparison group This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of norming in fairness is key, as it ensures relative interpretation to comparison groups. The correct answer works because it accurately identifies allowing interpretation relative to groups and its importance in test construction. A common distractor fails because it replaces standardization with judgment, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
16. In a normal distribution, a raw score at the 84th percentile corresponds to which T-score?
    Answer: 60 The 84th percentile is 1 standard deviation above the mean. T-scores have a mean of 50 and a standard deviation of 10, so 1 SD above the mean is 50 + 10 = 60. A T-score of 70 would be 2 SDs above the mean, about the 97.7th percentile, not the 84th. The tempting error is choosing 50, but that is the mean, which is the 50th percentile.
17. In standardization, what is the role of scoring rubrics?
    Answer: They define consistent criteria for assigning scores This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of scoring rubrics in standardization is key, as it ensures consistent score assignment. The correct answer works because it accurately identifies defining consistent criteria and its importance in test construction. A common distractor fails because it overstates rubrics as guaranteeing validity, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
18. In diagnostic screening, what does sensitivity describe?
    Answer: Correctly identifying individuals with the disorder This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of sensitivity in screening is key, as it ensures correct identification of those with the disorder. The correct answer works because it accurately identifies detecting individuals with the disorder and its importance in test construction. A common distractor fails because it confuses sensitivity with specificity for non-cases, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.
19. An item analysis reveals that for Item 25, 80% of high-scoring examinees answered correctly while 45% of low-scoring examinees answered correctly. What is the discrimination index, and how should this item be evaluated?
    Answer: $D = 0.35$; the item shows acceptable discrimination and should be retained in the test Discrimination index D = (proportion correct in high group) - (proportion correct in low group) = 0.80 - 0.45 = 0.35. This indicates acceptable discrimination (threshold typically 0.30+), meaning the item effectively differentiates between high and low ability examinees. Items with D ≥ 0.30 are generally retained. B: Incorrect calculation (0.80 × 0.45 rather than subtraction). C: Incorrect calculation and impossible value (D cannot exceed 1.0). D: Incorrect - uses low group proportion rather than calculating difference.
20. A key requirement for a high-quality norming sample is what?
    Answer: It should be representative of the intended test population This question tests the application of item analysis, standardization, norming, sensitivity, and specificity in psychological assessments. These concepts are crucial for ensuring the reliability and validity of psychological tests. Item analysis, for example, evaluates items based on difficulty and discrimination to refine test quality. In this question, understanding the role of norming sample quality is key, as it ensures representativeness for the target population. The correct answer works because it accurately identifies being representative and its importance in test construction. A common distractor fails because it limits to clinical cases only, which is a frequent misunderstanding. Teaching strategies include emphasizing the differences between these concepts through examples and encouraging practice in applying them to real-world scenarios, which helps in identifying subtle distinctions and preventing common errors.

Start this quiz