Reliability And Validity - Eppp Knowledge

1. Which reliability type is most relevant for performance ratings, behavioral coding, or diagnosis agreement?

   Interrater reliability. Crucial for subjective assessments where multiple evaluators must align to ensure objectivity.

2. What reliability type is most threatened when the construct or trait changes rapidly over time?

   Test–retest reliability. Vulnerable to actual fluctuations in unstable traits, leading to underestimated reliability.

3. Which reliability type is most relevant when two test versions are used to reduce practice effects?

   Alternate (parallel) forms reliability. Minimizes carryover from prior exposure by using comparable but distinct test versions.

4. Which broad validity framework integrates content, criterion, and construct evidence into one concept?

   Construct validity. Unifies diverse evidence sources to justify the theoretical interpretation of scores.

5. What is intrarater reliability designed to estimate?

   Consistency of the same rater across occasions. Measures stability of an individual's ratings over time, minimizing intra-observer variability.

6. What is interrater (interobserver) reliability designed to estimate?

   Agreement among different raters scoring the same targets. Quantifies consensus between independent observers, reducing subjectivity in scoring.

7. Which construct validity evidence is shown when a test has low correlations with measures of different constructs?

   Discriminant (divergent) validity evidence. Supports distinctiveness by weak associations with unrelated constructs, aiding multitrait-multimethod analysis.

8. What is alternate (parallel) forms reliability designed to estimate?

   Score equivalence across two comparable forms. Evaluates whether equivalent test versions yield similar results, controlling for form-specific errors.

9. Which validity form is supported when test scores correlate with an external criterion measured at the same time?

   Concurrent validity. Establishes criterion-related evidence by simultaneous correlation with established measures.

10. Which validity evidence is supported when the internal factor structure matches the theoretical model?

    Internal structure evidence (factorial validity). Validates the test's underlying dimensions through factor analysis matching expected structure.

11. Which construct validity evidence is shown when a test correlates strongly with measures of similar constructs?

    Convergent validity evidence. Confirms theoretical alignment by high correlations with conceptually related measures.

12. Which term refers to the degree to which evidence supports the intended interpretation of test scores?

    Validity. Ensures that inferences drawn from scores are appropriate and supported by evidence.

13. Which validity type refers to whether the test appears, on its surface, to measure what it claims?

    Face validity. Relies on superficial inspection to enhance test-taker acceptance, though not empirical validity.

14. What is the standard error of measurement (SEM) formula in terms of SD and reliability $r_{xx}$?

    $SEM = SD\sqrt{1 - r_{xx}}$. Estimates score precision, where lower values indicate higher reliability and less measurement error.

15. Which internal consistency coefficient is most appropriate for dichotomously scored items (0/1)?

    KR-20. Adapts Cronbach's alpha for binary responses, assuming item difficulty affects covariance.

16. Which validity form is supported when test scores predict a future outcome or criterion?

    Predictive validity. Demonstrates utility in forecasting future criteria, such as job performance from aptitude tests.

17. Which term refers to the consistency or stability of test scores across measurements?

    Reliability. Indicates the reproducibility of scores under similar conditions, essential for trustworthy measurements.

18. What is test–retest reliability designed to estimate?

    Score stability over time. Assesses temporal consistency by correlating scores from the same test given at different times.

19. Which internal consistency coefficient is based on the average of all possible split-halves?

    Cronbach's alpha. Computes reliability by averaging correlations from every possible test split, providing a comprehensive estimate.

20. Which reliability method divides a test into two halves and correlates the two half-scores?

    Split-half reliability. Provides an internal consistency estimate, often corrected with Spearman-Brown for full-test reliability.

21. What is internal consistency reliability designed to estimate?

    Consistency among items within a single test. Reflects how well items measure the same underlying construct, indicating test homogeneity.

22. Which validity form asks whether test content adequately samples the target domain?

    Content validity. Evaluates representativeness of items to the construct domain through expert judgment or blueprints.

23. Which principle states that validity cannot exceed reliability for a given measure?

    Validity is limited by reliability. An unreliable measure introduces error that caps the potential for accurate predictions or interpretations.

24. Which statistic corrects interrater agreement for chance when ratings are categorical?

    Cohen's kappa. Adjusts raw agreement percentages by subtracting expected chance agreement for categorical data.

25. Which validity concern focuses on whether score use and interpretation produce appropriate decisions and outcomes?

    Consequential validity. Examines intended and unintended impacts of test use on individuals and society.

Study these flashcards