Test Construction - Eppp Knowledge

1. Using Spearman-Brown, find $r_{new}$ if $r_{old}=0.60$ and length is doubled ($n=2$).

   $r_{new}=0.75$. Increasing length enhances reliability by reducing measurement error as per the prophecy formula.

2. What is sensitivity in diagnostic testing?

   $\frac{TP}{TP+FN}$. Sensitivity measures the proportion of actual positives correctly identified by the test.

3. What is the discrimination index ($D$) using upper and lower groups in item analysis?

   $D=p_U-p_L$. D quantifies an item's ability to differentiate high and low performers by comparing proportions correct in upper and lower groups.

4. Compute $z$ when $X=70$, $\mu=50$, and $\sigma=10$.

   $z=2$. Compute by subtracting the mean and dividing by the standard deviation to standardize.

5. What does a negative discrimination index ($D<0$) usually indicate about an item?

   Lower scorers outperformed higher scorers; likely flawed or miskeyed. Negative D implies the item inversely correlates with ability, often due to errors like incorrect keying.

6. What is item difficulty ($p$) in classical test theory for a dichotomous item?

   $p=\frac{\text{number correct}}{N}$. Item difficulty quantifies the proportion of test-takers who answer correctly, aiding in assessing item challenge level.

7. What is a percentile rank definition for a given raw score?

   Percent of norm group scoring at or below that raw score. Percentile rank positions a score within the norm group's distribution for relative comparison.

8. Compute sensitivity when $TP=45$ and $FN=5$.

   $0.90$. Divide true positives by the sum of true positives and false negatives.

9. What is specificity in diagnostic testing?

   $\frac{TN}{TN+FP}$. Specificity measures the proportion of actual negatives correctly identified by the test.

10. Compute $SEM$ when $SD=10$ and $r_{xx}=0.84$.

    $SEM=4$. Apply the formula to derive the expected error in an individual's observed score.

11. What is norming in test development?

    Deriving score meaning by comparing to a representative reference sample. Norming provides interpretive context by establishing score distributions from a representative sample.

12. Which sign of point-biserial suggests a problematic item: positive or negative?

    Negative point-biserial suggests a problematic item. Negative values indicate the item inversely relates to overall performance, flagging potential issues.

13. What does standardization mean in psychological test development?

    Uniform administration, scoring, and interpretation procedures. Standardization ensures test consistency and fairness across different administrations and users.

14. Compute specificity when $TN=81$ and $FP=9$.

    $0.90$. Divide true negatives by the sum of true negatives and false positives.

15. Which change typically increases sensitivity but decreases specificity: raising or lowering the cutoff?

    Lowering the cutoff increases sensitivity and decreases specificity. Lower cutoffs capture more true positives but increase false positives, trading specificity for sensitivity.

16. What is Cronbach's alpha ($\alpha$) primarily an estimate of?

    Internal consistency reliability. Alpha evaluates how consistently items measure the same underlying construct within a test.

17. What is the $z$-score formula for converting a raw score $X$ to standard units?

    $z=\frac{X-\mu}{\sigma}$. Z-score transforms raw scores to a standard normal distribution for comparability.

18. Identify the $D$ value: $p_U=0.80$ and $p_L=0.30$ for an item.

    $D=0.50$. Subtract $p_L$ from $p_U$ to compute D, indicating the item's discrimination power.

19. What is the key difference between a norm-referenced and a criterion-referenced interpretation?

    Norm: compare to others; Criterion: compare to a fixed performance standard. Norm-referenced evaluates relative standing, while criterion-referenced assesses absolute mastery.

20. Which direction of $p$ indicates a more difficult item: $p=0.20$ or $p=0.80$?

    $p=0.20$ indicates the more difficult item. Lower $p$ values reflect fewer correct responses, signifying greater item difficulty.

21. What is the standard error of measurement (SEM) formula using reliability $r_{xx}$?

    $SEM=SD\sqrt{1-r_{xx}}$. SEM estimates score precision by incorporating test reliability and observed score variability.

22. What is the Spearman-Brown prophecy formula for new reliability after changing test length?

    $r_{new}=\frac{nr_{old}}{1+(n-1)r_{old}}$. Spearman-Brown predicts adjusted reliability when altering test length by factor $n$.

23. What is the point-biserial correlation used for in item analysis?

    Correlation between item score ($0/1$) and total test score. Point-biserial assesses item discrimination by correlating binary item performance with continuous total scores.

24. What $p$-value range is typically targeted for maximal discrimination on many tests?

    $p\approx 0.50$ (often about $0.30$ to $0.70$). Mid-range $p$ maximizes item variance, optimizing differentiation of ability levels for better test discrimination.

25. What is the KR-20 formula for dichotomously scored items?

    $KR20=\frac{k}{k-1}\left(1-\frac{\sum pq}{\sigma_X^2}\right)$. KR-20 computes reliability for binary items using number of items, item variances, and total variance.

Study these flashcards