AP Statistics Flashcards: Potential Errors When Performing Tests

What this deck covers

This deck focuses on Potential Errors When Performing Tests, giving you a quick way to review the definitions, rules, and examples that matter most for AP Statistics.

How to use these flashcards

Work through these flashcards in short sessions. Try to answer each prompt before flipping the card, then revisit any cards you miss until the explanation feels automatic.

Flashcard 1: Identify the potential error: performing a $z$ test for a mean when $\sigma$ is unknown.

Answer: Use a $t$ test when $\sigma$ is unknown. Population SD rarely known; sample SD requires t-distribution for proper inference.

Flashcard 2: What is the Type I error in hypothesis testing, stated in terms of $H_0$?

Answer: Rejecting $H_0$ when $H_0$ is true. This is a false positive - incorrectly concluding an effect exists.

Flashcard 3: Identify the error: Treating a nonresponse-biased sample as if it were a random sample.

Answer: Nonresponse bias can invalidate inference. Missing responses may differ systematically from respondents.

Flashcard 4: What is the common formula error in a $z$ test for a proportion regarding standard error?

Answer: Use pooled $\sqrt{\frac{p_0(1-p_0)}{n}}$ under $H_0$, not sample $p$. Under $H_0$, use hypothesized $p_0$, not sample proportion.

Flashcard 5: Identify the potential error: running many tests at $\alpha=0.05$ and treating each as independent evidence.

Answer: Inflated overall Type I error rate (multiple comparisons problem). Each test has 5% error rate; multiple tests compound the overall error probability.

Flashcard 6: Which change increases power: increasing $n$, increasing $\alpha$, or increasing variability?

Answer: Increasing $n$ or increasing $\alpha$ (not increasing variability). Larger samples and higher $\alpha$ increase power; more variability decreases it.

Flashcard 7: Identify the error: using a $2$-sided $p$-value when the stated alternative is one-sided.

Answer: Use a one-sided $p$-value consistent with $H_a$. P-value calculation must match the alternative hypothesis direction specified.

Flashcard 8: Which option correctly describes how decreasing $\alpha$ affects Type I error risk?

Answer: It decreases the chance of a Type I error. Lower $\alpha$ means stricter criterion for rejection, reducing false positives.

Flashcard 9: What is the correct conclusion wording when $p>\alpha$?

Answer: Fail to reject $H_0$; insufficient evidence for $H_a$. Cannot prove $H_0$; only state that data doesn't provide strong evidence against it.

Flashcard 10: Identify the error: Using a $\chi^2$ test when expected counts are too small.

Answer: Expected counts condition violated; results may be unreliable. $\chi^2$ test requires all expected counts $\geq 5$.

Flashcard 11: Identify the error: Interpreting a $p$-value as $P(H_0\text{ is true})$.

Answer: A $p$-value is $P(\text{data or more extreme}\mid H_0)$. It's the probability of the data given $H_0$, not vice versa.

Flashcard 12: Identify the potential error: using a one-proportion $z$ test when $np_0<10$ or $n(1-p_0)<10$.

Answer: Normal approximation may be invalid; conditions not met. Success-failure condition ensures normal approximation is appropriate.

Flashcard 13: What is the key design error when claiming causation from an observational study?

Answer: Confounding; association does not imply causation. Without randomization, other variables may explain the relationship.

Flashcard 14: If sample size $n$ increases (all else fixed), what happens to power $1-\beta$?

Answer: Power increases. Larger samples reduce variability, making effects easier to detect.

Flashcard 15: Identify the error: Concluding $H_0$ is true because the $p$-value is large.

Answer: A large $p$-value means insufficient evidence, not proof $H_0$ is true. Failing to reject $H_0$ doesn't prove it's true.

Flashcard 16: Identify the error: Using a $t$ test on strongly skewed data with small $n$ and no checks.

Answer: Conditions not met; $t$ procedures may be invalid. $t$ procedures assume approximate normality or large $n$.

Flashcard 17: Identify the error: Using a one-sided test after seeing which direction favors $H_a$.

Answer: Post hoc tail choice invalidates the $p$-value. Test direction must be chosen before seeing data.

Flashcard 18: If $\alpha$ decreases while conditions stay the same, what happens to $\beta$?

Answer: $\beta$ increases. Lower $\alpha$ makes rejecting $H_0$ harder, increasing Type II errors.

Flashcard 19: What does the significance level $\alpha$ represent in a test of significance?

Answer: The probability of a Type I error. $\alpha$ is the threshold we set for rejecting $H_0$.

Flashcard 20: What is the standard condition error for two-sample inference when samples are dependent?

Answer: Using two-sample methods instead of paired methods. Paired data violates independence assumption of two-sample tests.

Flashcard 21: What is the Type II error in hypothesis testing, stated in terms of $H_0$?

Answer: Failing to reject $H_0$ when $H_0$ is false. This is a false negative - missing a real effect.

Flashcard 22: Identify the error: stating that the $p$-value is $P(H_0\text{ is true})$.

Answer: $p=P(\text{statistic as extreme as observed}\mid H_0)$. P-value measures extremeness of data given $H_0$, not probability $H_0$ is true.

Flashcard 23: What does a small $p$-value indicate about the data under $H_0$?

Answer: The observed result is unlikely if $H_0$ is true. Small p-values suggest the data would be surprising if $H_0$ were true.

Flashcard 24: Identify the error: Failing to state hypotheses in terms of population parameters.

Answer: Hypotheses must be about parameters (e.g., $\mu$, $p$), not statistics. We test population values, not sample statistics.

Flashcard 25: Which option correctly describes how decreasing $\alpha$ affects Type II error risk (all else fixed)?

Answer: It increases the chance of a Type II error. Harder to reject $H_0$ means more likely to miss real effects when they exist.

Flashcard 26: Which option is a correct decision rule using a $p$-value and $\alpha$?

Answer: Reject $H_0$ if $p\le\alpha$; otherwise fail to reject $H_0$. Compare p-value to significance level; reject when evidence is strong enough.

Flashcard 27: What is the correct conclusion wording when $p\le\alpha$?

Answer: Reject $H_0$; sufficient evidence for $H_a$. Data provides strong enough evidence to conclude the alternative is likely true.

Flashcard 28: What is the key design error when generalizing to a population from a convenience sample?

Answer: Selection bias; results may not generalize. Non-random samples may not represent the population.

Flashcard 29: What does the significance level $\alpha$ represent?

Answer: The probability of a Type I error. Set before testing; the maximum acceptable risk of making a Type I error.

Flashcard 30: Identify the potential error: treating nonrandom or biased sampling as if test results generalize to a population.

Answer: Inference may be invalid due to lack of random sampling/assignment. Statistical tests assume random sampling; bias limits generalizability.

Flashcard 31: Identify the error: Running many tests and treating one small $p$-value as strong evidence.

Answer: Multiple comparisons inflate the Type I error rate. Testing many hypotheses increases chance of false positives.

Flashcard 32: Identify the error: Claiming statistical significance implies practical importance.

Answer: Statistical significance does not imply practical importance. Small $p$-values can occur for trivial effects with large $n$.

Flashcard 33: What does power represent in a test, using $\beta$ notation?

Answer: $1-\beta$, the probability of rejecting a false $H_0$. Power measures test's ability to detect false null hypotheses.

Flashcard 34: Identify the error: choosing $H_a$ after seeing the data to match the observed direction.

Answer: Direction of $H_a$ must be set before analyzing data. Data snooping invalidates the test; hypotheses must be predetermined.

Flashcard 35: What is the power of a hypothesis test in terms of $\beta$?

Answer: $1-\beta$. Power is the probability of correctly rejecting a false null hypothesis.

Flashcard 36: Identify the error: concluding $H_0$ is true because you failed to reject $H_0$.

Answer: Failing to reject $H_0$ does not prove $H_0$ is true. Absence of evidence is not evidence of absence; we can only reject or not reject.

Flashcard 37: What does $\beta$ represent in a hypothesis test?

Answer: The probability of a Type II error. Depends on effect size, sample size, and $\alpha$; decreases as power increases.

Flashcard 38: Which error can be directly controlled by choosing $\alpha$: Type I or Type II?

Answer: Type I error. We set $\alpha$ directly; $\beta$ depends on other factors.