Potential Errors When Performing Tests - AP Statistics

1. A university tests whether the proportion of applicants who accept an admission offer is different from 0.60 after changing financial-aid packages. They test $H_0: p=0.60$ vs. $H_a: p\neq 0.60$ at $\alpha=0.05$. The university fails to reject $H_0$ and reports no convincing evidence the acceptance rate changed. In reality, the true acceptance rate is $p=0.52$. Which type of error was made?
   Answer: Type II error: failing to detect a real change in the acceptance rate This question tests understanding of Type II errors with two-sided tests. The university failed to reject H₀ (concluded no evidence of change) when the alternative hypothesis is actually true (p = 0.52 ≠ 0.60, the rate DID change). A Type II error occurs when we fail to reject a false null hypothesis - we miss detecting a real change. This matches the scenario: the acceptance rate actually dropped from 60% to 52%, but the test failed to detect this significant change. Type I errors involve rejecting a true H₀, which cannot occur when we fail to reject.
2. A city tests whether a new traffic signal timing reduces the mean commute time. They test $H_0: \mu=32$ minutes vs. $H_a: \mu<32$ at $\alpha=0.05$. The p-value is 0.01, so they reject $H_0$ and conclude mean commute time is reduced. In reality, the true mean commute time with the new timing is $\mu=30$ minutes. Which type of error was made?
   Answer: No error was made because they rejected a false null hypothesis This question tests understanding of correct decisions in hypothesis testing. The city rejected the null hypothesis (concluded commute time is reduced) and in reality the null hypothesis was indeed false (true mean is 30 minutes, which is less than 32). This represents a correct decision: rejecting a false null hypothesis. The p-value of 0.01 being less than α = 0.05 led to rejection, which was the correct conclusion given the true state of nature. No error was made because the test correctly identified that the new signal timing does reduce commute time, demonstrating the power of the test to detect true effects.
3. A pharmaceutical company tests whether the mean reduction in blood pressure from a new drug exceeds 5 mmHg. They use $H_0: \mu=5$ versus $H_a: \mu>5$ at $\alpha=0.05$. The test is not statistically significant, so they conclude there is not enough evidence the mean reduction exceeds 5 mmHg. In reality, the true population mean reduction is $\mu=6$ mmHg. Which type of error was made?
   Answer: Type II error (failing to reject a false null hypothesis) This question demonstrates a Type II error in pharmaceutical testing. The company failed to reject the null hypothesis (μ = 5 mmHg) when the alternative hypothesis was true (μ = 6 mmHg > 5 mmHg). This is a Type II error - failing to reject a false null hypothesis. The drug actually does provide greater blood pressure reduction than claimed in H₀, but the test failed to detect this. Type II errors in pharmaceutical contexts mean potentially beneficial treatments might not reach patients. The consequence of this error is missing an opportunity to identify an effective treatment. Understanding Type II errors helps students appreciate why adequate sample sizes and test power are crucial in medical research.
4. A city health department tests whether a new text-message reminder system increases the proportion of patients who show up for appointments. They test $H_0: p=0.80$ versus $H_a: p>0.80$ at significance level $\alpha=0.05$. Based on the sample, they reject $H_0$ and announce that the reminders increase the show-up rate. In truth, the reminder system does not increase the show-up rate (the true proportion is still $p=0.80$). Which type of error was made?
   Answer: Type I error: rejecting $H_0$ when $H_0$ is true This question tests understanding of Type I errors in hypothesis testing. The scenario describes rejecting H₀ (claiming the reminder system works) when in reality H₀ is true (the reminder system doesn't increase the show-up rate, p still equals 0.80). This is the classic definition of a Type I error: rejecting a true null hypothesis. Type I errors are also called "false positives" because we incorrectly claim an effect exists when it doesn't. The significance level α = 0.05 represents the probability of making this type of error. Remember: Type I error occurs when we reject H₀ but H₀ is actually true, while Type II error occurs when we fail to reject H₀ but H₀ is actually false.
5. A school district tests whether a new tutoring program changes the mean math score compared with the current mean of 75. They perform a two-sided test with $H_0: \mu=75$ versus $H_a: \mu\neq 75$ at $\alpha=0.10$. The test is not statistically significant, so they do not adopt the program, concluding there is not enough evidence of a change in mean score. In reality, the true population mean score with the tutoring program is $\mu=75$ exactly. Which type of error was made?
   Answer: No error was made This question demonstrates a correct decision in hypothesis testing. The school district failed to reject the null hypothesis (μ = 75), and in reality, the null hypothesis was true (μ = 75). This represents a correct decision - no error was made. When we fail to reject a true null hypothesis, we've made the right choice. This scenario helps students understand that not all hypothesis test outcomes involve errors. The two-sided alternative (μ ≠ 75) and higher significance level (α = 0.10) don't change the fact that the correct decision was made. Recognizing correct decisions is as important as identifying errors in understanding the complete framework of hypothesis testing outcomes.
6. A hospital tests a rapid screening tool for a rare infection. For a large group of patients, they conduct a hypothesis test about the infection rate: $H_0: p = 0.02$ versus $H_a: p > 0.02$ at $\alpha = 0.05$. Based on the sample, the hospital rejects $H_0$ and concludes the infection rate is higher than 2%. In reality, the true infection rate in the population is exactly 2%. Which type of error was made?
   Answer: Type I error: rejecting $H_0$ when $H_0$ is true The skill being tested is identifying Type I and Type II errors based on the outcome of a hypothesis test. In this case, the hospital rejected H0 (p = 0.02), but the true population proportion is exactly 0.02, meaning H0 is true. Rejecting a true H0 is a classic Type I error. Option C might distract by swapping the definitions, describing a Type II error as rejecting when H0 is true, which is incorrect. For a mini-lesson, Type I errors involve incorrectly rejecting the null (false alarm), with probability equal to α, while Type II errors involve failing to reject a false null (missed detection), with probability β that decreases with larger samples or effects.
7. A wildlife biologist tests whether the proportion of a fish species carrying a parasite is less than 0.30. The hypotheses are $H_0: p=0.30$ versus $H_a: p<0.30$ at $\alpha=0.05$. The test is statistically significant, so the biologist concludes the parasite rate is below 0.30. In reality, the true population proportion is $p=0.34$. Which type of error was made?
   Answer: Type I error (rejecting a true null hypothesis) This question presents a Type I error in a one-sided test context. The biologist rejected the null hypothesis (concluded p < 0.30) when in reality the null hypothesis was true (p = 0.34, which is not less than 0.30). This is a Type I error - rejecting a true null hypothesis. The actual proportion (0.34) is even higher than the null value (0.30), making this clearly a false rejection. Students should note that with a left-tailed test ($H_a$: p < 0.30), any true proportion at or above 0.30 makes H₀ true. Type I errors can occur in any direction of testing and represent false alarms where we claim an effect exists when it doesn't.
8. A manufacturer tests whether a machine is producing bolts with mean length 50 mm. The hypotheses are $H_0: \mu=50$ versus $H_a: \mu\neq 50$ at $\alpha=0.01$. The test result is to fail to reject $H_0$ and the company continues production without adjustment. In reality, the machine has drifted and the true mean length is $\mu=50.4$ mm. Which type of error was made?
   Answer: Type II error: failing to reject $H_0$ when $H_0$ is false This question tests understanding of Type II errors in hypothesis testing. The manufacturer failed to reject H₀ (concluded the machine is working properly at μ = 50 mm) when in reality H₀ is false (the true mean is μ = 50.4 mm, not 50 mm). This is a Type II error: failing to reject a false null hypothesis. Type II errors are also called "false negatives" because we miss detecting a real problem or effect. The company continues production thinking everything is fine when the machine has actually drifted from specifications. Remember the key distinction: Type I error = reject true H₀ (false alarm), while Type II error = fail to reject false H₀ (missed detection).
9. A public health team tests whether a new text-message reminder increases the proportion of patients who get a flu shot. They test $H_0: p=0.40$ vs. $H_a: p>0.40$ at significance level $\alpha=0.05$. After analyzing the sample, they reject $H_0$ and announce the reminders increase the flu-shot rate. In reality, the true population proportion with reminders is still $p=0.40$ (no increase). Which type of error was made?
   Answer: Type I error: concluding there is an increase when there is none This question tests understanding of Type I errors in hypothesis testing. The scenario describes rejecting H₀ (concluding there IS an increase) when the null hypothesis is actually true (p = 0.40, no increase). A Type I error occurs when we reject a true null hypothesis - we falsely detect an effect that doesn't exist. This is exactly what happened here: the health team concluded the reminders increase flu-shot rates when in reality they don't. Type II errors involve failing to reject a false null hypothesis, which is not the case here since H₀ was rejected.
10. A streaming service tests whether a new recommendation algorithm increases the mean number of hours watched per week from 6 hours. The hypotheses are $H_0: \mu = 6$ and $H_a: \mu > 6$. At $\alpha = 0.05$, the analysts fail to reject $H_0$. In reality, the algorithm truly increases the mean to $\mu = 6.5$. Which type of error was made?
    Answer: Type II error (failing to reject a false null hypothesis) This question assesses identifying errors in hypothesis testing, specifically Type II in a one-sided context. The decision was to fail to reject H0 (μ = 6), but reality μ = 6.5 > 6, making H0 false and Ha true, so this is a Type II error. Choice C correctly identifies it as Type II (failing to reject false H0). A common distractor is choice A, which is Type I but doesn't fit since H0 was not rejected. Another distractor is choice B, suggesting no error due to insufficient evidence, but since H0 is false, it is an error. In a mini-lesson, Type II error occurs when we miss detecting a true effect (failing to reject false H0), often due to small sample size or small effect; Type I is the opposite, rejecting true H0, controlled by the significance level α.
11. A food safety agency tests whether a new sterilization process reduces the mean number of bacteria colonies on packaged salad. The hypotheses are $H_0: \mu = 12$ colonies and $H_a: \mu < 12$. Using a significance level of $\alpha = 0.05$, the agency performs the test and fails to reject $H_0$. In reality, the sterilization process truly reduces the mean to $\mu = 10$. Which type of error was made?
    Answer: Type II error (failing to reject a false null hypothesis) This question assesses the skill of identifying potential errors in hypothesis testing, such as Type I and Type II errors in AP Statistics. The decision was to fail to reject H0 (μ = 12), but in reality μ = 10 < 12, making H0 false and Ha true. Therefore, failing to reject a false null hypothesis constitutes a Type II error. A common distractor is choice C, which claims no error was made simply because the decision was to fail to reject H0, ignoring that H0 is actually false. Another distractor is choice A, which describes Type I error but misapplies it here since H0 was not rejected. In a mini-lesson on error types, a Type I error occurs when we reject a true null hypothesis (false positive), while a Type II error happens when we fail to reject a false null hypothesis (false negative); the probability of Type I is controlled by α, and Type II by β, which depends on sample size and effect size.
12. A manufacturer tests whether a new machine produces washers with mean diameter 10.00 mm. The hypotheses are $H_0: \mu = 10.00$ versus $H_a: \mu \ne 10.00$ at $\alpha = 0.05$. The test result is to fail to reject $H_0$. In reality, the true mean diameter produced by the machine is 10.00 mm. Which type of error was made?
    Answer: No error was made because the decision matches the true state of the population This problem tests the skill of determining if an error occurred in hypothesis testing by comparing the test decision to reality. The manufacturer failed to reject H0 (μ = 10.00), and indeed, the true mean is 10.00, so H0 is true. This is a correct decision, meaning no error was made. Distractors like option A might tempt by defining Type I error, but since there was no rejection, it doesn't apply. As a mini-lesson, correct decisions include not rejecting a true H0 or rejecting a false one; errors are Type I (reject true H0) and Type II (fail to reject false H0). The power of a test is the probability of correctly rejecting a false H0.
13. A quality-control engineer tests whether the proportion of defective parts produced by a machine is greater than 0.02. They test $H_0: p=0.02$ versus $H_a: p>0.02$ at $\alpha=0.05$. The test is statistically significant, so they conclude the defect rate is greater than 0.02 and shut the machine down for maintenance. In reality, the true population defect proportion is $p=0.02$. Which type of error was made?
    Answer: Type I error (rejecting a true null hypothesis) This question illustrates a Type I error in quality control. The engineer rejected the null hypothesis (concluded p > 0.02) when the null hypothesis was actually true (p = 0.02). This is a Type I error - rejecting a true null hypothesis. The statistically significant result led to unnecessary machine maintenance when the defect rate was actually at the acceptable level. In quality control, Type I errors can lead to unnecessary downtime and costs. The irony is that while trying to maintain quality, the false alarm disrupted production unnecessarily. This example helps students understand that Type I errors have real-world consequences and why controlling the significance level α is important in balancing error risks.
14. A gym claims that the mean weight loss after an 8-week program is 6 lb. A statistician tests $H_0: \mu=6$ vs. $H_a: \mu\neq6$ at $\alpha=0.05$. The p-value is 0.31, so the statistician fails to reject $H_0$ and reports no convincing evidence that the mean weight loss differs from 6 lb. In reality, the true mean weight loss is $\mu=8$ lb. Which type of error was made?
    Answer: Type II error: failing to reject a false null hypothesis This question tests understanding of Type II errors in hypothesis testing. The statistician failed to reject the null hypothesis (concluded no evidence mean differs from 6 lb) when in reality the null hypothesis was false (true mean is 8 lb, which differs from 6 lb). This is the definition of a Type II error: failing to reject a false null hypothesis. The p-value of 0.31 being greater than α = 0.05 led to failing to reject, missing the real difference in weight loss effectiveness. Type II errors in program evaluation can lead to undervaluing effective interventions when the test fails to detect their true benefits.
15. A nutrition researcher tests whether the proportion of adults who meet a daily fiber recommendation is 0.30. The hypotheses are $H_0: p = 0.30$ versus $H_a: p \ne 0.30$ at $\alpha = 0.05$. After surveying a random sample, the researcher fails to reject $H_0$. In reality, the true proportion in the population is 0.24. Which type of error was made?
    Answer: Type II error: failing to reject $H_0$ when $H_0$ is false The skill targeted is recognizing Type I and Type II errors in the context of proportion testing. The researcher failed to reject H0 (p = 0.30), but the true proportion is 0.24, so H0 is false. Failing to reject a false H0 is a Type II error. Option A might confuse by defining Type I error, which requires rejection, not present here. For a mini-lesson, Type II errors are failures to detect real effects (controlled indirectly via power), while Type I errors are incorrect rejections of the null (directly set by α). Larger samples help reduce both error rates by increasing precision.
16. A school district tests whether a new tutoring program changes the mean math score. They use $H_0: \mu=70$ vs. $H_a: \mu\neq 70$ with $\alpha=0.10$. The district rejects $H_0$ and concludes the program changes the mean score. In truth, the program has no effect and the population mean remains $\mu=70$. Which type of error was made?
    Answer: Type I error: concluding the mean changed when it did not This question tests understanding of Type I errors with two-sided tests. The district rejected H₀ (concluded the mean changed) when the null hypothesis is actually true (μ = 70, no change). A Type I error occurs when we reject a true null hypothesis - we falsely conclude there's a difference when none exists. This is precisely what happened: the tutoring program has no effect, but the test incorrectly concluded it changes scores. Type II errors involve failing to reject H₀ when it's false, which doesn't apply here since H₀ was rejected.
17. A tech company tests whether a new battery lasts longer than 8 hours on average. The hypotheses are $H_0: \mu = 8$ versus $H_a: \mu > 8$, using $\alpha = 0.05$. The company runs the test and rejects $H_0$, concluding the mean battery life exceeds 8 hours. In reality, the true mean battery life is exactly 8 hours. Which type of error was made?
    Answer: Type I error: rejecting $H_0$ when $H_0$ is true This question assesses understanding of errors in hypothesis testing through the lens of battery life claims. The company rejected H0 (μ = 8), but the true mean is exactly 8, so H0 is true. This rejection of a true null is a Type I error. A distractor might be option A, which defines Type II but applies to failing to reject, not rejecting. Mini-lesson: Type I errors occur with probability α and represent false claims of significance; Type II errors, with probability β, mean missing true differences. Balancing these involves trade-offs, as lowering α increases β, and vice versa.
18. A company claims its light bulbs last an average of 1000 hours. A consumer group tests $H_0: \mu=1000$ vs. $H_a: \mu\neq1000$ at $\alpha=0.01$. The p-value is 0.18, so they fail to reject $H_0$ and report there is not convincing evidence the mean lifetime differs from 1000 hours. In reality, the true mean lifetime is $\mu=950$ hours. Which type of error was made?
    Answer: Type II error: failing to reject a false null hypothesis This question tests understanding of Type II errors in hypothesis testing. The consumer group failed to reject the null hypothesis (concluded no evidence of difference from 1000 hours) when in reality the null hypothesis was false (true mean is 950 hours). This is the definition of a Type II error: failing to reject a false null hypothesis. The p-value of 0.18 being greater than α = 0.01 led to failing to reject, missing the real difference that exists. Type II errors represent false negative results where we fail to detect a real effect, often occurring when sample sizes are too small or the effect size is modest.
19. A university tests whether the mean GPA of first-year students is 3.00. They test $H_0: \mu = 3.00$ versus $H_a: \mu \ne 3.00$ at $\alpha = 0.05$. The test result is to reject $H_0$. In reality, the true mean GPA of all first-year students is 3.00. Which type of error was made?
    Answer: Type I error: rejecting $H_0$ when $H_0$ is true This question tests the ability to identify errors in hypothesis testing for mean GPA. The university rejected H0 (μ = 3.00), but the true mean is 3.00, meaning H0 is true. Rejecting a true H0 constitutes a Type I error. Distractors like option A define Type II correctly but don't match the rejection decision. Mini-lesson: Type I errors are like convicting an innocent (reject true null), probability α; Type II are acquitting the guilty (fail to reject false null), probability β. Two-sided tests split α, potentially affecting error rates compared to one-sided tests.
20. An environmental group tests whether the mean lead level in a river exceeds the legal limit of 15 ppb. They test $H_0: \mu=15$ versus $H_a: \mu>15$ at $\alpha=0.10$. The test leads them to reject $H_0$ and they publicly claim the river exceeds the legal limit. In truth, the mean lead level is $\mu=14$ ppb. Which type of error was made?
    Answer: Type I error: rejecting $H_0$ when $H_0$ is true This question tests understanding of Type I errors in hypothesis testing. The environmental group rejected H₀ (claimed the lead level exceeds 15 ppb) when in reality H₀ is true (the actual mean is 14 ppb, which is less than 15 ppb). This is a Type I error: rejecting a true null hypothesis. The group made a false accusation about the river exceeding legal limits when it actually doesn't. The significance level α = 0.10 means they accepted a 10% chance of making this type of error. Type I errors can have serious consequences, such as unnecessary panic or costly interventions based on false alarms. Always remember: Type I error means we reject H₀ when it's actually true.

Start this quiz