Potential Errors When Performing Tests
Help Questions
AP Statistics › Potential Errors When Performing Tests
A manufacturer tests whether the mean lifetime of a battery is greater than 8 hours. They test $H_0: \mu=8$ vs. $H_a: \mu>8$ at $\alpha=0.05$. The test result is to reject $H_0$ and claim the mean lifetime exceeds 8 hours. In reality, the true mean lifetime is $\mu=8.6$ hours. Which type of error was made?
Type I error: concluding $\mu>8$ when in fact $\mu=8$
Type I error: failing to reject $H_0$ when $\mu>8$
No error: the conclusion matches the true state of the population
Type II error: rejecting $H_0$ when $\mu=8$
Type II error: failing to reject $H_0$ when $\mu>8$
Explanation
This question tests understanding of correct decisions in hypothesis testing. The manufacturer rejected H₀ (concluded μ > 8) when the alternative hypothesis is actually true (μ = 8.6 > 8). No error was made because the decision matches reality: the test correctly detected that the mean lifetime exceeds 8 hours. When we reject H₀ and H₀ is false (Ha is true), we've made a correct decision. Type I errors occur when we reject a true H₀, and Type II errors occur when we fail to reject a false H₀ - neither applies here.
A hospital tests whether the mean waiting time in the emergency room is less than 30 minutes after adding staff. They test $H_0: \mu=30$ vs. $H_a: \mu<30$ at $\alpha=0.05$. They fail to reject $H_0$ and state there is not convincing evidence the mean waiting time is below 30 minutes. In reality, the true mean waiting time is $\mu=30$ minutes. Which type of error was made?
Type I error: failing to reject $H_0$ when $\mu<30$
Type I error: concluding the mean is below 30 when it is not
No error: the decision is consistent with the true population mean
Type II error: failing to detect that the mean is below 30 when it is
Type II error: concluding the mean is below 30 when it is not
Explanation
This question tests understanding of correct decisions in hypothesis testing. The hospital failed to reject H₀ (concluded no evidence μ < 30) when the null hypothesis is actually true (μ = 30). No error was made because the decision matches reality: the test correctly concluded there's no evidence the mean is below 30 minutes, and indeed it equals 30. When we fail to reject H₀ and H₀ is true, we've made a correct decision. Type I errors occur when we reject a true H₀, and Type II errors occur when we fail to reject a false H₀ - neither applies here.
A hospital tests whether a new sterilization procedure reduces the mean number of bacteria colonies on instruments below 12 colonies. They use $H_0: \mu=12$ versus $H_a: \mu<12$ at $\alpha=0.01$. The test is not statistically significant, so they do not switch procedures. In reality, the true population mean under the new procedure is $\mu=9$ colonies. Which type of error was made?
No error was made because $\alpha=0.01$ is very strict
Type II error (rejecting a true null hypothesis)
Type I error (rejecting a true null hypothesis)
Type II error (failing to reject a false null hypothesis)
Type I error (failing to reject a false null hypothesis)
Explanation
This question demonstrates a Type II error in hypothesis testing. The hospital failed to reject the null hypothesis (μ = 12 colonies) when in reality the alternative hypothesis was true (μ = 9 colonies < 12 colonies). This is a Type II error - failing to reject a false null hypothesis. The new procedure actually was better (fewer bacteria colonies), but the test failed to detect this improvement. The very strict significance level (α = 0.01) makes Type II errors more likely because we require overwhelming evidence to reject H₀. This example shows how being too conservative (small α) can lead to missing real improvements, which is particularly concerning in medical contexts where better procedures could improve patient safety.
A streaming company tests whether a new recommendation algorithm increases the mean number of minutes watched per user above the current mean of 50 minutes. They test $H_0: \mu=50$ versus $H_a: \mu>50$ at $\alpha=0.05$. The result is statistically significant, so they conclude the algorithm increases watch time. In reality, the true population mean with the new algorithm is $\mu=50$ minutes (no change). Which type of error was made?
Type II error (rejecting a true null hypothesis)
Type I error (failing to reject a false null hypothesis)
Type I error (rejecting a true null hypothesis)
No error was made because the company used $\alpha=0.05$
Type II error (failing to reject a false null hypothesis)
Explanation
This question illustrates a Type I error in hypothesis testing. The streaming company rejected the null hypothesis (concluded the algorithm increases watch time) when the null hypothesis was actually true (μ = 50 minutes, no change). This is a Type I error - rejecting a true null hypothesis. The statistically significant result was a false positive, leading to an incorrect business decision. Type I errors occur with probability α = 0.05, meaning this outcome will happen about 5% of the time even when following proper procedures. Understanding that statistical significance doesn't guarantee practical truth helps students appreciate why we control but cannot eliminate Type I error risk.
A school district tests whether a new tutoring program changes the mean math score compared with the current mean of 75. They perform a two-sided test with $H_0: \mu=75$ versus $H_a: \mu\neq 75$ at $\alpha=0.10$. The test is not statistically significant, so they do not adopt the program, concluding there is not enough evidence of a change in mean score. In reality, the true population mean score with the tutoring program is $\mu=75$ exactly. Which type of error was made?
No error was made
Type II error (failing to reject a false null hypothesis)
Type I error (failing to reject a false null hypothesis)
Type II error (rejecting a true null hypothesis)
Type I error (rejecting a true null hypothesis)
Explanation
This question demonstrates a correct decision in hypothesis testing. The school district failed to reject the null hypothesis (μ = 75), and in reality, the null hypothesis was true (μ = 75). This represents a correct decision - no error was made. When we fail to reject a true null hypothesis, we've made the right choice. This scenario helps students understand that not all hypothesis test outcomes involve errors. The two-sided alternative (μ ≠ 75) and higher significance level (α = 0.10) don't change the fact that the correct decision was made. Recognizing correct decisions is as important as identifying errors in understanding the complete framework of hypothesis testing outcomes.
A quality-control engineer tests whether the proportion of defective parts produced by a machine is greater than 0.02. They test $H_0: p=0.02$ versus $H_a: p>0.02$ at $\alpha=0.05$. The test is statistically significant, so they conclude the defect rate is greater than 0.02 and shut the machine down for maintenance. In reality, the true population defect proportion is $p=0.02$. Which type of error was made?
Type I error (rejecting a true null hypothesis)
Type II error (failing to reject a false null hypothesis)
No error was made because maintenance is a safe choice
Type II error (rejecting a true null hypothesis)
Type I error (failing to reject a false null hypothesis)
Explanation
This question illustrates a Type I error in quality control. The engineer rejected the null hypothesis (concluded p > 0.02) when the null hypothesis was actually true (p = 0.02). This is a Type I error - rejecting a true null hypothesis. The statistically significant result led to unnecessary machine maintenance when the defect rate was actually at the acceptable level. In quality control, Type I errors can lead to unnecessary downtime and costs. The irony is that while trying to maintain quality, the false alarm disrupted production unnecessarily. This example helps students understand that Type I errors have real-world consequences and why controlling the significance level α is important in balancing error risks.
A city tests whether the proportion of commuters who use public transit is greater than 0.25 after a fare reduction. They test $H_0: p=0.25$ vs. $H_a: p>0.25$ at $\alpha=0.05$. The city fails to reject $H_0$ and states there is not convincing evidence the fare reduction increased transit use. In reality, the true proportion is $p=0.30$. Which type of error was made?
Type I error: failing to reject $H_0$ when $p>0.25$
Type II error: failing to detect an increase that is real
Type I error: concluding transit use increased when it did not
No error: failing to reject $H_0$ means $p=0.25$
Type II error: concluding transit use increased when it did not
Explanation
This question tests understanding of Type II errors in one-sided tests. The city failed to reject H₀ (concluded no evidence of increased transit use) when the alternative hypothesis is actually true (p = 0.30 > 0.25, transit use DID increase). A Type II error occurs when we fail to reject a false null hypothesis - we miss detecting a real effect. This matches the scenario: the fare reduction actually increased transit use to 30%, but the test failed to detect this increase. Type I errors involve rejecting a true H₀, which cannot occur when we fail to reject.
A pharmaceutical company tests whether a generic drug is less effective than the brand-name drug by comparing mean symptom-reduction scores. They set up $H_0: \mu_G-\mu_B=0$ vs. $H_a: \mu_G-\mu_B<0$ at $\alpha=0.05$. The analysis leads them to reject $H_0$ and claim the generic is less effective. In reality, the generic and brand are equally effective (the true difference is $\mu_G-\mu_B=0$). Which type of error was made?
Type II error: concluding the generic is less effective when it is not
Type II error: failing to detect that the generic is less effective
Type I error: concluding the generic is less effective when it is not
Type I error: failing to reject $H_0$ when the generic is less effective
No error: rejecting $H_0$ guarantees the generic is less effective
Explanation
This question tests understanding of Type I errors with difference tests. The company rejected H₀ (concluded the generic is less effective) when the null hypothesis is actually true (μG - μB = 0, drugs are equally effective). A Type I error occurs when we reject a true null hypothesis - we falsely detect a difference that doesn't exist. This is exactly the situation: the drugs are equally effective, but the test incorrectly concluded the generic is inferior. Type II errors involve failing to reject a false H₀, which doesn't apply when H₀ is rejected.
A tech company tests whether a new interface reduces average customer support time. They use $H_0: \mu=12$ minutes vs. $H_a: \mu<12$ minutes at $\alpha=0.05$. They reject $H_0$ and announce the interface reduced mean support time. In reality, the true mean support time with the new interface is still $\mu=12$ minutes. Which type of error was made?
Type II error: failing to detect a real reduction in support time
Type II error: concluding there is a reduction when there is none
No error: rejecting $H_0$ proves the mean is less than 12
Type I error: concluding there is a reduction when there is none
Type I error: failing to reject $H_0$ when the mean is less than 12
Explanation
This question tests understanding of Type I errors in one-sided tests. The company rejected H₀ (concluded support time was reduced) when the null hypothesis is actually true (μ = 12, no reduction). A Type I error occurs when we reject a true null hypothesis - we falsely claim an improvement that doesn't exist. This is precisely what happened: the new interface doesn't actually reduce support time, but the test incorrectly concluded it does. Type II errors involve failing to reject a false H₀, which doesn't apply when H₀ is rejected.
A quality-control engineer tests whether the mean fill amount of a cereal box is less than the labeled 500 g. The hypotheses are $H_0: \mu=500$ vs. $H_a: \mu<500$ at $\alpha=0.01$. The test result is to fail to reject $H_0$, so the engineer reports there is not convincing evidence of underfilling. In reality, the true mean fill is $\mu=495$ g (the boxes are underfilled). Which type of error was made?
Type II error: failing to detect underfilling when it is real
Type II error: concluding the boxes are underfilled when they are not
No error was made because failing to reject $H_0$ proves $\mu=500$
Type I error: failing to reject $H_0$ when $\mu<500$
Type I error: concluding the boxes are underfilled when they are not
Explanation
This question tests understanding of Type II errors in hypothesis testing. The engineer failed to reject H₀ (concluded no evidence of underfilling) when the alternative hypothesis is actually true (μ = 495 < 500, boxes ARE underfilled). A Type II error occurs when we fail to reject a false null hypothesis - we miss detecting a real effect. This matches the scenario perfectly: the boxes are actually underfilled but the test failed to detect this problem. Type I errors involve rejecting a true null hypothesis, which cannot happen when we fail to reject H₀.