Type I/II Errors & Power - Biostatistics

1. A researcher is testing whether a new drug reduces blood pressure more than a placebo. The study is designed with 80% power to detect a clinically meaningful difference of 10 mmHg, using α = 0.05. After the study, the researcher finds a p-value of 0.12 and concludes there is no significant difference. However, the observed difference was 8 mmHg favoring the new drug. What is the most accurate interpretation of this result?
   Answer: The result is consistent with either no true effect or a Type II error, but the observed effect size suggests the latter is plausible With p = 0.12 > 0.05, we fail to reject H₀. However, since we observed an 8 mmHg difference (close to the clinically meaningful 10 mmHg), this could represent either: (1) no true effect exists, or (2) a true effect exists but we failed to detect it (Type II error). The proximity of the observed effect to the target effect size makes a Type II error plausible. Choice A is wrong because we cannot definitively conclude a Type II error occurred without knowing the true effect. Choice B incorrectly identifies this as Type I error (which would be rejecting a true null). Choice D misunderstands power - the study was designed with adequate power for the target effect size.
2. A diagnostic test for a rare disease has a sensitivity of 0.90 and specificity of 0.85. In hypothesis testing terms, if the null hypothesis is 'patient does not have the disease' and the alternative is 'patient has the disease', what do sensitivity and specificity represent respectively?
   Answer: Sensitivity represents power of the test, specificity represents (1 - probability of Type I error) When you encounter questions linking diagnostic test characteristics to hypothesis testing, you need to map medical terminology onto statistical concepts by carefully defining what constitutes a "positive result" in the hypothesis testing framework. In this setup, the null hypothesis is "patient does not have the disease" and the alternative is "patient has the disease." A positive test result means rejecting the null hypothesis (concluding the patient has the disease). Sensitivity measures the probability of correctly identifying someone who actually has the disease - this means correctly rejecting a false null hypothesis. In hypothesis testing terms, this is the power of the test (the probability of correctly rejecting the null when it's false). Specificity measures the probability of correctly identifying someone who doesn't have the disease - this means correctly accepting a true null hypothesis, which equals (1 - probability of Type I error). Choice A reverses these concepts, incorrectly assigning sensitivity to (1 - Type I error) and specificity to power. Choice C confuses the probabilities themselves rather than their complements - sensitivity isn't the Type I error rate, and specificity isn't the Type II error rate. Choice D incorrectly states that specificity represents the Type I error probability itself, when it actually represents one minus that probability. The correct answer is B: sensitivity represents power, and specificity represents (1 - probability of Type I error). Study tip: Remember that sensitivity relates to detecting true positives (power), while specificity relates to avoiding false positives (avoiding Type I errors). Draw the 2×2 table linking disease status to test results to visualize these relationships.
3. A nutrition study tests whether a dietary supplement affects cholesterol levels. The researchers plan for 80% power to detect a 15 mg/dL reduction in cholesterol. After the study, they find a non-significant result (p = 0.18) but the actual mean reduction was 12 mg/dL. What most likely explains this outcome?
   Answer: A Type II error occurred because the actual effect was smaller than what the study was powered to detect When you encounter questions about study outcomes and statistical errors, focus on the relationship between what the study was designed to detect versus what actually happened. This scenario involves understanding statistical power and Type II errors. The study was powered to detect a 15 mg/dL reduction with 80% power, meaning there was an 80% chance of finding a statistically significant result if the true effect was 15 mg/dL or larger. However, the actual effect was only 12 mg/dL—smaller than what the study was designed to detect. Since the effect was smaller than anticipated, the study lacked sufficient power to reliably detect it, making a non-significant result likely even when a true effect exists. This describes a Type II error: failing to detect a real effect because the study wasn't adequately powered for that effect size. Option A incorrectly identifies this as a Type I error, which would be finding a significant result when no true effect exists—the opposite of what happened here. Option C is wrong because while the study design was reasonable for detecting 15 mg/dL differences, it wasn't adequate for detecting the smaller 12 mg/dL effect that actually occurred. Option D misunderstands power calculations—studies are designed to detect clinically meaningful differences, not any detectable change, and smaller effects require larger sample sizes to detect reliably. Remember: Type II errors often occur when the actual effect size is smaller than what the study was powered to detect. Always compare the observed effect size to the target effect size used in power calculations.
4. An educational intervention study uses a very large sample size (n = 5000) to test whether a new teaching method improves test scores. The researchers find a statistically significant result (p < 0.001) but the actual improvement is only 1 point on a 100-point scale. What concern should be raised about this finding?
   Answer: Type I error is likely because very large samples can detect trivial differences as significant When you encounter a study with very large sample sizes and statistically significant but practically small effects, you're dealing with a classic issue in statistical interpretation: the relationship between sample size, statistical significance, and practical importance. With extremely large samples (n = 5000), statistical tests become incredibly sensitive and can detect even tiny differences between groups as "statistically significant." Here, a 1-point improvement on a 100-point scale represents just 1% improvement - practically negligible in educational terms. Yet the massive sample size gave enough statistical power to detect this trivial difference with high confidence (p < 0.001). This illustrates how statistical significance doesn't guarantee practical significance. Option B correctly identifies that large samples can make Type I errors more likely by detecting differences that, while real, are so small they're meaningless in practice. The statistical test is working "too well" - finding significance where none should matter. Option A confuses the error types. Type II error means failing to detect a real effect, but here we did detect an effect (though trivial). Option C misses the key issue entirely - a low p-value alone doesn't address whether the finding matters practically. The statistical evidence against the null hypothesis is strong, but the effect itself is worthless. Option D incorrectly suggests both error types are present, when only the concern about detecting trivial differences (Type I error territory) applies. Remember: with large samples, always evaluate effect size alongside statistical significance. A tiny effect that's statistically significant may still be practically meaningless.
5. A pharmaceutical company conducts a superiority trial to show their new drug is better than standard care. They use α = 0.025 (one-sided test) and achieve 85% power. The trial results show p = 0.08 (one-sided), leading to a conclusion of no significant superiority. The company then conducts a post-hoc non-inferiority analysis using a margin of 10% and finds the drug is non-inferior (p < 0.05). How should this sequence of analyses be interpreted regarding potential errors?
   Answer: The post-hoc analysis represents data dredging that inflates the Type I error rate beyond the nominal level Conducting multiple analyses on the same data without appropriate adjustment inflates the overall Type I error rate above the nominal α level. The post-hoc non-inferiority analysis after failing superiority represents data dredging or 'fishing' for significant results. This practice undermines the error rate control that was established for the original superiority test. Choice A ignores the multiple testing issue. Choice B doesn't address the inflated error rate from sequential testing. Choice D incorrectly assumes the non-inferiority result must be a Type I error - non-inferiority and superiority test different hypotheses and can have different outcomes legitimately.
6. An environmental study tests whether pollution levels exceed safety standards. The consequences of incorrectly concluding pollution is safe when it's actually dangerous are severe, while incorrectly concluding pollution is dangerous when it's safe results in unnecessary but manageable costs. How should the researchers set their significance level?
   Answer: Use a large significance level (α = 0.10) to minimize Type II error since missing real danger is costly When you encounter questions about setting significance levels, think about the real-world consequences of each type of error. Type I error means rejecting a true null hypothesis (false positive), while Type II error means failing to reject a false null hypothesis (false negative). In this pollution study, the null hypothesis would be "pollution levels are safe." A Type I error means concluding pollution is dangerous when it's actually safe, leading to unnecessary cleanup costs. A Type II error means concluding pollution is safe when it's actually dangerous, potentially causing severe health consequences. Since missing real danger has severe consequences while false alarms only create manageable costs, you want to minimize Type II error. Using a larger significance level like $α = 0.10$ makes it easier to detect actual pollution problems by increasing your chance of rejecting the null hypothesis when pollution truly is dangerous. Answer A correctly identifies this approach. Answer B is backwards—using $α = 0.01$ minimizes Type I error (false alarms) but increases Type II error (missing real danger), which is exactly what you don't want here. Answer C incorrectly assumes $α = 0.05$ provides optimal balance regardless of context—the consequences clearly aren't equal in this scenario. Answer D misunderstands the relationship between significance levels and error types; you cannot minimize both simultaneously by choosing a very small alpha, as this actually increases Type II error. Study tip: Always identify which error type has more serious real-world consequences. When missing a problem is worse than a false alarm, use a larger significance level to reduce Type II error.
7. An industrial engineer tests whether a process modification reduces defect rates from the current 8% level. She observes 12 defects in a sample of 200 items (6% defect rate) and obtains p = 0.15 using a one-tailed test with α = 0.05. Unknown to her, the modification actually reduces defects to 5%. What error occurred, and what was the approximate power for detecting this true 3 percentage point reduction?
   Answer: Type II error occurred; power was approximately 45% based on the non-significant result pattern When analyzing hypothesis testing outcomes, you need to distinguish between the statistical decision and the true state of nature. Here, the engineer failed to reject the null hypothesis (p = 0.15 > α = 0.05), concluding no significant reduction in defects. However, the modification actually does reduce defects from 8% to 5% - meaning the null hypothesis is false but wasn't rejected. This scenario represents a Type II error: failing to reject a false null hypothesis. The engineer missed detecting a real improvement. Since we know the true effect size (3 percentage point reduction), we can estimate the power. With a sample size of 200, baseline rate of 8%, and true rate of 5%, the power for this one-tailed test would be approximately 45% - meaning there was only about a 45% chance of detecting this difference. The non-significant result (p = 0.15) is consistent with this moderate power level. Answer choice A incorrectly identifies a Type I error (rejecting a true null hypothesis), which didn't occur since the null wasn't rejected. The power estimate of 85% is also too high for this scenario. Choice C correctly identifies the error type but wrongly claims power can't be determined - it can be estimated when you know the true effect size. Choice D correctly identifies the Type II error but overestimates the power at 85%, when the actual power for detecting a 3-point reduction with n=200 is much lower. Remember: Type I errors occur when you reject a true null (false positive), while Type II errors occur when you fail to reject a false null (false negative).
8. A researcher plans a study with 80% power to detect a medium effect size at α = 0.05. After data collection, the p-value is 0.08. Given that the true effect size is indeed medium as hypothesized, what most likely occurred?
   Answer: A Type II error occurred because a true effect was not detected as significant When you encounter questions about statistical errors and study power, you need to distinguish between what the study was designed to detect versus what actually happened, and understand the relationship between power, effect size, and statistical significance. A Type II error occurs when there's truly an effect present, but your statistical test fails to detect it as significant. Here, you're told the true effect size is medium (as hypothesized), but the p-value of 0.08 exceeds the α = 0.05 threshold, so the result isn't statistically significant. This means a real effect went undetected – the definition of a Type II error. The 80% power tells us there was a 20% chance this would happen, and unfortunately, this study fell into that 20%. Looking at the wrong answers: (A) incorrectly identifies this as a Type I error, but Type I errors occur when you detect a significant effect that doesn't actually exist – the opposite situation. (C) misunderstands that proper powering doesn't guarantee significant results; even well-powered studies have a failure rate equal to (1 - power). (D) reflects a fundamental misconception that adequate power guarantees p < 0.05, but power only tells you the probability of detecting an effect if it exists. Remember that power calculations give you probabilities, not certainties. An 80% powered study still has a 20% chance of missing a true effect. When you see questions combining power, effect sizes, and p-values, always identify whether a true effect exists first, then determine if your test detected it.
9. In a behavioral study, researchers test whether a mindfulness intervention reduces anxiety scores. They use α = 0.05 and achieve 90% power to detect a moderate effect. The intervention truly has no effect, but due to measurement error and random variation, they obtain p = 0.03. What error occurs, and what is notable about the power level?
   Answer: Type I error occurs; high power is irrelevant since power only relates to Type II error When you encounter questions about statistical errors and power, focus on the fundamental definitions: Type I error occurs when you reject a true null hypothesis, while Type II error occurs when you fail to reject a false null hypothesis. Power is specifically the probability of correctly rejecting a false null hypothesis. In this scenario, the intervention truly has no effect (null hypothesis is true), but the researchers obtained p = 0.03 < 0.05, leading them to reject the null hypothesis. Since they rejected a true null hypothesis, this is a Type I error. The key insight is that power only relates to situations where the null hypothesis is false - it's the probability of avoiding Type II error. When the null hypothesis is actually true (as in this case), power becomes irrelevant to the outcome. Option A incorrectly suggests that high power increases the likelihood of Type I error. Power doesn't affect Type I error rates - that's controlled by your alpha level (0.05 here). Option C misidentifies this as Type II error, which would occur if they failed to detect a real effect. Option D is wrong because high power doesn't guarantee correct decisions when the null hypothesis is true - Type I errors can still occur at the rate specified by alpha. The correct answer is B because it properly identifies the Type I error and recognizes that power is irrelevant in this situation since power only applies when the alternative hypothesis is true. Study tip: Remember the power paradox - high power is great for detecting real effects, but it's completely irrelevant when there's no effect to detect. Type I error rates depend only on your alpha level, never on power.
10. Two studies examine the same research question with identical effect sizes, but Study A uses n=50 participants while Study B uses n=200 participants. Both use α = 0.05. Compared to Study A, Study B has:
    Answer: Same probability of Type I error and lower probability of Type II error When you encounter questions about sample size and error probabilities, focus on how Type I and Type II errors relate to study design parameters like sample size and significance level. Type I error probability (α) is the chance of rejecting a true null hypothesis - essentially finding a "false positive." This probability is set by the researcher when choosing the significance level. Since both studies use α = 0.05, they have identical Type I error probabilities of 5%, regardless of sample size. Type II error probability (β) is the chance of failing to reject a false null hypothesis - missing a real effect. Statistical power (1 - β) increases with larger sample sizes when the effect size and α remain constant. Study B's larger sample (n=200 vs n=50) provides greater power to detect the same effect size, meaning lower probability of Type II error. Answer C correctly identifies that Type I error probability stays the same (both studies use α = 0.05) while Type II error probability decreases with the larger sample size. Answer A incorrectly suggests Type I error probability changes with sample size - it doesn't, since α is fixed. Answer B wrongly claims Type I error decreases while Type II error increases, which contradicts how sample size affects power. Answer D correctly identifies unchanged Type I error probability but incorrectly states that Type II error increases - the opposite is true with larger samples. Remember: α (Type I error) is set by your significance level choice, while β (Type II error) decreases as sample size increases, assuming constant effect size and α.
11. In a meta-analysis of 10 studies testing the same hypothesis, 8 studies found non-significant results (p > 0.05) and 2 found significant results (p < 0.05). All studies had similar sample sizes and the overall meta-analysis shows a significant effect with substantial heterogeneity. What does this pattern most likely suggest about the individual studies?
    Answer: Type II errors likely occurred in several of the 8 non-significant studies due to insufficient individual study power When you encounter meta-analysis questions involving mixed results across studies, focus on statistical power and the ability to detect true effects. This scenario presents a classic pattern: individual studies with limited power failing to detect an effect that becomes apparent when combined. The key insight is that a significant overall meta-analysis with substantial heterogeneity suggests a true underlying effect exists. If 8 out of 10 similarly-sized studies failed to reach significance individually, but pooling all data reveals a significant effect, this strongly indicates that the individual studies lacked sufficient statistical power to detect the true effect size. These underpowered studies likely committed Type II errors (false negatives) by failing to reject a false null hypothesis. The 2 significant studies probably had slightly better conditions or sampling luck that allowed them to cross the significance threshold. Option A incorrectly assumes the majority result indicates truth, ignoring that multiple underpowered studies don't collectively prove no effect exists. Option B misunderstands publication bias - this bias typically suppresses non-significant results from being published, not causes Type II errors during study conduct. Option C wrongly suggests heterogeneity indicates equal error distribution, when substantial heterogeneity combined with an overall significant effect points specifically toward Type II errors in the non-significant studies. Remember: in meta-analysis questions, when individual studies show mixed results but pooled analysis reveals significance, suspect insufficient power in the negative studies rather than false positives in the significant ones. The meta-analysis capitalizes on increased sample size to detect effects individual studies missed.
12. A screening test for a rare disease is evaluated in a population where the true prevalence is 2%. The test has 95% sensitivity and 90% specificity. A researcher tests the hypothesis H₀: test sensitivity = 90% vs H₁: test sensitivity ≠ 90% using data from 1000 subjects and finds p = 0.001, rejecting H₀. What error consideration is most relevant here?
    Answer: No error occurred because the statistical test correctly identified that sensitivity differs from 90% When evaluating diagnostic tests, you need to distinguish between statistical significance and the types of errors that can occur in hypothesis testing. This question tests your understanding of Type I and Type II errors in the context of diagnostic test evaluation. The researcher tested whether the sensitivity equals 90% and found strong evidence (p = 0.001) that it doesn't. Since the true sensitivity is actually 95%, the statistical test correctly identified that the sensitivity differs from the hypothesized 90%. The null hypothesis was false, and the test correctly rejected it - this represents the desired outcome of hypothesis testing, not an error. Looking at the wrong answers: Option B misunderstands Type I error, which occurs when you reject a true null hypothesis, not when the null hypothesis contains an "incorrect" value. Option C incorrectly suggests Type II error (failing to reject a false null hypothesis) occurred, but the opposite happened - the false null was correctly rejected. The sample size of 1000 was actually quite adequate to detect a 5 percentage point difference in sensitivity. Option D incorrectly assumes that rare disease prevalence somehow violates test assumptions, but standard hypothesis tests for sensitivity can handle any prevalence level. The key insight is that statistical errors only occur when the test reaches the wrong conclusion. Here, a false null hypothesis was correctly rejected based on strong statistical evidence. Study tip: Remember that Type I error = rejecting a true null, and Type II error = accepting a false null. No error occurs when your statistical conclusion matches reality, regardless of effect sizes or prevalence rates.
13. A pharmaceutical company tests 100 different drug compounds, each in a separate hypothesis test with α = 0.05. Assuming all null hypotheses are true (no compounds are effective), approximately how many Type I errors would be expected, and what does this illustrate about multiple testing?
    Answer: About 5 Type I errors would occur, illustrating why multiple testing requires adjusted significance levels When you encounter multiple testing scenarios in biostatistics, think about the cumulative probability of making at least one error across all tests. This is a fundamental concept in controlling family-wise error rates. With 100 independent hypothesis tests at α = 0.05, and assuming all null hypotheses are true, each test has a 5% chance of producing a Type I error (false positive). The expected number of Type I errors is simply: $100 \times 0.05 = 5$ errors. This demonstrates why multiple testing is problematic—even when no real effects exist, you'll still get several "significant" results by chance alone, leading to false discoveries. Looking at the answer choices: Choice A correctly calculates 5 Type I errors but incorrectly suggests this illustrates the need for power analysis. Power analysis relates to Type II errors and detecting true effects, not controlling false positives. Choice C is mathematically wrong (1 error instead of 5) and incorrectly claims multiple testing reduces error rates—it actually inflates them. Choice D drastically overestimates the errors at 95, confusing the concept with the probability of avoiding any Type I errors. Choice B is correct because it properly calculates the expected 5 Type I errors and correctly identifies that this illustrates why multiple testing requires adjusted significance levels. Methods like Bonferroni correction, false discovery rate control, or other multiple comparison adjustments are essential to maintain the overall error rate at acceptable levels. Remember: multiple testing inflates your chance of false discoveries, so always consider whether significance level adjustments are needed when conducting numerous simultaneous tests.
14. A screening program for a genetic disorder uses a test with 95% sensitivity and 90% specificity. Public health officials want to minimize the chance of missing affected individuals, even if this means more false positives. In hypothesis testing framework, what error type are they prioritizing, and how should they adjust their decision threshold?
    Answer: Minimizing Type II error by lowering the threshold to make it easier to declare someone affected When you encounter screening test questions, think about the relationship between hypothesis testing errors and diagnostic outcomes. Type I error means rejecting a true null hypothesis (false positive), while Type II error means failing to reject a false null hypothesis (false negative). In medical screening, the null hypothesis is typically "the person is not affected." A Type II error would mean failing to detect someone who actually has the condition—exactly what the officials want to minimize. When you want to catch more affected individuals, you need to lower the decision threshold, making it easier to declare someone positive. This increases sensitivity but decreases specificity, resulting in more false positives but fewer missed cases. Answer B correctly identifies that minimizing Type II error (missing affected individuals) requires lowering the threshold to make positive declarations more likely. This trade-off accepts more false positives to reduce false negatives. Answer A incorrectly suggests minimizing Type I error and raising the threshold, which would actually increase missed cases. Answer C correctly identifies minimizing Type I error as the wrong priority but incorrectly states that lowering the threshold reduces false negatives—while true, the goal isn't to minimize Type I error here. Answer D wrongly suggests minimizing Type II error requires raising the threshold, which would actually increase Type II errors by making it harder to detect affected individuals. Remember: In screening contexts, when officials prioritize "not missing cases," they're willing to accept false positives to minimize false negatives—that's minimizing Type II error through lowered thresholds.
15. A researcher tests a new medication against a placebo to determine if it reduces blood pressure. The null hypothesis states there is no difference between treatments. If the researcher concludes the medication is effective when it actually has no effect, and separately, if the medication is truly effective but the researcher fails to detect this effect, what types of errors occur respectively?
    Answer: Type I error occurs in the first case, Type II error occurs in the second case When you encounter hypothesis testing questions, focus on the relationship between what's actually true versus what the researcher concludes. There are two fundamental errors that can occur when making statistical decisions. A Type I error happens when you reject a true null hypothesis - essentially finding an effect that doesn't actually exist (a "false positive"). A Type II error occurs when you fail to reject a false null hypothesis - missing a real effect that does exist (a "false negative"). In this blood pressure study, the first scenario describes concluding the medication works when it actually has no effect. Since the null hypothesis states "no difference between treatments," this null hypothesis is actually true, but the researcher incorrectly rejects it. This is a classic Type I error. The second scenario involves a truly effective medication that the researcher fails to detect. Here, the null hypothesis is false (there really is a difference), but the researcher fails to reject it, committing a Type II error. Answer A correctly identifies both error types. Answer B reverses them, which is a common mistake when students confuse "false positive" with "false negative." Answer C incorrectly suggests both scenarios involve Type I errors, missing that failing to detect a real effect is fundamentally different from falsely claiming an effect exists. Answer D makes the same mistake with Type II errors. Remember this simple framework: Type I = finding something that isn't there (false alarm), Type II = missing something that is there (missed detection). This distinction is crucial for interpreting research validity.
16. A researcher conducts a pilot study (n = 25) and finds a promising but non-significant trend (p = 0.08). She calculates that with the observed effect size, she would need n = 100 to achieve 80% power. If she conducts the larger study and the true effect size remains the same, what is the probability of obtaining a significant result?
    Answer: 0.80, since this represents the power to detect the true effect with n = 100 When you encounter power analysis questions, focus on understanding what each statistical concept represents in the context of detecting true effects. Statistical power is defined as the probability of correctly rejecting a false null hypothesis when there is a true effect present. In this scenario, the researcher has calculated that with n = 100, she would have 80% power to detect the observed effect size. This means that if the true effect size remains the same as observed in the pilot study, there's an 80% probability that the larger study will yield a statistically significant result (p < 0.05). The correct answer is B because power directly answers the question being asked - the probability of obtaining a significant result when a true effect exists is exactly what statistical power measures. Option A incorrectly confuses Type I error rate (α = 0.05) with power. Type I error is the probability of finding significance when there's no true effect, not when there is a true effect. Option C attempts to add probabilities inappropriately - you can't simply sum power, alpha, and an arbitrary value to get a meaningful probability. These aren't mutually exclusive outcomes that sum to 1. Option D represents 1 - α, which has no relevance to detecting true effects and would actually be the probability of not making a Type I error under the null hypothesis. Remember: Power analysis questions often test whether you can distinguish between Type I error (false positive rate) and power (true positive rate). Power tells you the probability of detecting real effects, while alpha tells you the probability of falsely detecting non-existent effects.
17. A medical researcher conducts a clinical trial to test a new treatment. The study has 70% power to detect the minimum clinically important difference. After analyzing the data, she finds a statistically significant result (p = 0.03) favoring the new treatment. What can be concluded about the possibility of error?
    Answer: A Type I error may have occurred with 5% probability based on the significance level used When you encounter questions about statistical significance and study power, you need to distinguish between the two types of statistical errors and understand what each probability represents. Since the researcher found a statistically significant result (p = 0.03), she rejected the null hypothesis. However, there's always a chance this rejection was incorrect—this would be a Type I error (falsely concluding there's an effect when there isn't one). The probability of Type I error equals the significance level used in the study. With p = 0.03, the researcher likely used α = 0.05, meaning there's a 5% chance the significant result occurred by chance alone. Answer B correctly identifies this possibility. Let's examine why the other options miss the mark. Answer A incorrectly assumes that statistical significance and adequate power eliminate all possibility of error—this is never true in hypothesis testing. Answer C confuses the error types: Type II error occurs when you fail to detect a real effect, but here we have a significant result, so Type II error didn't occur. The 70% power is actually reasonable, not "insufficient." Answer D misinterprets what the 30% represents—this would be the Type II error probability if we had failed to find significance, but it doesn't apply to our significant result. Remember this key distinction: when you find statistical significance, you risk Type I error (false positive) with probability equal to your alpha level. When you fail to find significance, you risk Type II error (false negative) with probability equal to 1 minus the power.
18. An investigator conducts 20 independent hypothesis tests, each with α = 0.05, to examine different biomarkers. Three tests yield p-values < 0.05. Assuming all null hypotheses are actually true, what is the most likely explanation for the observed results?
    Answer: Type I errors occurred in approximately 3 tests, which is consistent with the expected false positive rate If all null hypotheses are true and α = 0.05, we expect about 5% of tests to yield false positives (Type I errors). With 20 tests, we expect 20 × 0.05 = 1 false positive on average. Observing 3 significant results is close to this expectation and represents Type I errors. Choice A is wrong because Type II errors involve failing to reject false null hypotheses, but we're told all nulls are true. Choice C misinterprets the situation - power is irrelevant when null hypotheses are true. Choice D incorrectly identifies accepting true null hypotheses as Type I errors, when this is actually correct decisions.
19. A diagnostic test study aims to show that a new biomarker has sensitivity > 80%. The null hypothesis is H₀: sensitivity ≤ 80% vs H₁: sensitivity > 80%. The study finds sensitivity = 75% with a 95% confidence interval of (65%, 85%). The p-value is 0.25. If the true sensitivity is actually 82%, what type of error situation exists?
    Answer: Type II error occurred because the test failed to detect that true sensitivity exceeds 80% Statistical hypothesis testing involves two types of errors that can occur when making decisions about population parameters. Type I error happens when you reject a true null hypothesis, while Type II error occurs when you fail to reject a false null hypothesis. In this scenario, the study failed to reject $H_0$: sensitivity ≤ 80% because the p-value (0.25) exceeded the significance level (0.05). However, the true sensitivity is actually 82%, which means the null hypothesis is false and should have been rejected. When you fail to reject a false null hypothesis, you've committed a Type II error - missing a real effect that exists. Option A incorrectly identifies this as Type I error and misunderstands confidence intervals. The CI including values below 80% doesn't determine error type, and no Type I error occurred since we didn't reject $H_0$. Option B correctly notes that Type I error was avoided but wrongly concludes no error occurred - it misses that Type II error happened instead. Option C focuses on the confidence interval containing the true value, which indicates the CI was appropriately constructed, but this doesn't mean the hypothesis test reached the correct conclusion. The key insight is distinguishing between confidence interval interpretation and hypothesis testing outcomes. A CI can contain the true parameter value while the corresponding hypothesis test still makes an error due to insufficient power or unlucky sampling. Remember: Type II error occurs when your test lacks sufficient evidence to detect a real effect. Always check whether the null hypothesis should have been rejected given the true parameter value.
20. A clinical trial comparing two treatments has 90% power to detect a hazard ratio of 0.7 with α = 0.05. The true hazard ratio is actually 0.8. If this study were repeated 100 times under identical conditions, approximately how many times would the researchers correctly conclude there is a significant difference between treatments?
    Answer: Between 50-70 times, because the actual power is lower than 90% but higher than 50% The study has 90% power to detect HR = 0.7, but the true HR = 0.8 (smaller effect). Power decreases as the true effect size gets smaller, so the actual power is less than 90% but still substantial since 0.8 is not too far from 0.7. The exact power would be between 50-70%, so we'd expect to correctly reject the null hypothesis in 50-70 studies out of 100. Choice A is wrong because this scenario involves detecting a true effect (power), not Type I error. Choice B incorrectly assumes a simple proportional relationship. Choice D is wrong because power depends on the actual effect size, not the design target.

Start this quiz