Business Statistics Quiz: Post Hoc Comparisons
20 questions · exam conditions
0:00
Post Hoc ComparisonsQuestion 1 of 20

In post-hoc tests such as Tukey's HSD and Scheffé's, the Mean Square Within (MSW), also known as Mean Square Error (MSE), from the overall ANOVA table is used to calculate the test statistic. What is the primary statistical advantage of using this MSW value?

It provides a more reliable estimate of the common population variance by pooling information from all the experimental groups.
It adjusts the degrees of freedom to account for the number of comparisons being made, thereby controlling the family-wise error rate.
It is used only when the sample sizes of the groups are unequal to provide a weighted average of the variances.
It corrects for any lack of statistical independence between the pairwise comparisons being performed.
← Back to quizzes

Business Statistics Quiz

Business Statistics Quiz: Post Hoc Comparisons

Practice Post Hoc Comparisons in Business Statistics with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Post Hoc Comparisons, giving you a quick way to practice the rules, question types, and explanations that matter most for Business Statistics.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

In post-hoc tests such as Tukey's HSD and Scheffé's, the Mean Square Within (MSW), also known as Mean Square Error (MSE), from the overall ANOVA table is used to calculate the test statistic. What is the primary statistical advantage of using this MSW value?

  1. It provides a more reliable estimate of the common population variance by pooling information from all the experimental groups. (correct answer)
  2. It adjusts the degrees of freedom to account for the number of comparisons being made, thereby controlling the family-wise error rate.
  3. It is used only when the sample sizes of the groups are unequal to provide a weighted average of the variances.
  4. It corrects for any lack of statistical independence between the pairwise comparisons being performed.
Explanation: When you encounter questions about post-hoc tests following ANOVA, focus on understanding why we use the pooled variance estimate from the original analysis rather than calculating separate variances for each comparison. The key insight is that ANOVA operates under the assumption of equal population variances across all groups (homoscedasticity). When this assumption holds, each group's sample variance is estimating the same underlying population parameter. The Mean Square Within (MSW) takes advantage of this by pooling variance information from all groups simultaneously, creating a more stable and reliable estimate than any single group's variance would provide. This pooled estimate has more degrees of freedom (df=N−kdf = N - k where NN is total sample size and kk is number of groups), making it more precise and less susceptible to sampling variability. Option B incorrectly describes degrees of freedom adjustment for multiple comparisons - that's handled through critical value adjustments, not through MSW itself. Option C mischaracterizes MSW's purpose; while it does weight by sample sizes when pooling, this isn't specifically about unequal groups but rather about optimal estimation. Option D confuses the issue of statistical dependence between comparisons with variance estimation - MSW doesn't address the dependence problem inherent in multiple pairwise tests. Study tip: Remember that post-hoc tests inherit their error term (MSW) from the original ANOVA because they're essentially breaking down the overall analysis into component parts. The multiple comparison adjustments happen in the critical values, not in the variance estimate itself.

Question 2

A consulting firm analyzes the employee turnover rates for 5 different industry sectors. The ANOVA is significant (p < .01). A post-hoc analysis using a method that controls the family-wise error rate at α = 0.05 is performed. The 95% confidence interval for the mean difference between the Finance and Technology sectors is [-0.5%, 7.5%], while the interval for the difference between Finance and Healthcare is [1.2%, 8.8%]. Which conclusion is most appropriate?

  1. There is a statistically significant difference between Finance and Healthcare, but not between Finance and Technology. (correct answer)
  2. The Finance sector has a significantly higher turnover rate than both the Technology and Healthcare sectors.
  3. No conclusions can be drawn because the confidence intervals are too wide, indicating a lack of practical significance.
  4. Since the overall ANOVA was significant, both comparisons should be considered statistically significant despite the intervals.
Explanation: When you encounter post-hoc analysis after a significant ANOVA, you're examining specific pairwise comparisons while controlling for multiple testing. The key is interpreting confidence intervals for mean differences: if the interval contains zero, the difference is not statistically significant. For the Finance vs. Technology comparison, the 95% confidence interval is [-0.5%, 7.5%]. Since this interval includes zero, you cannot conclude there's a statistically significant difference between these sectors' turnover rates. The difference could plausibly be zero. For the Finance vs. Healthcare comparison, the interval is [1.2%, 8.8%]. This interval does not contain zero—all plausible values are positive, indicating Finance has a significantly higher turnover rate than Healthcare. Choice A correctly identifies this pattern: there's a significant difference between Finance and Healthcare but not between Finance and Technology. Choice B is wrong because while Finance does have higher turnover than Healthcare, the comparison with Technology is not statistically significant. Choice C misunderstands the analysis—wide confidence intervals don't negate statistical significance, and practical significance isn't the primary concern here. Choice D reflects a common misconception: a significant overall ANOVA doesn't automatically make all pairwise comparisons significant. That's precisely why post-hoc testing with family-wise error control exists. Remember: in post-hoc confidence intervals for mean differences, zero inclusion equals no significance. This rule helps you quickly interpret multiple comparison results without getting distracted by interval width or the overall ANOVA result.

Question 3

A market researcher conducted ANOVA comparing customer satisfaction scores across four retail locations and found F(3,76) = 5.12, p = 0.003. She wants to compare each of the three branch stores against the flagship store (control) using an appropriate post-hoc test. The mean satisfaction scores are: Flagship = 7.8, Branch A = 6.9, Branch B = 7.2, Branch C = 8.4. If the pooled standard error for comparisons is 0.35 and she uses α = 0.05, which multiple comparison procedure should she use and why?

  1. Tukey's HSD because it's designed for all pairwise comparisons and provides optimal power for this design
  2. Bonferroni correction because it provides the most conservative approach for protecting against Type I error
  3. Dunnett's test because it's specifically designed for multiple comparisons against a single control group (correct answer)
  4. Fisher's LSD because the omnibus ANOVA was significant, providing sufficient Type I error protection
Explanation: When you encounter post-hoc testing scenarios in ANOVA, the key is matching the comparison structure to the appropriate test. Here, the researcher wants to compare three branch stores specifically against one flagship store (control), not make all possible pairwise comparisons between groups. Dunnett's test (C) is specifically designed for this exact scenario - multiple comparisons against a single control group. It controls the familywise error rate while maintaining better statistical power than other methods when your research question focuses on comparing treatments to a control. Since the researcher has a clear control group (flagship store) and wants to test each branch against it, Dunnett's test is the optimal choice. Option A is incorrect because Tukey's HSD is designed for all pairwise comparisons (comparing every group to every other group). While it would work, it's unnecessarily conservative here since you're not interested in comparing Branch A to Branch B, for example. Option B is wrong because the Bonferroni correction, while conservative, isn't specifically designed for control-versus-treatment comparisons. It's a general adjustment that sacrifices power unnecessarily in this structured comparison situation. Option D is incorrect because Fisher's LSD doesn't provide adequate Type I error protection when making multiple comparisons, even after a significant omnibus test. The "protected" LSD approach is generally considered insufficient for controlling familywise error rates. Study tip: Match your post-hoc test to your comparison structure: Tukey's for all pairwise comparisons, Dunnett's for multiple groups versus one control, and consider the balance between Type I error protection and statistical power.

Question 4

An analyst performed ANOVA on sales performance across five regions and found F(4,45) = 4.67, p = 0.003. When conducting Scheffé post-hoc tests, she found that a complex contrast comparing the average of Regions 1 and 2 against the average of Regions 4 and 5 yielded a test statistic of F = 3.84. Given that the critical value for Scheffé's test is F(4,45) × 4 = 18.68, and individual pairwise critical value for Tukey would be 2.86, what should she conclude?

  1. The contrast is significant because F = 3.84 > 2.86, indicating a meaningful difference between the regional groupings
  2. The contrast is marginally significant and requires additional analysis using a less conservative post-hoc method
  3. The test is invalid because Scheffé's method cannot be applied to complex contrasts involving multiple regions
  4. The contrast is not significant because F = 3.84 < 18.68, suggesting no difference between these regional groupings (correct answer)
Explanation: When you encounter ANOVA followed by post-hoc tests, you're dealing with multiple comparisons that require adjusted critical values to control for Type I error inflation. The key is understanding which critical value applies to each specific test method. The correct answer is D because Scheffé's method requires comparing the test statistic F = 3.84 against the Scheffé critical value of 18.68. Since 3.84 < 18.68, the contrast is not statistically significant, meaning there's insufficient evidence of a difference between the average of Regions 1&2 versus Regions 4&5. Option A incorrectly applies Tukey's critical value (2.86) to judge a Scheffé test. While 3.84 > 2.86, you must use the appropriate critical value for the method being employed. Tukey's method is designed for pairwise comparisons, not complex contrasts. Option B suggests the result is "marginally significant," but there's no such interpretation in hypothesis testing—results are either significant or not based on the chosen α level and critical value. The suggestion to use a "less conservative" method would compromise the error control that post-hoc tests are designed to provide. Option C is factually wrong. Scheffé's method is specifically designed to handle any linear contrast, including complex ones involving multiple group comparisons. This flexibility is actually Scheffé's main advantage over other post-hoc methods. Study tip: Always match your critical value to your test method. Scheffé uses Fcritical=(k−1)×F(k−1,dferror)F_{critical} = (k-1) \times F_{(k-1,df_{error})}, while Tukey uses different critical values for pairwise comparisons only.

Question 5

A clinical researcher is studying the effectiveness of four different therapy approaches for anxiety reduction. After conducting ANOVA and finding significant differences (F(3,96) = 7.89, p < 0.001), she needs to perform post-hoc comparisons. She has three specific research questions: (1) Is cognitive therapy better than no treatment? (2) Is behavioral therapy better than no treatment? (3) Is the combination therapy better than either individual therapy alone?

Considering the researcher's specific questions and the need to control Type I error appropriately, what would be the most statistically sound approach for these comparisons?

  1. Use three planned orthogonal contrasts without multiple comparison adjustment since these are theory-driven hypotheses specified before data collection (correct answer)
  2. Apply Dunnett's test for questions 1 and 2, then use a separate planned contrast for question 3 with Bonferroni adjustment
  3. Perform all possible pairwise comparisons using Tukey's HSD to comprehensively evaluate all treatment differences
  4. Use Bonferroni correction with α = 0.05/3 = 0.017 for each of the three specific comparisons to control familywise error
Explanation: The three questions represent planned, theory-driven contrasts that appear to be orthogonal (independent). When contrasts are planned a priori and orthogonal, no adjustment for multiple comparisons is needed, maximizing statistical power. Choice B unnecessarily mixes methods. Choice C goes beyond the specific research questions. Choice D applies unnecessary correction to planned orthogonal contrasts.

Question 6

A marketing firm runs a one-way ANOVA to compare the mean customer engagement scores of five different advertising campaigns. The result is statistically significant (p < .05). A junior analyst suggests that the firm should now conduct independent samples t-tests for all possible pairs of campaigns to see which specific campaigns differ. Which of the following is the most critical statistical flaw in the analyst's suggestion?

  1. The t-tests would have lower statistical power to detect differences compared to a proper post-hoc procedure.
  2. The approach inflates the family-wise error rate, increasing the probability of making at least one Type I error. (correct answer)
  3. The t-tests are only valid if the initial ANOVA result was not statistically significant.
  4. The assumption of equal variances across groups is more likely to be violated by multiple t-tests than by an ANOVA.
Explanation: The primary reason not to run multiple t-tests after an ANOVA is the problem of multiple comparisons, which inflates the family-wise error rate (FWER). With five campaigns, there are 10 possible pairwise comparisons. If each is tested at α = 0.05, the probability of incorrectly rejecting at least one null hypothesis (a Type I error) across the 'family' of tests becomes much higher than 0.05. Post-hoc tests are specifically designed to control this FWER.

Question 7

A financial analyst compares the mean annual returns of investment portfolios from four different firms. After a significant ANOVA result, the analyst decides to perform post-hoc tests to compare all possible pairs of firms. If both Tukey's HSD and a Bonferroni-corrected set of t-tests are conducted, which statement most accurately describes the likely relationship between their results?

  1. The Bonferroni correction will generally have greater statistical power than Tukey's HSD for this type of analysis.
  2. Tukey's HSD is specifically optimized for all pairwise comparisons and will likely have more power to detect differences than the Bonferroni method. (correct answer)
  3. Both tests are guaranteed to identify the exact same pairs of firms as being significantly different from one another.
  4. Tukey's HSD should only be used for a small number of planned comparisons, whereas Bonferroni is designed for all possible pairwise comparisons.
Explanation: Tukey's HSD (Honestly Significant Difference) test is specifically designed to control the family-wise error rate for a set of all possible pairwise comparisons. The Bonferroni correction is a general method that can be applied to any set of tests, but it is often overly conservative (i.e., has lower statistical power) when the number of comparisons is large. For all pairwise comparisons following an ANOVA, Tukey's HSD is typically more powerful than Bonferroni, meaning it is more likely to find a true difference if one exists.

Question 8

A researcher conducts an experiment with 5 independent groups and, after finding a significant ANOVA F-statistic, performs all 10 possible pairwise comparisons. A colleague criticizes the analysis, arguing that the family-wise error rate was not controlled. Which statement provides the strongest support for this criticism?

  1. The failure to find a significant difference for some pairs proves that the overall ANOVA result was a Type I error.
  2. By conducting 10 tests, each at an alpha of 0.05, the probability of making at least one Type I error is substantially greater than 0.05. (correct answer)
  3. The per-comparison error rate for each of the 10 tests is, by definition, higher than the family-wise error rate.
  4. The statistical power for each individual comparison is too low to be meaningful when so many tests are performed.
Explanation: The family-wise error rate (FWER) is the probability of making at least one Type I error in a family, or set, of statistical tests. When multiple comparisons are made without correction, this rate inflates. With 10 tests, each at α = 0.05, the FWER is 1 - (1 - 0.05)^10 ≈ 0.40. This means there is a 40% chance of a false positive, which is unacceptably high and the core reason for using post-hoc procedures that control the FWER.

Question 9

A supply chain manager analyzes shipping times for four different carriers (A, B, C, D). After a significant ANOVA, she becomes interested in exploring an unplanned, complex comparison: whether the average shipping time for carriers A and B differs from the average for carriers C and D. Which post-hoc test is specifically designed to maintain control over the family-wise error rate for any and all possible linear comparisons, including complex ones like this?

  1. Tukey's HSD test
  2. Fisher's LSD test
  3. Dunnett's test
  4. The Scheffé test (correct answer)
Explanation: The Scheffé test is the most flexible and also the most conservative post-hoc test. Its key feature is that it controls the family-wise error rate for the set of all possible linear contrasts (or comparisons), not just pairwise comparisons. This includes complex comparisons like comparing the average of one subset of groups to the average of another, making it the appropriate choice for this type of exploratory analysis.

Question 10

A market research team plans an experiment to compare customer satisfaction for three new product features (X, Y, Z) against the current feature (Control). Before collecting data, their only hypotheses are that X will be better than Control, Y will be better than Control, and Z will be better than Control. They have no interest in comparing the new features to each other. Which analytical approach is most statistically powerful for testing these specific hypotheses?

  1. Conduct a one-way ANOVA, and if significant, follow up with Tukey's HSD to test all six pairwise comparisons.
  2. Conduct a one-way ANOVA, and if significant, follow up with the Scheffé test to check the three comparisons of interest.
  3. Use Dunnett's test to directly compare each of the new features against the control group, which is more powerful than testing all pairs. (correct answer)
  4. Run three separate t-tests without any correction, since the comparisons were planned in advance of the data collection.
Explanation: This scenario involves planned comparisons of several treatment groups to a single control group. Dunnett's test is specifically designed for this situation. It is more powerful than post-hoc tests that examine all possible pairs (like Tukey's) or all possible contrasts (like Scheffé's) because it focuses the statistical power on the small number of comparisons of interest. Running uncorrected t-tests would inflate the family-wise error rate, even for planned comparisons.

Question 11

A significant ANOVA result for five groups leads to a post-hoc analysis using Tukey's HSD. The results show that the mean of Group 1 is significantly higher than the means of Groups 3, 4, and 5. However, no other pairwise comparisons (e.g., Group 1 vs 2, Group 3 vs 4) are significant. What is the most rigorous and defensible conclusion?

  1. The means of Groups 2, 3, 4, and 5 can be considered statistically identical to each other.
  2. Some factor unique to Group 1 must have caused its mean to be higher than the means of the other groups.
  3. The initial significant ANOVA result must have been a Type I error because so few post-hoc comparisons were significant.
  4. The evidence suggests a difference between the population mean of Group 1 and those of Groups 3, 4, and 5, while for other pairs, a difference cannot be ruled out or confirmed. (correct answer)
Explanation: When you encounter ANOVA followed by post-hoc testing, remember that statistical significance tells you whether differences exist, but non-significance doesn't prove equality—it simply means you lack sufficient evidence to detect a difference. The correct interpretation focuses on what the data actually demonstrate. Tukey's HSD found significant differences between Group 1 and Groups 3, 4, and 5, providing evidence that Group 1's population mean truly differs from these three groups. For all other comparisons (like Group 1 vs. 2, or Group 3 vs. 4), the lack of significance means you cannot conclude either that differences exist or that they don't exist—the evidence is simply insufficient either way. Answer A incorrectly assumes that non-significant results prove the groups are "statistically identical." Non-significance never proves equality; it only indicates insufficient evidence for a difference. Answer B makes an unjustified causal leap—statistical differences don't necessarily imply causation, and the data doesn't support claims about what "caused" Group 1's higher mean. Answer C misunderstands the relationship between ANOVA and post-hoc tests. A significant ANOVA indicates at least one difference exists among the groups, which the post-hoc analysis confirmed (Group 1 vs. Groups 3, 4, 5). Finding fewer significant pairs than possible doesn't invalidate the overall ANOVA. Answer D correctly captures the nuanced interpretation: evidence exists for specific differences (Group 1 vs. Groups 3, 4, 5), while other potential differences remain neither confirmed nor ruled out. Remember: in hypothesis testing, non-significance means "insufficient evidence," not "no difference."

Question 12

Two separate studies are conducted. Study A compares mean outputs across 3 production lines. Study B compares mean outputs across 10 production lines. Both studies find a significant ANOVA result and plan to conduct post-hoc analyses of all pairwise differences using Tukey's HSD, controlling the family-wise error rate at α = 0.05. How does the larger number of groups in Study B affect the post-hoc analysis compared to Study A?

  1. The statistical power of each individual pairwise comparison will be greater in Study B.
  2. The underlying assumptions of the post-hoc test are more likely to be met in Study B due to the larger number of groups.
  3. The risk of a Type I error for any single comparison is higher in Study B, even though the family-wise rate is controlled.
  4. The minimum observed difference between two sample means required to declare statistical significance will be larger in Study B. (correct answer)
Explanation: When you encounter ANOVA post-hoc analysis questions, focus on how the number of comparisons affects the statistical thresholds needed for significance. Tukey's HSD controls the family-wise error rate by adjusting the critical value based on the total number of pairwise comparisons. With 3 groups, Study A has (32)=3\binom{3}{2} = 3 comparisons, while Study B with 10 groups has (102)=45\binom{10}{2} = 45 comparisons. As the number of comparisons increases, Tukey's HSD requires a larger observed difference between sample means to declare significance. This protects against inflated Type I error rates when making multiple comparisons. The correct answer is D because Tukey's HSD becomes more conservative (requires larger differences) as the number of groups increases. The critical value grows with more comparisons, so Study B will need larger observed differences than Study A to reach statistical significance. Answer A is wrong because statistical power actually decreases in Study B due to the more stringent significance threshold required for each comparison. Answer B is incorrect because the number of groups doesn't affect whether assumptions like normality or equal variances are met—these depend on the data characteristics, not the study design. Answer C misunderstands how family-wise error control works: Tukey's HSD keeps the probability of any Type I error at α = 0.05 regardless of group number, meaning individual comparison error rates actually become smaller as more groups are added. Remember: More groups in ANOVA post-hoc testing means more conservative tests and larger differences needed for significance, even though overall error rates remain controlled.

Question 13

Following a significant one-way ANOVA on sales data from four regions, a post-hoc test reveals that mean sales in the North region are 'significantly higher' than in the South region (p = 0.02). What is the correct interpretation of the term 'significantly higher' in this business context?

  1. The observed difference in mean sales is unlikely to be a result of random chance if the true regional sales means were actually equal. (correct answer)
  2. The difference in sales between the North and South regions is large enough to be considered strategically important for the company.
  3. Sales in the North region are guaranteed to outperform sales in the South region in all subsequent business cycles.
  4. The marketing strategy employed in the North region has been proven to be the direct cause of the superior sales results.
Explanation: When you encounter post-hoc test results following ANOVA, you're dealing with statistical significance, which has a very specific meaning that's often misunderstood in business contexts. The key is distinguishing between statistical significance and practical importance. Statistical significance (p = 0.02) tells you about the reliability of your finding, not its magnitude or cause. A p-value of 0.02 means there's only a 2% probability of observing this difference (or larger) if the regional means were actually equal in the population. This makes option A correct—the observed difference is unlikely due to random chance alone. Option B confuses statistical significance with practical significance. A statistically significant difference could actually be quite small in dollar terms and have little strategic value. Statistical tests don't measure business importance. Option C represents a classic overgeneralization error. Statistical significance never guarantees future outcomes—it only tells you about the reliability of your current sample results. Future performance could vary due to countless factors. Option D falls into the causation trap. Post-hoc tests only reveal which groups differ significantly; they provide zero information about why those differences exist. The superior North region performance could stem from demographics, competition, economic conditions, or numerous other factors beyond marketing strategy. Study tip: Remember that "statistically significant" always means "probably not due to chance"—nothing more, nothing less. It doesn't indicate size, importance, future certainty, or causation. Keep this distinction sharp to avoid common ANOVA interpretation errors.

Question 14

A human resources director compared job satisfaction scores across four departments using ANOVA (F(3,156) = 6.23, p < 0.001). She conducted post-hoc analysis and obtained the following p-values for pairwise comparisons using different methods: Sales vs. Marketing (Fisher's LSD: p = 0.023, Tukey's HSD: p = 0.089), Engineering vs. Marketing (Fisher's LSD: p = 0.041, Tukey's HSD: p = 0.156), Sales vs. HR (Fisher's LSD: p = 0.018, Tukey's HSD: p = 0.071). If she wants to minimize the risk of falsely concluding differences exist while maintaining reasonable statistical power, which approach should she choose?

  1. Use Fisher's LSD results since the omnibus ANOVA was highly significant, providing adequate Type I error control
  2. Use Fisher's LSD for exploratory analysis but require replication before making organizational decisions
  3. Average the p-values from both methods to balance Type I error control with statistical power considerations
  4. Use Tukey's HSD results since they properly control familywise error rate while maintaining reasonable power (correct answer)
Explanation: When you encounter ANOVA followed by post-hoc comparisons, you're dealing with the multiple comparisons problem - the more statistical tests you run, the higher your chance of finding false positives (Type I errors). The key decision is choosing between methods that prioritize different error control strategies. Tukey's HSD (Honestly Significant Difference) is specifically designed to control the familywise error rate, meaning it keeps your overall probability of making any Type I error across all comparisons at your chosen alpha level (typically 0.05). This makes it the gold standard when you want to minimize false discoveries while still maintaining reasonable power to detect true differences. In this scenario, where the HR director wants to "minimize the risk of falsely concluding differences exist," Tukey's more stringent approach is appropriate, making D correct. A is incorrect because while Fisher's LSD does provide some protection when the omnibus ANOVA is significant, it doesn't control familywise error rate - it only controls per-comparison error rate, leading to inflated Type I error risk across multiple tests. B suggests a overly cautious approach that doesn't directly address the statistical methodology question and essentially sidesteps the multiple comparisons issue. C is statistically invalid - you cannot simply average p-values from different methods. This approach has no theoretical justification and would produce meaningless results. Study tip: Remember that Tukey's HSD trades some statistical power for better Type I error control, while Fisher's LSD does the opposite. Choose based on whether avoiding false positives or detecting true differences is more critical.

Question 15

A pharmaceutical company tested five different drug formulations for pain relief using a randomized controlled trial with 50 participants per group. The ANOVA revealed significant differences among treatments (F(4,245) = 8.23, p < 0.001). The research team wants to make specific comparisons based on their hypotheses: (1) compare the new formulation against placebo, (2) compare two established drugs against each other, and (3) test if the average of two combination therapies differs from the single-agent therapy.

Given the research team's specific planned comparisons described above, which post-hoc approach would be most statistically appropriate and efficient?

  1. Use Tukey's HSD for all pairwise comparisons since it controls familywise error rate comprehensively
  2. Apply Bonferroni correction to the three planned contrasts using α/3 = 0.017 for each test
  3. Conduct the three planned orthogonal contrasts without adjustment since they were specified a priori (correct answer)
  4. Use Scheffé's method since it accommodates complex contrasts including the combination therapy comparison
Explanation: When contrasts are planned a priori and are orthogonal (independent), no multiple comparison adjustment is needed, providing maximum statistical power. The described comparisons appear to be pre-planned based on research hypotheses. Choice A is unnecessarily conservative for planned comparisons. Choice B applies unnecessary correction to planned contrasts. Choice D (Scheffé) is overly conservative for a small number of planned comparisons.

Question 16

A production manager analyzed defect rates across three shifts using ANOVA and found significant differences (p = 0.012). She calculated the following confidence intervals for pairwise differences using Tukey's HSD: Day vs. Evening: [-2.1, -0.3], Evening vs. Night: [-0.8, 1.4], Day vs. Night: [-2.9, -0.1]. Based on these results, which interpretation is correct?

  1. Day shift has significantly lower defect rates than both Evening and Night shifts, while Evening and Night shifts don't differ significantly (correct answer)
  2. All three pairwise comparisons show significant differences since the overall ANOVA was significant
  3. Only the Day vs. Evening comparison is significant because it has the largest effect size
  4. Evening shift has significantly higher defect rates than both other shifts, with Day and Night shifts being equivalent
Explanation: Confidence intervals that don't contain zero indicate significant differences. Day vs. Evening [-2.1, -0.3] and Day vs. Night [-2.9, -0.1] don't contain zero (significant), while Evening vs. Night [-0.8, 1.4] contains zero (not significant). This means Day shift differs from both others. Choice B incorrectly assumes ANOVA significance means all pairwise comparisons are significant. Choice C misinterprets which comparisons are significant. Choice D misreads the pattern of differences.

Question 17

A manager is reviewing a statistical report that compared the productivity of 4 different sales teams. The report states: "A one-way ANOVA was significant (F(3, 76) = 4.5, p = 0.006). Subsequently, all six possible pairwise comparisons were made using uncorrected t-tests at α = 0.05. Two of these comparisons were found to be statistically significant." Based on the methodology described, what is the most pressing concern regarding the report's conclusion?

  1. The ANOVA p-value of 0.006 is not low enough to justify any follow-up tests; a p-value less than 0.001 should be required.
  2. The reported significant differences may be spurious (Type I errors) because the family-wise error rate was not controlled. (correct answer)
  3. The sample size (n=80 total) was likely too small to have sufficient power for the six pairwise comparisons.
  4. The use of t-tests after ANOVA is inappropriate; chi-square tests should have been used for pairwise comparisons instead.
Explanation: The most significant flaw in the described analysis is the use of uncorrected t-tests for post-hoc comparisons. This procedure fails to control for the inflation of the family-wise error rate. With six comparisons at α = 0.05, the probability of finding at least one significant result by chance alone is much higher than 5%. Therefore, the two 'significant' findings could easily be Type I errors, and the conclusion is suspect.

Question 18

A pharmaceutical company compares a new drug against three existing drugs. The consequences of a Type I error (incorrectly claiming the new drug is superior) are severe. However, a Type II error (failing to detect a real improvement) would be a major missed opportunity. After a significant ANOVA, the goal is to conduct all pairwise comparisons. Which post-hoc procedure offers the best balance by effectively controlling the family-wise error rate while maintaining reasonable statistical power for this specific goal?

  1. Fisher's LSD, because maximizing power to avoid a Type II error is the most important consideration in drug development.
  2. The Scheffé test, because it offers the absolute strongest protection against Type I error, which is the stated primary concern.
  3. Tukey's HSD, as it is designed for all pairwise comparisons and is considered a good compromise between controlling Type I error and retaining power. (correct answer)
  4. A series of uncorrected t-tests, as this maximizes the chance of finding a significant result and avoiding a missed market opportunity.
Explanation: The scenario requires a balance between controlling Type I error (high priority) and maintaining power (also important). The Scheffé test is often too conservative for simple pairwise comparisons, greatly reducing power. Fisher's LSD and uncorrected t-tests do not adequately control the family-wise error rate. Tukey's HSD is specifically designed for all pairwise comparisons and is widely considered to be the best-balanced procedure for this common scenario, offering strong FWER control without being overly conservative.

Question 19

An operations manager performs a one-way ANOVA to compare the mean production times of four different assembly line configurations. The F-test yields a p-value of 0.09 at an alpha level of 0.05. What is the most appropriate next step for the manager to take?

  1. Perform Fisher's LSD test because it is the most powerful post-hoc test for finding any potential differences.
  2. Perform the Scheffé test because it is the most conservative and suitable for any unplanned comparisons.
  3. Conclude there is insufficient evidence of a difference among the means and do not perform post-hoc tests. (correct answer)
  4. Increase the sample size for each configuration and re-run the ANOVA until a significant result is obtained.
Explanation: A fundamental prerequisite for conducting most post-hoc tests (like Tukey's, Scheffé's, etc.) is a statistically significant overall ANOVA F-test. Since the p-value (0.09) is greater than the alpha level (0.05), the null hypothesis of equal means cannot be rejected. Therefore, there is no statistical justification for proceeding to find which specific means are different. The correct conclusion is that there is insufficient evidence to claim any difference exists.

Question 20

An analyst conducts a study with six groups and obtains a significant ANOVA F-test. They are considering two post-hoc options for pairwise comparisons: Tukey's HSD or Fisher's LSD. Which of the following statements provides the strongest rationale for choosing Tukey's HSD over Fisher's LSD in this situation?

  1. Tukey's HSD has greater statistical power, making it more likely to detect true differences between the six groups.
  2. Fisher's LSD is invalid in this case because the initial F-test was significant.
  3. Tukey's HSD maintains control of the family-wise error rate across all 15 pairwise comparisons, whereas Fisher's LSD does not. (correct answer)
  4. Fisher's LSD requires a much larger sample size per group to be effective compared to Tukey's HSD.
Explanation: The key distinction between these tests is the control of the family-wise error rate (FWER). For more than three groups, Fisher's LSD does not maintain the FWER at the nominal alpha level (e.g., 0.05). With six groups, there are 15 pairwise comparisons, and the FWER for Fisher's LSD would be considerably higher than 0.05. Tukey's HSD is specifically designed to control the FWER at the specified alpha level for all pairwise comparisons, making it the appropriate choice for maintaining Type I error control.