Business Statistics Quiz: Hypothesis Testing Framework
20 questions · exam conditions
0:00
Hypothesis Testing FrameworkQuestion 1 of 20

A software company tests whether their app's average session time equals 12 minutes. They obtain a p-value of 0.08 using α=0.05\alpha = 0.05. The CEO asks: 'What would happen to our conclusion if we had chosen α=0.10\alpha = 0.10 from the beginning?' Assuming all other aspects of the study remain identical, how should the statistician respond?

The p-value would need to be recalculated using α=0.10\alpha = 0.10, so the conclusion cannot be determined from the current information
The conclusion would remain 'fail to reject H0H_0' because changing the significance level after seeing the results violates the principles of hypothesis testing
The conclusion would change from 'fail to reject H0H_0' to 'reject H0H_0' because 0.08 < 0.10, indicating statistical significance at the higher α level
The conclusion would remain unchanged because p-values between 0.05 and 0.10 are considered marginally significant regardless of the chosen α level
← Back to quizzes

Business Statistics Quiz

Business Statistics Quiz: Hypothesis Testing Framework

Practice Hypothesis Testing Framework in Business Statistics with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Hypothesis Testing Framework, giving you a quick way to practice the rules, question types, and explanations that matter most for Business Statistics.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

A software company tests whether their app's average session time equals 12 minutes. They obtain a p-value of 0.08 using α=0.05\alpha = 0.05. The CEO asks: 'What would happen to our conclusion if we had chosen α=0.10\alpha = 0.10 from the beginning?' Assuming all other aspects of the study remain identical, how should the statistician respond?

  1. The p-value would need to be recalculated using α=0.10\alpha = 0.10, so the conclusion cannot be determined from the current information
  2. The conclusion would remain 'fail to reject H0H_0' because changing the significance level after seeing the results violates the principles of hypothesis testing
  3. The conclusion would change from 'fail to reject H0H_0' to 'reject H0H_0' because 0.08 < 0.10, indicating statistical significance at the higher α level (correct answer)
  4. The conclusion would remain unchanged because p-values between 0.05 and 0.10 are considered marginally significant regardless of the chosen α level
Explanation: When you encounter hypothesis testing questions involving significance levels and p-values, focus on the fundamental decision rule: reject the null hypothesis when the p-value is less than your chosen significance level (α\alpha). In this scenario, the p-value of 0.08 remains fixed regardless of which significance level you choose—it's determined by the data and test statistic. At α=0.05\alpha = 0.05, since 0.08 > 0.05, you fail to reject H0H_0. However, if the significance level had been set at α=0.10\alpha = 0.10 from the beginning, then 0.08 < 0.10, leading you to reject H0H_0. The conclusion legitimately changes because you're applying a different decision criterion to the same evidence. Option A is incorrect because p-values don't need recalculation when you change α\alpha—the p-value depends only on your data and test statistic. Option B misunderstands the CEO's hypothetical question; they're asking "what if we had chosen α=0.10\alpha = 0.10 initially," not suggesting they change it post-hoc. While changing α\alpha after seeing results is indeed problematic, that's not what's happening here. Option D is wrong because there's no universal "marginal significance" category that overrides your chosen significance level—statistical significance is determined solely by comparing p-value to α\alpha. Study tip: Remember that the p-value is fixed by your data, but your conclusion depends on the significance level you choose. Always compare p-value to α\alpha directly: p < α\alpha means reject H0H_0, p ≥ α\alpha means fail to reject H0H_0.

Question 2

An analyst tests H0:μ=100H_0: \mu = 100 versus H1:μ≠100H_1: \mu \neq 100 and obtains a p-value of 0.03. Her supervisor suggests that a p-value of 0.03 means there is only a 3% chance that the null hypothesis is true. How should the analyst respond to this interpretation?

  1. The supervisor is correct; a p-value of 0.03 indicates a 97% probability that the alternative hypothesis is true
  2. The interpretation is incorrect; the p-value is the probability of observing the sample result or more extreme, assuming the null hypothesis is true (correct answer)
  3. The supervisor's interpretation is approximately correct for small p-values, but becomes less accurate as p-values approach 0.50
  4. The interpretation is backwards; a p-value of 0.03 actually indicates a 97% chance that the null hypothesis is true
Explanation: The p-value is P(observing sample result or more extreme | H₀ is true), not P(H₀ is true | sample result). This is a common misconception confusing conditional probabilities. The p-value does not give the probability that either hypothesis is true. Choice A perpetuates the same error as the supervisor. Choice C incorrectly suggests the misinterpretation is sometimes valid. Choice D compounds the error by reversing the probabilities incorrectly.

Question 3

A pharmaceutical company claims their new drug is effective in at least 75% of patients. In a clinical trial of 200 patients, 142 showed improvement. The test statistic is z=−1.31z = -1.31 with a p-value of 0.095. Which statement correctly interprets this result in the context of the hypothesis testing framework?

  1. At α=0.05\alpha = 0.05, we reject the null hypothesis and conclude there is sufficient evidence that the drug's effectiveness rate is less than 75%
  2. At α=0.10\alpha = 0.10, we reject the null hypothesis and conclude there is sufficient evidence that the drug's effectiveness rate is less than 75% (correct answer)
  3. At α=0.05\alpha = 0.05, we fail to reject the null hypothesis, meaning we cannot conclude the drug's effectiveness rate differs significantly from 75%
  4. At α=0.10\alpha = 0.10, we fail to reject the null hypothesis, indicating the sample provides strong evidence supporting the company's claim
Explanation: The null hypothesis is H₀: p ≥ 0.75 (company's claim) and alternative is H₁: p < 0.75. With p-value = 0.095 and α = 0.10, we reject H₀ since 0.095 < 0.10, concluding insufficient evidence for the company's claim. Choice A is wrong because 0.095 > 0.05, so we don't reject at α = 0.05. Choice C misinterprets the alternative hypothesis direction. Choice D incorrectly states we fail to reject when p-value < α, and failing to reject H₀ doesn't provide strong evidence for the claim.

Question 4

An operations manager tests whether machine downtime has decreased from the historical average of 4.2 hours per week. Using α=0.05\alpha = 0.05 and a sample of 40 weeks, she obtains xˉ=3.8\bar{x} = 3.8 hours and calculates a test statistic of z=−2.12z = -2.12. Without calculating the exact p-value, what can be concluded about this hypothesis test?

  1. Since ∣z∣=2.12>1.96|z| = 2.12 > 1.96, we reject H0H_0 and conclude downtime has significantly decreased from 4.2 hours per week
  2. The negative test statistic indicates the sample mean is in the wrong direction, so we automatically fail to reject H0H_0 regardless of its magnitude
  3. Since z=−2.12z = -2.12 falls between -2.58 and 2.58, we fail to reject H0H_0 at the 0.01 significance level commonly used for operational decisions
  4. Since z=−2.12<−1.645z = -2.12 < -1.645, we reject H0H_0 and conclude downtime has significantly decreased from 4.2 hours per week (correct answer)
Explanation: When you encounter hypothesis testing questions about operations, focus on identifying the type of test and matching your critical value to the alternative hypothesis direction. This is a left-tailed test because the manager wants to know if downtime has decreased from 4.2 hours. The alternative hypothesis is Ha:μ<4.2H_a: \mu < 4.2, so we're only concerned with the left tail of the distribution. For a left-tailed test at α=0.05\alpha = 0.05, the critical value is z=−1.645z = -1.645. Since our test statistic z=−2.12z = -2.12 is less than -1.645 (falls further into the rejection region), we reject H0H_0 and conclude downtime has significantly decreased. Answer A incorrectly treats this as a two-tailed test by using ±1.96\pm 1.96 and taking the absolute value. This misses that we're testing for a specific direction of change. Answer B shows a fundamental misunderstanding—negative test statistics aren't "wrong" when testing for decreases; they're exactly what we expect and want to see. Answer C uses the wrong significance level (α=0.01\alpha = 0.01 with critical value ±2.58\pm 2.58) and applies two-tailed logic when the problem clearly states α=0.05\alpha = 0.05. The key strategy here is recognizing directional language in the problem. Words like "decreased," "increased," "greater than," or "less than" signal one-tailed tests. Always match your critical value to both the significance level given AND the direction being tested. Left-tailed tests use negative critical values; right-tailed tests use positive ones.

Question 5

A researcher conducts three separate hypothesis tests on the same dataset: (1) a two-tailed test for μ=50\mu = 50 with p-value = 0.084, (2) a one-tailed test for μ>50\mu > 50 with p-value = 0.958, and (3) a one-tailed test for μ<50\mu < 50 with p-value = 0.042. At α=0.05\alpha = 0.05, what can be concluded about the relationship between these tests?

  1. Only test (3) leads to rejection of the null hypothesis, and the sample mean must be significantly less than 50 (correct answer)
  2. Tests (1) and (3) both lead to rejection since their p-values sum to less than the two-tailed p-value from test (1)
  3. All three tests fail to reject their respective null hypotheses because the two-tailed test has the highest p-value
  4. Tests (2) and (3) contradict each other since one p-value is very high while the other is very low using identical data
Explanation: At α = 0.05, only test (3) has p-value < 0.05 (0.042), so only H₀: μ ≥ 50 is rejected, concluding μ < 50. The p-values are consistent: 0.042 + 0.958 = 1.000, and 0.042 × 2 = 0.084 (two-tailed p-value). Choice B misunderstands p-value relationships and incorrectly states test (1) rejects H₀. Choice C ignores that test (3) does reject H₀. Choice D incorrectly sees contradiction in mathematically consistent complementary p-values.

Question 6

A marketing team tests a new website layout against the current one to see if it increases user engagement. After running an A/B test, they calculate a p-value of 0.049. The team's pre-determined significance level is α=0.05\alpha = 0.05. Which of the following is the most prudent course of action?

  1. Conclude that the new layout is definitively superior and immediately roll it out to all users.
  2. Declare the result statistically significant but recommend a follow-up study or further analysis due to the result's proximity to the significance threshold. (correct answer)
  3. Conclude that there is no difference between the layouts because the p-value is practically equal to the significance level.
  4. Increase the significance level to α=0.10\alpha = 0.10 to make the evidence for the new layout appear stronger, then proceed with the rollout.
Explanation: The p-value of 0.049 is less than the significance level of 0.05, so the result is technically statistically significant. However, because it is so close to the threshold, the evidence is not overwhelmingly strong. Prudent business practice would involve acknowledging the statistical significance while also considering the practical significance and the borderline nature of the result before committing to a costly implementation. A follow-up study could provide more conclusive evidence.

Question 7

Two independent research teams are testing the same null hypothesis (H₀: μ=100\mu = 100) against the same two-tailed alternative (Hₐ: μ≠100\mu \neq 100) using the same significance level. Team A obtains a test statistic of z=2.15z = 2.15. Team B obtains a test statistic of z=2.60z = 2.60.

Based on this information, which conclusion about the p-values from the two studies is correct?

  1. The p-value for Team A's test will be smaller than the p-value for Team B's test.
  2. The p-value for Team B's test will be smaller than the p-value for Team A's test. (correct answer)
  3. The relationship between the p-values cannot be determined without knowing the sample sizes used by each team.
  4. The relationship between the p-values cannot be determined without knowing the pre-determined significance level (α\alpha).
Explanation: The p-value represents the probability of observing a test statistic as extreme as or more extreme than the one calculated, assuming the null hypothesis is true. A more extreme test statistic (further from zero) will correspond to a smaller tail area on the standard normal distribution curve. Since ∣2.60∣>∣2.15∣|2.60| > |2.15|, the test statistic from Team B is more extreme, and therefore its associated p-value will be smaller.

Question 8

A pharmaceutical company is testing a new drug designed to lower blood pressure. The null hypothesis is that the drug has no effect on blood pressure (H₀: μchange=0\mu_{change} = 0), while the alternative hypothesis is that the drug lowers blood pressure (Hₐ: μchange<0\mu_{change} < 0).

From the company's perspective, what would be the consequence of making a Type I error?

  1. The company fails to market a drug that is actually effective, losing potential revenue and the opportunity to help patients.
  2. The company markets a drug that is ineffective, leading to wasted resources, potential patient harm, and reputational damage. (correct answer)
  3. The company correctly concludes the drug is effective and successfully brings a beneficial product to market.
  4. The company correctly concludes the drug is ineffective, saving the costs associated with a full-scale launch.
Explanation: A Type I error occurs when we reject a true null hypothesis. In this context, the null hypothesis is that the drug has no effect. Rejecting this true null hypothesis means concluding the drug is effective when it is not. This would lead the company to invest in producing and marketing an ineffective product, which is a costly mistake with potentially severe consequences.

Question 9

A researcher conducts a hypothesis test using a significance level of α=0.05\alpha = 0.05 and obtains a p-value of 0.07. Finding the result is not statistically significant, the researcher considers re-evaluating the data using a significance level of α=0.10\alpha = 0.10. What is the primary implication of this change?

  1. The p-value of the test will decrease, making the result more likely to be significant.
  2. The probability of making a Type II error (β\beta) increases, while the power of the test decreases.
  3. The conclusion of the test changes, and the probability of making a Type I error increases. (correct answer)
  4. The standard error of the estimate will decrease, resulting in a narrower confidence interval for the effect.
Explanation: Changing the significance level α\alpha from 0.05 to 0.10 does not change the p-value, which is calculated from the data. However, it does change the decision rule. With α=0.10\alpha = 0.10, the p-value of 0.07 is now less than α\alpha, so the conclusion changes from 'fail to reject H₀' to 'reject H₀'. The significance level α\alpha is, by definition, the probability of making a Type I error. Therefore, increasing α\alpha from 0.05 to 0.10 increases the risk of a Type I error.

Question 10

A factory supervisor tests whether a new production method is faster than the old one. The hypotheses are H₀: μnew≥μold\mu_{new} \geq \mu_{old} and Hₐ: μnew<μold\mu_{new} < \mu_{old}. The supervisor chooses a significance level of α=0.01\alpha = 0.01 to be very certain before investing in the new method. The test yields a p-value of 0.025. What is the correct conclusion and the type of risk the supervisor is accepting?

  1. Reject H₀ and risk a Type I error by possibly implementing a method that is not truly faster.
  2. Reject H₀ and risk a Type II error by possibly implementing a method that is not truly faster.
  3. Fail to reject H₀ and risk a Type I error by possibly missing out on a faster method.
  4. Fail to reject H₀ and risk a Type II error by possibly missing out on a faster method. (correct answer)
Explanation: The decision rule is to reject H₀ if the p-value is less than α\alpha. Here, the p-value (0.025) is greater than the chosen significance level (α=0.01\alpha = 0.01). Therefore, the supervisor must fail to reject the null hypothesis. A Type II error is the failure to reject a false null hypothesis. In this context, if the new method is actually faster (meaning H₀ is false), the supervisor's decision to not reject H₀ would be a Type II error, causing the company to miss out on an improvement.

Question 11

An online retail giant tests a minor change to its website's font using a very large sample of 2,000,000 users. The test shows that the new font increases the average time spent on the site by 0.1 seconds, with a p-value < 0.0001. Which of the following is the most appropriate business interpretation of this result?

  1. Since the p-value is extremely small, the result is highly significant and the new font should be implemented immediately to increase user engagement.
  2. The result is statistically significant, but the effect size (0.1 seconds) is so small that it may not be practically meaningful for the business. (correct answer)
  3. The p-value is so low that it must be the result of a calculation error or a flawed experimental design; the test should be disregarded.
  4. The test proves that the new font causes users to stay on the site longer, justifying its implementation regardless of the magnitude of the increase.
Explanation: With a very large sample size, even a minuscule effect can become statistically significant (i.e., have a very small p-value). The p-value indicates that the 0.1-second increase is unlikely to be due to random chance. However, statistical significance does not automatically imply practical or business significance. An average increase of 0.1 seconds is likely too small to have any real impact on business outcomes, and may not be worth the cost or effort of implementation.

Question 12

A company policy states that all A/B tests must use a significance level of α=0.01\alpha = 0.01 for making decisions. The product team runs a test on a new feature and obtains a p-value of 0.03. Which of the following statements is the most accurate?

  1. Although the p-value is small, according to company policy, the result is not statistically significant. (correct answer)
  2. The result is statistically significant because the p-value is less than the conventional threshold of 0.05.
  3. The company policy should be changed to α=0.05\alpha = 0.05 to allow this promising result to be considered significant.
  4. The p-value of 0.03 indicates that there is strong evidence for the new feature's effectiveness, overriding the company policy.
Explanation: The framework of hypothesis testing requires the significance level (α\alpha) to be set before conducting the test. This α\alpha level serves as the decision threshold. In this case, the company policy sets α=0.01\alpha = 0.01. The obtained p-value of 0.03 is greater than 0.01. Therefore, according to the pre-established rule, the null hypothesis cannot be rejected, and the result is not considered statistically significant. Other conventional thresholds or the perceived strength of the p-value are irrelevant to the decision based on the stated policy.

Question 13

An analyst is investigating whether a new advertising campaign has increased weekly sales. The null hypothesis states that the mean weekly sales have not increased. After collecting data, the analyst observes a substantial increase in the sample mean sales. Which of the following statements best describes the logical next step in the hypothesis testing framework?

  1. Conclude that the advertising campaign was successful because the sample mean sales increased.
  2. Calculate the probability of the null hypothesis being true given the observed increase in sales.
  3. Determine how likely the observed increase in sample sales would be if the campaign actually had no effect. (correct answer)
  4. Accept the alternative hypothesis because the initial evidence aligns with the desired outcome.
Explanation: The core logic of frequentist hypothesis testing is not to directly evaluate the probability of a hypothesis. Instead, we assume the null hypothesis (no effect) is true and then calculate the probability of observing our sample data (or more extreme data) under this assumption. This probability is the p-value. A substantial increase in the sample mean is not sufficient evidence on its own; we need to assess if this increase is too large to be explained by random sampling variability. Choice C perfectly describes this process of calculating the p-value.

Question 14

A pilot study with a sample of n=40n=40 is conducted to test for a difference in customer satisfaction scores. The result is not statistically significant, with a p-value of 0.15. The manager believes a true difference exists and authorizes a larger study. Assuming the true effect size is non-zero and the second study with n=250n=250 finds the same sample effect size as the first, what is the most probable p-value for the second study?

  1. It will be smaller than 0.15. (correct answer)
  2. It will be larger than 0.15.
  3. It will remain approximately 0.15.
  4. It will be equal to 0.15×(40/250)0.15 \times (40/250).
Explanation: When you encounter questions about sample size and statistical significance, remember that larger samples provide more precise estimates and greater power to detect true effects. This question tests your understanding of how sample size affects p-values when the underlying effect remains constant. The key insight is that p-values depend on both effect size and sample size. When the true effect size stays the same but you dramatically increase your sample size (from 40 to 250), you're giving your statistical test much more power to detect that effect. A larger sample reduces the standard error of your estimate, making your test statistic larger and your p-value smaller. Since the manager suspects a true difference exists and you're using the same effect size with a much larger sample, the p-value will decrease from 0.15, making it more likely to reach statistical significance. Answer A is correct because the larger sample size will produce a smaller p-value than 0.15. Answer B suggests the opposite relationship between sample size and p-values, which contradicts fundamental statistical principles. Answer C incorrectly assumes that p-values remain constant regardless of sample size changes. Answer D presents a fabricated formula that has no basis in statistical theory—p-values don't scale linearly with sample size ratios. Study tip: Remember the inverse relationship between sample size and p-values: when effect size is held constant, larger samples yield smaller p-values and greater statistical power. This principle appears frequently on business statistics exams, especially in questions about study design and power analysis.

Question 15

A bank is creating a statistical model to screen loan applications. The null hypothesis is that the applicant will not default on the loan (H₀: Applicant will not default). A Type I error means rejecting a non-defaulter (denying a good applicant), resulting in lost interest income. A Type II error means failing to reject a defaulter (approving a bad applicant), resulting in the loss of the entire principal.

If the cost of a Type II error is considered vastly greater than the cost of a Type I error, how should the bank set its significance level, α\alpha?

  1. Set α\alpha to a relatively high value (e.g., 0.10) to be more likely to identify and reject potential defaulters. (correct answer)
  2. Set α\alpha to a very low value (e.g., 0.01) to minimize the number of rejected good applicants.
  3. Set α\alpha at the conventional 0.05 level to balance the risks of both error types equally.
  4. The value of α\alpha is irrelevant, as it only affects Type I errors, not Type II errors.
Explanation: When you encounter hypothesis testing problems involving error costs, focus on the relationship between the significance level (α\alpha) and the two types of errors. Type I error occurs when you reject a true null hypothesis, while Type II error occurs when you fail to reject a false null hypothesis. The significance level α\alpha directly controls the probability of Type I error. In this banking scenario, Type II errors (approving defaulters) cost much more than Type I errors (rejecting good applicants) because losing the entire principal exceeds lost interest income. To minimize devastating Type II errors, you need to be more aggressive about rejecting applicants, which means increasing your willingness to make Type I errors. Answer A is correct because setting a higher α\alpha (like 0.10) makes you more likely to reject the null hypothesis ("will not default"), thus catching more potential defaulters. This reduces Type II errors at the cost of increasing Type I errors—exactly what you want when Type II errors are vastly more expensive. Answer B is wrong because a very low α\alpha (0.01) would minimize Type I errors but increase Type II errors, which is backwards given the cost structure. Answer C is incorrect because the conventional 0.05 level doesn't account for the dramatically different error costs—you shouldn't balance them equally when one is much more expensive. Answer D is false because α\alpha indirectly affects Type II errors through the trade-off relationship between error types. Remember: when one error type is much costlier, adjust α\alpha to minimize that error, even if it increases the less costly error type.

Question 16

A government agency is testing a food manufacturer for compliance with a law stating that the average sodium content in its canned soup must not exceed 470 mg per serving. The agency will issue a warning if there is sufficient evidence that the manufacturer is exceeding this limit. From the agency's perspective, what are the correct null and alternative hypotheses?

  1. H₀: μ=470\mu = 470; Hₐ: μ≠470\mu \neq 470
  2. H₀: μ≥470\mu \geq 470; Hₐ: μ<470\mu < 470
  3. H₀: μ≤470\mu \leq 470; Hₐ: μ>470\mu > 470 (correct answer)
  4. H₀: xˉ≤470\bar{x} \leq 470; Hₐ: xˉ>470\bar{x} > 470
Explanation: The agency is looking for evidence to take action, which is issuing a warning for exceeding the limit. Therefore, the claim to be proven, the alternative hypothesis, is that the mean sodium content is greater than 470 mg (Hₐ: μ>470\mu > 470). The null hypothesis represents the state of compliance or innocence, which is that the mean sodium content is less than or equal to the limit (H₀: μ≤470\mu \leq 470). The burden of proof is on the agency to show that the manufacturer is out of compliance. Choice D is incorrect because hypotheses are about the population parameter μ\mu, not the sample statistic xˉ\bar{x}.

Question 17

In the context of hypothesis testing, suppose a test is conducted with α=0.05\alpha = 0.05 and results in a p-value of 0.04. Unknown to the researcher, the null hypothesis is, in reality, true. Which of the following has occurred?

  1. A correct decision was made, as the data provided sufficient evidence to reject the null hypothesis.
  2. A Type I error was made. (correct answer)
  3. A Type II error was made.
  4. No conclusion can be drawn without knowing the power of the test.
Explanation: First, we determine the decision based on the data. Since the p-value (0.04) is less than the significance level (α=0.05\alpha = 0.05), the decision is to reject the null hypothesis. Second, we evaluate this decision against the unknown reality. The problem states that the null hypothesis is actually true. A Type I error is defined as rejecting a true null hypothesis. Therefore, a Type I error has occurred.

Question 18

To estimate a p-value via simulation, a computer program was set to simulate 10,000 datasets under the assumption that the null hypothesis is true. A real-world experiment yielded a test statistic of 25. The simulation found that 150 of the 10,000 simulated datasets produced a test statistic of 25 or greater. For a one-tailed test where larger values are more extreme, what is the estimated p-value?

  1. 0.0025
  2. 0.015 (correct answer)
  3. 0.150
  4. 150
Explanation: The p-value is the probability of observing a result as extreme as or more extreme than the one from the actual experiment, assuming the null hypothesis is true. The simulation does exactly this. It found that 150 out of 10,000 trials under the null hypothesis resulted in a test statistic of 25 or greater. The estimated p-value is the proportion of these occurrences: p=150/10,000=0.015p = 150 / 10,000 = 0.015.

Question 19

A quality control engineer at a manufacturing plant conducts a hypothesis test to see if a recent process adjustment has changed the mean length of a component. The test yields a p-value of 0.02. Which of the following statements is the correct interpretation of this p-value?

  1. There is a 2% probability that the process adjustment had no effect on the mean length of the component.
  2. There is a 98% probability that the process adjustment had an effect on the mean length of the component.
  3. If the process adjustment truly had no effect, there is a 2% chance of observing a sample mean as far from the original mean as, or farther than, the one observed. (correct answer)
  4. If this experiment were repeated 100 times, the null hypothesis would be incorrectly rejected approximately two times.
Explanation: The p-value is a conditional probability. It is the probability of obtaining the observed sample result, or a more extreme result, given that the null hypothesis is true. Choice C correctly states this definition. Choices A and B are common misinterpretations that incorrectly state the p-value as the probability of the null or alternative hypothesis being true. Choice D incorrectly defines the p-value as the Type I error rate for future experiments; the Type I error rate is α\alpha, not the p-value.

Question 20

A research paper reports a statistically significant finding with p < 0.05. A second paper investigating a similar effect in a different context reports its finding with p = 0.04. What is the most valid comparison between these two results based solely on the p-values reported?

  1. The evidence against the null hypothesis is stronger in the first paper than in the second.
  2. The evidence against the null hypothesis is stronger in the second paper than in the first.
  3. Both results provide a similar level of evidence against their respective null hypotheses, but the second paper's reporting is more precise.
  4. It is impossible to compare the strength of evidence because the first paper did not report an exact p-value. (correct answer)
Explanation: While both results meet the α=0.05\alpha = 0.05 threshold for significance, we cannot compare the strength of the evidence. The first paper's result, p < 0.05, could be p = 0.049 or it could be p = 0.0001. Since we do not know the exact p-value, we cannot say whether the evidence is stronger, weaker, or similar to the second paper's p = 0.04. Any comparison of strength would be pure speculation. The only valid conclusion is that a precise comparison is not possible.