Hypothesis Testing Framework - Business Statistics

A financial analyst tests the hypothesis that a new investment strategy produces returns different from the historical average. The test results in a p-value of 0.35. A significance level of $\alpha = 0.05$ was used. Which statement is the most accurate conclusion?
Answer: The analyst has failed to find sufficient evidence that the new investment strategy's returns differ from the historical average. Hypothesis testing is based on the logic of rejecting or failing to reject the null hypothesis. A high p-value (0.35 > 0.05) means we fail to reject the null hypothesis. This does not mean we have proven the null hypothesis is true; it simply means we lack sufficient evidence to conclude the alternative is true. This is often summarized as 'the absence of evidence is not evidence of absence'. Choice C correctly reflects this nuance. Choices A and B wrongly claim proof or evidence for the null. Choice D is a common misinterpretation of the p-value.
A researcher conducts a hypothesis test and reports: 'We reject $H_0: \mu = 25$ at the 0.05 significance level with a p-value of 0.032.' A colleague questions whether this conclusion would change if they used $\alpha = 0.01$ instead. Based on the information provided, what is the most appropriate response?
Answer: The conclusion would definitely change to 'fail to reject $H_0$ ' because the p-value 0.032 is greater than $\alpha = 0.01$ When you encounter hypothesis testing questions involving different significance levels, the key is understanding how p-values and alpha levels work together in decision-making. The decision rule is straightforward: reject the null hypothesis when the p-value is less than your chosen significance level ( $\alpha$ ). In this scenario, the researcher found a p-value of 0.032 and rejected $H_0$ at $\alpha = 0.05$ . This makes sense because 0.032 < 0.05. However, when considering $\alpha = 0.01$ , you must compare the same p-value (0.032) to the new significance level. Since 0.032 > 0.01, you would fail to reject $H_0$ at the stricter significance level. The conclusion definitely changes, making option C correct. Option A is wrong because the p-value comparison method works regardless of whether the test is one-tailed or two-tailed, and sample size doesn't affect this decision rule. Option B contains a dangerous misconception—statistical significance is not universal across all significance levels. What's significant at $\alpha = 0.05$ may not be significant at $\alpha = 0.01$ . Option D is incorrect because p-values are absolutely dependent on your chosen significance level for decision-making; that's the entire basis of hypothesis testing. Remember this simple rule: stricter significance levels (smaller $\alpha$ values) make it harder to reject the null hypothesis. If your p-value falls between your old and new alpha levels, your conclusion will change. Always compare the p-value directly to your chosen $\alpha$ .
A logistics manager wants to test if a new delivery route is faster than the current route, which has a known mean delivery time of 45 minutes. The manager is only interested in adopting the new route if it demonstrates a statistically significant reduction in time. Which of the following represents the correct null and alternative hypotheses for this test?
Answer: H₀: $\mu \geq 45$ ; Hₐ: $\mu < 45$ The manager is looking for evidence of a reduction in time, so the alternative hypothesis (what the manager wants to prove) should be that the mean time is less than 45 minutes (Hₐ: $\mu < 45$ ). This is a one-tailed test. The null hypothesis represents the status quo or the absence of the effect being tested for. Therefore, it should state that the mean time is greater than or equal to 45 minutes (H₀: $\mu \geq 45$ ). Choice D is incorrect because hypotheses are always formulated in terms of population parameters ( $\mu$ ), not sample statistics ( $\bar{x}$ ).
A medical researcher tests a new treatment's effectiveness and obtains a p-value of 0.18. She concludes: 'Since p > 0.05, we accept the null hypothesis that the treatment has no effect.' What is the primary issue with this conclusion in the context of hypothesis testing framework?
Answer: She should 'fail to reject' the null hypothesis rather than 'accept' it, since the test doesn't prove the null hypothesis is true When you encounter hypothesis testing questions, remember that statistical tests can only provide evidence against the null hypothesis—they cannot prove it's true. This is a fundamental limitation of the framework that many students misunderstand. The correct interpretation here is option D. When p > α (significance level), you should "fail to reject" the null hypothesis rather than "accept" it. A p-value of 0.18 means there's an 18% chance of observing results this extreme or more extreme if the null hypothesis were true. This isn't strong enough evidence to reject the null, but it also doesn't prove the treatment has no effect. The treatment might still work—you simply don't have sufficient evidence to conclude it does. Option A is wrong because while medical research sometimes uses stricter significance levels, 0.05 is perfectly acceptable and commonly used. The choice of α depends on the specific study context, not a universal rule. Option B is incorrect because confidence intervals, while valuable, don't address the fundamental error in language. You could report a confidence interval and still make the same mistake of "accepting" versus "failing to reject." Option C misses the point entirely. The p-value allows for a clear decision within the hypothesis testing framework—it's above the significance level, so you fail to reject. More data might be useful, but it's not required to reach a conclusion. Remember this key distinction: statistical tests can disprove hypotheses but cannot prove them true. Always use "fail to reject" language to maintain this crucial conceptual accuracy.
An analyst tests $H_0: \mu = 100$ versus $H_1: \mu \neq 100$ and obtains a p-value of 0.03. Her supervisor suggests that a p-value of 0.03 means there is only a 3% chance that the null hypothesis is true. How should the analyst respond to this interpretation?
Answer: The interpretation is incorrect; the p-value is the probability of observing the sample result or more extreme, assuming the null hypothesis is true The p-value is P(observing sample result or more extreme | H₀ is true), not P(H₀ is true | sample result). This is a common misconception confusing conditional probabilities. The p-value does not give the probability that either hypothesis is true. Choice A perpetuates the same error as the supervisor. Choice C incorrectly suggests the misinterpretation is sometimes valid. Choice D compounds the error by reversing the probabilities incorrectly.
From the company's perspective, what would be the consequence of making a Type I error?
Answer: The company markets a drug that is ineffective, leading to wasted resources, potential patient harm, and reputational damage. A Type I error occurs when we reject a true null hypothesis. In this context, the null hypothesis is that the drug has no effect. Rejecting this true null hypothesis means concluding the drug is effective when it is not. This would lead the company to invest in producing and marketing an ineffective product, which is a costly mistake with potentially severe consequences.
To estimate a p-value via simulation, a computer program was set to simulate 10,000 datasets under the assumption that the null hypothesis is true. A real-world experiment yielded a test statistic of 25. The simulation found that 150 of the 10,000 simulated datasets produced a test statistic of 25 or greater. For a one-tailed test where larger values are more extreme, what is the estimated p-value?
Answer: 0.015 The p-value is the probability of observing a result as extreme as or more extreme than the one from the actual experiment, assuming the null hypothesis is true. The simulation does exactly this. It found that 150 out of 10,000 trials under the null hypothesis resulted in a test statistic of 25 or greater. The estimated p-value is the proportion of these occurrences: $p = 150 / 10,000 = 0.015$ .
A data scientist performs a two-tailed z-test for a difference in means and obtains a test statistic of $z = 2.0$ . The corresponding two-tailed p-value is approximately 0.0456. Her manager points out that the research question was only concerned with whether the new process was better, not just different. If the analysis is correctly changed to a one-tailed test (Hₐ: $\mu_{new} > \mu_{old}$ ), what would be the new p-value?
Answer: 0.0228 A two-tailed p-value is the sum of the areas in both tails of the distribution that are as extreme as or more extreme than the test statistic. A one-tailed p-value is the area in just one of those tails. Since the observed test statistic (z=2.0) is positive and aligns with the direction of the new one-tailed alternative hypothesis (greater than), the one-tailed p-value is simply half of the two-tailed p-value. Therefore, the new p-value is 0.0456 / 2 = 0.0228.
In the context of hypothesis testing, suppose a test is conducted with $\alpha = 0.05$ and results in a p-value of 0.04. Unknown to the researcher, the null hypothesis is, in reality, true. Which of the following has occurred?
Answer: A Type I error was made. First, we determine the decision based on the data. Since the p-value (0.04) is less than the significance level ( $\alpha = 0.05$ ), the decision is to reject the null hypothesis. Second, we evaluate this decision against the unknown reality. The problem states that the null hypothesis is actually true. A Type I error is defined as rejecting a true null hypothesis. Therefore, a Type I error has occurred.
A quality control manager tests whether the mean weight of packages differs from the target of 500 grams. After collecting a sample of 36 packages with a sample mean of 497 grams and sample standard deviation of 12 grams, she calculates a test statistic of $t = -1.5$ and obtains a p-value of 0.1423. If she had instead used a one-tailed test with the same sample data to test whether packages are underweight, what would be the approximate p-value?
Answer: 0.0712, since the one-tailed p-value is half the two-tailed p-value when the sample mean is in the direction of the alternative hypothesis For a two-tailed test, the p-value of 0.1423 represents the probability of getting a test statistic as extreme as ±1.5. Since the sample mean (497) is less than the target (500), it supports the one-tailed alternative hypothesis that packages are underweight. The one-tailed p-value is half the two-tailed p-value: 0.1423/2 = 0.0712. Choice B incorrectly assumes p-values are the same for one- and two-tailed tests. Choice C incorrectly doubles the p-value. Choice D incorrectly calculates the complement.
A government agency is testing a food manufacturer for compliance with a law stating that the average sodium content in its canned soup must not exceed 470 mg per serving. The agency will issue a warning if there is sufficient evidence that the manufacturer is exceeding this limit. From the agency's perspective, what are the correct null and alternative hypotheses?
Answer: H₀: $\mu \leq 470$ ; Hₐ: $\mu > 470$ The agency is looking for evidence to take action, which is issuing a warning for exceeding the limit. Therefore, the claim to be proven, the alternative hypothesis, is that the mean sodium content is greater than 470 mg (Hₐ: $\mu > 470$ ). The null hypothesis represents the state of compliance or innocence, which is that the mean sodium content is less than or equal to the limit (H₀: $\mu \leq 470$ ). The burden of proof is on the agency to show that the manufacturer is out of compliance. Choice D is incorrect because hypotheses are about the population parameter $\mu$ , not the sample statistic $\bar{x}$ .
An analyst is investigating whether a new advertising campaign has increased weekly sales. The null hypothesis states that the mean weekly sales have not increased. After collecting data, the analyst observes a substantial increase in the sample mean sales. Which of the following statements best describes the logical next step in the hypothesis testing framework?
Answer: Determine how likely the observed increase in sample sales would be if the campaign actually had no effect. The core logic of frequentist hypothesis testing is not to directly evaluate the probability of a hypothesis. Instead, we assume the null hypothesis (no effect) is true and then calculate the probability of observing our sample data (or more extreme data) under this assumption. This probability is the p-value. A substantial increase in the sample mean is not sufficient evidence on its own; we need to assess if this increase is too large to be explained by random sampling variability. Choice C perfectly describes this process of calculating the p-value.
A factory supervisor tests whether a new production method is faster than the old one. The hypotheses are H₀: $\mu_{new} \geq \mu_{old}$ and Hₐ: $\mu_{new} < \mu_{old}$ . The supervisor chooses a significance level of $\alpha = 0.01$ to be very certain before investing in the new method. The test yields a p-value of 0.025. What is the correct conclusion and the type of risk the supervisor is accepting?
Answer: Fail to reject H₀ and risk a Type II error by possibly missing out on a faster method. The decision rule is to reject H₀ if the p-value is less than $\alpha$ . Here, the p-value (0.025) is greater than the chosen significance level ( $\alpha = 0.01$ ). Therefore, the supervisor must fail to reject the null hypothesis. A Type II error is the failure to reject a false null hypothesis. In this context, if the new method is actually faster (meaning H₀ is false), the supervisor's decision to not reject H₀ would be a Type II error, causing the company to miss out on an improvement.
A company policy states that all A/B tests must use a significance level of $\alpha = 0.01$ for making decisions. The product team runs a test on a new feature and obtains a p-value of 0.03. Which of the following statements is the most accurate?
Answer: Although the p-value is small, according to company policy, the result is not statistically significant. The framework of hypothesis testing requires the significance level ( $\alpha$ ) to be set before conducting the test. This $\alpha$ level serves as the decision threshold. In this case, the company policy sets $\alpha = 0.01$ . The obtained p-value of 0.03 is greater than 0.01. Therefore, according to the pre-established rule, the null hypothesis cannot be rejected, and the result is not considered statistically significant. Other conventional thresholds or the perceived strength of the p-value are irrelevant to the decision based on the stated policy.
A researcher conducts three separate hypothesis tests on the same dataset: (1) a two-tailed test for $\mu = 50$ with p-value = 0.084, (2) a one-tailed test for $\mu > 50$ with p-value = 0.958, and (3) a one-tailed test for $\mu < 50$ with p-value = 0.042. At $\alpha = 0.05$ , what can be concluded about the relationship between these tests?
Answer: Only test (3) leads to rejection of the null hypothesis, and the sample mean must be significantly less than 50 At α = 0.05, only test (3) has p-value < 0.05 (0.042), so only H₀: μ ≥ 50 is rejected, concluding μ < 50. The p-values are consistent: 0.042 + 0.958 = 1.000, and 0.042 × 2 = 0.084 (two-tailed p-value). Choice B misunderstands p-value relationships and incorrectly states test (1) rejects H₀. Choice C ignores that test (3) does reject H₀. Choice D incorrectly sees contradiction in mathematically consistent complementary p-values.
A finding from a well-designed and properly executed study is reported as statistically significant. This conclusion, on its own, provides direct evidence for which of the following statements?
Answer: The effect observed in the sample is unlikely to be the result of random sampling variation alone. Statistical significance (e.g., a low p-value) has a very specific meaning. It indicates that if the null hypothesis were true, the observed sample data would be surprising or unlikely. It is a statement about the incompatibility of the data with the null hypothesis due to random chance. It does not, by itself, guarantee practical importance (A), prove the alternative hypothesis (B), or validate the overall quality and design of the study (C).
A quality control engineer at a manufacturing plant conducts a hypothesis test to see if a recent process adjustment has changed the mean length of a component. The test yields a p-value of 0.02. Which of the following statements is the correct interpretation of this p-value?
Answer: If the process adjustment truly had no effect, there is a 2% chance of observing a sample mean as far from the original mean as, or farther than, the one observed. The p-value is a conditional probability. It is the probability of obtaining the observed sample result, or a more extreme result, given that the null hypothesis is true. Choice C correctly states this definition. Choices A and B are common misinterpretations that incorrectly state the p-value as the probability of the null or alternative hypothesis being true. Choice D incorrectly defines the p-value as the Type I error rate for future experiments; the Type I error rate is $\alpha$ , not the p-value.
A software company tests whether their app's average session time equals 12 minutes. They obtain a p-value of 0.08 using $\alpha = 0.05$ . The CEO asks: 'What would happen to our conclusion if we had chosen $\alpha = 0.10$ from the beginning?' Assuming all other aspects of the study remain identical, how should the statistician respond?
Answer: The conclusion would change from 'fail to reject $H_0$ ' to 'reject $H_0$ ' because 0.08 < 0.10, indicating statistical significance at the higher α level When you encounter hypothesis testing questions involving significance levels and p-values, focus on the fundamental decision rule: reject the null hypothesis when the p-value is less than your chosen significance level ( $\alpha$ ). In this scenario, the p-value of 0.08 remains fixed regardless of which significance level you choose—it's determined by the data and test statistic. At $\alpha = 0.05$ , since 0.08 > 0.05, you fail to reject $H_0$ . However, if the significance level had been set at $\alpha = 0.10$ from the beginning, then 0.08 < 0.10, leading you to reject $H_0$ . The conclusion legitimately changes because you're applying a different decision criterion to the same evidence. Option A is incorrect because p-values don't need recalculation when you change $\alpha$ —the p-value depends only on your data and test statistic. Option B misunderstands the CEO's hypothetical question; they're asking "what if we had chosen $\alpha = 0.10$ initially," not suggesting they change it post-hoc. While changing $\alpha$ after seeing results is indeed problematic, that's not what's happening here. Option D is wrong because there's no universal "marginal significance" category that overrides your chosen significance level—statistical significance is determined solely by comparing p-value to $\alpha$ . Study tip: Remember that the p-value is fixed by your data, but your conclusion depends on the significance level you choose. Always compare p-value to $\alpha$ directly: p < $\alpha$ means reject $H_0$ , p ≥ $\alpha$ means fail to reject $H_0$ .
Based on this information, which conclusion about the p-values from the two studies is correct?
Answer: The p-value for Team B's test will be smaller than the p-value for Team A's test. The p-value represents the probability of observing a test statistic as extreme as or more extreme than the one calculated, assuming the null hypothesis is true. A more extreme test statistic (further from zero) will correspond to a smaller tail area on the standard normal distribution curve. Since $|2.60| > |2.15|$ , the test statistic from Team B is more extreme, and therefore its associated p-value will be smaller.
A pharmaceutical company claims their new drug is effective in at least 75% of patients. In a clinical trial of 200 patients, 142 showed improvement. The test statistic is $z = -1.31$ with a p-value of 0.095. Which statement correctly interprets this result in the context of the hypothesis testing framework?
Answer: At $\alpha = 0.10$ , we reject the null hypothesis and conclude there is sufficient evidence that the drug's effectiveness rate is less than 75% The null hypothesis is H₀: p ≥ 0.75 (company's claim) and alternative is H₁: p < 0.75. With p-value = 0.095 and α = 0.10, we reject H₀ since 0.095 < 0.10, concluding insufficient evidence for the company's claim. Choice A is wrong because 0.095 > 0.05, so we don't reject at α = 0.05. Choice C misinterprets the alternative hypothesis direction. Choice D incorrectly states we fail to reject when p-value < α, and failing to reject H₀ doesn't provide strong evidence for the claim.

Start this quiz