Carrying Out a Test for the Difference of Two Population Means - AP Statistics

SECTION 1

Historical Context & Motivation

The need to compare two groups is one of the oldest questions in scientific inquiry: does a new treatment outperform an old one, do students taught by one method perform differently from those taught by another, does crop yield change when fertilizer is applied? Before formal statistical testing existed, researchers relied on subjective judgment — eyeballing averages and hoping the differences looked "big enough." The development of the two-sample t-test gave scientists a principled way to quantify the evidence against the hypothesis that two population means are equal, accounting for both the size of the observed difference and the variability within each sample.

1908 • Student's t-Distribution

William Sealy Gosset, publishing under the pseudonym "Student," derived the t-distribution to handle small samples at the Guinness Brewery, laying the groundwork for all t-based inference.

1935 • Fisher's Design of Experiments

Ronald Fisher formalized the randomized experiment and showed how randomization justifies the use of probability-based inference, including comparisons of two group means.

1938 • The Behrens–Fisher Problem

Statisticians recognized that comparing means from populations with unequal variances posed a theoretical challenge, motivating approximations that would later become standard practice.

1946 • Welch's Approximation

Bernard Welch published an approximate degrees-of-freedom formula that does not assume equal variances, producing the procedure now recommended by the AP Statistics curriculum.

Today, when an AP Statistics student encounters two independent groups and a quantitative response variable, the central question remains the same one Gosset and Welch wrestled with: Is the difference we observe in sample means large enough, relative to sampling variability, to constitute convincing evidence of a real difference in population means? This lesson walks through every step of carrying out that test.

SECTION 2

Core Principles & Definitions

Before computing any test statistic, it is essential to understand the conceptual architecture of a two-sample t-test. The procedure compares two independent random samples — one of size n₁ from Population 1 and another of size n₂ from Population 2 — and asks whether the observed difference in sample means, x̄₁ − x̄₂, is statistically significant. The five foundational ideas below govern the entire process.

Null & Alternative Hypotheses

The null hypothesis H₀: μ₁ − μ₂ = 0 asserts no difference in population means. The alternative (Hₐ) may be two-sided (≠ 0) or one-sided (> 0 or < 0), chosen before data are collected.

Conditions for Inference

Three conditions must hold: (1) Random — data come from random samples or a randomized experiment; (2) Independent — the two samples are independent of each other and each sample is ≤ 10% of its population; (3) Normal — each population is approximately normal, or each sample size is large enough (n ≥ 30) for the CLT to apply.

Test Statistic (t)

The test statistic measures how many estimated standard errors the observed difference lies from the hypothesized difference. A larger absolute value of t provides stronger evidence against H₀.

Degrees of Freedom

The Welch approximation produces a non-integer df based on both sample sizes and standard deviations. Calculators handle this automatically; on the exam, you may use the conservative df = min(n₁ − 1, n₂ − 1) when using a table.

P-Value & Conclusion

The p-value is the probability of obtaining a test statistic at least as extreme as the observed one, assuming H₀ is true. If p ≤ α, we reject H₀ and conclude there is convincing evidence of a difference in population means.

✦

KEY TAKEAWAY

Think of the two-sample t-test like a signal-to-noise ratio in engineering. The signal is the observed difference between the two sample means (x̄₁ − x̄₂). The noise is the standard error — a measure of how much the difference would vary from sample to sample. When the signal overwhelms the noise, the t-statistic is large and the p-value is small, pointing toward a real population-level difference rather than mere chance.

SECTION 3

Visual Explanation: Anatomy of the Two-Sample t-Test

The flowchart traces the four-step framework used on the AP exam: (1) state hypotheses, (2) verify conditions, (3) compute the test statistic and degrees of freedom, and (4) obtain the p-value and state a conclusion in context. Notice the decision fork at the bottom — the conclusion always depends on whether the p-value falls at or below the predetermined significance level α.

The flowchart above encapsulates the procedure tested on every AP Statistics exam free-response question involving two means. Each box corresponds to a scoring component: stating the hypotheses earns credit only when both H₀ and Hₐ are expressed using correct notation (μ₁ − μ₂ or μ₁ and μ₂ individually) and defined in context. Checking conditions earns credit when all three are named and verified with reference to the given information. The test statistic box requires the correct formula and numerical substitution, while the conclusion box demands a linkage between the p-value, α, and the real-world context of the problem.

SECTION 4

Mathematical Framework

The mathematical core of the two-sample t-test rests on a single ratio: the observed difference in sample means divided by its estimated standard error. This section unpacks each component.

TWO-SAMPLE T-STATISTIC

t = (x̄₁ − x̄₂) − (μ₁ − μ₂)₀ / √(s₁² / n₁ + s₂² / n₂)

x̄₁, x̄₂ = sample means; s₁, s₂ = sample standard deviations; n₁, n₂ = sample sizes; (μ₁ − μ₂)₀ = hypothesized difference (usually 0). Under H₀ with (μ₁ − μ₂)₀ = 0, the numerator simplifies to x̄₁ − x̄₂.

STANDARD ERROR OF THE DIFFERENCE

SE = √(s₁² / n₁ + s₂² / n₂)

Because the two samples are independent, the variances of the individual sampling distributions add. We estimate each population variance σ² with the sample variance s², then take the square root to return to the scale of the original measurements.

WELCH DEGREES OF FREEDOM

df = (s₁²/n₁ + s₂²/n₂)² / [ (s₁²/n₁)²/(n₁−1) + (s₂²/n₂)²/(n₂−1) ]

This formula yields a non-integer that is always between min(n₁−1, n₂−1) and n₁+n₂−2. Your calculator computes this automatically. On the AP exam, you are not expected to compute Welch df by hand — but you should recognize it when calculator output reports a non-integer df.

💡

AP Exam Tip

On the AP exam, you must name the procedure (two-sample t-test) and show the formula for the test statistic, or clearly reference the calculator function used (e.g., 2-SampTTest). Always report the test statistic, degrees of freedom, and p-value from your calculator output.

Connecting the formula to the logic: the numerator captures the observed departure from the null hypothesis, and the denominator captures the expected variability of that departure under repeated sampling. A t-statistic of 2.5, for example, means the observed difference is 2.5 standard errors away from zero — placing it well into the tails of the t-distribution and yielding a small p-value.

SECTION 5

Detailed Breakdown: Checking Conditions

No hypothesis test is valid unless its underlying conditions are met. For the two-sample t-test, the AP exam rubric explicitly awards points for checking each condition with reference to the data described in the problem. The table below summarizes the three conditions and how to verify each one.

Condition	What It Requires	How to Check / Justify
Random	Data come from two independent random samples, or from two treatment groups in a randomized experiment.	State how the data were collected. For surveys, cite random sampling. For experiments, cite random assignment to treatment groups.
Independent	Observations within and between groups are independent. For sampling without replacement, each sample should be ≤ 10% of its population.	Verify the 10% condition: n₁ ≤ 0.10 × N₁ and n₂ ≤ 0.10 × N₂. For experiments, independence follows from random assignment.
Normal / Large Sample	Each population distribution is approximately normal, OR each sample size is large enough for the Central Limit Theorem to apply (n ≥ 30 per group is a common guideline).	If sample sizes are small, examine dotplots, boxplots, or normal probability plots for severe skewness or outliers. If n₁ ≥ 30 and n₂ ≥ 30, cite the CLT.

The three conditions for valid two-sample t-inference

Under the null hypothesis, the sampling distribution of x̄₁ − x̄₂ is centered at 0 and follows a t-distribution with Welch degrees of freedom. The observed difference (pink dot) determines the location on this distribution, and the shaded right tail represents the p-value for a one-sided test (Hₐ: μ₁ − μ₂ > 0). For a two-sided test, the p-value would include both tails.

The diagram above illustrates the key geometric intuition: the further the observed difference falls into the tail of the null distribution, the smaller the p-value and the stronger the evidence against H₀. A two-sided test doubles the tail area (or equivalently, compares |t| to critical values in both tails), while a one-sided test uses only the tail in the direction specified by Hₐ.

SECTION 6

Worked Example

A researcher wants to know whether a new study-skills workshop improves exam performance. A random sample of 35 students who attended the workshop scored a mean of 78.2 with a standard deviation of 9.4 on a final exam. An independent random sample of 40 students who did not attend scored a mean of 73.6 with a standard deviation of 11.1. At the α = 0.05 significance level, is there convincing evidence that the workshop is associated with a higher mean exam score?

Two-Sample t-Test: Workshop vs. No Workshop

1
Step 1 — State Hypotheses

Let μ₁ = the true mean exam score for the population of students who attend the workshop, and μ₂ = the true mean exam score for the population of students who do not attend. H₀: μ₁ − μ₂ = 0 (no difference). Hₐ: μ₁ − μ₂ > 0 (workshop students score higher on average). This is a one-sided (right-tailed) test because the researcher specifically suspects the workshop improves scores.

H₀: μ₁ − μ₂ = 0; Hₐ: μ₁ − μ₂ > 0
2
Step 2 — Check Conditions

Random: The problem states that both groups are independent random samples. ✓ Independent: The two samples are independent of each other. It is reasonable to assume that 35 workshop attendees are fewer than 10% of all potential workshop attendees, and 40 non-attendees are fewer than 10% of all potential non-attendees. ✓ Normal/Large Sample: Both sample sizes exceed 30 (n₁ = 35, n₂ = 40), so the Central Limit Theorem assures us that the sampling distribution of x̄₁ − x̄₂ is approximately normal. ✓

All three conditions are satisfied.
3
Step 3 — Compute the Test Statistic

SE = √(s₁²/n₁ + s₂²/n₂) = √(9.4²/35 + 11.1²/40) = √(88.36/35 + 123.21/40) = √(2.5246 + 3.0803) = √5.6049 ≈ 2.3675. The test statistic is t = (x̄₁ − x̄₂ − 0)/SE = (78.2 − 73.6)/2.3675 = 4.6/2.3675 ≈ 1.943.

t ≈ 1.943
4
Step 4 — Degrees of Freedom & P-Value

Using a TI-84 2-SampTTest (or the Welch formula), the calculator returns df ≈ 72.8. For a one-sided test (right tail), p-value = P(t > 1.943 | df ≈ 72.8) ≈ 0.028. Alternatively, using the conservative df = min(34, 39) = 34 with a t-table, 1.943 falls between the 0.025 and 0.05 one-tail critical values, confirming 0.025 < p < 0.05.

p ≈ 0.028, df ≈ 72.8
5
Step 5 — Conclusion in Context

Because the p-value (≈ 0.028) is less than α = 0.05, we reject H₀. There is convincing evidence that the true mean exam score for students who attend the study-skills workshop is higher than the true mean exam score for students who do not attend. However, because this is an observational study (random samples, not a randomized experiment), we cannot conclude that the workshop caused the improvement — only that a statistically significant association exists.

Reject H₀. Evidence supports μ₁ > μ₂.

SECTION 7

Common Errors & Scoring Pitfalls

On the AP Statistics exam, partial credit is awarded according to a strict rubric. Understanding the most frequent mistakes — and how they cost points — is as important as understanding the procedure itself.

Common Error	Why It's Wrong	How to Avoid It
Writing hypotheses with x̄ instead of μ	Hypotheses concern population parameters (μ), not sample statistics (x̄). Using x̄ means testing a claim about a known number, which is nonsensical.	Always define μ₁ and μ₂ in words and use them in H₀ and Hₐ.
Using a one-sample or paired procedure	The one-sample t-test handles a single mean; the paired t-test handles matched pairs. Applying either to two independent groups uses incorrect standard error and df.	Ask: are the two groups independent of each other, or is each observation in group 1 matched to one in group 2?
Failing to check conditions	Rubrics consistently award a dedicated point for condition verification. Skipping this step forfeits an easy point.	Always address Random, Independent, and Normal/Large Sample explicitly, citing information from the problem.
Saying "accept H₀" instead of "fail to reject H₀"	A hypothesis test can only reject or fail to reject H₀. Failing to find evidence against H₀ does not prove it true.	Use the phrasing: "We do not have convincing evidence that…" when p > α.
Claiming causation from an observational study	Only randomized experiments allow causal conclusions. If the problem describes random sampling (not random assignment), the conclusion must be limited to association.	Read the problem carefully: does it say random assignment or random selection? Tailor your language accordingly.

Frequent mistakes and how to avoid them on the AP exam

✦

KEY TAKEAWAY

Think of the four-step process as a legal argument: your hypotheses are the charges, conditions establish that the court has jurisdiction, the test statistic and p-value present the evidence, and the conclusion is the verdict — rendered in the context of the original question, never in abstract statistical language alone.

SECTION 8

Connection to Confidence Intervals & Advanced Methods

The two-sample t-test and the two-sample t-confidence interval are two sides of the same coin. The test asks whether zero is a plausible value for μ₁ − μ₂, while the confidence interval provides a range of plausible values. On the AP exam, you may be asked to use either approach — and results should be consistent: if you reject H₀ at α = 0.05 in a two-sided test, then the corresponding 95% confidence interval for μ₁ − μ₂ should not contain zero.

Feature	Two-Sample t-Test	Two-Sample t-Interval
Question answered	Is there convincing evidence that μ₁ ≠ μ₂ (or μ₁ > μ₂, etc.)?	What range of values is plausible for μ₁ − μ₂?
Output	Test statistic (t), degrees of freedom, p-value	Interval: (x̄₁ − x̄₂) ± t* × SE
Decision rule	Reject H₀ if p ≤ α	Conclude significance if 0 is not in the interval
When to prefer	When the problem asks "Is there evidence that...?" or asks for a significance test	When the problem asks "Estimate the difference" or asks for a confidence interval

Comparing the two-sample t-test with the two-sample t-interval

Looking beyond the AP curriculum, the two-sample t-test generalizes to analysis of variance (ANOVA) when comparing more than two groups, and to multiple regression when controlling for additional variables. The core logic, however, remains the same: compare the observed effect size to the variability expected under a null model. Mastering the two-sample case builds the conceptual foundation for every more advanced comparison you will encounter in further statistics coursework.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A student conducts a two-sample t-test and obtains a p-value of 0.03 with α = 0.05. Which of the following is the most appropriate conclusion?

A. We accept the alternative hypothesis and prove that the two population means are different.
B. We reject H₀. There is convincing evidence that the two population means are different.
C. We fail to reject H₀ because 0.03 is a small probability, meaning the difference likely occurred by chance.
D. We accept H₀ because 0.03 is close to zero.
E. No conclusion can be drawn without knowing the exact value of the test statistic.

Show answer (B)

The correct answer is B. Since p = 0.03 < α = 0.05, we reject H₀ and conclude there is convincing evidence that the population means differ. Choice A is wrong because we never "prove" a hypothesis — we only conclude that there is convincing evidence; we also never say we "accept Hₐ." Choice C misinterprets the p-value; a small p-value is evidence against H₀, not for it. Choice D is wrong because we never "accept H₀," and the small p-value actually works against H₀. Choice E is incorrect because the p-value alone, compared to α, is sufficient to reach a decision about the hypotheses.

Question 2: BASIC CALCULATION

Two independent random samples yield: x̄₁ = 52, s₁ = 6, n₁ = 25, and x̄₂ = 48, s₂ = 8, n₂ = 30. Assuming H₀: μ₁ − μ₂ = 0, what is the value of the two-sample t-statistic?

A. t ≈ 2.13
B. t ≈ 2.18
C. t ≈ 1.83
D. t ≈ 2.50
E. t ≈ 1.60

Show answer (A)

The correct answer is A. SE = √(6²/25 + 8²/30) = √(36/25 + 64/30) = √(1.44 + 2.1333) = √3.5733 ≈ 1.8903. Then t = (52 − 48)/1.8903 = 4/1.8903 ≈ 2.116, which rounds to approximately 2.12. Among the choices, 2.13 is the closest. Choice B (2.18) overshoots the true value due to excessive intermediate rounding. Choice C (1.83) and E (1.60) result from arithmetic errors such as forgetting to square the standard deviations before dividing by sample sizes. Choice D (2.50) incorrectly uses too small a denominator.

Question 3: INTERMEDIATE

A researcher conducts a two-sample t-test and obtains t = −2.47 with df = 58 and a two-sided alternative Hₐ: μ₁ − μ₂ ≠ 0. Using a t-table, the researcher finds that the critical value for a two-tailed test at α = 0.05 with 50 df is t* = 2.009 and with 60 df is t* = 2.000. Which of the following is the most appropriate conclusion?

A. Fail to reject H₀ because the test statistic is negative.
B. Reject H₀ because |t| = 2.47 exceeds the critical value of approximately 2.00.
C. Fail to reject H₀ because df = 58 is not in the table.
D. Reject H₀ only if the test is one-sided; the two-sided test is inconclusive.
E. The test cannot be completed because the Welch df was not reported as a whole number.

Show answer (B)

The correct answer is B. For a two-sided test, we compare |t| to the critical value. Since |−2.47| = 2.47, and the critical value at df = 58 lies between 2.009 (df = 50) and 2.000 (df = 60) — approximately 2.00 — we have 2.47 > 2.00, so we reject H₀. Choice A is wrong because the sign of t does not determine significance — we use the absolute value in a two-sided test. Choice C is wrong because when the exact df is not in the table, we use the nearest smaller df (conservative approach) or interpolate; either way, the conclusion is clear. Choice D incorrectly suggests that a two-sided test requires a different approach — it simply uses both tails. Choice E is wrong because non-integer df is standard with the Welch method and does not prevent the test from being carried out.

Question 4: APPLIED

4 pts

An agricultural scientist randomly assigns 20 plots to a new fertilizer and 20 plots to a standard fertilizer. After the growing season, the new-fertilizer plots have a mean yield of 142.3 bushels/acre (s = 12.6) and the standard-fertilizer plots have a mean yield of 134.7 bushels/acre (s = 14.1). Carry out a complete significance test at α = 0.05 to determine whether the new fertilizer produces a higher mean yield. Include all four steps: hypotheses (with parameter definitions), conditions, test statistic and p-value, and a conclusion in context. If applicable, address whether a causal conclusion is warranted.

Show answer

Point 1 — Hypotheses: Let μ₁ = the true mean yield (bushels/acre) for plots treated with the new fertilizer, and μ₂ = the true mean yield for plots treated with the standard fertilizer. H₀: μ₁ − μ₂ = 0. Hₐ: μ₁ − μ₂ > 0. (One point for correctly stated hypotheses with defined parameters.) Point 2 — Conditions: Random: Plots were randomly assigned to treatment groups. ✓ Independent: The two groups are independent because of random assignment, and each group of 20 is independent within itself (no repeated measures). ✓ Normal/Large Sample: With n₁ = n₂ = 20 (each below 30), we need to assume the population distributions of yield are approximately normal, or verify from dotplots/boxplots that there are no strong skewness or outliers. Given no information to the contrary, we proceed with caution. ✓ (One point for naming and verifying all three conditions.) Point 3 — Test statistic and p-value: SE = √(12.6²/20 + 14.1²/20) = √(158.76/20 + 198.81/20) = √(7.938 + 9.9405) = √17.8785 ≈ 4.229. t = (142.3 − 134.7)/4.229 = 7.6/4.229 ≈ 1.797. Using a calculator with Welch df ≈ 37.4, the one-sided p-value is P(t > 1.797) ≈ 0.040. (One point for correct test statistic, df, and p-value.) Point 4 — Conclusion: Because p ≈ 0.040 < α = 0.05, we reject H₀. There is convincing evidence that the true mean yield for plots treated with the new fertilizer is higher than the true mean yield for plots treated with the standard fertilizer. Because the study used random assignment to treatments, a causal conclusion is appropriate: the new fertilizer causes an increase in mean yield compared to the standard fertilizer. (One point for a correct decision linked to the p-value and α, stated in context, with an appropriate causal claim.)

Question 5: CRITICAL THINKING

5 pts

A health researcher compares systolic blood pressure between a treatment group (n₁ = 15, x̄₁ = 128, s₁ = 18) and a control group (n₂ = 15, x̄₂ = 136, s₂ = 10). She performs a two-sample t-test and obtains t ≈ −1.52 with p ≈ 0.072 (one-sided, Hₐ: μ₁ < μ₂) at α = 0.05. (a) State and justify the conclusion of the test. (b) The researcher is concerned that the small sample sizes and unequal standard deviations may have affected the result. Explain how each of these concerns relates to the validity and power of the test. (c) If the researcher could redesign the study with larger samples (n₁ = n₂ = 50) but the same sample means and standard deviations, explain how the test statistic, p-value, and conclusion would likely change and why. (d) A colleague suggests that because the blood pressure data may be skewed, a nonparametric test such as the Wilcoxon rank-sum test should be used instead. Under what conditions would you agree with this suggestion?

Show answer

Point 1 — Part (a): Because p ≈ 0.072 > α = 0.05, we fail to reject H₀. There is not convincing evidence that the true mean systolic blood pressure for the treatment group is lower than the true mean systolic blood pressure for the control group. (One point for correct decision, linkage to p and α, and conclusion in context.) Point 2 — Part (b), validity concern: With n₁ = n₂ = 15, the Normal/Large Sample condition requires that each population distribution is approximately normal. If blood pressure distributions are substantially skewed in either population, the t-distribution may not be a good model for the test statistic, which could lead to inaccurate p-values. The unequal standard deviations (18 vs. 10) make the Welch approximation especially important — it adjusts the degrees of freedom to account for this inequality. The pooled t-test (which assumes σ₁ = σ₂) would be inappropriate here and could yield misleading results. (One point for explaining how small samples affect the normality condition and how unequal SDs affect df and the choice between Welch and pooled procedures.) Point 3 — Part (b), power concern: Power is the probability of correctly rejecting H₀ when Hₐ is true. Small sample sizes result in a larger standard error, which produces a smaller |t| for any given true difference, making it harder to detect real effects. With n = 15 per group, the test may simply lack the statistical power to detect a clinically meaningful difference. This means the failure to reject H₀ should not be interpreted as evidence that the treatment has no effect — only that the study was unable to detect one. (One point for correctly explaining the relationship between sample size, standard error, and power.) Point 4 — Part (c): If n₁ = n₂ = 50 with the same x̄ and s values, SE would decrease substantially: SE = √(18²/50 + 10²/50) = √(6.48 + 2.00) = √8.48 ≈ 2.91, compared to the original SE ≈ √(18²/15 + 10²/15) ≈ 5.27. The test statistic would become t = (128 − 136)/2.91 ≈ −2.75, which is much more extreme. The p-value would decrease well below 0.05, and the researcher would reject H₀, concluding that there is convincing evidence of a lower mean blood pressure in the treatment group. The larger sample size also increases the degrees of freedom, making the t-distribution closer to standard normal and slightly reducing the p-value further. (One point for correct quantitative reasoning about how larger n changes SE, t, and the p-value.) Point 5 — Part (d): The Wilcoxon rank-sum test (Mann-Whitney U test) does not assume normality because it is based on ranks rather than raw values. If the blood pressure distributions are strongly skewed or have heavy tails — especially with such small samples (n = 15 per group), where the CLT does not provide adequate protection — a nonparametric test would be more appropriate because it does not rely on the distributional assumption that the t-test requires. However, if the data appear roughly symmetric with no extreme outliers, the t-test is preferred because it is more powerful (more likely to detect a real difference) when its assumptions are approximately met. You would agree with the colleague if examination of the sample data (via dotplots, boxplots, or normal probability plots) revealed substantial departures from normality. (One point for correctly identifying when the nonparametric alternative is preferable and why the t-test retains an advantage when its conditions are met.)

SUMMARY

Summary

The two-sample t-test determines whether an observed difference in sample means provides convincing evidence of a difference in population means. The procedure follows four steps: (1) state null and alternative hypotheses using population parameters μ₁ and μ₂; (2) verify the Random, Independent, and Normal/Large Sample conditions; (3) compute the test statistic t = (x̄₁ − x̄₂) / √(s₁²/n₁ + s₂²/n₂) using Welch degrees of freedom; and (4) obtain the p-value and state a conclusion in context. Reject H₀ when p ≤ α, and always link the decision back to the real-world scenario described in the problem.

Remember that a causal conclusion is only justified when the data come from a randomized experiment; observational studies support only claims of association. The two-sample t-test and the two-sample t-confidence interval are complementary tools: one tests a specific hypothesis, while the other estimates the magnitude of the difference. Together, they form the backbone of inference for comparing two quantitative populations.