Justifying a Claim Based on a Confidence Interval for a Difference of Population Proportions - AP Statistics

SECTION 1

Historical Context & Motivation

Comparing two groups on a categorical outcome—whether a new drug reduces infection rates more than a placebo, or whether voter turnout differs between two districts—is one of the most fundamental questions in applied statistics. For centuries, researchers relied on intuition and raw percentages to draw such comparisons, but the lack of a formal framework meant that sampling variability could easily masquerade as a genuine difference. The development of confidence intervals for the difference of two proportions gave scientists and policymakers a principled tool for quantifying uncertainty and justifying claims about population-level differences.

1900s • Karl Pearson's Proportion Tests

Karl Pearson developed early chi-squared tests for categorical data, laying the groundwork for comparing proportions across groups with formal statistical procedures.

1934 • Jerzy Neyman's Confidence Framework

Neyman introduced the concept of a confidence interval, providing a repeated-sampling interpretation that allowed researchers to attach a confidence level to interval estimates rather than relying solely on hypothesis tests.

1940s • Large-Sample Normal Approximations

Statisticians formalized the use of the Central Limit Theorem to approximate the sampling distribution of p̂₁ − p̂₂, enabling straightforward construction of two-proportion z-intervals.

1998 • Agresti–Caffo Adjusted Interval

Alan Agresti and Brian Caffo proposed adding pseudo-observations to improve coverage properties of the Wald interval for small samples, refining how confidence intervals for proportion differences are constructed in practice.

The central question this lesson addresses is: once you have computed a confidence interval for the difference p₁ − p₂, how do you use it to justify—or refuse to justify—a claim about whether the two population proportions are equal, different, or ordered in a specific direction? Mastering this reasoning is essential for the AP Statistics exam, where free-response questions routinely ask you to interpret confidence intervals in context and draw defensible conclusions.

SECTION 2

Core Principles & Definitions

Before diving into the mechanics of justification, it is essential to understand the foundational ideas that make confidence-interval-based reasoning valid. These principles connect the mathematical machinery of interval estimation to the inferential logic that lets us make statements about populations from samples.

Difference of Population Proportions

The parameter of interest is p₁ − p₂, the true difference between two population proportions. We estimate it with p̂₁ − p̂₂, the difference of sample proportions.

Confidence Interval Interpretation

A C% confidence interval is a range of plausible values for p₁ − p₂. If we repeated sampling many times, approximately C% of the constructed intervals would capture the true parameter.

Zero as a Benchmark

Because p₁ − p₂ = 0 means the two population proportions are equal, the position of zero relative to the interval is the key to justifying claims about a difference.

Consistency with Hypothesis Testing

A two-sided C% confidence interval that does not contain 0 corresponds to rejecting H₀: p₁ − p₂ = 0 at the α = 1 − C significance level. Interval-based reasoning and testing are two sides of the same coin.

✦

KEY TAKEAWAY

Think of a confidence interval like a net cast around the true value of p₁ − p₂. If zero falls inside the net, you cannot rule out that the two proportions are equal—the "catch" includes equality as a plausible outcome. If zero falls outside the net, you have convincing evidence that the proportions genuinely differ, because equality is no longer among the plausible values.

SECTION 3

Visual Explanation

Three Possible Scenarios for the Confidence Interval

The diagram below illustrates the three qualitatively different outcomes when a confidence interval for p₁ − p₂ is plotted on a number line. In each case, the position of the interval relative to zero determines the conclusion you can draw.

In Scenario A, the entire interval lies above 0, providing convincing evidence that p₁ > p₂. In Scenario B, the interval straddles 0, so zero is a plausible value for p₁ − p₂ and we lack convincing evidence of a difference. In Scenario C, the entire interval lies below 0, providing convincing evidence that p₁ < p₂.

The critical reasoning pattern is straightforward: examine whether 0 falls inside or outside the interval. When the interval is entirely positive (lower bound > 0), every plausible value for p₁ − p₂ is positive, meaning p₁ is plausibly greater than p₂ at the given confidence level. Conversely, an interval that is entirely negative (upper bound < 0) supports the claim that p₁ < p₂. An interval that contains 0 means we cannot distinguish the two proportions at that confidence level—note that this does not prove they are equal, only that the data are insufficient to rule out equality.

SECTION 4

Mathematical Framework

The confidence interval for the difference of two population proportions uses the normal approximation to the sampling distribution of p̂₁ − p̂₂. Understanding each component of the formula clarifies where the interval's boundaries come from and why its width is governed by sample sizes and variability.

POINT ESTIMATE

p̂₁ − p̂₂

where p̂₁ = x₁ / n₁ and p̂₂ = x₂ / n₂ are the sample proportions, with x₁ and x₂ representing the number of successes in each sample.

STANDARD ERROR

SE = √[ p̂₁(1 − p̂₁) / n₁ + p̂₂(1 − p̂₂) / n₂ ]

This formula uses each sample proportion in its own variance term because we are not assuming p₁ = p₂ (no pooling). This is the appropriate standard error for a confidence interval, as opposed to the pooled SE used in a significance test under H₀.

CONFIDENCE INTERVAL

(p̂₁ − p̂₂) ± z* × SE

z* is the critical value from the standard normal distribution corresponding to the desired confidence level C. For 95% confidence, z* ≈ 1.960; for 90%, z* ≈ 1.645; for 99%, z* ≈ 2.576.

Conditions for Validity

✓ Random: Both samples must be randomly selected (or randomly assigned in an experiment).
✓ Independence (10% condition): When sampling without replacement, each sample must be less than 10% of its respective population: n₁ < 0.10 N₁ and n₂ < 0.10 N₂.
✓ Large Counts (Normal condition): n₁p̂₁ ≥ 10, n₁(1 − p̂₁) ≥ 10, n₂p̂₂ ≥ 10, and n₂(1 − p̂₂) ≥ 10, ensuring the sampling distribution of p̂₁ − p̂₂ is approximately normal.

⚠

Pooled vs. Unpooled SE

A common source of error is using the pooled standard error (which combines the two samples under the null hypothesis assumption p₁ = p₂) when constructing a confidence interval. The pooled SE is reserved for significance tests of H₀: p₁ = p₂. For confidence intervals, always use the unpooled (separate-sample) SE shown above.

SECTION 5

Decision Logic for Justifying Claims

Once you have constructed the confidence interval, justifying a claim requires a structured argument. The AP Statistics exam expects you to connect the interval's contents to a conclusion stated in context. Below is a decision flowchart followed by a detailed breakdown of each pathway.

This flowchart summarizes the decision logic. After computing the confidence interval, check whether 0 is contained. If not, the sign of the interval tells you the direction of the difference. Always state your conclusion in context.

Interval Location	Contains 0?	Justified Conclusion
Entirely positive (L > 0)	No	We are C% confident that p₁ − p₂ > 0, i.e., p₁ is greater than p₂. There is convincing evidence of a difference.
Entirely negative (U < 0)	No	We are C% confident that p₁ − p₂ < 0, i.e., p₂ is greater than p₁. There is convincing evidence of a difference.
Spans zero (L < 0 < U)	Yes	Because the interval contains 0, there is not convincing evidence that p₁ ≠ p₂ at the C% confidence level.

Summary of conclusions based on interval position relative to zero

💡

Common Language Pitfall

When the interval contains 0, do not write "we accept that p₁ = p₂" or "we proved there is no difference." The correct phrasing is: "The data do not provide convincing evidence of a difference." A confidence interval that contains 0 does not prove equality—it merely fails to rule it out.

SECTION 6

Worked Example

A university researcher wants to know whether the proportion of students who prefer online classes differs between freshmen and seniors. In a random sample of 200 freshmen, 118 prefer online classes. In an independent random sample of 150 seniors, 72 prefer online classes. Construct a 95% confidence interval for p₁ − p₂ (freshmen minus seniors) and use it to justify whether there is convincing evidence of a difference.

Two-Proportion z-Interval: Online Class Preference

1
Step 1 — State the Parameter

Let p₁ = the true proportion of all freshmen who prefer online classes and p₂ = the true proportion of all seniors who prefer online classes. The parameter of interest is p₁ − p₂.
2
Step 2 — Check Conditions

Random: Both samples are stated to be random. Independence (10%): It is reasonable that 200 < 10% of all freshmen and 150 < 10% of all seniors at the university. Large Counts: n₁p̂₁ = 118 ≥ 10, n₁(1−p̂₁) = 82 ≥ 10, n₂p̂₂ = 72 ≥ 10, n₂(1−p̂₂) = 78 ≥ 10. All conditions are satisfied.
3
Step 3 — Compute Sample Proportions

p̂₁ = 118/200 = 0.590 and p̂₂ = 72/150 = 0.480.

p̂₁ − p̂₂ = 0.590 − 0.480 = 0.110
4
Step 4 — Compute the Standard Error

SE = √[ (0.590)(0.410)/200 + (0.480)(0.520)/150 ] = √[ 0.0012095 + 0.001664 ] = √0.0028735 ≈ 0.05361.

SE ≈ 0.0536
5
Step 5 — Construct the Interval

For 95% confidence, z* = 1.960. The interval is 0.110 ± 1.960 × 0.0536 = 0.110 ± 0.1051, giving (0.0049, 0.2151).

95% CI: (0.005, 0.215)
6
Step 6 — Justify the Claim

Because the entire 95% confidence interval lies above 0 (the lower bound is approximately 0.005, which is positive), 0 is not contained in the interval. We are 95% confident that the true difference p₁ − p₂ is between 0.005 and 0.215. Since all plausible values are positive, there is convincing evidence that the proportion of freshmen who prefer online classes is greater than the proportion of seniors who prefer online classes.

📝

Interpreting Practical Significance

Notice that the lower bound is very close to zero (0.005). While we have statistical evidence of a difference, the difference could be as small as half a percentage point—hardly meaningful in practice. On the AP exam, you may be asked to comment on whether the difference is practically significant, not just statistically significant.

SECTION 7

Strengths, Limitations & Common Mistakes

Strengths	Limitations
Provides a range of plausible values, giving richer information than a simple reject/fail-to-reject decision.	Relies on the large-sample normal approximation; coverage can be poor when sample sizes are small or proportions are near 0 or 1.
Directly addresses the magnitude and direction of the difference, supporting practical significance judgments.	A wide interval (small samples) may contain 0 even when a true difference exists, leading to an inconclusive result rather than evidence of no difference.
Duality with hypothesis testing: a CI that excludes 0 mirrors a significant test result at the corresponding α level.	Cannot establish causation from observational data, even when the interval excludes 0. Causation requires random assignment.
Easy to communicate to non-statisticians: 'We are 95% confident the difference is between ___ and ___.'	The confidence level applies to the procedure, not to a specific interval. Saying 'There is a 95% probability that p₁ − p₂ is in this interval' is incorrect.

Comparing strengths and limitations of CI-based reasoning for two proportions

✦

KEY TAKEAWAY

A confidence interval for p₁ − p₂ provides more nuanced information than a hypothesis test alone. Think of the hypothesis test as a binary verdict—guilty or not guilty—while the confidence interval is the full evidence report, showing the range of differences consistent with the data. On the AP exam, always use the interval to state both the direction and the plausible magnitude of the difference in your justification.

SECTION 8

Connection to Hypothesis Testing & Advanced Methods

Confidence intervals and significance tests for the difference of two proportions are deeply intertwined but differ in important ways that AP Statistics students must understand. The table below highlights the relationship and the distinctions between the two approaches, along with a look toward more advanced methods you may encounter in college-level statistics courses.

Feature	CI for p₁ − p₂	z-Test for p₁ − p₂
Purpose	Estimate the range of plausible values for p₁ − p₂	Test whether p₁ − p₂ = 0 (or some other hypothesized value)
Standard Error	Unpooled: uses p̂₁ and p̂₂ separately	Pooled: uses p̂c = (x₁ + x₂)/(n₁ + n₂) under H₀
Output	An interval (L, U)	A z-statistic and p-value
Conclusion format	"We are C% confident that p₁ − p₂ is between L and U."	"At significance level α, we reject/fail to reject H₀."
Duality	If 0 ∉ CI at level C, the test rejects H₀ at α = 1 − C	If p-value < α, then 0 is outside the corresponding CI

CI vs. Hypothesis Test for the difference of two proportions

In more advanced coursework, you will encounter methods that refine the Wald interval presented here. The Agresti–Caffo interval adds one success and one failure to each sample before computing the interval, which improves coverage probability for small samples. Bayesian methods replace the fixed-confidence interpretation with posterior probability distributions over p₁ − p₂, allowing direct probability statements about the parameter. Logistic regression extends the comparison of proportions to multiple groups and adjusts for confounding variables—an important step beyond the two-sample inference framework. However, the fundamental logic of examining whether zero is contained in the interval of plausible values remains the backbone of all these approaches.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A 95% confidence interval for p₁ − p₂ is (−0.08, 0.14). Which of the following is the best interpretation of this result?

A. There is a 95% probability that the true difference p₁ − p₂ lies between −0.08 and 0.14.
B. We are 95% confident that the true difference p₁ − p₂ is between −0.08 and 0.14, so there is convincing evidence that p₁ > p₂.
C. We are 95% confident that the true difference p₁ − p₂ is between −0.08 and 0.14; because the interval contains 0, there is not convincing evidence of a difference between p₁ and p₂.
D. Because the interval contains 0, we can conclude that p₁ = p₂.
E. We are 95% confident that the sample difference p̂₁ − p̂₂ is between −0.08 and 0.14.

Show answer (C)

C is correct. The interval (−0.08, 0.14) contains 0, which means 0 is a plausible value for p₁ − p₂, so we lack convincing evidence that the two proportions differ. A is wrong because the confidence level refers to the procedure, not the probability that a specific interval captures the parameter. B is wrong because the interval contains both negative and positive values, so we cannot claim p₁ > p₂. D is wrong because failing to find evidence of a difference is not the same as proving equality. E is wrong because the confidence interval estimates the population parameter p₁ − p₂, not the sample statistic.

Question 2: BASIC CALCULATION

In a study, 84 out of 300 teenagers in City A reported regular exercise, and 60 out of 200 teenagers in City B reported regular exercise. What is the point estimate of p₁ − p₂ (City A minus City B)?

A. −0.110
B. −0.020
C. 0.020
D. 0.110
E. 0.290

Show answer (B)

p̂₁ = 84/300 = 0.280 and p̂₂ = 60/200 = 0.300. The point estimate is p̂₁ − p̂₂ = 0.280 − 0.300 = −0.020. The correct answer is B.

Question 3: INTERMEDIATE

A 90% confidence interval for p₁ − p₂ is (0.03, 0.17). A researcher claims that the two population proportions are equal. Based on this interval, which of the following statements is most appropriate?

A. The interval supports the researcher's claim because 0.10, a reasonable guess for equality, is in the interval.
B. The interval contradicts the researcher's claim because the entire interval is positive, meaning 0 is not a plausible value for p₁ − p₂ at the 90% confidence level.
C. The interval neither supports nor contradicts the claim because a 90% confidence level is not high enough to draw conclusions.
D. The interval supports the researcher's claim because the margin of error is small.
E. The interval cannot be used to evaluate the claim because a hypothesis test is required.

Show answer (B)

B is correct. The entire interval (0.03, 0.17) lies above zero, so zero is not among the plausible values for p₁ − p₂ at the 90% confidence level. This contradicts the researcher's claim that p₁ = p₂ (which requires p₁ − p₂ = 0). A is wrong because equality corresponds to p₁ − p₂ = 0, not to any arbitrary value in the interval. C is wrong because 90% is a standard and perfectly valid confidence level for drawing conclusions. D is wrong because a small margin of error is irrelevant to whether the interval contains 0. E is wrong because confidence intervals and hypothesis tests are dual approaches—a CI that excludes 0 provides the same information as rejecting H₀ at the corresponding significance level.

Question 4: APPLIED

4 pts

A pharmaceutical company tests a new vaccine. In a randomized experiment, 420 out of 1,500 participants in the treatment group contracted a mild illness, while 510 out of 1,500 participants in the control group contracted the same illness. (a) Verify the conditions for constructing a confidence interval for p₁ − p₂ (treatment minus control). (b) Construct a 95% confidence interval for p₁ − p₂. (c) Based on your interval, is there convincing evidence that the vaccine reduces the proportion who contract the illness? Justify your answer. (d) A company executive claims the vaccine reduces the illness rate by at least 10 percentage points. Does your interval support this claim? Explain.

Show answer

Rubric (4 points): Point 1 — Conditions (1 point): Random: Participants were randomly assigned to treatment and control groups. Independence: Because this is an experiment with random assignment (not sampling from a larger population), independence is satisfied by the randomization. Large Counts: n₁p̂₁ = 420 ≥ 10, n₁(1−p̂₁) = 1080 ≥ 10, n₂p̂₂ = 510 ≥ 10, n₂(1−p̂₂) = 990 ≥ 10. All conditions met. Point 2 — Interval construction (1 point): p̂₁ = 420/1500 = 0.280, p̂₂ = 510/1500 = 0.340. Point estimate: 0.280 − 0.340 = −0.060. SE = √[(0.280)(0.720)/1500 + (0.340)(0.660)/1500] = √[0.0001344 + 0.0001496] = √0.0002840 ≈ 0.01685. ME = 1.960 × 0.01685 ≈ 0.03303. 95% CI: (−0.060 − 0.033, −0.060 + 0.033) = (−0.093, −0.027). Point 3 — Justification of claim about difference (1 point): Because the entire 95% confidence interval (−0.093, −0.027) lies below 0, zero is not a plausible value for p₁ − p₂. We are 95% confident that the true difference in illness rates (treatment minus control) is between −0.093 and −0.027. Since all plausible values are negative, there is convincing evidence that the vaccine reduces the proportion who contract the illness. Point 4 — Evaluating the executive's claim (1 point): A 10-percentage-point reduction means p₁ − p₂ = −0.10. Our interval is (−0.093, −0.027), and −0.10 is not contained in this interval (it lies below the lower bound). Therefore, the interval does not support the claim of at least a 10-percentage-point reduction; in fact, all plausible values for the difference are greater than −0.093, suggesting the reduction is less than 10 percentage points.

Question 5: CRITICAL THINKING

4 pts

Two independent surveys are conducted. Survey 1 produces a 95% confidence interval for p₁ − p₂ of (−0.02, 0.12). Survey 2, with larger sample sizes, produces a 95% confidence interval for the same parameter of (0.01, 0.09). (a) Explain why the two surveys can produce different intervals even though they estimate the same parameter. (b) A policy analyst wants to claim that p₁ > p₂. Can this claim be justified using Survey 1? Can it be justified using Survey 2? Explain each. (c) Describe a general principle about sample size, interval width, and the ability to detect a true difference. (d) Suppose you knew the true difference was p₁ − p₂ = 0.05. Explain why Survey 1 failed to provide convincing evidence while Survey 2 succeeded, connecting your explanation to the concept of statistical power.

Show answer

Rubric (4 points): Point 1 — Different intervals for same parameter (1 point): Confidence intervals are computed from sample data, and different random samples yield different sample proportions due to sampling variability. Survey 1 and Survey 2 drew different samples, producing different point estimates and different standard errors, which result in different interval endpoints even though the true parameter p₁ − p₂ is the same. Point 2 — Evaluating the claim for each survey (1 point): Survey 1: The interval (−0.02, 0.12) contains 0, so the data do not provide convincing evidence that p₁ > p₂ at the 95% confidence level. Some plausible values for the difference are negative or zero. Survey 2: The interval (0.01, 0.09) lies entirely above 0, so there is convincing evidence at the 95% confidence level that p₁ > p₂, because every plausible value of the difference is positive. Point 3 — Sample size, width, and detection (1 point): Larger sample sizes reduce the standard error (because SE is inversely related to the square root of n), which produces a narrower confidence interval. A narrower interval is more likely to exclude 0 when a true nonzero difference exists, making it easier to detect a real difference. Survey 2's larger samples produced a narrower interval that was precise enough to exclude 0. Point 4 — Connection to statistical power (1 point): Statistical power is the probability of correctly detecting a true difference (rejecting H₀ when H₀ is false). If the true difference is p₁ − p₂ = 0.05, a study with small samples has a wide interval that may contain 0 despite the true difference being positive—this is a failure to detect the difference (low power). Survey 1's wider interval (width 0.14) was too imprecise to exclude 0, while Survey 2's narrower interval (width 0.08) was precise enough to capture only positive values. Larger samples increase power by reducing the standard error, making the interval narrow enough to distinguish a true difference from zero.

SUMMARY

Lesson Summary

Summary

A confidence interval for p₁ − p₂ is constructed as (p̂₁ − p̂₂) ± z* × SE, where the standard error uses unpooled sample proportions. Before constructing the interval, verify the Random, Independence (10% rule), and Large Counts conditions. The resulting interval provides a range of plausible values for the true difference in population proportions.

To justify a claim, examine the interval's position relative to zero. If the interval is entirely positive, there is convincing evidence that p₁ > p₂. If entirely negative, there is convincing evidence that p₁ < p₂. If the interval contains 0, the data do not provide convincing evidence of a difference—but this does not prove the proportions are equal. Always state conclusions in context, reference the confidence level, and describe both the direction and plausible magnitude of the difference.