CI for Difference of Proportions — Confidence Interval for...

SECTION 1

Historical Context & Motivation

The need to compare proportions across two groups is one of the oldest inferential problems in statistics. From early medical trials asking whether a new treatment outperforms an existing one, to sociological surveys comparing attitudes between demographic groups, researchers have long required a rigorous way to quantify how different two population proportions really are — and how much uncertainty surrounds that estimate. The confidence interval for the difference of proportions provides exactly that framework, delivering a range of plausible values for p₁ − p₂ rather than a single point estimate.

1710

Arbuthnot's Two-Sample Comparison

John Arbuthnot published one of the earliest statistical comparisons of proportions, examining birth ratios of boys and girls in London parish records over 82 years — a pioneering two-sample analysis.

1900

Pearson's χ² Test

Karl Pearson introduced the chi-squared goodness-of-fit test, providing a formal framework for testing differences between observed and expected proportions in categorical data.

1934

Neyman's Confidence Interval Theory

Jerzy Neyman formalized the concept of confidence intervals as repeated-sampling guarantees, establishing the theoretical foundation that underpins all modern interval estimation — including intervals for differences of proportions.

1998

Agresti–Caffo Adjusted Interval

Alan Agresti and Brian Caffo proposed adding pseudo-observations to improve small-sample coverage of the Wald interval for the difference of two proportions, sparking renewed interest in interval refinements.

The central question this concept addresses is both simple and profound: given two independent random samples from two populations with unknown success probabilities p₁ and p₂, how can we construct an interval that captures the true difference p₁ − p₂ with a specified level of confidence? The answer draws on the Central Limit Theorem, properties of independent random variables, and the normal approximation to the binomial distribution.

SECTION 2

Core Principles & Definitions

Before constructing the interval, it is essential to understand the foundational ideas that justify it. The confidence interval for the difference of proportions rests on the independence of the two samples, the normal approximation enabled by sufficiently large sample sizes, and the additive property of variances for independent random variables. Each of these principles contributes a distinct element to the formula and its interpretation.

Independent Random Samples

The two samples must be drawn independently from their respective populations. Observations in one sample must not influence or be paired with observations in the other. This independence is what allows us to add variances when computing the standard error of the difference.

Normal Approximation

By the Central Limit Theorem, each sample proportion p̂ is approximately normal for large n. Since a linear combination of independent normal random variables is also normal, p̂₁ − p̂₂ is approximately normally distributed.

Pooling Is Not Used Here

Unlike the two-proportion z-test (which pools under H₀: p₁ = p₂), the confidence interval does not assume the two proportions are equal. Each sample proportion contributes its own variance estimate to the standard error.

Conditions for Validity

The success–failure condition requires n₁p̂₁ ≥ 10, n₁(1 − p̂₁) ≥ 10, n₂p̂₂ ≥ 10, and n₂(1 − p̂₂) ≥ 10. Additionally, each sample should be no more than 10% of its population to ensure approximate independence within each group.

Interpretation of the Interval

A 95% confidence interval means: if we were to repeat the sampling process many times and build an interval each time, approximately 95% of those intervals would contain the true difference p₁ − p₂. It does not mean there is a 95% probability that the particular interval we computed contains the parameter.

✦ KEY TAKEAWAY

Think of each sample proportion as a measurement taken with a slightly fuzzy instrument. When you subtract two fuzzy measurements, the fuzziness compounds — the uncertainty in the difference is always larger than the uncertainty in either individual estimate alone. This is precisely why the standard error of p̂₁ − p̂₂ combines the individual variances by addition, not subtraction. It is analogous to two independent measurement errors in a physics experiment: you add the squared uncertainties (variances) and then take the square root.

SECTION 3

Visual Explanation

The diagram below illustrates the conceptual architecture of the confidence interval for the difference of two proportions. Two independent sampling distributions — one for p̂₁ and one for p̂₂ — combine into a single sampling distribution of the difference p̂₁ − p̂₂, which is centered at the true difference p₁ − p₂. The confidence interval captures the middle (1 − α) × 100% of this distribution.

The two individual sampling distributions (violet for p̂₁, cyan for p̂₂) each have their own standard errors. When we form the difference p̂₁ − p̂₂, the variances add, producing the wider green distribution at the bottom. The gold dashed lines mark the boundaries of the 95% confidence interval.

Notice that the distribution of the difference is wider than either individual sampling distribution. This reflects the fundamental statistical reality that comparing two estimates always introduces more uncertainty than estimating a single parameter. The confidence interval captures the plausible range for the true difference p₁ − p₂ by extending z* standard errors in both directions from the observed difference p̂₁ − p̂₂. If the interval contains zero, we lack evidence that the two population proportions differ at the chosen confidence level.

SECTION 4

Mathematical Framework

The derivation proceeds from the sampling distribution of p̂₁ − p̂₂. Let X₁ ~ Binomial(n₁, p₁) and X₂ ~ Binomial(n₂, p₂) be independent, with sample proportions p̂₁ = X₁/n₁ and p̂₂ = X₂/n₂. The expected value and variance of the difference follow directly from the linearity of expectation and the additive property of variances for independent variables.

POINT ESTIMATE

p̂₁ − p̂₂

where p̂₁ = x₁/n₁ is the sample proportion of successes in sample 1, and p̂₂ = x₂/n₂ is the sample proportion in sample 2.

STANDARD ERROR OF THE DIFFERENCE

SE = √( p̂₁(1 − p̂₁)/n₁ + p̂₂(1 − p̂₂)/n₂ )

This is the unpooled standard error. Because we are not assuming p₁ = p₂ (as a hypothesis test under H₀ would), each proportion's variance is estimated separately.

CONFIDENCE INTERVAL FORMULA

(p̂₁ − p̂₂) ± z* · √( p̂₁(1 − p̂₁)/n₁ + p̂₂(1 − p̂₂)/n₂ )

z* is the critical value from the standard normal distribution corresponding to the desired confidence level (e.g., z* = 1.96 for 95%, z* = 2.576 for 99%, z* = 1.645 for 90%).

MARGIN OF ERROR

E = z* · √( p̂₁(1 − p̂₁)/n₁ + p̂₂(1 − p̂₂)/n₂ )

The margin of error E controls the half-width of the interval. Larger samples, proportions closer to 0 or 1, or lower confidence levels all decrease E.

💡 Why Variances Add

For independent random variables, Var(A − B) = Var(A) + Var(B). The minus sign in the difference does not become a minus sign for variances because variance measures squared deviations from the mean, and squaring eliminates the sign. This is why the standard error of the difference is always larger than either individual standard error.

SECTION 5

Conditions & Detailed Breakdown

The validity of the normal-approximation confidence interval depends on several conditions being satisfied. When these conditions fail, the actual coverage probability can deviate substantially from the nominal level. The diagram below organizes the required checks into a decision flowchart, and the subsequent table summarizes common rules of thumb and what to do when conditions are violated.

A step-by-step decision flowchart for verifying the conditions of the two-proportion z-interval. If any condition fails, the diagram indicates the appropriate remedial approach on the right side.

Summary of conditions and remedies for the two-proportion z-interval

Condition	Rule of Thumb	If Violated
Independence	Two separate populations; no matched pairs	Use McNemar's test or paired confidence interval for dependent proportions
Randomization	Simple random sample or random assignment to groups	Results may not generalize; interpret with caution
10% Condition	n₁ ≤ 0.10 × N₁ and n₂ ≤ 0.10 × N₂	Apply finite population correction factor √((N − n)/(N − 1)) to each SE
Success–Failure	All four counts (n₁p̂₁, n₁q̂₁, n₂p̂₂, n₂q̂₂) ≥ 10	Use Agresti–Caffo adjustment, Wilson score method, or exact Clopper–Pearson intervals

SECTION 6

Worked Example

A university health center wants to compare the flu vaccination rates among undergraduate and graduate students. A random sample of 400 undergraduates reveals that 140 were vaccinated, while a separate random sample of 250 graduate students reveals that 120 were vaccinated. Construct a 95% confidence interval for the difference in vaccination proportions (graduate − undergraduate).

95% CI for Difference in Flu Vaccination Rates

Step 1 — Identify Given ValuesLet population 1 = graduate students and population 2 = undergraduates. We have n₁ = 250, x₁ = 120, n₂ = 400, x₂ = 140. The sample proportions are p̂₁ = 120/250 = 0.48 and p̂₂ = 140/400 = 0.35.

p̂₁ = 0.48, p̂₂ = 0.35

Step 2 — Check ConditionsIndependence: the two samples are drawn from separate populations with no overlap. Randomization: both are random samples. 10% condition: university populations are well over 2,500 and 4,000 respectively. Success–failure: n₁p̂₁ = 120 ≥ 10, n₁(1 − p̂₁) = 130 ≥ 10, n₂p̂₂ = 140 ≥ 10, n₂(1 − p̂₂) = 260 ≥ 10. All conditions are satisfied.

All conditions met ✓

Step 3 — Compute the Point EstimateThe point estimate of the difference is p̂₁ − p̂₂ = 0.48 − 0.35 = 0.13. This suggests the graduate vaccination rate is 13 percentage points higher than the undergraduate rate in our samples.

p̂₁ − p̂₂ = 0.13

Step 4 — Calculate the Standard ErrorSE = √( p̂₁(1 − p̂₁)/n₁ + p̂₂(1 − p̂₂)/n₂ ) = √( 0.48 × 0.52 / 250 + 0.35 × 0.65 / 400 ) = √( 0.2496/250 + 0.2275/400 ) = √( 0.0009984 + 0.00056875 ) = √0.00156715 ≈ 0.03959.

SE ≈ 0.0396

Step 5 — Find the Critical Value and Margin of ErrorFor a 95% confidence level, z* = 1.96. The margin of error is E = 1.96 × 0.0396 ≈ 0.0776.

E ≈ 0.0776

Step 6 — Construct the Interval and InterpretCI = (p̂₁ − p̂₂) ± E = 0.13 ± 0.0776, giving the interval (0.0524, 0.2076). We are 95% confident that the true difference in vaccination rates (graduate minus undergraduate) lies between approximately 0.052 and 0.208. Because the entire interval is positive, we have evidence at the 95% level that the graduate vaccination rate exceeds the undergraduate rate.

95% CI: (0.052, 0.208)

SECTION 7

Strengths, Limitations & Comparisons

The Wald interval for the difference of proportions is the most commonly taught and applied method, but it is not without trade-offs. Understanding its strengths and weaknesses helps you decide when to use it confidently and when to consider alternatives.

Trade-offs of the Wald confidence interval for difference of proportions

Strengths	Limitations
Simple, closed-form formula that requires only basic arithmetic and a z-table	Coverage can be poor (below nominal) when sample sizes are small or proportions are near 0 or 1
Intuitive interpretation: point estimate ± margin of error	Interval can extend below −1 or above +1, which are impossible values for a difference of proportions
Performs well for large samples with moderate proportions (away from 0 and 1)	Relies on the normal approximation; the success–failure condition can be restrictive
Directly connects to hypothesis testing via duality of tests and CIs	Does not incorporate continuity correction; the Newcombe hybrid score interval often has better coverage

⚖️ PRACTICAL GUIDANCE

In practice, the Wald interval is perfectly adequate for large-sample, moderate-proportion scenarios — which covers the majority of public health, marketing, and social science applications. However, if your sample sizes are below roughly 30 per group, or if either proportion is below 0.05 or above 0.95, consider the Agresti–Caffo adjustment (add 1 success and 1 failure to each sample) or the Newcombe score interval for improved coverage.

SECTION 8

Connection to Advanced Theory

The two-proportion z-interval is a gateway to several more advanced inferential methods. Understanding its structure prepares you for logistic regression (where the difference of proportions becomes a log-odds ratio), stratified analyses (Mantel–Haenszel methods), and Bayesian approaches to comparing binomial parameters. The table below highlights key connections between the basic CI and its more sophisticated relatives.

From the basic two-proportion CI to advanced inferential methods

Feature	Wald CI (This Lesson)	Advanced Extension
Effect measure	p₁ − p₂ (absolute risk difference)	Relative risk (p₁/p₂) or odds ratio — often via log transformation for CI construction
Number of groups	Exactly two independent groups	Chi-squared test or logistic regression for k ≥ 2 groups
Confounders	Not accounted for (crude comparison)	Stratified analysis (Mantel–Haenszel) or multiple logistic regression for adjusted comparisons
Inference paradigm	Frequentist (repeated-sampling guarantee)	Bayesian credible interval using Beta priors on p₁ and p₂
Small-sample behavior	Coverage can be liberal; relies on CLT approximation	Exact methods (Fisher's exact test) or profile-likelihood CIs with guaranteed coverage

One particularly important extension is the transition from absolute differences to relative measures. In epidemiology, for instance, the risk difference tells you how many additional cases per unit population occur in the exposed group, while the relative risk tells you how many times more likely the exposed group is to develop the outcome. Both are built on the same underlying binomial sampling model, and confidence intervals for relative risk and odds ratio follow analogous logic — albeit using log transformations to achieve approximate normality.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

A 95% confidence interval for p₁ − p₂ is (−0.08, 0.14). A student claims this proves the two population proportions are equal. Evaluate this claim and explain what the interval actually tells us.

PROBLEM 2 — BASIC CALCULATION

In a study, 64 out of 200 patients in Group A recovered within a week, while 90 out of 300 patients in Group B recovered within a week. Compute the 95% confidence interval for p_A − p_B.

PROBLEM 3 — INTERMEDIATE

A marketing analyst wants a 99% confidence interval for the difference in click-through rates between two ad designs. Design 1 had 215 clicks out of 1,000 impressions; Design 2 had 258 clicks out of 1,200 impressions. Construct the interval and state whether there is evidence of a difference at the 1% significance level.

PROBLEM 4 — APPLIED

A public health researcher surveys two cities about mask usage. In City X (n = 520), 338 residents report regular mask use. In City Y (n = 480), 264 report regular mask use. Construct a 90% confidence interval for the difference in mask-usage proportions (City X − City Y), verify all conditions, and interpret the result in context.

PROBLEM 5 — CRITICAL THINKING

Suppose you construct a 95% Wald interval for p₁ − p₂ and obtain (−0.02, 0.48). A colleague notes that this interval is very wide and the lower bound is close to zero. She proposes increasing the confidence level to 99% to 'make the result more convincing.' Critique this proposal. Then discuss at least two alternative strategies that would more effectively narrow the interval.

SUMMARY

Lesson Summary

The confidence interval for the difference of proportions estimates the true value of p₁ − p₂ by combining a point estimate (p̂₁ − p̂₂) with a margin of error (z* × SE). The standard error is computed as √(p̂₁q̂₁/n₁ + p̂₂q̂₂/n₂), reflecting the fact that variances of independent random variables add. Four conditions must hold: independence between samples, random sampling, the 10% condition, and the success–failure condition.

If the interval contains zero, there is insufficient evidence to conclude a difference at the chosen confidence level; if it lies entirely above or below zero, evidence of a difference exists. The Wald interval works well for large samples with moderate proportions, but for small samples or extreme proportions, the Agresti–Caffo adjustment or the Newcombe score interval provides improved coverage. This method serves as a foundation for more advanced comparative techniques such as relative risk, odds ratios, and logistic regression.