CI for Differences — Compute CI for difference in means and proportions - Biostatistics

SECTION 1

Historical Context & Motivation

The need to compare two populations — whether treatment versus control, male versus female, or exposed versus unexposed — lies at the very heart of scientific inquiry. Long before formal statistical methods existed, physicians and epidemiologists recognized that a single point estimate of a difference between groups could be misleading without some measure of precision. The development of confidence intervals for differences arose from a century-long effort to move beyond subjective judgment and toward rigorous, reproducible inference about how two populations truly differ.

1908 • Student's t-Distribution

William Sealy Gosset, publishing under the pseudonym "Student," derived the t-distribution while working at the Guinness brewery. His work provided the theoretical basis for inference about means in small samples, a critical building block for comparing two groups.

1934 • Neyman's Confidence Interval Framework

Jerzy Neyman formalized the concept of the confidence interval, defining it as a random interval that, under repeated sampling, contains the true parameter a specified proportion of the time. This framework unified interval estimation for means, proportions, and their differences.

1943 • Welch's Approximation

Bernard Welch proposed an approximate solution for comparing two means when population variances are unequal (the Behrens–Fisher problem). His adjustment to degrees of freedom became the default in modern statistical software.

1998 • APA & CONSORT Reporting Guidelines

Major scientific organizations began requiring confidence intervals alongside p-values. The CONSORT guidelines for clinical trials mandated CIs for treatment effects, cementing their role in biostatistics and evidence-based medicine.

The central question this lesson addresses is both simple and profound: when we observe a difference between two sample statistics — be it a difference in means or a difference in proportions — how do we construct an interval that, with a stated level of confidence, captures the true population-level difference? This question pervades clinical trials, epidemiological studies, and virtually every biostatistical comparison.

SECTION 2

Core Principles & Definitions

Before diving into formulas, it is essential to understand the conceptual architecture underlying confidence intervals for differences. A confidence interval provides a range of plausible values for an unknown population parameter, constructed so that if we were to repeat the sampling process many times, a stated percentage (typically 95%) of such intervals would contain the true parameter. When we extend this idea to differences between two populations, the parameter of interest becomes μ₁ − μ₂ (for means) or p₁ − p₂ (for proportions), and the standard error must account for variability in both samples.

Point Estimate of the Difference

The observed difference between two sample statistics (x̄₁ − x̄₂ or p̂₁ − p̂₂) serves as the best single estimate of the true population difference. It forms the center of the confidence interval.

Standard Error of the Difference

Because two independent samples each contribute variability, the standard error of the difference combines the individual standard errors using the sum-of-variances rule: SE(diff) = √(SE₁² + SE₂²).

Critical Value (z* or t*)

The critical value depends on the desired confidence level and the sampling distribution. Use z* when sample sizes are large or population variances are known; use t* when estimating variances from small samples.

Independence Assumption

The formulas presented here assume the two samples are independently drawn. Paired or matched designs require a different approach (CI for paired differences), which modifies the standard error calculation.

Interpreting the Interval

If the interval for μ₁ − μ₂ excludes zero, we have evidence at the given confidence level that the two populations differ. If zero is inside the interval, the data are consistent with no difference.

✦

KEY TAKEAWAY

Think of constructing a CI for a difference like measuring the gap between two moving targets with a ruler that itself has some wobble. Each target (sample mean or proportion) has its own measurement uncertainty, and the wobble in the gap measurement reflects both sources of variability combined. The confidence interval tells you the range of gap sizes that are consistent with your wobbly measurements at a chosen confidence level.

SECTION 3

Visualizing the Confidence Interval for a Difference

The diagram below illustrates the conceptual process of constructing a confidence interval for the difference in two means. Two independent sampling distributions, one for each group mean, are shown at the top. Their difference yields a single sampling distribution of x̄₁ − x̄₂, centered at the observed point estimate. The confidence interval is the shaded region spanning the critical values around that center.

The two sampling distributions at the top each represent uncertainty in their respective group means. Their difference produces a single distribution (center), and the green band marks the 95% confidence interval centered on the observed point estimate x̄₁ − x̄₂.

Notice that the width of the confidence interval depends on three factors: the variability within each group (which inflates the standard error), the sample sizes (larger samples narrow the interval), and the chosen confidence level (higher confidence demands a wider interval). If the green band spans zero, the data are consistent with the possibility that the two population means are equal.

SECTION 4

Mathematical Framework

CI for the Difference in Two Means (Independent Samples)

When two samples are drawn independently from populations with means μ₁ and μ₂, the general form of the confidence interval for the difference μ₁ − μ₂ follows the familiar point-estimate-plus-or-minus-margin structure. The choice of critical value (z* versus t*) and the formula for the standard error depend on whether the population standard deviations are known and whether sample sizes are large.

CI FOR DIFFERENCE IN MEANS (σ KNOWN OR LARGE n)

(x̄₁ − x̄₂) ± z* × √(σ₁²/n₁ + σ₂²/n₂)

x̄₁, x̄₂ = sample means; σ₁², σ₂² = population variances (or use s₁², s₂² when n is large); n₁, n₂ = sample sizes; z* = critical value from the standard normal distribution (e.g., 1.96 for 95% CI).

CI FOR DIFFERENCE IN MEANS (σ UNKNOWN, SMALL n — WELCH'S t)

(x̄₁ − x̄₂) ± t*_df × √(s₁²/n₁ + s₂²/n₂)

s₁², s₂² = sample variances; t*_df = critical value from the t-distribution with Welch–Satterthwaite degrees of freedom: df = (s₁²/n₁ + s₂²/n₂)² / [(s₁²/n₁)²/(n₁−1) + (s₂²/n₂)²/(n₂−1)]. This approximation handles unequal variances and unequal sample sizes.

CI for the Difference in Two Proportions

When the outcome is binary (e.g., disease present versus absent), each group's outcome is summarized by a sample proportion p̂. The confidence interval for p₁ − p₂ relies on the normal approximation to the binomial, which is valid when the sample sizes are large enough that each group has at least five successes and five failures.

CI FOR DIFFERENCE IN PROPORTIONS (WALD METHOD)

(p̂₁ − p̂₂) ± z* × √[p̂₁(1 − p̂₁)/n₁ + p̂₂(1 − p̂₂)/n₂]

p̂₁ = x₁/n₁, p̂₂ = x₂/n₂ where x₁ and x₂ are the number of successes in each group. Note: unlike a hypothesis test (which pools proportions under H₀), the CI uses each group's own estimated proportion because the interval does not assume equality.

⚠

Pooled vs. Unpooled SE

A common source of confusion: when performing a hypothesis test for H₀: p₁ = p₂, you pool the proportions to estimate one common p. When constructing a confidence interval, you do not assume p₁ = p₂, so each group's proportion is used separately in the standard error. Always verify which context you are in.

SECTION 5

Assumptions, Conditions, & Decision Guide

Choosing the correct formula and assessing whether the interval is valid requires checking a set of conditions. The diagram below provides a decision flowchart that maps your data characteristics — type of outcome, sample size, and knowledge of population variance — to the appropriate confidence interval method.

This flowchart guides you from the type of outcome (continuous or binary) through sample-size and variance-knowledge checks to the appropriate CI formula. Always verify the independence and sampling conditions listed at the bottom.

Condition	Difference in Means	Difference in Proportions
Independence	Samples drawn independently from each population; random assignment or random selection.	Same requirement; each individual belongs to exactly one group.
Normality / Large-Sample	Each population is approximately normal, OR each n ≥ 30 (CLT). Check with histograms or Q-Q plots.	n₁p̂₁ ≥ 5, n₁(1−p̂₁) ≥ 5, and same for group 2 (normal approximation to binomial).
Equal Variances?	If assumed equal → pooled t-test; if not → Welch's t. Default to Welch's unless strong evidence of equality.	Not applicable (each proportion has its own variance determined by p̂).
10% Condition	Each sample is less than 10% of its respective population (ensures independence when sampling without replacement).	Same condition applies.

Summary of conditions for valid CIs for differences

SECTION 6

Worked Examples

Example A: CI for Difference in Means

A clinical researcher compares systolic blood pressure (mmHg) between a treatment group and a placebo group. The treatment group (n₁ = 45) has x̄₁ = 128.4 and s₁ = 14.2. The placebo group (n₂ = 50) has x̄₂ = 135.1 and s₂ = 16.8. Construct a 95% confidence interval for the difference in mean systolic blood pressure (Treatment − Placebo).

95% CI for μ₁ − μ₂ (Blood Pressure)

1
Step 1 — Identify Given Values

x̄₁ = 128.4, s₁ = 14.2, n₁ = 45 (treatment); x̄₂ = 135.1, s₂ = 16.8, n₂ = 50 (placebo). The point estimate of the difference is x̄₁ − x̄₂ = 128.4 − 135.1 = −6.7 mmHg.

Point estimate = −6.7 mmHg
2
Step 2 — Check Conditions

Both n₁ = 45 and n₂ = 50 are ≥ 30, so by the Central Limit Theorem the sampling distribution of the difference is approximately normal. The samples are independent (separate randomized groups). Population standard deviations are unknown, but with samples this large, the z-approximation is acceptable; alternatively, we can use Welch's t. Here we use z* = 1.96 for the 95% level.
3
Step 3 — Compute Standard Error

SE = √(s₁²/n₁ + s₂²/n₂) = √(14.2²/45 + 16.8²/50) = √(201.64/45 + 282.24/50) = √(4.4809 + 5.6448) = √10.1257 ≈ 3.182.

SE ≈ 3.182
4
Step 4 — Compute the Margin of Error

ME = z* × SE = 1.96 × 3.182 ≈ 6.237.

ME ≈ 6.24
5
Step 5 — Construct the Interval

CI = (x̄₁ − x̄₂) ± ME = −6.7 ± 6.24 = (−12.94, −0.46).

95% CI: (−12.94, −0.46) mmHg
6
Step 6 — Interpret

We are 95% confident that the true difference in mean systolic blood pressure (Treatment − Placebo) lies between −12.94 mmHg and −0.46 mmHg. Because the entire interval is negative and does not contain zero, there is evidence at the 95% confidence level that the treatment lowers systolic blood pressure relative to placebo.

Example B: CI for Difference in Proportions

In a vaccine efficacy trial, 18 out of 200 participants in the vaccine group developed the infection (p̂₁ = 0.09), while 40 out of 200 in the placebo group developed the infection (p̂₂ = 0.20). Construct a 95% confidence interval for p₁ − p₂.

95% CI for p₁ − p₂ (Vaccine Efficacy)

1
Step 1 — Identify Given Values

p̂₁ = 18/200 = 0.09, n₁ = 200 (vaccine); p̂₂ = 40/200 = 0.20, n₂ = 200 (placebo). Point estimate: p̂₁ − p̂₂ = 0.09 − 0.20 = −0.11.

Point estimate = −0.11
2
Step 2 — Check Conditions

Vaccine group: n₁p̂₁ = 18 ≥ 5, n₁(1−p̂₁) = 182 ≥ 5. Placebo group: n₂p̂₂ = 40 ≥ 5, n₂(1−p̂₂) = 160 ≥ 5. All conditions satisfied; the normal approximation is valid.
3
Step 3 — Compute Standard Error

SE = √[p̂₁(1−p̂₁)/n₁ + p̂₂(1−p̂₂)/n₂] = √[0.09 × 0.91/200 + 0.20 × 0.80/200] = √[0.0819/200 + 0.16/200] = √[0.0004095 + 0.0008] = √0.0012095 ≈ 0.03478.

SE ≈ 0.0348
4
Step 4 — Compute Margin of Error & Interval

ME = 1.96 × 0.0348 ≈ 0.0682. CI = −0.11 ± 0.0682 = (−0.1782, −0.0418).

95% CI: (−0.178, −0.042)
5
Step 5 — Interpret

We are 95% confident that the true difference in infection rates (Vaccine − Placebo) is between −17.8 percentage points and −4.2 percentage points. The entire interval is negative and excludes zero, providing strong evidence that the vaccine reduces infection rates compared to placebo.

SECTION 7

Strengths, Limitations & Method Comparisons

Feature	z-Based CI for Differences	t-Based (Welch) CI for Differences
When to use	Population σ known, or both n ≥ 30 (CLT justification). Also used for proportions.	Population σ unknown, at least one sample is small (n < 30), and data are approximately normal.
Strengths	Simple to compute; no degrees-of-freedom calculation; widely understood.	Robust to unequal variances; more accurate coverage in small samples; widely implemented in software.
Limitations	Under-covers (intervals too narrow) in small samples because it ignores estimation error in σ.	Satterthwaite df is an approximation; still requires approximate normality of the data (or large n).
For proportions	Wald interval; requires success/failure condition; can have poor coverage near 0 or 1.	Not typically used; alternative: Agresti-Caffo adjusted CI improves coverage for small samples.

z-based vs. t-based CIs for differences

✦

KEY TAKEAWAY

In modern biostatistical practice, Welch's t-interval is the default recommendation for comparing two means because it does not assume equal variances and converges to the z-interval as sample sizes grow. For proportions, the Wald interval remains standard, but be aware that Agresti-Caffo or Newcombe's hybrid method offers better coverage when proportions are extreme or samples are small. Always report the confidence interval alongside any p-value; the interval communicates both the direction and the practical magnitude of the effect.

SECTION 8

Connections to Advanced Theory

The confidence intervals for differences in means and proportions presented in this lesson serve as the foundation for more sophisticated inference methods encountered in advanced biostatistics coursework. Understanding how these basic intervals generalize will help you appreciate the broader inferential landscape.

This Lesson (Basic CI for Differences)	Advanced Extension
CI for μ₁ − μ₂ (two independent groups)	ANOVA simultaneous CIs (Tukey, Bonferroni) for pairwise differences among k > 2 groups; linear contrasts in regression.
CI for p₁ − p₂ (risk difference)	CIs for relative risk (RR) and odds ratios (OR) via log transformations; logistic regression CIs for adjusted ORs.
z* or t* with normal/t distribution	Bootstrap CIs (percentile, BCa) that do not require distributional assumptions; Bayesian credible intervals.
Unpooled SE for CI; pooled SE for test	Profile likelihood–based CIs that automatically handle nuisance parameters; score-based (Wilson) CIs for proportions.

How basic CIs for differences connect to advanced methods

An especially important extension in epidemiology is the transition from the risk difference (p₁ − p₂) to the relative risk (p₁/p₂) and odds ratio [(p₁/(1−p₁))/(p₂/(1−p₂))]. Each measure answers a slightly different clinical question, and their CIs require log-transformations to achieve approximate normality. The conceptual machinery — point estimate ± critical value × standard error — remains the same, but it operates on the log scale and is back-transformed via exponentiation.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A 95% confidence interval for the difference in mean cholesterol levels (Drug A − Drug B) is (−8.2, 3.5) mg/dL. A researcher concludes: "There is a 95% probability that the true difference lies in this interval." Identify and correct the error in this interpretation.

Show answer

The error lies in assigning probability to the parameter. The true difference μ₁ − μ₂ is a fixed (unknown) constant; it either lies inside the interval or it does not. The correct interpretation is: we are 95% confident that the interval (−8.2, 3.5) contains the true difference, meaning that if we repeated the study many times, about 95% of such intervals would capture the parameter. Additionally, because the interval contains zero, the data are consistent with no difference between the two drugs at the 95% confidence level.

Question 2: BASIC CALCULATION

Group 1 (n₁ = 60): x̄₁ = 72.0, s₁ = 10.0. Group 2 (n₂ = 55): x̄₂ = 68.5, s₂ = 12.0. Compute the 95% CI for μ₁ − μ₂ using the z-approximation.

Show answer

Point estimate: 72.0 − 68.5 = 3.5. Standard error: SE = √(10.0²/60 + 12.0²/55) = √(100/60 + 144/55) = √(1.6667 + 2.6182) = √4.2849 ≈ 2.070. Margin of error: 1.96 × 2.070 ≈ 4.057. Therefore, 95% CI = 3.5 ± 4.057 = (−0.56, 7.56). Since the interval includes zero, we cannot conclude at the 95% level that the means differ.

Question 3: INTERMEDIATE

In a study comparing the proportion of patients experiencing side effects, 22 of 150 patients in the new drug group and 35 of 160 patients in the standard drug group reported side effects. Construct a 99% confidence interval for the difference in side-effect rates (New − Standard) and interpret it clinically.

Show answer

p̂₁ = 22/150 = 0.1467, p̂₂ = 35/160 = 0.2188. Point estimate: 0.1467 − 0.2188 = −0.0721. Standard error: SE = √[0.1467 × 0.8533/150 + 0.2188 × 0.7812/160] = √[0.001252/150 × 150... let's compute: 0.1467 × 0.8533 = 0.12518, divided by 150 = 0.000835; 0.2188 × 0.7812 = 0.17085, divided by 160 = 0.001068. SE = √(0.000835 + 0.001068) = √0.001903 ≈ 0.04362. For 99% CI, z* = 2.576. ME = 2.576 × 0.04362 ≈ 0.1124. CI = −0.0721 ± 0.1124 = (−0.1845, 0.0403). Interpretation: we are 99% confident the true difference in side-effect rates lies between −18.5 and +4.0 percentage points. Since the interval includes zero, we lack sufficient evidence at the 99% level to conclude that the new drug has a different side-effect rate from the standard drug.

Question 4: APPLIED

A randomized trial evaluates a new cognitive-behavioral therapy (CBT) for depression. The Beck Depression Inventory (BDI) scores at 12 weeks are: CBT group (n₁ = 24, x̄₁ = 14.3, s₁ = 6.1) and usual care group (n₂ = 28, x̄₂ = 20.7, s₂ = 8.9). Both samples are too small for z; use Welch's t. Compute the Welch–Satterthwaite degrees of freedom, find the appropriate t* for a 95% CI, and construct the interval. (Hint: round df down to the nearest integer.)

Show answer

Point estimate: 14.3 − 20.7 = −6.4. Variance terms: s₁²/n₁ = 37.21/24 = 1.5504; s₂²/n₂ = 79.21/28 = 2.8289. SE = √(1.5504 + 2.8289) = √4.3793 ≈ 2.0927. Welch–Satterthwaite df: numerator = (1.5504 + 2.8289)² = (4.3793)² = 19.178; denominator = (1.5504)²/(24−1) + (2.8289)²/(28−1) = 2.4037/23 + 8.0027/27 = 0.10451 + 0.29640 = 0.40091. df = 19.178/0.40091 ≈ 47.84, rounded down to 47. From the t-table, t*₄₇,0.025 ≈ 2.012. ME = 2.012 × 2.0927 ≈ 4.211. CI = −6.4 ± 4.211 = (−10.61, −2.19). We are 95% confident the true mean difference (CBT − Usual Care) in BDI scores lies between −10.61 and −2.19 points. The interval excludes zero, suggesting CBT significantly reduces depression scores relative to usual care.

Question 5: CRITICAL THINKING

A biostatistician constructs both a 95% CI for the risk difference (p₁ − p₂) and a 95% CI for the log relative risk ln(p₁/p₂) from the same data, then back-transforms the latter to a CI for the relative risk. She notices that the risk-difference CI contains zero but the relative-risk CI excludes 1. Is this a contradiction? Explain why or why not, and discuss which measure might be more informative and under what study design.

Show answer

This is not a contradiction. The risk difference and relative risk are different parameters that measure association on different scales (additive versus multiplicative). Their confidence intervals are derived from different standard error formulas and, critically, the null values differ: zero for the risk difference and one for the relative risk. Because the transformations are nonlinear, it is entirely possible for one interval to exclude its null value while the other does not. This situation is most likely when the baseline risk p₂ is very small: even a modest absolute difference can correspond to a large relative effect. For example, if p̂₁ = 0.003 and p̂₂ = 0.001, the risk difference (0.002) might have a CI containing zero due to wide absolute-scale uncertainty, while the relative risk (3.0) may have a CI excluding 1 because the log-scale standard error is smaller. In cohort studies and clinical trials, both measures are informative, but the risk difference communicates clinical importance in absolute terms (e.g., number needed to treat), whereas the relative risk communicates the multiplicative fold-change. In case-control studies, only the odds ratio is estimable directly. The choice of measure should align with the study design and the clinical question.

SUMMARY

Lesson Summary

Summary

This lesson covered the construction and interpretation of confidence intervals for the difference in two means and confidence intervals for the difference in two proportions. The general formula follows the structure point estimate ± critical value × standard error, where the standard error of the difference combines variability from both samples using the sum-of-variances rule. For means, use z* when population variances are known or samples are large, and Welch's t* with Satterthwaite degrees of freedom when variances are unknown and at least one sample is small. For proportions, the Wald interval uses unpooled standard errors and requires each group to satisfy the success/failure condition.

Interpretation is critical: if the interval for μ₁ − μ₂ excludes zero, we have evidence of a real difference at the stated confidence level; if it contains zero, the data are consistent with no difference. The same logic applies to p₁ − p₂. These intervals form the foundation for more advanced comparative measures, including relative risk, odds ratios, and ANOVA-based contrasts. Always check assumptions — independence, normality or large-sample conditions, and the 10% rule — before computing and interpreting the interval.