Bias & Confounding — Identify common biases (selection, recall, measurement) and confounding - Biostatistics

SECTION 1

Historical Context & Motivation

From the earliest days of clinical observation, physicians and scientists have struggled with a deceptively simple question: does the evidence we collect actually reflect reality, or are we being misled by the way we gathered it? The formal study of bias and confounding emerged as epidemiology matured into a quantitative discipline during the twentieth century. Landmark studies—some famous for their successes, others for their failures—revealed that systematic errors can masquerade as true associations, leading to flawed clinical guidelines, wasted resources, and patient harm. Understanding these threats to validity is therefore not merely an academic exercise; it is foundational to every aspect of evidence-based medicine and public health policy.

1747 • Lind's Scurvy Trial

James Lind conducted one of the earliest controlled experiments aboard HMS Salisbury, comparing six treatments for scurvy. Although groundbreaking, the trial suffered from selection bias due to its tiny, non-random sample of twelve sailors.

1948 • Framingham Heart Study Begins

This landmark cohort study enrolled over 5,000 residents of Framingham, Massachusetts. Its design explicitly addressed confounding by measuring multiple cardiovascular risk factors simultaneously, enabling researchers to disentangle the independent effects of hypertension, cholesterol, and smoking.

1951 • Doll & Hill — Smoking and Lung Cancer

Richard Doll and A. Bradford Hill published their case-control study linking smoking to lung cancer. Critics initially argued the association was due to confounding by occupational exposures or genetic predisposition, forcing the investigators to develop rigorous methods for controlling confounders.

1979 • Sackett's Catalogue of Biases

David Sackett published a seminal paper cataloguing over 35 distinct biases in analytic research, providing the first systematic taxonomy that organized selection bias, information bias, and confounding into a coherent framework still used today.

2000s • DAGs and Modern Causal Inference

Judea Pearl, Miguel Hernán, and James Robins popularized directed acyclic graphs (DAGs) as formal tools for identifying confounders and biases before data collection, transforming confounding analysis from intuition into a rigorous graphical calculus.

This historical arc reveals a recurring theme: even well-intentioned studies can produce misleading conclusions when bias or confounding goes unrecognized. The central question this lesson addresses is: How do we identify, classify, and mitigate the systematic errors that threaten the internal validity of biomedical research?

SECTION 2

Core Principles & Definitions

Before dissecting individual bias types, it is essential to establish the conceptual landscape. In epidemiological research, the observed association between an exposure and an outcome can deviate from the true association for three fundamental reasons: chance (random error), bias (systematic error), and confounding (mixing of effects). While random error can be reduced by increasing sample size, bias and confounding require careful study design and analytical strategy. The following core principles provide a framework for understanding these threats.

Selection Bias

Occurs when the participants who enter (or remain in) a study differ systematically from the target population with respect to the exposure-outcome relationship. Common subtypes include volunteer bias, loss-to-follow-up bias, and Berkson's bias (hospital-based selection).

Recall Bias

A type of information bias in which participants' recollections of past exposures are influenced by their disease status. Cases often search their memory more thoroughly for potential causes than controls, leading to differential misclassification of exposure.

Measurement Bias

Systematic error arising from inaccurate ascertainment or classification of exposure, outcome, or covariates. Subtypes include observer/interviewer bias and instrument bias. May be differential (varying by group) or non-differential (affecting all groups equally).

Confounding

A mixing of effects that occurs when a third variable is associated with both the exposure and the outcome, is not on the causal pathway between them, and creates a spurious or masked association. Unlike bias, confounding reflects a real biological relationship involving an extraneous variable.

✦

KEY TAKEAWAY

Think of bias like a warped ruler—every measurement you take with it will be off in the same direction, no matter how many measurements you make. Confounding, by contrast, is like trying to weigh yourself while holding a cat: the scale reading is technically accurate, but it reflects two things mixed together. Increasing sample size fixes random error but does nothing for either of these systematic problems—only better design and analysis can address them.

SECTION 3

Visual Explanation — Taxonomy of Threats to Validity

This taxonomy illustrates the three threats to internal validity. Random error (left) is addressable by increasing sample size, while bias (center) and confounding (right) require design-based or analytical solutions. Note how bias branches into selection, recall, and measurement categories, each with specific subtypes.

The diagram above provides a hierarchical view of the key threats we will explore throughout this lesson. Notice the fundamental distinction: bias is a property of the study design or data collection process, meaning it introduces systematic error that cannot be corrected simply by collecting more data. Confounding, on the other hand, reflects a real relationship among three or more variables in the population and can often be addressed analytically (through stratification, regression, or matching) if the confounder is measured. The three criteria for confounding—association with exposure, independent effect on outcome, and not being on the causal pathway—serve as a practical checklist when evaluating whether a variable should be treated as a confounder in analysis.

SECTION 4

Mathematical Framework — Quantifying Bias & Confounding

While bias and confounding are conceptual threats, they can be expressed quantitatively. The mathematical framework below formalizes how these distortions affect observed measures of association, particularly the odds ratio (OR) and relative risk (RR). By understanding these relationships, you can determine the direction and magnitude of distortion.

CRUDE VS. ADJUSTED MEASURE

Bias = OR_crude − OR_true

Where OR_crude is the observed odds ratio and OR_true is the unbiased odds ratio. If Bias > 0, the association is overestimated; if Bias < 0, it is underestimated.

NON-DIFFERENTIAL MISCLASSIFICATION EFFECT

OR_observed = 1 + (OR_true − 1) × (Se + Sp − 1)²

For a dichotomous exposure with sensitivity Se and specificity Sp of the exposure classification tool. When misclassification is non-differential (same in cases and controls), the observed OR is biased toward the null (OR = 1). This simplified formula assumes equal misclassification probabilities in both exposure groups.

CONFOUNDING — PERCENTAGE CHANGE METHOD

% Confounding = ((OR_crude − OR_adjusted) / OR_adjusted) × 100

A change of ≥ 10% between the crude and adjusted estimates is conventionally interpreted as evidence of meaningful confounding. OR_adjusted is obtained after stratifying by or adjusting for the suspected confounder (e.g., via Mantel–Haenszel method or logistic regression).

MANTEL–HAENSZEL ADJUSTED ODDS RATIO

OR_MH = Σ(aᵢdᵢ / Tᵢ) / Σ(bᵢcᵢ / Tᵢ)

Computed across i strata of the confounder, where a, b, c, d are cells of the 2×2 table and T is the stratum total. This weighted average removes the effect of the confounding variable while preserving statistical efficiency.

⚠

Direction of Bias

Non-differential misclassification of a dichotomous exposure biases toward the null (OR → 1), while differential misclassification can bias in either direction—toward or away from the null—making it especially dangerous because you cannot predict the direction without additional information.

SECTION 5

Detailed Classification of Biases & Confounding Scenarios

Having established the conceptual and mathematical framework, we can now examine each bias category in greater depth and connect them to the study designs in which they most commonly arise. The diagram below illustrates how each bias type maps onto the stages of a typical epidemiological study, from participant selection through data analysis.

This flowchart maps bias and confounding to the stage of study conduct where each enters. Selection bias enters during sampling, information bias (including recall and measurement bias) enters during data collection, and confounding must be addressed at both the design and analysis stages.

Bias / Threat	Study Designs Most Affected	Direction of Effect	Primary Prevention Strategy
Selection bias	Case-control, retrospective cohort	Either direction	Population-based sampling, clearly defined source population
Recall bias	Case-control (retrospective)	Usually away from null	Use objective records; prospective design
Non-differential measurement bias	All observational designs	Toward the null (for dichotomous exposure)	Validated instruments, calibration, blinding
Differential measurement bias	Unblinded trials, chart reviews	Either direction	Blinding, standardized assessment protocols
Confounding	All observational designs	Either direction	Randomization (design), stratification / regression (analysis)

Summary of major biases and confounding: affected designs, direction of effect, and prevention

SECTION 6

Worked Example — Coffee, Smoking, and Pancreatic Cancer

Consider the following classic scenario: a case-control study examines whether coffee consumption is associated with pancreatic cancer. The investigators recruit 200 pancreatic cancer cases from a hospital and 200 controls from the same hospital (patients admitted for other non-cancer diagnoses). They find a crude odds ratio of 2.5, suggesting coffee drinkers have 2.5 times the odds of pancreatic cancer. However, smoking is far more prevalent among coffee drinkers, and smoking is an established risk factor for pancreatic cancer. We need to determine whether the coffee-cancer association is real or confounded by smoking.

Assessing Confounding by Smoking in a Coffee–Cancer Study

1
Step 1 — Identify the Crude Association

From the overall 2×2 table, we compute the crude odds ratio for the coffee–cancer relationship. Using the cell frequencies: a = 120 (cases exposed to coffee), b = 80 (cases not exposed), c = 80 (controls exposed), d = 120 (controls not exposed). OR_crude = (a × d) / (b × c) = (120 × 120) / (80 × 80) = 14,400 / 6,400 = 2.25.

OR_crude = 2.25
2
Step 2 — Check Confounder Criteria for Smoking

We verify three conditions. (1) Is smoking associated with coffee drinking? Yes—70% of coffee drinkers smoke versus 30% of non-coffee drinkers. (2) Is smoking an independent risk factor for pancreatic cancer? Yes—well established in the literature. (3) Is smoking on the causal pathway between coffee and cancer? No—smoking does not mediate the biological effect of coffee. All three criteria are met, so smoking qualifies as a potential confounder.
3
Step 3 — Stratify by Smoking Status

We create separate 2×2 tables for smokers and non-smokers. Among smokers: a₁ = 84, b₁ = 24, c₁ = 56, d₁ = 16. OR_smokers = (84 × 16) / (24 × 56) = 1,344 / 1,344 = 1.0. Among non-smokers: a₂ = 36, b₂ = 56, c₂ = 24, d₂ = 104. OR_non-smokers = (36 × 104) / (56 × 24) = 3,744 / 1,344 ≈ 2.79. Note: if we use a simpler illustrative example where both strata yield OR ≈ 1.0, the confounding is complete.

Stratum-specific ORs differ from crude OR
4
Step 4 — Compute the Mantel–Haenszel Adjusted OR

Using the MH formula: OR_MH = Σ(aᵢdᵢ / Tᵢ) / Σ(bᵢcᵢ / Tᵢ). For smokers (T₁ = 180): (84 × 16) / 180 = 7.47. For non-smokers (T₂ = 220): (36 × 104) / 220 = 17.02. Numerator = 7.47 + 17.02 = 24.49. For smokers: (24 × 56) / 180 = 7.47. For non-smokers: (56 × 24) / 220 = 6.11. Denominator = 7.47 + 6.11 = 13.58. OR_MH = 24.49 / 13.58 ≈ 1.80.

OR_MH ≈ 1.80
5
Step 5 — Assess Degree of Confounding

Percent confounding = ((OR_crude − OR_adjusted) / OR_adjusted) × 100 = ((2.25 − 1.80) / 1.80) × 100 = 25%. Since this exceeds the conventional 10% threshold, we conclude that smoking meaningfully confounded the crude association between coffee and pancreatic cancer. The adjusted estimate of 1.80 is a more accurate reflection of the coffee–cancer relationship, though residual confounding by unmeasured variables may still exist.

25% confounding — smoking is an important confounder

SECTION 7

Strengths & Limitations of Bias Control Methods

No single strategy eliminates all threats to validity. Each method of controlling bias and confounding has inherent strengths and limitations, and the optimal approach depends on the study design, available resources, and nature of the threat. The table below contrasts the primary methods used at both the design and analysis stages.

Control Method	Strengths	Limitations
Randomization	Controls for both known and unknown confounders; gold standard for causal inference; eliminates selection bias in treatment allocation	Only applicable to experimental designs; may not achieve balance in small samples; ethical/feasibility constraints for many exposures
Restriction	Simple to implement; completely eliminates confounding by the restricted variable; cheap	Reduces sample size and generalizability; cannot control for multiple confounders simultaneously; residual confounding still possible
Matching	Efficient control of strong confounders; increases statistical efficiency in case-control studies; intuitive	Cannot study matched variable as risk factor; overmatching can introduce bias; computationally difficult with many variables
Stratification (MH)	Transparent and intuitive; allows assessment of effect modification; no modeling assumptions	Limited to few confounders (sparse data with many strata); cannot handle continuous confounders well
Multivariable Regression	Can adjust for many confounders simultaneously; handles continuous and categorical variables; widely available	Model misspecification risk; assumes correct functional form; cannot control for unmeasured confounders; "garbage in, garbage out"
Blinding	Prevents measurement/observer bias and differential recall; reduces Hawthorne effect	Not always feasible (surgical interventions, lifestyle exposures); unblinding can occur

Comparison of strategies for controlling bias and confounding

✦

KEY TAKEAWAY

Think of controlling for bias and confounding like building a house: design-phase methods (randomization, restriction, matching, blinding) are your foundation—once the foundation is poured, you cannot fix it retroactively. Analysis-phase methods (stratification, regression) are your interior renovations—they can improve the structure, but only if the foundation is sound. A study with irreparable selection bias cannot be salvaged by a sophisticated statistical model.

SECTION 8

Connection to Advanced Causal Inference

The concepts of bias and confounding introduced in this lesson form the foundation for more advanced frameworks in causal inference. Modern epidemiology increasingly relies on directed acyclic graphs (DAGs) to formalize the relationships among variables, making it possible to identify confounders, mediators, and colliders with mathematical precision rather than intuition alone. Understanding these advanced tools begins with mastery of the fundamental concepts covered here.

This Lesson (Foundational)	Advanced Extension
Confounder identified by three checklist criteria	DAG-based backdoor criterion identifies minimal sufficient adjustment sets algorithmically
Stratification by suspected confounders (Mantel–Haenszel)	Inverse probability weighting (IPW) and marginal structural models for time-varying confounding
Selection bias as a design flaw	Collider-stratification bias: conditioning on a common effect of exposure and outcome opens a non-causal path
Sensitivity analysis for unmeasured confounding (qualitative)	E-value and quantitative bias analysis formally bound the impact of unmeasured confounders
Effect modification detected via stratified analysis	Interaction on additive vs. multiplicative scales; sufficient-component cause models

Foundational concepts and their advanced causal inference extensions

💡

Important Distinction: Confounding vs. Effect Modification

Students frequently confuse confounding with effect modification (interaction). In confounding, the stratum-specific ORs are similar to each other but different from the crude OR—we want to remove confounding. In effect modification, the stratum-specific ORs differ meaningfully from each other—we want to report effect modification rather than pool the strata. Recognizing this distinction is crucial for correct interpretation.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A case-control study of childhood leukemia asks mothers of cases and mothers of healthy controls to recall their child's exposure to household pesticides during pregnancy. Mothers of leukemia cases report significantly higher pesticide exposure than mothers of controls. What type of bias is most likely present, and in which direction would it distort the odds ratio?

Show answer

This scenario exemplifies recall bias. Mothers of children with leukemia are more likely to search their memories extensively for potential causes of their child's illness (a phenomenon sometimes called "rumination bias"). This differential recall leads to systematic over-reporting of exposure among cases relative to controls. The result is a spuriously inflated odds ratio (biased away from the null), because the exposure-outcome association appears stronger than it truly is. Using prospective exposure records (e.g., medical charts, pesticide purchase databases) rather than maternal recall would mitigate this bias.

Question 2: BASIC CALCULATION

An exposure classification instrument has sensitivity = 0.80 and specificity = 0.90. If the true OR for the exposure-disease association is 3.0, estimate the observed OR under non-differential misclassification using the formula: OR_observed = 1 + (OR_true − 1) × (Se + Sp − 1)².

Show answer

Substituting the values: OR_observed = 1 + (3.0 − 1) × (0.80 + 0.90 − 1)² = 1 + 2.0 × (0.70)² = 1 + 2.0 × 0.49 = 1 + 0.98 = 1.98. The observed OR (1.98) is substantially attenuated compared to the true OR (3.0), demonstrating how non-differential misclassification of a dichotomous exposure biases the estimate toward the null. The "bias factor" of (Se + Sp − 1)² = 0.49 shows that nearly half of the true excess odds ratio is lost due to misclassification. Improving measurement accuracy (increasing Se and Sp toward 1.0) would recover the true association.

Question 3: INTERMEDIATE

A cohort study finds a crude RR of 1.8 for the association between oral contraceptive (OC) use and myocardial infarction (MI). After adjusting for smoking (a known confounder), the adjusted RR is 1.2. Calculate the percent confounding and interpret whether confounding is meaningful.

Show answer

Using the percent confounding formula: % Confounding = ((RR_crude − RR_adjusted) / RR_adjusted) × 100 = ((1.8 − 1.2) / 1.2) × 100 = (0.6 / 1.2) × 100 = 50%. This exceeds the conventional 10% threshold, indicating that smoking is a substantial confounder in the OC–MI relationship. The crude estimate overstated the association because OC users in this study were more likely to smoke, and smoking independently increases MI risk. The adjusted RR of 1.2, while still above 1.0, suggests a much weaker association once confounding by smoking is removed.

Question 4: APPLIED

You are designing a case-control study to examine whether a new occupational chemical exposure causes bladder cancer. You plan to recruit cases from a referral hospital and controls from the same hospital's orthopedic ward. Identify at least two specific biases that could arise from this design and propose concrete modifications to minimize each.

Show answer

First, Berkson's bias (selection bias) is a concern because hospital-based controls may not represent the exposure distribution in the source population. Orthopedic patients might have higher occupational exposure rates (e.g., manual laborers at higher injury risk), which would attenuate the OR toward the null. To mitigate this, you could use population-based controls identified through random-digit dialing or population registries. Second, recall bias is likely because bladder cancer cases may recall their occupational exposures more thoroughly than controls who have no reason to reflect on chemical exposures. Using employment records, industrial hygiene databases, or job-exposure matrices instead of self-reported exposure histories would reduce this bias. Third, confounding by smoking must be addressed since smoking is a strong risk factor for bladder cancer and may be associated with certain occupations. Collecting detailed smoking histories and adjusting via stratification or regression would control this confounder.

Question 5: CRITICAL THINKING

A prospective cohort study reports that physical activity reduces the risk of death from cardiovascular disease (RR = 0.6), even after adjusting for age, sex, smoking, BMI, and socioeconomic status. A critic argues that the finding could still be explained by confounding due to pre-existing subclinical disease ("reverse causation")—people who are already becoming ill exercise less. Evaluate this critique. What additional analyses or design features could address this concern, and could any form of bias (not just confounding) also explain the result?

Show answer

The critic raises a valid concern about reverse causation, which functions as confounding by baseline health status: subclinical disease both reduces physical activity and increases mortality risk. Several approaches can address this. A lag analysis that excludes deaths occurring in the first 2–5 years of follow-up would reduce the influence of pre-existing disease. Alternatively, restricting the analysis to participants who reported good health at baseline provides another check. Beyond confounding, selection bias (healthy worker/volunteer effect) could also explain the result: individuals who join studies and report high physical activity may systematically differ from non-participants in health-seeking behaviors, diet quality, and access to healthcare. Non-differential measurement bias in self-reported physical activity would generally bias toward the null, so it cannot explain an observed protective effect—but differential measurement bias could, if healthier participants more accurately recall exercise. Finally, computing the E-value would quantify the minimum strength an unmeasured confounder would need to fully explain the observed RR of 0.6, providing a formal sensitivity analysis for residual confounding.

SUMMARY

Lesson Summary

Summary

This lesson examined the three major systematic threats to internal validity in epidemiological research. Selection bias arises when the study sample differs systematically from the target population due to how participants are recruited or retained—common subtypes include volunteer bias, loss-to-follow-up, Berkson's bias, and the healthy worker effect. Recall bias is a form of information bias in which disease status differentially affects participants' recollections of past exposures, typically inflating the observed association in case-control studies. Measurement bias encompasses systematic errors in the ascertainment of exposure, outcome, or covariates—when non-differential, it biases dichotomous exposure associations toward the null; when differential, it can distort in either direction.

Confounding occurs when a third variable is associated with both exposure and outcome and is not on the causal pathway, creating a spurious or masked association. Unlike bias, confounding can be addressed both at the design stage (randomization, restriction, matching) and at the analysis stage (stratification, Mantel–Haenszel adjustment, multivariable regression). The percent change method (≥ 10% threshold) provides a practical criterion for assessing whether confounding is meaningful. Mastery of these concepts is essential before advancing to directed acyclic graphs (DAGs), inverse probability weighting, and other modern causal inference tools that extend the foundational principles covered here.