Checking Regression Assumptions — Check assumptions (linearity, independence, normality, equal variance) - Biostatistics

SECTION 1

Historical Context & Motivation

Regression analysis is among the most widely used statistical methods in biomedical research, yet the validity of its results hinges on a set of mathematical conditions that must be satisfied for standard inference—confidence intervals, p-values, and prediction intervals—to hold. The formal study of these conditions traces back to the early development of ordinary least squares (OLS) estimation, and researchers have spent over a century refining both the theoretical foundations and the practical diagnostic tools used to evaluate them. Understanding why these assumptions exist, and how violations distort results, is essential for any researcher conducting or interpreting regression-based analyses in the health sciences.

1805 • Legendre Introduces Least Squares

Adrien-Marie Legendre publishes the method of least squares for fitting linear models to astronomical data, implicitly assuming a linear relationship and well-behaved errors.

1809 • Gauss Connects Least Squares to Normality

Carl Friedrich Gauss derives the least squares estimator under the assumption of normally distributed errors, establishing the theoretical link between the method and the normality assumption.

1922 • Gauss–Markov Theorem Formalized

The Gauss–Markov theorem proves that OLS yields the best linear unbiased estimator (BLUE) when errors are uncorrelated and have equal variance—formalizing the independence and homoscedasticity assumptions.

1973 • Anscombe's Quartet Published

Francis Anscombe demonstrates four datasets with identical summary regression statistics but dramatically different scatter plots, powerfully illustrating that numerical summaries alone cannot validate assumptions.

1990s–Present • Modern Diagnostic Tools

Software packages in R, SAS, and Python make residual diagnostics, Q–Q plots, and formal tests (e.g., Breusch–Pagan, Durbin–Watson) accessible to applied researchers across biostatistics and epidemiology.

The central question this lesson addresses is: How do we verify that the mathematical conditions underlying a regression model are met, and what do we do when they are not? In biostatistics, where models inform clinical decisions, drug dosage calculations, and epidemiological policy, a regression model that violates its assumptions can yield misleading coefficient estimates, inflated confidence, and ultimately flawed conclusions about biological relationships.

SECTION 2

Core Principles — The LINE Assumptions

The four classical assumptions of ordinary least squares regression are commonly remembered by the mnemonic LINE: Linearity, Independence, Normality, and Equal variance. Each assumption pertains to properties of the residuals (the differences between observed and predicted values), not to the raw data themselves. When all four hold simultaneously, OLS provides the best linear unbiased estimator, and the standard formulas for standard errors, t-tests, and F-tests are valid.

Linearity

The true relationship between the predictor(s) and the outcome is linear in the parameters. The expected value of Y given X falls on a straight line (or hyperplane in multiple regression). Violations produce systematic patterns in residual plots.

Independence

The residuals are independent of one another—knowing one residual provides no information about another. This assumption is violated in time-series data, clustered designs, or repeated-measures studies without appropriate modeling.

Normality

The residuals follow a normal (Gaussian) distribution at each level of the predictor. This assumption is most critical for inference in small samples; with large n, the Central Limit Theorem provides robustness.

Equal Variance (Homoscedasticity)

The variance of the residuals remains constant across all levels of the predictor(s). When variance changes systematically—called heteroscedasticity—standard errors are biased, making confidence intervals and hypothesis tests unreliable.

✦

KEY TAKEAWAY

Think of the LINE assumptions as the fine print on a contract. The OLS method promises to give you the best, most efficient estimates—but only if the four conditions are met. Just as an insurance policy becomes void if terms are violated, the guarantees of OLS (unbiasedness, minimum variance, valid inference) break down when assumptions fail. The regression output might still appear numerical and precise, but the numbers you trust—p-values, confidence intervals—may be quietly misleading.

SECTION 3

Visual Explanation — Residual Diagnostic Plots

The primary tool for evaluating regression assumptions is the residual plot—a scatter plot of residuals (ê) on the vertical axis against fitted values (ŷ) or predictor values on the horizontal axis. A well-behaved model produces a residual plot with no discernible pattern: points scatter randomly around zero in a roughly constant horizontal band. The diagram below contrasts a model that satisfies assumptions with three common violation patterns.

Four residual plot patterns. Top-left: random scatter indicates assumptions are met. Top-right: a curved (U-shaped) pattern signals nonlinearity. Bottom-left: a fanning pattern reveals heteroscedasticity (unequal variance). Bottom-right: extreme outlier residuals suggest non-normal error distributions.

In practice, you should generate these plots immediately after fitting any regression model. The residual-versus-fitted plot (top panels) addresses linearity and equal variance simultaneously: a systematic curve indicates non-linearity, while a fanning pattern indicates heteroscedasticity. Extreme residuals far from the horizontal band (bottom-right panel) raise concerns about normality. The independence assumption typically cannot be assessed from a residual-versus-fitted plot alone; it requires knowledge of the study design or time-ordered residual plots.

SECTION 4

Mathematical Framework

The simple linear regression model is expressed as a data-generating process with explicit conditions on the error term. These conditions are precisely the LINE assumptions expressed in mathematical notation. The model and its assumptions together determine the statistical properties of the OLS estimators β̂₀ and β̂₁.

SIMPLE LINEAR REGRESSION MODEL

Yᵢ = β₀ + β₁Xᵢ + εᵢ

Yᵢ = observed response for the i-th observation; β₀ = intercept; β₁ = slope; Xᵢ = predictor value; εᵢ = random error (residual) for observation i.

LINEARITY ASSUMPTION

E(Yᵢ | Xᵢ) = β₀ + β₁Xᵢ

The conditional expectation of Y given X is a linear function of the parameters. Equivalently, E(εᵢ | Xᵢ) = 0 for all i. If the true relationship is curvilinear, this expectation is not zero and the linear model is misspecified.

INDEPENDENCE ASSUMPTION

Cov(εᵢ, εⱼ) = 0 for all i ≠ j

The covariance between any two distinct error terms is zero, meaning the error for one observation is unrelated to the error for any other observation. Violations commonly arise in longitudinal or spatially correlated data.

NORMALITY AND EQUAL VARIANCE

εᵢ ~ N(0, σ²) for all i

Each error term follows a normal distribution with mean zero and constant variance σ². The constant σ² across all observations encodes the homoscedasticity (equal variance) assumption. When σ² varies with X—i.e., Var(εᵢ | Xᵢ) = σᵢ²—standard errors from OLS are biased.

A compact way to express all four conditions simultaneously is: ε ~ N(0, σ²I), where ε is the n × 1 vector of errors, 0 is a zero vector, and σ²I is a scalar variance times the identity matrix. The identity matrix enforces both equal variance (all diagonal elements are σ²) and independence (all off-diagonal elements are zero). This elegant formulation makes it clear that a single violation—say, correlated errors—changes the covariance matrix from σ²I to some other structure Σ, invalidating the standard OLS inference formulas.

SECTION 5

Detailed Diagnostic Methods for Each Assumption

Each of the four LINE assumptions has both graphical and formal testing procedures. In practice, graphical methods are preferred as the primary diagnostic because formal tests can be overly sensitive in large samples (rejecting trivial violations) or underpowered in small samples (failing to detect meaningful violations). The diagram below presents a decision flowchart for the complete assumption-checking process.

A complete assumption-checking flowchart. After fitting the model and computing residuals, each of the four LINE assumptions is assessed through both graphical tools and formal hypothesis tests. When violations are detected, remedial strategies (transformations, robust methods, or alternative models) should be applied before interpreting coefficients.

Assumption	Primary Graphical Diagnostic	Formal Test	What to Look For
Linearity	Residual vs. fitted values plot; partial regression plots	Ramsey RESET test; lack-of-fit F-test	Curved or U-shaped pattern in residuals; systematic departure from zero line
Independence	Residuals vs. time/order plot; autocorrelation function (ACF) plot	Durbin–Watson test (d statistic); Ljung–Box test	Runs of positive/negative residuals; sinusoidal pattern; d ≈ 2 indicates no autocorrelation
Normality	Normal Q–Q plot; histogram with normal curve overlay	Shapiro–Wilk test; Anderson–Darling test; Kolmogorov–Smirnov test	Points deviating from the 45° reference line on Q–Q plot; skewness or heavy tails in histogram
Equal Variance	Scale-location plot (√\|standardized residuals\| vs. fitted); residual vs. fitted for fan shape	Breusch–Pagan test; White's test; Levene's test (for grouped data)	Increasing or decreasing spread in residuals; megaphone/fan pattern; significant BP test p-value

Summary of diagnostic tools for each LINE assumption

SECTION 6

Worked Example — Blood Pressure and Age

A biostatistician fits a simple linear regression predicting systolic blood pressure (SBP, in mmHg) from age (in years) using data from n = 50 adults in a cross-sectional study. The fitted model is SBP = 98.7 + 0.97 × Age. Before interpreting the slope coefficient, the analyst must verify all four LINE assumptions. We walk through each check using the model's residuals.

Checking LINE Assumptions for the SBP–Age Model

1
Step 1 — Compute Residuals

For each of the 50 observations, compute ê_i = SBP_i − (98.7 + 0.97 × Age_i). For example, if patient 1 is 45 years old with SBP = 145 mmHg, then ŷ₁ = 98.7 + 0.97(45) = 142.35, so ê₁ = 145 − 142.35 = 2.65.

ê₁ = 2.65 mmHg
2
Step 2 — Check Linearity

Plot all 50 residuals against the fitted values. In this example, the scatter appears randomly distributed around zero with no systematic curvature. We confirm this impression with the Ramsey RESET test: the test statistic is F = 1.42 with p = 0.24, so we fail to reject the null hypothesis of correct specification.

✓ Linearity assumption satisfied (RESET p = 0.24 > 0.05)
3
Step 3 — Check Independence

Since this is a cross-sectional study (not time-series), observations were sampled independently from different individuals. We verify that no clustering structure exists (e.g., participants from the same family or clinic). The Durbin–Watson statistic is d = 1.89, which is close to 2 and falls within the non-rejection bounds.

✓ Independence satisfied (cross-sectional design, DW = 1.89 ≈ 2)
4
Step 4 — Check Normality

Construct a normal Q–Q plot of the standardized residuals. The points fall closely along the 45° reference line with minor deviations in the tails. The Shapiro–Wilk test yields W = 0.976, p = 0.41. With p well above 0.05, we have no evidence to reject normality. A histogram of the residuals is approximately bell-shaped and centered at zero.

✓ Normality satisfied (Shapiro–Wilk p = 0.41)
5
Step 5 — Check Equal Variance

Examine the residual-versus-fitted plot for a fanning or trumpet pattern. In this case, we notice that the residual spread appears slightly wider for older patients (higher fitted values). The Breusch–Pagan test gives χ² = 5.83, p = 0.016, suggesting statistically significant heteroscedasticity. This is the one assumption that appears violated.

✗ Equal variance violated (Breusch–Pagan p = 0.016 < 0.05)
6
Step 6 — Remediation

Because heteroscedasticity is detected, the analyst has several options: (a) use heteroscedasticity-consistent (HC) standard errors (also called robust or sandwich standard errors) to obtain valid inference without changing the model; (b) apply a variance-stabilizing transformation such as log(SBP); or (c) use weighted least squares (WLS) if the variance function can be modeled. In this example, using HC standard errors increases the standard error of β̂₁ from 0.18 to 0.23, widening the 95% CI from (0.61, 1.33) to (0.51, 1.43). The coefficient remains statistically significant, but the confidence interval more honestly reflects the uncertainty.

Corrected 95% CI for slope: (0.51, 1.43) using robust SE

SECTION 7

Consequences of Violations & Remedial Strategies

Not all assumption violations are equally damaging. Understanding the specific consequences of each violation helps researchers prioritize diagnostics and select appropriate remedial measures. The table below summarizes what goes wrong when each assumption fails and the most common corrective strategies used in biostatistical practice.

Violated Assumption	Consequence	Remedial Strategy
Linearity	Biased coefficient estimates; predicted values systematically too high or too low in certain regions; R² underestimates explanatory power	Add polynomial terms (X², X³); apply splines; transform X or Y (e.g., log, square root); use generalized additive models (GAMs)
Independence	Standard errors are underestimated (positive autocorrelation) or overestimated; inflated Type I error rates; confidence intervals are too narrow	Use mixed-effects models for clustered data; employ GEE (generalized estimating equations); fit time-series models (ARIMA) for temporal data; include random effects
Normality	Hypothesis tests and confidence intervals become inaccurate, especially in small samples; prediction intervals are unreliable; coefficient estimates remain unbiased	Apply Box–Cox or log transformation to Y; use bootstrapped confidence intervals; rely on CLT for large samples (n > 30–50); use nonparametric regression
Equal Variance	OLS is no longer efficient; standard errors are biased (can be inflated or deflated); confidence intervals have incorrect coverage probability	Use robust (sandwich/HC) standard errors; weighted least squares (WLS); variance-stabilizing transformations (log, sqrt); model the variance function explicitly

Consequences and remedies for each LINE assumption violation

✦

KEY TAKEAWAY

A useful hierarchy for prioritizing assumption checks is: linearity first (because a misspecified functional form makes everything else meaningless), then independence (because correlated errors can dramatically distort inference), then equal variance (because it affects efficiency and standard errors), and finally normality (because it is least consequential in large samples thanks to the Central Limit Theorem). Think of it like building a house: linearity is the foundation, independence is the framing, equal variance is the insulation, and normality is the paint—you can tolerate imperfect paint far more than a cracked foundation.

SECTION 8

Connection to Advanced Theory — Beyond OLS

When one or more LINE assumptions are substantially violated and simple remedial measures are insufficient, biostatisticians turn to more sophisticated modeling frameworks. These advanced methods relax specific assumptions while preserving valid inference. Understanding these extensions positions you to select the right tool for complex biomedical data structures where standard OLS falls short.

Feature	Standard OLS Regression	Advanced Alternatives
Error distribution	Assumes normal (Gaussian) errors	GLMs allow binomial, Poisson, gamma, and other distributions via a link function
Functional form	Linear in parameters; polynomial terms added manually	GAMs fit smooth nonparametric functions; spline regression captures flexible curves
Correlated errors	Assumes independent errors (Cov = 0)	Mixed models and GEE handle clustering, repeated measures, and hierarchical data
Heteroscedasticity	Assumes constant variance σ²	WLS weights inversely by variance; robust SE provides valid inference without reweighting
Outlier sensitivity	Highly sensitive; single outlier can shift regression line	Robust regression (M-estimation, LTS) downweights influential points automatically

OLS versus advanced regression alternatives

In modern biostatistical practice, the assumption-checking workflow is not merely a pass/fail gate but a diagnostic process that guides model selection. A researcher who discovers non-normality and heteroscedasticity in residuals from a model of hospital costs, for example, might switch to a generalized linear model (GLM) with a gamma family and log link, which simultaneously addresses both violations by modeling the conditional mean on a log scale with a variance proportional to the mean. Similarly, researchers analyzing clinical trial data with repeated measures on each patient should recognize the independence violation and adopt a linear mixed-effects model from the outset, rather than attempting OLS and retrospectively diagnosing correlated residuals.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A colleague tells you, "I ran a regression and the R² is 0.85, so the model is definitely valid." Explain why a high R² does not guarantee that the regression assumptions are met. Which specific assumption violations could produce a misleadingly high R²?

Show answer

R² measures the proportion of variance explained by the model, but it does not assess the distributional properties of the residuals. A model with a high R² can still violate linearity (the relationship may be curvilinear, and a quadratic model might yield an even higher R²), independence (autocorrelated data can inflate R²), normality (skewed residuals do not affect R² directly but invalidate inference), or equal variance (heteroscedasticity biases standard errors even when R² is high). Anscombe's Quartet famously demonstrates that four very different datasets can produce identical R² values. Always check residual plots regardless of R².

Question 2: BASIC CALCULATION

A Durbin–Watson test on a time-series regression with n = 30 observations and k = 2 predictors yields d = 0.95. Using the critical values d_L = 1.28 and d_U = 1.57 at α = 0.05, what do you conclude about the independence assumption?

Show answer

The Durbin–Watson statistic ranges from 0 to 4, with d ≈ 2 indicating no autocorrelation, d < 2 suggesting positive autocorrelation, and d > 2 suggesting negative autocorrelation. Since d = 0.95 is less than

d_L

= 1.28, we reject the null hypothesis of no positive autocorrelation at the 5% level. This means the independence assumption is violated—consecutive residuals are positively correlated, which typically inflates R² and deflates standard errors. The researcher should consider fitting a time-series model (e.g., including lagged terms) or using Newey–West standard errors to correct for autocorrelation.

Question 3: INTERMEDIATE

A researcher regresses BMI on daily caloric intake for n = 200 adults. The residual-versus-fitted plot shows a clear funnel (fan) shape—residuals are more spread out at higher fitted values. The Breusch–Pagan test gives χ² = 12.4, p = 0.0004. (a) Which assumption is violated and why? (b) The researcher proposes taking the log of caloric intake as a fix. Is this the most direct remedy? Suggest an alternative approach and explain when each is preferable.

Show answer

Part (a): The equal variance (homoscedasticity) assumption is violated. The fan shape indicates that residual variance increases with the level of the fitted values, a hallmark of heteroscedasticity. The Breusch–Pagan test confirms this with p = 0.0004, strongly rejecting the null hypothesis of constant variance. Part (b): Transforming the predictor (log of caloric intake) may or may not fix heteroscedasticity because the fan shape relates to the response scale, not the predictor scale. A more direct remedy is to either (i) transform the response—taking log(BMI)—which is a variance-stabilizing transformation when variance increases with the mean, or (ii) use heteroscedasticity-consistent (HC/robust) standard errors, which provide valid inference without altering the model specification. Transformation is preferred when a log-linear relationship is biologically meaningful, while robust SE is preferred when the researcher wants to preserve the original scale for clinical interpretability.

Question 4: APPLIED

An epidemiologist studies the relationship between PM₂.₅ air pollution exposure (μg/m³) and forced expiratory volume (FEV₁, in liters) across 15 monitoring sites, with multiple patients measured at each site. She fits a simple OLS regression of FEV₁ on PM₂.₅. The residual-versus-fitted plot looks fine, and the Q–Q plot is approximately linear. However, a colleague points out that patients within the same monitoring site share unmeasured environmental factors. (a) Which LINE assumption is most likely violated, and why? (b) What modeling approach would you recommend?

Show answer

Part (a): The independence assumption is most likely violated. Patients within the same monitoring site share unmeasured environmental confounders (temperature, humidity, socioeconomic neighborhood effects), creating within-cluster correlation. The residuals for patients at the same site are more similar to each other than to patients at different sites, so Cov(εᵢ, εⱼ) ≠ 0 for patients i and j within the same cluster. This form of clustering cannot be detected from standard residual-versus-fitted plots, which is precisely why understanding the study design is critical. Part (b): A linear mixed-effects model with a random intercept for monitoring site is the appropriate approach: FEV₁ᵢⱼ = β₀ + β₁PM₂.₅ⱼ + uⱼ + εᵢⱼ, where uⱼ ~ N(0, σ²_u) is the site-level random effect. This partitions residual variation into between-site and within-site components, correctly accounting for the clustered structure and producing valid standard errors for the PM₂.₅ effect.

Question 5: CRITICAL THINKING

Consider two scenarios: (A) A Shapiro–Wilk test on residuals from a regression with n = 25 yields p = 0.04. (B) The same test on residuals from a regression with n = 5,000 yields p = 0.001, but the Q–Q plot shows only trivial deviation from normality. In each case, discuss whether you would be concerned about the normality assumption and what action, if any, you would take. How does sample size affect your interpretation of formal normality tests versus graphical diagnostics?

Show answer

Scenario A (n = 25): With a small sample, the normality assumption matters substantially because the Central Limit Theorem provides little protection—the sampling distribution of the coefficient estimators has not yet converged to normality. The significant Shapiro–Wilk result (p = 0.04) combined with small n warrants genuine concern. The researcher should examine the Q–Q plot to characterize the departure (skewness? heavy tails? outliers?) and consider remedies such as a Box–Cox transformation, bootstrapped confidence intervals, or nonparametric alternatives. Scenario B (n = 5,000): With a large sample, formal tests of normality have enormous statistical power and will reject H₀ for trivially small departures that have no practical significance. The Q–Q plot showing only minor deviations is far more informative here—it suggests the residuals are approximately normal enough that, combined with the CLT, inference on coefficients will be reliable. The p = 0.001 result is a case of statistical significance without practical significance. No remedial action is needed. The key insight: formal tests answer 'Is the departure from normality exactly zero?' while graphical diagnostics answer 'Is the departure large enough to matter?' In small samples, both tools are important; in large samples, graphical assessment should take priority over formal tests.

SUMMARY

Lesson Summary

Summary

The validity of ordinary least squares regression rests on four foundational conditions known as the LINE assumptions: linearity (the true relationship between predictors and outcome is linear in the parameters), independence (residuals are uncorrelated with one another), normality (residuals follow a Gaussian distribution), and equal variance (residual variance is constant across all levels of the predictor). These conditions are assessed primarily through residual diagnostic plots—residual-versus-fitted plots for linearity and homoscedasticity, Q–Q plots for normality, and time-order plots or study design review for independence—supplemented by formal tests such as the Breusch–Pagan, Durbin–Watson, and Shapiro–Wilk tests.

When violations are detected, remedial strategies include variable transformations (log, square root, Box–Cox) for non-linearity and non-normality, robust standard errors or weighted least squares for heteroscedasticity, and mixed-effects models or generalized estimating equations for correlated observations. Remember that graphical diagnostics are generally more informative than formal tests, especially in very large or very small samples, and that assumption checking should be viewed as a routine, non-negotiable step in every regression analysis—not an afterthought.