Variance & Standard Deviation — Compute and interpret variance and standard deviation - Biostatistics

SECTION 1

Historical Context & Motivation

The need to measure how data values scatter around a central value is as old as empirical science itself. Early astronomers and surveyors recognized that repeated measurements of the same quantity rarely agreed perfectly, and they sought systematic ways to characterize that disagreement. The formal mathematical machinery we now call variance and standard deviation grew out of this centuries-long quest to separate signal from noise — a quest that remains central to modern biostatistics, where biological variability is both a nuisance and a source of scientific insight.

1733 • De Moivre's Bell Curve

Abraham de Moivre derived the normal curve as an approximation to the binomial distribution, introducing the concept of a 'measure of dispersion' that would later evolve into the standard deviation.

1809 • Gauss and Least Squares

Carl Friedrich Gauss published his method of least squares in Theoria Motus, formalizing the idea of minimizing squared deviations — the conceptual foundation of variance.

1893 • Pearson Coins 'Standard Deviation'

Karl Pearson, working on biometric problems, introduced the term 'standard deviation' in a lecture to the Royal Society, replacing Gauss's cumbersome 'mean error' terminology and anchoring the concept in biological research.

1908 • Student's t-Distribution

William Sealy Gosset (writing as 'Student') published his t-distribution while working at the Guinness brewery, showing that variance estimated from small samples requires a correction factor — the origin of the n − 1 denominator in the sample variance.

1918 • Fisher and ANOVA

Ronald A. Fisher introduced the analysis of variance (ANOVA) in a landmark paper on genetics, partitioning total variance into components attributable to different sources — a framework that revolutionized experimental design in biology and medicine.

From Gauss's planetary orbits to Fisher's crop yields, the unifying question has remained: how spread out are these observations, and what does that spread mean? In biostatistics, the answer shapes everything from confidence intervals to hypothesis tests to clinical trial design. The sections that follow develop the computational and interpretive tools you need to engage with this question rigorously.

SECTION 2

Core Principles & Definitions

Before diving into formulas, it is essential to understand the conceptual architecture that variance and standard deviation rest upon. Both measures quantify dispersion — the degree to which individual data points in a distribution differ from the central tendency, typically the arithmetic mean. A distribution where every observation is identical has zero dispersion; the more the observations scatter, the larger the variance and standard deviation become. Understanding why we square deviations, why we sometimes divide by n versus n − 1, and why we take a square root at the end are the three conceptual pillars of this topic.

Deviation from the Mean

Each observation xᵢ has a deviation defined as xᵢ − x̄. The sum of all deviations equals zero by construction, which is why raw deviations cannot serve as a measure of spread and must be transformed — typically by squaring.

Squaring Eliminates Sign

Squaring each deviation (xᵢ − x̄)² ensures all terms are non-negative and penalizes large deviations more heavily than small ones. The resulting quantity — the sum of squares (SS) — is the raw material from which variance is computed.

Population vs. Sample

When data represent the entire population, we divide SS by N. When data are a sample, we divide by n − 1 (Bessel's correction) to obtain an unbiased estimator of the population variance.

Variance vs. Standard Deviation

Variance is expressed in squared units (e.g., kg²), which complicates interpretation. Taking the positive square root yields the standard deviation, which shares the original measurement units (e.g., kg) and is therefore more intuitive for describing spread.

Sensitivity to Outliers

Because deviations are squared, a single extreme value can dramatically inflate variance. In biostatistics, this sensitivity motivates exploratory checks for outliers and sometimes leads researchers to report robust alternatives such as the interquartile range.

✦

KEY TAKEAWAY

Think of the mean as the balance point of a seesaw and the standard deviation as a measure of how far, on average, the children (data points) are sitting from that balance point. If every child sits right at the center, the seesaw is perfectly still (SD = 0). The more spread out they are, the more the seesaw would wobble — and the larger the standard deviation. Variance is simply that wobble measure before you convert it back into the units everyone understands.

SECTION 3

Visualizing Spread Around the Mean

A powerful way to internalize variance and standard deviation is to see how two distributions with the same mean can look radically different when their spreads differ. The diagram below compares two hypothetical datasets — systolic blood pressure readings from two patient cohorts — both centered at 120 mmHg but exhibiting markedly different variability. The shaded bands represent ±1 and ±2 standard deviations from the mean.

Both curves are centered at 120 mmHg, but Cohort A (σ = 6, solid cyan) clusters tightly around the mean while Cohort B (σ = 15, dashed pink) spreads broadly. The shaded rectangles approximate the ±1 SD bands. About 68% of data in a normal distribution falls within ±1 SD, illustrating why a larger SD implies a wider range of typical observations.

Notice that Cohort A's narrow curve peaks much higher than Cohort B's — the total area under both curves is the same (representing 100% of observations), so a tighter spread forces a taller peak. In clinical practice, you would interpret these distributions very differently: a patient in Cohort A with a reading of 135 mmHg is an extreme outlier (more than 2.5 SDs above the mean), whereas the same reading in Cohort B falls within 1 SD and would be considered unremarkable. This is precisely why reporting a mean without its standard deviation is incomplete — the SD provides essential context for judging whether any individual value is ordinary or exceptional.

SECTION 4

Mathematical Framework

We now formalize the intuition from the previous sections into precise mathematical definitions. All derivations begin with the concept of a deviation — the signed difference between an observation and the mean. Because the sum of deviations is identically zero (a consequence of the mean being the least-squares estimator of location), we square each deviation before averaging. The resulting quantity depends on whether we are describing a known population or estimating from a sample.

POPULATION VARIANCE

σ² = (1/N) Σᵢ₌₁ᴺ (xᵢ − μ)²

σ² = population variance; N = population size; xᵢ = individual observation; μ = population mean. Every member of the population is included, so we divide by N.

POPULATION STANDARD DEVIATION

σ = √[ (1/N) Σᵢ₌₁ᴺ (xᵢ − μ)² ]

The positive square root of the population variance restores the original measurement units. If observations are in mmHg, σ is also in mmHg.

SAMPLE VARIANCE (BESSEL-CORRECTED)

s² = (1/(n − 1)) Σᵢ₌₁ⁿ (xᵢ − x̄)²

s² = sample variance; n = sample size; x̄ = sample mean. Dividing by n − 1 corrects the negative bias that arises because the sample mean x̄ is itself estimated from the data, consuming one degree of freedom.

SAMPLE STANDARD DEVIATION

s = √[ (1/(n − 1)) Σᵢ₌₁ⁿ (xᵢ − x̄)² ]

Although s is a consistent estimator of σ, note that s is actually a slightly biased estimator of σ (unlike s² which is unbiased for σ²). The bias is negligible for moderate to large n.

ℹ

Why n − 1?

The sample mean x̄ is computed from the same data used to calculate deviations. Because the deviations must sum to zero once x̄ is fixed, only n − 1 of them are free to vary — hence n − 1 degrees of freedom. Dividing by n would systematically underestimate the true population variance, especially in small samples common in biostatistics (e.g., pilot clinical trials with n < 30).

A computationally convenient rearrangement, sometimes called the shortcut formula, avoids computing each deviation individually: s² = [Σxᵢ² − (Σxᵢ)²/n] / (n − 1). While algebraically equivalent, this form can introduce rounding errors with large numbers; modern software typically uses the definitional form with careful numerical algorithms.

SECTION 5

Detailed Breakdown — Step-by-Step Computation

To solidify the computational procedure, let us trace through every stage of computing the sample variance and sample standard deviation for a small biostatistical dataset. The diagram below maps the five-step workflow, and the table that follows applies each step to a concrete example of white blood cell (WBC) counts (×10³/μL) from six patients.

The workflow proceeds left-to-right through five stages. The table below the flowchart traces each patient's contribution to the sum of squares (SS = 15.50). Dividing by n − 1 = 5 yields s² = 3.10, and the final standard deviation is s ≈ 1.76 × 10³/μL.

Several features of this computation deserve emphasis. First, observe that Patient 2 (xᵢ = 7.1, very close to x̄ = 7.0) contributes almost nothing to the sum of squares, whereas Patient 5 (xᵢ = 9.5) contributes 6.25 — over 40% of the total. This illustrates the quadratic weighting inherent in variance: values far from the mean dominate. Second, note that the cumulative SS column makes it easy to check arithmetic by verifying that the final entry equals the sum of all individual squared deviations. Finally, the result s ≈ 1.76 means that a 'typical' deviation of a patient's WBC count from the sample mean is roughly 1,760 cells/μL — a clinically meaningful scale of variability for this assay.

SECTION 6

Worked Example — Hemoglobin Levels

A clinical researcher measures hemoglobin concentrations (g/dL) in a sample of eight adult patients enrolled in an anemia screening study. The observed values are: 12.1, 13.4, 11.8, 14.2, 12.9, 13.0, 11.5, 13.1. Compute the sample variance and sample standard deviation, and interpret the result in clinical terms.

Hemoglobin Variability

1
Step 1 — Compute the Sample Mean

Sum all observations: 12.1 + 13.4 + 11.8 + 14.2 + 12.9 + 13.0 + 11.5 + 13.1 = 102.0. Divide by n = 8 to obtain x̄ = 102.0 / 8 = 12.75 g/dL.

x̄ = 12.75 g/dL
2
Step 2 — Compute Deviations

Subtract x̄ from each value: (12.1 − 12.75) = −0.65, (13.4 − 12.75) = 0.65, (11.8 − 12.75) = −0.95, (14.2 − 12.75) = 1.45, (12.9 − 12.75) = 0.15, (13.0 − 12.75) = 0.25, (11.5 − 12.75) = −1.25, (13.1 − 12.75) = 0.35. Verification: the sum of deviations is −0.65 + 0.65 − 0.95 + 1.45 + 0.15 + 0.25 − 1.25 + 0.35 = 0.00 ✓.

Sum of deviations = 0 (expected)
3
Step 3 — Square Each Deviation

0.4225, 0.4225, 0.9025, 2.1025, 0.0225, 0.0625, 1.5625, 0.1225.

Eight squared deviations computed
4
Step 4 — Compute the Sum of Squares and Sample Variance

SS = 0.4225 + 0.4225 + 0.9025 + 2.1025 + 0.0225 + 0.0625 + 1.5625 + 0.1225 = 5.62. Divide by n − 1 = 7: s² = 5.62 / 7 ≈ 0.803 (g/dL)².

s² ≈ 0.803 (g/dL)²
5
Step 5 — Take the Square Root

s = √0.803 ≈ 0.896 g/dL. This means that a typical patient's hemoglobin level in this sample deviates from the sample mean by roughly 0.9 g/dL.

s ≈ 0.90 g/dL
6
Step 6 — Clinical Interpretation

The sample mean of 12.75 g/dL sits within the normal adult range (12–17 g/dL for males, 12–15 g/dL for females). The standard deviation of 0.90 g/dL indicates relatively homogeneous hemoglobin levels: approximately 68% of patients (if the distribution is roughly normal) are expected to fall between 11.85 and 13.65 g/dL. No patient deviates by more than about 1.6 SD from the mean, suggesting no extreme outliers in this small sample.

68% of patients expected between 11.85 and 13.65 g/dL

SECTION 7

Strengths, Limitations & Comparison with Other Measures

Variance and standard deviation are the most widely used measures of dispersion in biostatistics, but they are not the only options. Understanding their strengths and limitations — and when an alternative measure might be more informative — is essential for thoughtful data analysis. The table below compares the standard deviation with two common alternatives: the range and the interquartile range (IQR).

Feature	Standard Deviation (s)	Range	IQR
Uses all data points?	Yes — every observation contributes to the sum of squares	No — only the minimum and maximum	No — only the 25th and 75th percentiles
Sensitivity to outliers	High — squared deviations amplify extreme values	Very high — a single outlier redefines the range	Low — quartiles are resistant to extreme values
Mathematical tractability	Excellent — integrates into inferential methods (t-tests, ANOVA, regression)	Poor — no clean algebraic properties	Moderate — used in non-parametric methods and boxplots
Best used when	Distribution is approximately symmetric/normal and outlier-free	Quick, rough sense of spread is sufficient	Distribution is skewed or contains outliers
Units	Same as the data (variance is in squared units)	Same as the data	Same as the data

Comparison of common measures of dispersion in biostatistics

🔍

WHEN TO CHOOSE WHAT

Think of dispersion measures as different lenses for the same photograph. The standard deviation is a high-resolution, mathematically versatile lens — ideal when the data behave well (symmetric, no extreme outliers). The IQR is a robust, scratch-resistant lens — it may sacrifice some detail but delivers a reliable picture even when the data are messy. The range is a snapshot thumbnail — fast and easy, but too crude for formal analysis. In peer-reviewed biostatistics, reporting mean ± SD is the norm for normally distributed data, while median (IQR) is preferred for skewed distributions.

SECTION 8

Connection to Inferential Statistics

The sample standard deviation is far more than a descriptive summary — it is the engine that powers many of the inferential procedures you will encounter throughout biostatistics. The standard error of the mean (SEM), confidence intervals, t-tests, ANOVA, and regression all depend critically on the sample variance or its derivatives. Understanding how descriptive dispersion connects to inferential reasoning bridges the gap between simply summarizing data and drawing statistically valid conclusions about populations.

Concept	Descriptive Level (This Lesson)	Inferential Extension
Spread measure	s = sample SD quantifies variability among individual observations	SEM = s / √n quantifies uncertainty of the sample mean as an estimator of μ
Interval	x̄ ± 1s captures ~68% of individual data points (for normal data)	x̄ ± t·SEM gives a 95% confidence interval for the population mean μ
Comparison	Comparing SDs across groups informally assesses homogeneity of variance	Levene's or Bartlett's tests formally assess equal variances as an ANOVA assumption
Effect size	s provides scale for understanding raw differences between group means	Cohen's d = (x̄₁ − x̄₂) / s_pooled standardizes the difference in SD units
Relative variability	Coefficient of variation CV = (s / x̄) × 100% allows comparison across different scales	CV is used to assess assay reproducibility and measurement precision in lab settings

How descriptive measures of variability extend into inferential biostatistics

A critical distinction that frequently confuses students is the difference between SD and SEM. The SD describes the variability of individual observations, whereas the SEM describes the precision with which the sample mean estimates the population mean. Because SEM = s / √n, increasing the sample size shrinks the SEM (and thus tightens confidence intervals) without changing the SD. In journal articles, you will frequently see 'mean ± SD' used for descriptive purposes and 'mean ± SEM' or 'mean (95% CI)' used for inferential conclusions. Conflating the two is a common reporting error flagged in biostatistics peer review.

💡

Looking Ahead

The variance concept extends directly into multivariate settings through the variance-covariance matrix, where diagonal elements are individual variances and off-diagonal elements capture linear co-variation between pairs of variables. This matrix is foundational for principal component analysis (PCA), multivariate regression, and mixed-effects models — all workhorses of modern biostatistical analysis.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

Explain why dividing by n − 1 rather than n when computing the sample variance produces an unbiased estimator of σ². In your answer, reference the concept of degrees of freedom and describe what would happen if we used n instead.

Show answer

When we compute the sample mean x̄ from the same data used to calculate deviations, we impose a constraint: the deviations must sum to zero. This means that once n − 1 deviations are known, the last one is determined automatically, leaving only n − 1 independent pieces of information — i.e., n − 1 degrees of freedom. Dividing by n would systematically produce an estimate that is, on average, smaller than the true population variance (a negative bias), because the deviations are measured from the sample mean (which minimizes the sum of squares for that particular sample) rather than from the true population mean. The Bessel correction (dividing by n − 1) inflates the estimate just enough to counteract this bias, yielding E[s²] = σ².

Question 2: BASIC CALCULATION

A researcher records the serum cholesterol levels (mg/dL) for five patients: 190, 210, 205, 195, 200. Compute the sample variance and sample standard deviation.

Show answer

First, compute x̄ = (190 + 210 + 205 + 195 + 200) / 5 = 1000 / 5 = 200 mg/dL. Next, compute deviations: −10, 10, 5, −5, 0. Square each: 100, 100, 25, 25, 0. Sum of squares SS = 250. Divide by n − 1 = 4: s² = 250 / 4 = 62.5 (mg/dL)². Finally, s = √62.5 ≈ 7.91 mg/dL. The typical deviation of a patient's cholesterol from the group mean is approximately 7.91 mg/dL.

Question 3: INTERMEDIATE

Two laboratory assays measure fasting blood glucose (mg/dL) in the same 10 patients. Assay A yields x̄ = 95.0, s = 8.2; Assay B yields x̄ = 94.5, s = 3.1. Both means are clinically equivalent, but the standard deviations differ. (a) Compute the coefficient of variation (CV) for each assay. (b) Which assay demonstrates better measurement precision, and why?

Show answer

(a) CV_A = (8.2 / 95.0) × 100% ≈ 8.63%; CV_B = (3.1 / 94.5) × 100% ≈ 3.28%. (b) Assay B demonstrates substantially better measurement precision because its CV is less than half that of Assay A. The CV standardizes the standard deviation relative to the mean, making it a dimensionless measure of relative variability. A lower CV indicates tighter clustering of repeated measurements around the central value, which is a key desideratum in clinical chemistry. In practice, regulatory agencies often require CVs below 5–10% for acceptable assay reproducibility, so Assay B meets this threshold comfortably while Assay A is at the border.

Question 4: APPLIED

A clinical trial enrolls 30 patients and measures their systolic blood pressure (SBP) before and after a new antihypertensive drug. The mean change (post − pre) is −12.3 mmHg with a sample standard deviation of changes

s_d

= 6.8 mmHg. (a) Compute the standard error of the mean change. (b) Construct an approximate 95% confidence interval for the population mean change μ_d. (c) Does this interval suggest the drug has a statistically significant effect at α = 0.05?

Show answer

(a) SEM =

s_d

/ √n = 6.8 / √30 = 6.8 / 5.477 ≈ 1.24 mmHg. (b) For 29 degrees of freedom, the critical t-value for a 95% CI is approximately 2.045. The 95% CI is x̄_d ± t × SEM = −12.3 ± 2.045 × 1.24 = −12.3 ± 2.54, yielding (−14.84, −9.76) mmHg. (c) Yes, the drug has a statistically significant effect: the entire 95% confidence interval lies below zero, meaning we can reject the null hypothesis H₀: μ_d = 0 at α = 0.05. The data provide strong evidence that the drug reduces SBP by between approximately 9.8 and 14.8 mmHg on average. Note how the sample SD of the changes (6.8 mmHg) quantified patient-to-patient variability in treatment response, while the SEM (1.24 mmHg) quantified uncertainty about the mean effect.

Question 5: CRITICAL THINKING

A biostatistician discovers that the distribution of hospital length-of-stay (LOS) in days for a sample of 200 trauma patients is heavily right-skewed with x̄ = 5.8 days, s = 7.2 days, median = 3.0 days, and IQR = 1.5 to 6.0 days. (a) What does the fact that s > x̄ tell you about the shape of the distribution? (b) Would you report mean ± SD or median (IQR) for this dataset, and why? (c) If the researcher insists on using the mean and SD, what transformation might make the data more symmetric, and how would you back-transform the results for clinical interpretation?

Show answer

(a) When the SD exceeds the mean for a variable that cannot take negative values (length of stay ≥ 0), it signals extreme positive skew. The interval x̄ − 1s = 5.8 − 7.2 = −1.4 extends into impossible negative territory, confirming that the normal-distribution assumptions underlying mean ± SD reporting are violated. (b) The appropriate summary is median (IQR) = 3.0 (1.5–6.0) days. This is more informative because the median is resistant to the influence of a few patients with very long stays who inflate the mean, and the IQR captures the central 50% of the distribution without being distorted by the long right tail. Most biostatistics style guides (e.g., STROBE, CONSORT) recommend median (IQR) for skewed continuous outcomes. (c) A natural logarithm transformation (ln LOS) often symmetrizes right-skewed positive data like hospital LOS. The researcher would compute x̄_ln and s_ln on the log-transformed data, then back-transform by exponentiation: exp(x̄_ln) gives the geometric mean in days, and exp(x̄_ln ± 1.96 × s_ln) gives an approximate 95% reference range. This approach preserves the mathematical convenience of the mean and SD while respecting the distributional properties of the data.

SUMMARY

Summary

Variance (σ² or s²) measures the average of the squared deviations from the mean, while the standard deviation (σ or s) is its positive square root, restoring the original measurement units. For sample data, we divide the sum of squares by n − 1 (Bessel's correction) to obtain an unbiased estimator of the population variance. This correction accounts for the loss of one degree of freedom when x̄ is estimated from the data.

In biostatistics, the SD serves dual roles: descriptively, it quantifies individual-level variability; inferentially, it feeds into the standard error (SEM = s/√n), confidence intervals, and hypothesis tests. Reporting conventions dictate mean ± SD for approximately normal data and median (IQR) for skewed distributions. Because variance is sensitive to outliers (due to squaring), exploratory screening of data quality should always precede computation. Mastering these measures provides the foundation for every parametric inferential method you will encounter in biostatistics.