Standard Distribution Results — Recognize and apply standard distribution results (chi-square, t, F) - Statistics Graduate Level

SECTION 1

Historical Context & Motivation

Statistical inference rests on knowing the sampling distribution of a test statistic under a null hypothesis, and three families of distributions—the chi-square, the Student t, and the F—form the backbone of virtually all classical parametric testing. Each arose from distinct practical problems: estimating error in astronomical observations, ensuring quality in small-batch brewing, and comparing variances across experimental treatments. Understanding their historical genesis reveals why these distributions appear so persistently across statistics, and why their derivations from the normal distribution provide a unified theoretical framework for inference. The story spans roughly a century, from the early work of Helmert and Pearson through the innovations of Gosset and Fisher, each contribution building on the algebraic structure of sums of squared standard normals.

1876 • Helmert's Chi-Square Foundation

Friedrich Robert Helmert derived the distribution of sums of squared standard normal variables while analyzing geodetic measurement errors, laying the groundwork for what Pearson would later formalize as the chi-square distribution.

1900 • Pearson's Goodness-of-Fit Test

Karl Pearson published his celebrated chi-square goodness-of-fit test, establishing the χ² statistic as the principal tool for comparing observed frequencies against theoretical expectations in categorical data analysis.

1908 • Gosset's t-Distribution

William Sealy Gosset, publishing under the pseudonym "Student" while employed at the Guinness Brewery, derived the t-distribution for inference with small samples where the population variance is unknown.

1924 • Fisher's F-Distribution

Ronald A. Fisher developed the analysis of variance (ANOVA) framework and introduced the F-distribution as the ratio of two independent chi-square variables divided by their respective degrees of freedom, enabling comparison of multiple group means simultaneously.

1935 • Fisher's Design of Experiments

Fisher's landmark text consolidated the theoretical connections among the chi-square, t, and F distributions and demonstrated their central role in experimental design, cementing these distributions as the standard toolkit for parametric inference.

The central question these distributions collectively address is this: given that we observe data from a normal population, what is the exact distribution of functions of sample statistics—such as sample variances, standardized means, and variance ratios—that we use for inference? Answering this question requires tracing how sums and ratios of squared normals generate entirely new distribution families, each indexed by degrees of freedom that encode the dimension of the underlying quadratic form.

SECTION 2

Core Principles & Definitions

All three standard distributions arise from a single generative mechanism: independent standard normal random variables combined through sums of squares and ratios. The chi-square distribution emerges as the sum of squared independent standard normals. The t-distribution arises when a standard normal is divided by the square root of an independent chi-square divided by its degrees of freedom. The F-distribution materializes as the ratio of two independent chi-square random variables, each scaled by their respective degrees of freedom. These relationships are not coincidental but reflect the deep algebraic structure of quadratic forms in multivariate normal theory.

Chi-Square (χ²) Distribution

If Z₁, Z₂, …, Z_k are independent standard normals, then Q = ΣZ²ᵢ follows a χ² distribution with k degrees of freedom. It is supported on [0, ∞), right-skewed, and additive: independent chi-squares sum to a chi-square.

Student's t-Distribution

If Z ~ N(0,1) and V ~ χ²(ν) are independent, then T = Z / √(V/ν) follows a t-distribution with ν degrees of freedom. It is symmetric about zero, bell-shaped, and heavier-tailed than the standard normal, converging to N(0,1) as ν → ∞.

F-Distribution

If U ~ χ²(d₁) and V ~ χ²(d₂) are independent, then F = (U/d₁) / (V/d₂) follows an F-distribution with (d₁, d₂) degrees of freedom. It is right-skewed, supported on [0, ∞), and satisfies F ~ 1/F when the df are swapped.

Interconnection Principle

The three distributions are algebraically linked: T² ~ F(1, ν), and χ²(k)/k is F(k, ∞) in the limit. These identities mean that understanding one distribution immediately gives insight into the others, and proofs often reduce to moment-generating function (MGF) or transformation arguments.

✦

KEY TAKEAWAY

Think of the standard normal as the fundamental building block—like a single LEGO brick. The chi-square distribution is a tower built by stacking squared bricks. The t-distribution is a ratio that measures how tall a single brick stands relative to the average height of the tower. The F-distribution compares the average height of two different towers. Every classical parametric test is, at its core, an exercise in assembling and comparing these constructions.

SECTION 3

Visual Explanation — Density Comparison

The chi-square density shifts rightward and becomes more symmetric as the degrees of freedom k increase. For k = 1, the density has a pole at zero; by k = 10, the distribution approximates a skewed bell, consistent with the central limit theorem prediction that χ²(k) approaches normality for large k.

The diagram above illustrates a fundamental property of the chi-square family: the shape parameter k (degrees of freedom) governs both the mode and the skewness. When k = 1, the density is a monotonically decreasing function with a singularity at the origin, reflecting the fact that squaring a single standard normal concentrates probability mass near zero. As k grows, the mode shifts to k − 2 and the distribution becomes progressively more Gaussian, a direct consequence of the central limit theorem applied to the sum Z₁² + Z₂² + ⋯ + Z_k. This visual intuition is essential when interpreting chi-square test statistics: a goodness-of-fit statistic with 3 degrees of freedom has a very different rejection region geometry than one with 30.

SECTION 4

Mathematical Framework — Definitions, Densities & MGFs

Chi-Square Distribution

CHI-SQUARE DEFINITION

Q = Z₁² + Z₂² + ⋯ + Z_k where Zᵢ ~iid N(0,1) ⟹ Q ~ χ²(k)

The moment-generating function is M_Q(t) = (1 − 2t)^−k/2 for t < 1/2, from which E[Q] = k and Var(Q) = 2k follow immediately by differentiation.

CHI-SQUARE DENSITY

f(x; k) = [1 / (2^(k/2) Γ(k/2))] × x^(k/2 − 1) × e^(−x/2), x > 0

Here Γ(·) is the gamma function. Note that χ²(k) is simply the Gamma(k/2, 2) distribution, connecting it to the broader exponential family.

Student's t-Distribution

T-DISTRIBUTION DEFINITION

T = Z / √(V/ν) where Z ~ N(0,1), V ~ χ²(ν), Z ⊥ V ⟹ T ~ t(ν)

The density is f(t; ν) = [Γ((ν+1)/2) / (√(νπ) Γ(ν/2))] × (1 + t²/ν)^−(ν+1)/2. The t-distribution has no MGF for ν ≤ 1 (infinite first moment for ν = 1), but E[T] = 0 for ν > 1 and Var(T) = ν/(ν − 2) for ν > 2.

F-Distribution

F-DISTRIBUTION DEFINITION

F = (U/d₁) / (V/d₂) where U ~ χ²(d₁), V ~ χ²(d₂), U ⊥ V ⟹ F ~ F(d₁, d₂)

The density involves a Beta-type kernel: f(x; d₁, d₂) = [1/B(d₁/2, d₂/2)] × (d₁/d₂)^d₁/2 × x^{d₁/2 − 1} × (1 + d₁x/d₂)^{−(d₁+d₂)/2}. The mean is d₂/(d₂ − 2) for d₂ > 2.

💡

Critical Identity

If T ~ t(ν), then T² ~ F(1, ν). This identity connects two-sided t-tests to one-way ANOVA F-tests with a single numerator degree of freedom, and its proof follows directly by squaring the t-definition: T² = Z²/(V/ν) = (χ²(1)/1)/(χ²(ν)/ν) ~ F(1, ν).

SECTION 5

Key Derivations & Structural Results

A rigorous understanding of these distributions requires working through the fundamental derivation that connects the sample mean and sample variance from a normal population. This result, due to Fisher, is the cornerstone upon which the t-test and all ANOVA procedures rest.

Fisher's Theorem (Cochran's Theorem Application)

Let X₁, X₂, …, X_n be iid N(μ, σ²). Define the sample mean X̄ = (1/n)ΣXᵢ and the sample variance S² = [1/(n−1)]Σ(Xᵢ − X̄)². Then (i) X̄ ~ N(μ, σ²/n), (ii) (n−1)S²/σ² ~ χ²(n−1), and (iii) X̄ and S² are independent. The independence assertion is the non-obvious part and follows from Cochran's theorem: the total sum of squares decomposes into orthogonal components in the n-dimensional space of observations. The projection onto the one-dimensional subspace spanned by 1 = (1,1,…,1)ᵀ gives nX̄², while the projection onto the (n−1)-dimensional orthogonal complement gives Σ(Xᵢ − X̄)². Since these projections are onto complementary subspaces, the resulting quadratic forms are independent chi-squares by the multivariate normal rotation argument.

Flowchart showing the construction of the chi-square, t, and F distributions from independent standard normal random variables, together with their interconnections and primary applications.

Deriving the Sample t-Statistic

From Fisher's theorem, (X̄ − μ)/(σ/√n) ~ N(0,1) and (n−1)S²/σ² ~ χ²(n−1), independently. Forming the t-ratio: T = [(X̄ − μ)/(σ/√n)] / √[(n−1)S²/σ²/(n−1)] = (X̄ − μ)/(S/√n). The σ cancels, yielding a pivotal quantity—a function of the data and the parameter μ whose distribution does not depend on any unknown parameters. This pivot is the basis of the one-sample t-test and the t-confidence interval for the mean.

Additivity of Chi-Squares and the F-Statistic in ANOVA

In one-way ANOVA with g groups and nᵢ observations per group, the total sum of squares SST = Σᵢⱼ(Xᵢⱼ − X̄..)² decomposes as SSB + SSW, where SSB (between groups) has g−1 degrees of freedom and SSW (within groups) has N − g degrees of freedom. Under H₀ (all group means equal), Cochran's theorem guarantees that SSB/σ² ~ χ²(g−1) and SSW/σ² ~ χ²(N−g) independently, so F = [SSB/(g−1)] / [SSW/(N−g)] ~ F(g−1, N−g). Large values of F indicate that between-group variability exceeds within-group variability beyond what chance alone would produce.

SECTION 6

Worked Example — One-Sample t-Test with Connection to F

A quality engineer collects n = 16 tensile-strength measurements from a new alloy. The sample mean is X̄ = 452 MPa and the sample standard deviation is S = 18 MPa. Test whether the population mean differs from the specification value μ₀ = 440 MPa at the α = 0.05 significance level, then verify the T² = F identity.

One-Sample t-Test and F-Identity Verification

1
Step 1 — State Hypotheses and Identify Distribution

H₀: μ = 440 vs H₁: μ ≠ 440 (two-sided). Under H₀ and normality of the data, the test statistic T = (X̄ − μ₀)/(S/√n) follows a t-distribution with ν = n − 1 = 15 degrees of freedom.
2
Step 2 — Compute the Test Statistic

T = (452 − 440) / (18/√16) = 12 / (18/4) = 12 / 4.5 = 2.667.

T = 2.667
3
Step 3 — Determine the Critical Value

For a two-sided test at α = 0.05 with 15 df, the critical values from the t-table are t_0.025,15 = ±2.131. Since |T| = 2.667 > 2.131, we reject H₀.

Reject H₀ at α = 0.05; the mean tensile strength differs significantly from 440 MPa.
4
Step 4 — Compute the p-value

Using a t-distribution calculator (or software), P(|T₁₅| > 2.667) ≈ 2 × 0.0088 = 0.0176. Since p = 0.0176 < 0.05, this confirms rejection.

p ≈ 0.0176
5
Step 5 — Verify the T² = F(1, 15) Identity

T² = (2.667)² = 7.111. Under the identity T² ~ F(1, ν), the equivalent F-test has F = 7.111 with (1, 15) degrees of freedom. The critical value F_0.05,1,15 = (2.131)² = 4.543. Since 7.111 > 4.543, we again reject—exactly as the t-test concluded. The p-values are identical, confirming the algebraic equivalence of the two-sided t-test and the F-test with one numerator degree of freedom.

T² = 7.111 = F(1, 15) statistic; both tests yield p ≈ 0.0176

SECTION 7

Comparison of Properties & Common Pitfalls

Property	χ²(k)	t(ν)	F(d₁, d₂)
Support	[0, ∞)	(−∞, ∞)	[0, ∞)
Symmetry	Right-skewed	Symmetric about 0	Right-skewed
Mean	k	0 (ν > 1)	d₂/(d₂ − 2) (d₂ > 2)
Variance	2k	ν/(ν − 2) (ν > 2)	2d₂²(d₁+d₂−2) / [d₁(d₂−2)²(d₂−4)] (d₂ > 4)
MGF Exists?	Yes, $(1−2t)^{−k/2}$	No (use CF instead)	No
Additivity	Yes: χ²(a)+χ²(b)=χ²(a+b)	No	No
Limiting Form	≈ N(k, 2k) as k → ∞	→ N(0,1) as ν → ∞	d₁F → χ²(d₁) as d₂ → ∞

Comparative properties of the three standard sampling distributions

⚠

COMMON PITFALLS

The most frequent error in applying these distributions is violating the independence assumption. For instance, using a chi-square result for (n−1)S²/σ² requires that S² be independent of X̄, which holds only under normality. In non-normal populations, Cochran's theorem fails, and bootstrap or permutation methods may be needed. Another common pitfall is confusing degrees of freedom in nested ANOVA or regression contexts—always trace each df back to the dimension of the quadratic form's subspace.

SECTION 8

Connection to Advanced Theory

The standard distribution results extend naturally into several advanced areas of statistical theory. The noncentral versions of all three distributions arise when the null hypothesis is false. For example, if X₁, …, X_n ~ N(μ, σ²) and we test H₀: μ = μ₀ when in fact μ ≠ μ₀, then (X̄ − μ₀)/(S/√n) follows a noncentral t-distribution with noncentrality parameter δ = (μ − μ₀)/(σ/√n). Power analysis for t-tests, F-tests in ANOVA, and regression tests all require these noncentral distributions.

Standard (Central)	Noncentral Extension	Use Case
χ²(k)	χ²(k, λ) where λ = Σμᵢ²	Power of goodness-of-fit and variance tests
t(ν)	t(ν, δ) where δ = (μ−μ₀)/(σ/√n)	Sample size determination for t-tests
F(d₁, d₂)	F(d₁, d₂, λ) where λ = Σnᵢ(μᵢ−μ̄)²/σ²	Power of ANOVA and regression tests

Central vs. noncentral distribution extensions

Beyond noncentrality, these distributions connect to the multivariate theory through Wishart matrices (the multivariate generalization of the chi-square) and Hotelling's T² (the multivariate generalization of Student's t). In Bayesian statistics, conjugate prior families for normal likelihoods—the inverse-chi-square for variance and the normal-inverse-chi-square for joint (μ, σ²) inference—are direct descendants of these sampling distributions. The Wilks' lambda test in MANOVA generalizes the F-test to multiple response variables, and its null distribution can be expressed in terms of products of independent Beta random variables, each related to F-statistics through the Beta-F transformation F = (β⁻¹ − 1) × (d₂/d₁).

ℹ

Looking Ahead

In likelihood ratio testing, the statistic −2 ln(Λ) is asymptotically χ²(r) under H₀, where r is the difference in dimension between the full and reduced parameter spaces (Wilks' theorem). This connects the finite-sample chi-square results studied here to large-sample theory and provides a unified framework for testing nested models across regression, GLMs, and mixed models.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

Explain why the t-distribution has heavier tails than the standard normal distribution. Your explanation should reference the ratio structure of the t-statistic and the role of the chi-square denominator.

Show answer

The t-statistic T = Z/√(V/ν) involves dividing a standard normal Z by the square root of an independent chi-square divided by its degrees of freedom. When V/ν is smaller than its expected value of 1 (which happens with non-negligible probability, especially for small ν), the ratio T becomes inflated, producing extreme values more frequently than a standard normal would. Conversely, when V/ν is large, T is shrunk toward zero. This additional source of variability from the random denominator inflates the tails of the distribution. As ν → ∞, V/ν converges in probability to 1 by the law of large numbers, and the extra variability disappears, so t(ν) → N(0,1).

Question 2: BASIC CALCULATION

Let X₁, X₂, …, X₂₅ be iid N(50, 9). Compute the probability P((n−1)S²/σ² > 36.42) using the chi-square distribution.

Show answer

Here n = 25 and σ² = 9, so (n−1)S²/σ² = 24S²/9 ~ χ²(24). We need P(χ²(24) > 36.42). Consulting a chi-square table or using software, the 0.05 upper critical value for χ²(24) is 36.415. Since 36.42 is essentially equal to this critical value, P(χ²(24) > 36.42) ≈ 0.05. This means there is approximately a 5% chance that the scaled sample variance exceeds 36.42 when sampling from the specified normal population.

Question 3: INTERMEDIATE

Two independent samples from normal populations yield S₁² = 24.5 (n₁ = 10) and S₂² = 8.7 (n₂ = 13). Test H₀: σ₁² = σ₂² versus H₁: σ₁² ≠ σ₂² at α = 0.10 using the F-test.

Show answer

The test statistic is F = S₁²/S₂² = 24.5/8.7 ≈ 2.816, with (n₁−1, n₂−1) = (9, 12) degrees of freedom. For a two-sided test at α = 0.10, we need the upper 0.05 and lower 0.05 critical values. From tables, F₀.₀₅,₉,₁₂ ≈ 2.80 and F₀.₉₅,₉,₁₂ = 1/F₀.₀₅,₁₂,₉ ≈ 1/3.07 ≈ 0.326. Since F = 2.816 > 2.80, we are just barely in the rejection region, so we reject H₀ at the 10% level and conclude there is evidence that the population variances differ. The test relies critically on the assumption that both populations are normal; moderate departures from normality can inflate the Type I error rate of the F-test for variance equality.

Question 4: APPLIED

In a one-way ANOVA with g = 4 groups and n = 8 observations per group, the between-group mean square is MSB = 120 and the within-group mean square is MSW = 30. Compute the F-statistic, determine whether to reject H₀ at α = 0.01, and estimate the noncentrality parameter λ assuming the observed MSB reflects the true effect.

Show answer

The F-statistic is F = MSB/MSW = 120/30 = 4.00 with (g−1, N−g) = (3, 28) degrees of freedom. The critical value F₀.₀₁,₃,₂₈ ≈ 4.57 from tables. Since 4.00 < 4.57, we fail to reject H₀ at α = 0.01 (though we would reject at α = 0.05 since F₀.₀₅,₃,₂₈ ≈ 2.95). For the noncentrality parameter, under the alternative E[MSB] = σ² + nΣ(μᵢ−μ̄)²/(g−1), so Σnᵢ(μᵢ−μ̄)²/σ² ≈ (g−1)(MSB−MSW)/MSW × (MSW/MSW) but more directly, λ = (g−1)(F−1) × n/g... A simpler estimator is λ̂ = d₁(F−1) = 3(4−1) = 9, giving an estimated noncentrality of λ = 9 for power calculations.

Question 5: CRITICAL THINKING

Prove that if T ~ t(ν), then T² ~ F(1, ν). State clearly where you use the independence of Z and V in the definition T = Z/√(V/ν), and discuss what happens when this independence assumption fails.

Show answer

Proof: Let T = Z/√(V/ν) where Z ~ N(0,1) and V ~ χ²(ν) are independent. Then T² = Z²/(V/ν). Since Z ~ N(0,1), we have Z² ~ χ²(1). Independence of Z and V implies independence of Z² and V (since Z² is a function of Z alone). Therefore T² = (Z²/1)/(V/ν) = (U/d₁)/(V/d₂) where U = Z² ~ χ²(1), d₁ = 1, d₂ = ν, and U ⊥ V. By the definition of the F-distribution, T² ~ F(1, ν). The independence of Z and V is essential at two points: first, to ensure Z² and V are independent (required for the F-definition), and second, to guarantee that Z² ~ χ²(1) is unaffected by conditioning on V. If Z and V were dependent—as might occur if the normality assumption fails and the sample mean and sample variance become correlated—then T² would not follow an F-distribution, and the resulting tests would have incorrect size. In practice, this manifests as inflated Type I error rates when applying t-tests to heavily skewed or heavy-tailed data with small samples.

SUMMARY

Summary & Review

Summary

The three standard sampling distributions form a tightly interconnected family built from independent standard normal random variables. The chi-square distribution χ²(k) arises as the sum of k squared standard normals and governs inference about population variances and goodness-of-fit testing. The Student's t-distribution t(ν) emerges when a standard normal is divided by the square root of an independent chi-square scaled by its degrees of freedom, providing the pivotal quantity for inference about a normal mean with unknown variance. The F-distribution F(d₁, d₂) is the ratio of two independent scaled chi-squares and underpins ANOVA, regression significance tests, and variance comparisons.

The central structural result connecting them is T² ~ F(1, ν), which unifies two-sided t-tests with one-numerator-df F-tests. Cochran's theorem guarantees the independence of the sample mean and sample variance under normality, making these distribution results valid for exact inference. Extensions to noncentral distributions enable power analysis and sample size determination, while multivariate generalizations (Wishart, Hotelling's T²) carry the same algebraic structure into higher dimensions. Mastery of these results equips you to derive test statistics from first principles, verify their null distributions, and recognize when assumptions break down.