Chi-Square Goodness-of-Fit - College Statistics

SECTION 1

Historical Context & Motivation

Statistical inference for continuous data was well established by the late nineteenth century, but researchers lacked a rigorous method for testing hypotheses about categorical data — data that fall into discrete groups rather than lying on a numerical continuum. When a biologist counted phenotypes, or a sociologist tallied survey responses, the available tools for assessing whether those counts conformed to a theoretical model were crude at best. The breakthrough came from Karl Pearson, who in 1900 introduced a test statistic that elegantly quantified the discrepancy between observed and expected frequencies. His chi-square goodness-of-fit test became one of the most widely used inferential procedures in the sciences, social sciences, and industry.

1857 • Mendel's Pea Experiments

Gregor Mendel recorded phenotypic ratios in pea plants and compared them to predicted 3:1 and 9:3:3:1 ratios, but lacked a formal test of how well his data fit the theoretical expectations.

1900 • Pearson's Chi-Square Paper

Karl Pearson published 'On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can Be Reasonably Supposed to Have Arisen from Random Sampling,' establishing the χ² test statistic and its approximate sampling distribution.

1922 • Fisher's Correction

R. A. Fisher clarified the correct degrees of freedom when parameters are estimated from the data, refining Pearson's original framework and sparking an important methodological debate.

1934 • Neyman–Pearson Framework

Jerzy Neyman and Egon Pearson embedded the goodness-of-fit test within their broader hypothesis-testing framework, formalizing concepts of Type I error, power, and critical regions for the χ² statistic.

The central question Pearson's test addresses is deceptively simple: given a set of observed counts across several categories, can we conclude that the underlying population distribution matches a specific theoretical distribution, or is the discrepancy large enough to reject that claim? This question arises constantly — testing whether a die is fair, whether genetic offspring ratios follow Mendelian predictions, whether customer preferences are uniformly distributed, or whether defect types occur at historically established rates. The chi-square goodness-of-fit test provides a principled, quantitative answer.

SECTION 2

Core Principles & Definitions

The chi-square goodness-of-fit test rests on a small set of foundational ideas that connect observed data to theoretical expectations. Understanding these principles is essential before examining the formula or interpreting output, because each one constrains when and how the test may be legitimately applied.

Observed vs. Expected Frequencies

The test compares observed counts (O) from sample data to expected counts (E) derived from the hypothesized distribution. Expected counts are computed as E = n × p, where n is the total sample size and p is the hypothesized proportion for each category.

Null and Alternative Hypotheses

H₀ states that the population distribution of the categorical variable matches the specified distribution. Hₐ states that at least one category's proportion differs from the specified value. The test is inherently non-directional (two-sided in spirit).

The χ² Test Statistic

The statistic aggregates squared deviations between observed and expected counts, each normalized by the expected count. Large values signal poor fit; small values suggest the data are consistent with the hypothesized distribution.

Degrees of Freedom

With k categories and no parameters estimated from the data, df = k − 1. Each estimated parameter reduces df by one. The chi-square reference distribution with df degrees of freedom is used to obtain the p-value.

Conditions for Valid Inference

Three conditions must hold: (1) the data come from a random sample or randomized experiment, (2) observations are independent, and (3) every expected count is at least 5. When expected counts are small, categories may be combined.

✦

KEY TAKEAWAY

Think of the goodness-of-fit test like a quality-control inspector on a production line. The inspector has a blueprint (the expected distribution) and measures the finished product (the observed data). Small deviations are expected due to natural variation, but if the product deviates too far from the blueprint, the inspector flags it. The χ² statistic quantifies 'how far is too far' by accumulating every category's normalized squared deviation into a single number, and the chi-square distribution tells us how likely that number is under normal variation alone.

SECTION 3

Visual Explanation

A bar chart overlaying observed and expected frequencies is the most intuitive way to see how well data fit a hypothesized distribution. The diagram below shows a scenario in which a die is rolled 120 times and the observed counts are compared against the uniform expectation of 20 per face. Notice that some faces exceed the expected count while others fall short; the chi-square statistic measures whether these deviations, taken together, are larger than chance alone would produce.

Observed frequencies (solid cyan bars) versus expected frequencies (dashed violet bars) for 120 rolls of a die. The dashed pink line marks the uniform expectation of 20 rolls per face. The chi-square statistic aggregates the squared differences between each pair of bars, normalized by the expected count.

In the diagram above, Face 4 has the largest positive deviation (O − E = 6) and Face 6 has the largest negative deviation (O − E = −6). Yet no single bar tells the whole story; the test aggregates information from all six categories into one summary statistic. Because the deviations are squared before summing, positive and negative deviations contribute equally, and larger deviations receive disproportionately more weight. The normalization by E ensures that a deviation of 6 when E = 20 is treated differently from a deviation of 6 when E = 200.

SECTION 4

Mathematical Framework

The mathematical backbone of the goodness-of-fit test is elegant in its simplicity. A single formula captures the intuition that we should penalize large deviations from expectation while accounting for the scale of each expected count. Below, we develop the test statistic, its sampling distribution, and the decision rule.

CHI-SQUARE TEST STATISTIC

χ² = Σ (Oᵢ − Eᵢ)² / Eᵢ

where Oᵢ = observed count in category i, Eᵢ = expected count in category i (Eᵢ = n × pᵢ), and the summation runs over all k categories. The statistic is always non-negative; χ² = 0 only when every Oᵢ equals Eᵢ exactly.

EXPECTED COUNT

Eᵢ = n × pᵢ

where n is the total sample size and pᵢ is the hypothesized probability for category i under H₀. The expected counts must sum to n, which is guaranteed because the pᵢ values sum to 1.

SAMPLING DISTRIBUTION & DEGREES OF FREEDOM

χ² ~ χ²(df), where df = k − 1 − m

Under H₀ and given that the expected count conditions are met, the test statistic follows an approximate chi-square distribution with df degrees of freedom. Here k is the number of categories and m is the number of parameters estimated from the data. In the simplest case (all proportions fully specified by H₀), m = 0 and df = k − 1.

DECISION RULE

Reject H₀ if χ² ≥ χ²(α, df) or equivalently if p-value ≤ α

The test is always right-tailed. Large values of χ² indicate poor fit between the data and the hypothesized distribution. The critical value χ²(α, df) is obtained from chi-square distribution tables or software, and α is the pre-specified significance level (commonly 0.05).

ℹ

Why Right-Tailed?

Every squared deviation contributes a non-negative term to χ². When the data agree with H₀, each (Oᵢ − Eᵢ)² / Eᵢ is small, and the total statistic is small. When the data disagree with H₀, at least some deviations are large, pushing χ² into the right tail. There is no concept of a 'too good' fit being evidence against H₀ in standard practice (although Fisher himself noted suspiciously small χ² values in some of Mendel's data).

SECTION 5

The Chi-Square Distribution & Critical Regions

To convert the computed χ² statistic into a decision, we need the chi-square distribution, a family of right-skewed probability distributions indexed by degrees of freedom. As df increases, the distribution shifts rightward, its mean increases (mean = df), and its shape becomes more symmetric. For the goodness-of-fit test, the relevant distribution has df = k − 1, and the rejection region lies entirely in the right tail. The diagram below illustrates the chi-square density curve for df = 5, with the critical region shaded at α = 0.05.

The chi-square density curve for df = 5. The red-shaded region to the right of χ² = 11.07 represents the critical region at α = 0.05. Any computed test statistic falling in this region leads to rejection of H₀. The mean of the distribution equals the degrees of freedom (here, 5).

df	α = 0.10	α = 0.05	α = 0.01
1	2.706	3.841	6.635
2	4.605	5.991	9.210
3	6.251	7.815	11.345
4	7.779	9.488	13.277
5	9.236	11.070	15.086
6	10.645	12.592	16.812

Selected chi-square critical values for common significance levels

As the table illustrates, the critical value increases with both the degrees of freedom and the stringency of the significance level. For a given df, moving from α = 0.10 to α = 0.01 roughly doubles the critical threshold, reflecting the higher evidentiary bar required to declare statistical significance at more demanding levels.

SECTION 6

Worked Example

A genetics researcher crosses two heterozygous plants and observes the phenotypes of 200 offspring. According to Mendelian theory, the expected phenotypic ratio is 9:3:3:1 across four categories (Round-Yellow, Round-Green, Wrinkled-Yellow, Wrinkled-Green). The observed counts are 112, 48, 27, and 13 respectively. Does the data support the Mendelian model at the α = 0.05 significance level?

Testing Mendelian Ratios (9:3:3:1)

1
Step 1 — State the Hypotheses

H₀: The population distribution of phenotypes follows the 9:3:3:1 ratio, i.e., p₁ = 9/16, p₂ = 3/16, p₃ = 3/16, p₄ = 1/16. Hₐ: At least one phenotype proportion differs from the Mendelian prediction.
2
Step 2 — Compute Expected Counts

With n = 200: E₁ = 200 × (9/16) = 112.5, E₂ = 200 × (3/16) = 37.5, E₃ = 200 × (3/16) = 37.5, E₄ = 200 × (1/16) = 12.5. All expected counts exceed 5, so the condition for the chi-square approximation is satisfied.

E = (112.5, 37.5, 37.5, 12.5)
3
Step 3 — Compute the χ² Statistic

χ² = (112 − 112.5)²/112.5 + (48 − 37.5)²/37.5 + (27 − 37.5)²/37.5 + (13 − 12.5)²/12.5 = (−0.5)²/112.5 + (10.5)²/37.5 + (−10.5)²/37.5 + (0.5)²/12.5 = 0.25/112.5 + 110.25/37.5 + 110.25/37.5 + 0.25/12.5 = 0.0022 + 2.94 + 2.94 + 0.02 = 5.9022

χ² ≈ 5.90
4
Step 4 — Determine Degrees of Freedom

There are k = 4 categories and no parameters were estimated from the data (all proportions were specified by H₀), so df = k − 1 = 4 − 1 = 3.

df = 3
5
Step 5 — Find the Critical Value and Make a Decision

From the chi-square table (or software), χ²(0.05, 3) = 7.815. Since our computed statistic 5.90 < 7.815, we fail to reject H₀. Alternatively, the p-value for χ² = 5.90 with 3 df is approximately 0.117, which exceeds α = 0.05.

Fail to reject H₀. The data are consistent with the 9:3:3:1 Mendelian ratio at the 0.05 significance level.
6
Step 6 — Interpret in Context

There is not sufficient evidence at the 5% significance level to conclude that the offspring phenotype distribution departs from the expected 9:3:3:1 Mendelian ratio. Note, however, that the largest contributions to χ² come from the Round-Green and Wrinkled-Yellow categories, each contributing about 2.94. If the researcher were concerned about these specific deviations, further experimentation with a larger sample size would increase the test's power to detect a genuine departure from the model.

SECTION 7

Assumptions, Strengths & Limitations

Like every statistical procedure, the chi-square goodness-of-fit test operates under specific assumptions that, if violated, can compromise the validity of its conclusions. Understanding both the strengths and limitations of the test is essential for responsible application.

Aspect	Strengths	Limitations
Generality	Applicable to any fully specified discrete distribution — uniform, binomial, Poisson, or custom ratios.	Cannot be used for continuous data without first binning into categories, which introduces arbitrary choices.
Sample Size	Works well for moderate to large samples. No distributional assumption on the raw data (only on counts).	Requires all expected counts ≥ 5. With small expected counts, the chi-square approximation breaks down; Fisher's exact test or simulation methods may be preferable.
Directionality	The omnibus nature detects any departure from H₀, regardless of which category is the source.	Being omnibus, it does not indicate which specific category (or categories) deviate from expectation. Post-hoc residual analysis is needed.
Simplicity	The formula is straightforward and can be computed by hand or with any statistical software.	Simplicity can lead to misuse — applying the test to non-independent observations (e.g., repeated measures on the same subjects) is a common error.
Power	Power increases with sample size. For large n, even subtle departures from H₀ become detectable.	With very large samples, trivially small departures become statistically significant — effect size and practical significance must also be considered.

Strengths and limitations of the chi-square goodness-of-fit test

✦

KEY TAKEAWAY

The chi-square goodness-of-fit test is a versatile first-line tool for categorical data, much like a general-purpose wrench in an engineer's toolkit. It handles a wide range of problems admirably, but for specialized jobs — very small samples, sparse categories, or situations requiring directional sensitivity — more specialized instruments (exact tests, likelihood-ratio tests, or Bayesian methods) may be warranted. Always verify the expected-count condition and independence assumption before trusting the results.

SECTION 8

Connection to Advanced Chi-Square Methods

The goodness-of-fit test is only one member of a broader family of chi-square procedures used in categorical data analysis. Understanding where it sits relative to its siblings — the chi-square test of independence and the chi-square test of homogeneity — helps clarify when each is appropriate and reveals a unifying structure. Beyond these classical tests, more advanced approaches like the likelihood-ratio test (G-test) and log-linear models extend the core ideas to multidimensional contingency tables and complex model comparisons.

Feature	Goodness-of-Fit	Test of Independence	Test of Homogeneity
Number of Variables	One categorical variable	Two categorical variables	One categorical variable across multiple populations
Data Layout	One-way frequency table (1 × k)	Two-way contingency table (r × c)	Two-way contingency table (r × c)
H₀	Data follow a specified distribution	Two variables are independent in one population	Distribution of one variable is the same across populations
Degrees of Freedom	k − 1	(r − 1)(c − 1)	(r − 1)(c − 1)
Sampling Design	One sample, one variable	One sample, cross-classified	Independent samples from each population

Comparison of the three classical chi-square tests

As you advance through categorical data analysis, you will encounter the likelihood-ratio statistic G², defined as G² = 2 Σ Oᵢ ln(Oᵢ / Eᵢ). Under the same conditions, G² has an asymptotically equivalent chi-square distribution, but it is preferred in model-selection contexts because likelihood-ratio statistics are additive — the difference between nested models' G² values is itself a chi-square random variable, enabling systematic model comparison. Log-linear models generalize this idea to multi-way tables, modeling the logarithm of expected cell counts as a linear function of main effects and interactions, much as ANOVA models decompose variance for continuous outcomes. These advanced tools all trace their lineage back to Pearson's original goodness-of-fit framework.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A researcher performs a chi-square goodness-of-fit test and obtains χ² = 0.42 with df = 3. She reports a p-value of approximately 0.936 and fails to reject H₀. Her colleague claims this proves the null hypothesis is true. Evaluate this claim and explain what the result actually tells us.

Show answer

The colleague's claim conflates 'failing to reject H₀' with 'proving H₀ is true,' which is a fundamental misunderstanding of hypothesis testing. A high p-value (0.936) indicates that the observed data are highly consistent with the null distribution, but it does not rule out other distributions that might also produce similar data. We cannot affirm the null hypothesis; we can only state that the data provide no evidence against it. Additionally, the test may lack sufficient power — if the sample size is small, even a meaningfully different distribution could produce a non-significant result. The correct interpretation is: 'At the chosen significance level, there is insufficient evidence to conclude that the population distribution differs from the hypothesized distribution.'

Question 2: BASIC CALCULATION

A bag of candy claims to contain 30% red, 20% blue, 25% green, 15% yellow, and 10% orange candies. You randomly select 200 candies and observe: Red = 72, Blue = 34, Green = 46, Yellow = 28, Orange = 20. Calculate the chi-square test statistic.

Show answer

First, compute the expected counts: E(Red) = 200 × 0.30 = 60, E(Blue) = 200 × 0.20 = 40, E(Green) = 200 × 0.25 = 50, E(Yellow) = 200 × 0.15 = 30, E(Orange) = 200 × 0.10 = 20. All expected counts exceed 5. Now compute each term: (72 − 60)²/60 = 144/60 = 2.40; (34 − 40)²/40 = 36/40 = 0.90; (46 − 50)²/50 = 16/50 = 0.32; (28 − 30)²/30 = 4/30 ≈ 0.133; (20 − 20)²/20 = 0/20 = 0. Summing: χ² = 2.40 + 0.90 + 0.32 + 0.133 + 0 = 3.753. With df = 5 − 1 = 4, the critical value at α = 0.05 is 9.488. Since 3.753 < 9.488, we fail to reject H₀ — the observed distribution is consistent with the manufacturer's claim.

Question 3: INTERMEDIATE

A hospital administrator hypothesizes that emergency room visits are uniformly distributed across the seven days of the week. Over a sample of 350 visits, the observed counts are: Mon = 62, Tue = 45, Wed = 40, Thu = 48, Fri = 55, Sat = 58, Sun = 42. Conduct the full goodness-of-fit test at α = 0.05 and identify which day(s) contribute most to the test statistic using standardized residuals.

Show answer

Under the uniform hypothesis, each day's expected count is E = 350/7 = 50. The df = 7 − 1 = 6. Computing each component: Mon: (62 − 50)²/50 = 2.88; Tue: (45 − 50)²/50 = 0.50; Wed: (40 − 50)²/50 = 2.00; Thu: (48 − 50)²/50 = 0.08; Fri: (55 − 50)²/50 = 0.50; Sat: (58 − 50)²/50 = 1.28; Sun: (42 − 50)²/50 = 1.28. The total χ² = 2.88 + 0.50 + 2.00 + 0.08 + 0.50 + 1.28 + 1.28 = 8.52. The critical value χ²(0.05, 6) = 12.592. Since 8.52 < 12.592, we fail to reject H₀. The standardized residuals (O − E)/√E are: Mon = 12/√50 ≈ 1.70, Tue = −0.71, Wed = −1.41, Thu = −0.28, Fri = 0.71, Sat = 1.13, Sun = −1.13. Monday (1.70) and Wednesday (−1.41) have the largest absolute standardized residuals, suggesting these days deviate most from the uniform pattern, though none exceed the conventional threshold of ±2.

Question 4: APPLIED

A quality engineer at a semiconductor factory inspects 500 chips and classifies defects into five categories. Historical data suggest the proportions should be: Solder 35%, Alignment 25%, Contamination 20%, Crack 12%, Other 8%. The observed counts are: Solder = 195, Alignment = 108, Contamination = 95, Crack = 72, Other = 30. Test at α = 0.01 whether the current defect distribution has shifted from the historical pattern, and discuss the practical implications.

Show answer

Expected counts: E(Solder) = 175, E(Alignment) = 125, E(Contamination) = 100, E(Crack) = 60, E(Other) = 40. All exceed 5. The χ² components: (195 − 175)²/175 = 400/175 = 2.286; (108 − 125)²/125 = 289/125 = 2.312; (95 − 100)²/100 = 25/100 = 0.250; (72 − 60)²/60 = 144/60 = 2.400; (30 − 40)²/40 = 100/40 = 2.500. Total χ² = 2.286 + 2.312 + 0.250 + 2.400 + 2.500 = 9.748. With df = 4, the critical value at α = 0.01 is 13.277. Since 9.748 < 13.277, we fail to reject H₀ at the 1% level. However, the p-value is approximately 0.045, which is significant at α = 0.05. Practically, the engineer should note that Crack defects (standardized residual ≈ 1.55) and Other defects (residual ≈ −1.58) show the most movement. Even though the overall shift is not significant at the stringent 1% level, the increase in crack defects warrants monitoring, as crack defects may have more severe reliability consequences than other types.

Question 5: CRITICAL THINKING

A political scientist surveys 1,000 voters and records their party preference. She wants to test whether the current distribution matches last election's results (Party A: 40%, Party B: 35%, Party C: 15%, Independent: 10%). She observes: A = 380, B = 370, C = 140, Ind = 110. (a) Conduct the test at α = 0.05. (b) Now suppose she had surveyed 10,000 voters and observed exactly the same proportions (i.e., A = 3800, B = 3700, C = 1400, Ind = 1100). Without redoing the full calculation, predict how the test statistic and conclusion change, and explain why. (c) Discuss the implications of this phenomenon for interpreting statistical significance with large samples.

Show answer

(a) Expected counts for n = 1,000: E(A) = 400, E(B) = 350, E(C) = 150, E(Ind) = 100. χ² = (380 − 400)²/400 + (370 − 350)²/350 + (140 − 150)²/150 + (110 − 100)²/100 = 400/400 + 400/350 + 100/150 + 100/100 = 1.00 + 1.143 + 0.667 + 1.00 = 3.810. With df = 3, χ²(0.05, 3) = 7.815. Since 3.810 < 7.815, we fail to reject H₀. (b) With n = 10,000 and identical proportions, every observed count is multiplied by 10 and every expected count is multiplied by 10. In the χ² formula, the numerator (Oᵢ − Eᵢ)² scales by 10² = 100 while the denominator Eᵢ scales by 10, so each term scales by 100/10 = 10. The new χ² = 10 × 3.810 = 38.10, which far exceeds the critical value 7.815, yielding a highly significant result. (c) This illustrates a fundamental principle: with sufficiently large samples, even trivially small departures from H₀ become statistically significant. The proportional deviations (e.g., 38% vs. 40% for Party A) are modest and may be practically meaningless. Researchers should supplement the chi-square test with effect-size measures, such as Cramér's V or Cohen's w (here w = √(χ²/n) ≈ 0.062 for n = 1,000), which remain constant regardless of sample size and indicate that the departure from the hypothesized distribution is quite small.

SUMMARY

Lesson Summary

Summary

The chi-square goodness-of-fit test evaluates whether a sample of categorical data conforms to a hypothesized distribution. Developed by Karl Pearson in 1900, it computes the test statistic χ² = Σ (Oᵢ − Eᵢ)² / Eᵢ, which aggregates the squared, normalized deviations between observed counts and expected counts. Under H₀, this statistic follows a chi-square distribution with df = k − 1 degrees of freedom, and the test is always right-tailed.

Valid application requires a random sample, independent observations, and all expected counts ≥ 5. The test is an omnibus procedure — it detects any departure from H₀ but does not identify which specific categories are responsible; standardized residuals serve that diagnostic role. The goodness-of-fit test is the foundation for more advanced chi-square methods, including the test of independence, the test of homogeneity, and log-linear models for multi-way contingency tables.