Chi-Square Test of Independence - College Statistics

SECTION 1

Historical Context & Motivation

Statistical inquiry has long grappled with a fundamental question: when we observe that two categorical attributes appear to co-vary in a sample, does that pattern reflect a genuine relationship in the population, or is it merely the product of sampling variability? Before the development of formal hypothesis tests for categorical data, researchers relied on intuitive judgments about tabulated frequencies—an approach that was subjective and unreliable. The chi-square test of independence emerged at the turn of the twentieth century to provide a principled, quantitative answer to precisely this question. Its development was intertwined with the broader formalization of mathematical statistics and marked a turning point in how scientists analyzed categorical data across disciplines from genetics to sociology.

1900 • Pearson's Chi-Square Statistic

Karl Pearson published 'On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling,' introducing the χ² goodness-of-fit statistic that would become the foundation for all chi-square tests.

1922 • Fisher's Degrees of Freedom Correction

Ronald A. Fisher clarified the correct degrees of freedom for contingency tables, correcting an error in Pearson's original formulation and establishing the (r − 1)(c − 1) rule that remains standard today.

1934 • Fisher's Exact Test for Small Samples

Fisher proposed an exact test for 2 × 2 tables, providing an alternative when sample sizes were too small for the chi-square approximation to hold reliably. This underscored the asymptotic nature of the χ² distribution.

1954 • Yates' Continuity Correction

Frank Yates formalized a continuity correction for 2 × 2 tables, addressing the discrete-to-continuous approximation inherent in comparing integer-valued frequencies to a continuous χ² distribution.

Modern Era • Computational & Simulation-Based Approaches

With the advent of computing, permutation tests and Monte Carlo simulations now supplement the classical chi-square test, but Pearson's χ² statistic remains the dominant first-line tool taught in statistics courses worldwide.

The central question that motivated Pearson's work remains the question at the heart of every chi-square test of independence: given observed frequencies in a two-way contingency table, is there sufficient evidence to conclude that the row variable and the column variable are associated, or can the observed pattern be attributed to chance alone? Understanding this test equips you with one of the most widely used tools in the analyst's toolkit for categorical data.

SECTION 2

Core Principles & Definitions

Before performing a chi-square test of independence, it is essential to understand the conceptual architecture that supports it. The test operates on data organized in a contingency table (also called a cross-tabulation or two-way table), where each cell contains the frequency count for a specific combination of levels from two categorical variables. The logic of the test mirrors hypothesis testing in general: we posit a null hypothesis, compute a test statistic that measures deviation from that hypothesis, and then compare the statistic to a reference distribution to obtain a p-value.

Contingency Table

A matrix of observed frequencies with r rows and c columns. Each row represents a level of one categorical variable, each column a level of the other. The entry in cell (i, j) is the count O_ij of observations falling in that combination.

Null Hypothesis (H₀)

The two categorical variables are independent in the population—knowing the level of one variable provides no information about the distribution of the other. Formally: P(A ∩ B) = P(A) × P(B) for all combinations of levels.

Expected Frequencies

Under H₀, the expected count E_ij for cell (i, j) is computed as (Row i total × Column j total) / Grand total. These represent the counts we would anticipate if the variables were truly independent.

Chi-Square Statistic (χ²)

A single number summarizing how far the observed frequencies depart from the expected frequencies. Large values of χ² indicate substantial discrepancies, providing evidence against independence.

Degrees of Freedom

For an r × c table, df = (r − 1)(c − 1). This parameter controls the shape of the reference chi-square distribution used to compute the p-value. It reflects the number of independent pieces of information in the table.

✦

KEY TAKEAWAY

Think of the chi-square test like a quality-control inspector comparing a manufactured product to its blueprint. The expected frequencies are the blueprint—what the data should look like if the two variables truly don't interact. The observed frequencies are the actual product off the assembly line. The χ² statistic measures the total 'defect score'—the cumulative, standardized gap between what was expected and what was observed. If the defect score exceeds a critical threshold, the inspector rejects the null hypothesis and concludes the two variables are not operating independently.

⚠

Assumptions & Conditions

The chi-square test of independence requires: (1) data collected via a random sampling mechanism, (2) each observation classified into exactly one cell, (3) observations are independent of each other, and (4) all expected cell counts are at least 5 (some texts relax this to 80% of cells ≥ 5 with none below 1). If these conditions are violated, consider Fisher's exact test or a simulation-based approach.

SECTION 3

Visual Explanation: The Contingency Table

The diagram below illustrates the structure of a generic r × c contingency table and how the expected frequencies are derived from marginal totals. Understanding this visual layout is critical because every computation in the chi-square test flows directly from the observed and expected values in each cell of this table.

A generic 2 × 2 contingency table showing observed counts O_ij in cyan, marginal row and column totals in amber, and the grand total N in pink. Expected frequencies are computed from the marginals using the formula shown below the table.

In the diagram, each interior cell of the table contains both the observed count O_ij and, in parentheses, the expected count E_ij. The marginal totals along the right edge (R_i) and bottom edge (C_j) are essential because the expected frequency for any cell is simply the product of its row total and column total divided by the grand total N. This formula operationalizes the null hypothesis: if the two variables are independent, the proportion of observations in row i should be the same across all columns, and the expected count reflects that proportional distribution.

SECTION 4

Mathematical Framework

The mathematical machinery of the chi-square test of independence involves three key formulas: computing expected frequencies under H₀, calculating the χ² test statistic, and determining the degrees of freedom that specify which chi-square distribution to use as the reference. Each formula plays a distinct role in the logical chain from data to decision.

EXPECTED FREQUENCY

Eᵢⱼ = (Rᵢ × Cⱼ) / N

Where E_ij is the expected count in cell (i, j), R_i is the total for row i, C_j is the total for column j, and N is the grand total of all observations.

CHI-SQUARE TEST STATISTIC

χ² = Σ (Oᵢⱼ − Eᵢⱼ)² / Eᵢⱼ

The summation runs over all r × c cells in the contingency table. O_ij is the observed count and E_ij is the expected count. Squaring the difference ensures all deviations contribute positively, and dividing by E_ij standardizes each term so that a deviation of 5 in a cell with an expected count of 10 is weighted more heavily than the same deviation in a cell with an expected count of 100.

DEGREES OF FREEDOM

df = (r − 1)(c − 1)

Where r is the number of rows and c is the number of columns. Once the marginal totals are fixed, only (r − 1)(c − 1) cell frequencies are free to vary—the rest are determined by the constraint that rows and columns must sum to their respective marginals.

Under the null hypothesis and given sufficiently large expected counts, the test statistic χ² follows approximately a chi-square distribution with (r − 1)(c − 1) degrees of freedom. The p-value is the probability of observing a χ² value at least as large as the one computed from the data, assuming H₀ is true. Because chi-square distributions are right-skewed, the test is always one-tailed (right tail): only large values of χ² provide evidence against independence.

💡

Why Divide by Eᵢⱼ?

Dividing by the expected count is what makes the chi-square statistic a standardized measure. Consider two cells: one expects 200 observations and the other expects 10. A discrepancy of 8 is unremarkable in the first cell (8/200 = 4% deviation) but dramatic in the second (8/10 = 80% deviation). Division by E_ij ensures that each cell's contribution to χ² reflects its relative, not absolute, departure from expectation.

SECTION 5

Decision Process & the χ² Distribution

Once you have computed the χ² test statistic and identified the degrees of freedom, the final step is to locate the computed value on the appropriate chi-square distribution curve and determine whether it falls in the rejection region. The following diagram illustrates this decision process for a test conducted at the α = 0.05 significance level.

The chi-square distribution with df = 4, showing the critical value χ²_crit = 9.488 at α = 0.05. The red shaded region in the right tail represents the rejection region. If the computed χ² falls in this region, we reject the null hypothesis of independence.

The curve shown is the probability density function for a χ² distribution with 4 degrees of freedom. As degrees of freedom increase, the distribution shifts to the right and becomes more symmetric, approaching a normal distribution for large df. Notice that the distribution is right-skewed and bounded below by zero—this reflects the fact that χ², being a sum of squared terms, can never be negative. The critical value of 9.488 is the point beyond which only 5% of the distribution's area lies, so if the computed χ² exceeds this threshold, we conclude that the observed data would be very unlikely under the assumption of independence.

Step 1: State H₀ (variables are independent) and H₁ (variables are not independent).
Step 2: Choose a significance level α (commonly 0.05).
Step 3: Construct the contingency table and compute all expected frequencies.
Step 4: Verify that all expected counts are ≥ 5.
Step 5: Compute χ² = Σ (O − E)² / E and the degrees of freedom.
Step 6: Compare χ² to χ²_crit or compute the p-value. If p < α, reject H₀.

SECTION 6

Worked Example

A university health center surveyed 500 students about their sleep quality (Good or Poor) and their academic performance (Above Average, Average, Below Average). The observed data are shown below. At the α = 0.05 significance level, test whether sleep quality and academic performance are independent.

Sleep Quality	Above Avg	Average	Below Avg	Row Total
Good	120	100	30	250
Poor	80	100	70	250
Col Total	200	200	100	500

Observed Frequencies: Sleep Quality vs. Academic Performance

Chi-Square Test of Independence: Sleep Quality vs. Academic Performance

1
Step 1 — State Hypotheses

H₀: Sleep quality and academic performance are independent. H₁: Sleep quality and academic performance are not independent (they are associated).
2
Step 2 — Compute Expected Frequencies

For each cell, E_ij = (Row total × Column total) / Grand total. For Good & Above Avg: E = (250 × 200) / 500 = 100. For Good & Average: E = (250 × 200) / 500 = 100. For Good & Below Avg: E = (250 × 100) / 500 = 50. For Poor & Above Avg: E = (250 × 200) / 500 = 100. For Poor & Average: E = (250 × 200) / 500 = 100. For Poor & Below Avg: E = (250 × 100) / 500 = 50.

Expected frequencies: 100, 100, 50, 100, 100, 50. All E_ij ≥ 5 ✓
3
Step 3 — Compute the χ² Statistic

χ² = Σ (O − E)² / E = (120 − 100)²/100 + (100 − 100)²/100 + (30 − 50)²/50 + (80 − 100)²/100 + (100 − 100)²/100 + (70 − 50)²/50 = 400/100 + 0/100 + 400/50 + 400/100 + 0/100 + 400/50 = 4 + 0 + 8 + 4 + 0 + 8 = 24.0

χ² = 24.0
4
Step 4 — Determine Degrees of Freedom

df = (r − 1)(c − 1) = (2 − 1)(3 − 1) = 1 × 2 = 2.

df = 2
5
Step 5 — Find Critical Value and Make Decision

From the chi-square table, χ²_crit at α = 0.05 with df = 2 is 5.991. Since our computed χ² = 24.0 is much greater than 5.991, we reject H₀. Equivalently, the p-value is approximately 6.1 × 10⁻⁶, which is far below 0.05.

Reject H₀: There is statistically significant evidence that sleep quality and academic performance are associated (χ² = 24.0, df = 2, p < 0.001).
6
Step 6 — Interpret in Context

Students with good sleep quality are overrepresented in the 'Above Average' category (120 observed vs. 100 expected) and underrepresented in the 'Below Average' category (30 observed vs. 50 expected). The reverse pattern holds for poor sleepers. This suggests that better sleep quality is associated with higher academic performance in this sample, though the chi-square test does not establish causation.

SECTION 7

Strengths, Limitations & Common Pitfalls

Like any statistical tool, the chi-square test of independence has important strengths and limitations that determine when it should—and should not—be used. A thoughtful analyst understands these boundaries and selects the appropriate alternative when the chi-square test's assumptions are violated.

Strengths	Limitations
Works with nominal and ordinal categorical data—no distributional assumption about the variables themselves.	Requires sufficiently large expected counts (all E ≥ 5); breaks down with sparse data.
Computationally straightforward and easy to interpret: a single test statistic and p-value.	Does not indicate the direction or strength of association—only that one exists.
Applicable to tables of any dimension (r × c), not just 2 × 2.	Cannot establish causation; association may be confounded by lurking variables.
Well-understood asymptotic theory with widely available critical-value tables.	Sensitive to sample size: very large N can yield statistical significance for trivially small associations.
Nonparametric—no requirement for normally distributed underlying data.	Ignores the ordinal structure of variables; treats all categories as purely nominal.

Strengths and Limitations of the Chi-Square Test of Independence

✦

KEY TAKEAWAY

The chi-square test of independence is a powerful first-line screening tool—it tells you whether an association exists but not how strong it is or what drives it. Think of it as a smoke alarm: it alerts you that something is happening, but you still need to investigate. For effect size, follow up with measures like Cramér's V or the contingency coefficient. For direction, examine standardized residuals in individual cells.

⚠

Common Pitfall: Using Percentages Instead of Counts

The chi-square statistic must be computed from raw frequency counts, never from percentages or proportions. If you input proportions into the formula, the resulting χ² value will be deflated by a factor of N, leading to incorrect conclusions. Always verify that your contingency table entries represent actual counts of observations.

SECTION 8

Connection to Related Tests & Advanced Theory

The chi-square test of independence does not exist in isolation. It belongs to a broader family of chi-square tests and connects to several other inferential procedures. Understanding these connections clarifies when to use the test of independence versus its relatives and how it serves as a gateway to more sophisticated analyses.

Test / Measure	Purpose	Relationship to χ² Independence Test
χ² Goodness-of-Fit	Tests whether a single categorical variable follows a hypothesized distribution.	Same χ² formula but applied to a one-way table (one variable, k categories) rather than a two-way table.
Fisher's Exact Test	Tests independence in 2 × 2 tables when expected counts are small.	An exact alternative that does not rely on the large-sample χ² approximation; preferred when any E < 5.
G-Test (Likelihood Ratio)	Tests independence using a log-likelihood ratio statistic.	Asymptotically equivalent to Pearson's χ²; preferred in some advanced settings due to additive decomposability.
Cramér's V	Measures the strength of association between two categorical variables.	Computed from χ² as V = √(χ² / (N × min(r−1, c−1))); provides effect size context the test alone lacks.
Log-Linear Models	Model the cell counts in multi-way contingency tables as functions of main effects and interactions.	Generalize the independence test to three or more variables; the two-way χ² test is a special case of a saturated log-linear model comparison.

Related Tests and Measures in Categorical Data Analysis

For students continuing into advanced categorical data analysis, the chi-square test of independence can be viewed as a special case of the likelihood ratio framework central to generalized linear models. In that framework, the null hypothesis of independence corresponds to a log-linear model with no interaction term, and rejecting H₀ motivates fitting models with interaction effects. Additionally, when one variable is naturally a predictor and the other a response, the chi-square test of independence is equivalent to comparing proportions across groups—linking it directly to logistic regression for binary outcomes. These connections illustrate that the chi-square test is not merely a standalone procedure but the entry point into a rich hierarchy of models for categorical data.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

Explain why the chi-square test of independence is always a right-tailed test. Why can the χ² statistic never be negative, and what would a χ² value of exactly zero indicate about the relationship between the two variables?

Show answer

The χ² statistic is a sum of squared differences divided by positive expected counts: χ² = Σ (O − E)² / E. Since every term in this sum involves a squared quantity in the numerator and a positive quantity in the denominator, each term is non-negative, making the total sum non-negative. The test is right-tailed because only large positive values of χ² indicate large deviations from the independence model; small values suggest the data are consistent with H₀. A χ² value of exactly zero would mean that every observed count equals its expected count perfectly—O_ij = E_ij for all cells—indicating the data are in perfect agreement with the assumption of independence.

Question 2: BASIC CALCULATION

A researcher collects data on handedness (Left, Right) and eye dominance (Left, Right) for 200 subjects. The observed counts are: Left-hand/Left-eye = 40, Left-hand/Right-eye = 20, Right-hand/Left-eye = 50, Right-hand/Right-eye = 90. Compute all expected frequencies and verify that the assumptions for the chi-square test are met.

Show answer

Row totals: Left-hand = 60, Right-hand = 140. Column totals: Left-eye = 90, Right-eye = 110. Grand total N = 200. Expected frequencies: E(Left/Left) = (60 × 90)/200 = 27, E(Left/Right) = (60 × 110)/200 = 33, E(Right/Left) = (140 × 90)/200 = 63, E(Right/Right) = (140 × 110)/200 = 77. All expected counts (27, 33, 63, 77) exceed 5, so the chi-square approximation is appropriate. The data were collected from a random sample and each subject is classified into exactly one cell, satisfying the independence and mutual exclusivity assumptions.

Question 3: INTERMEDIATE

Using the data from Problem 2 (O: 40, 20, 50, 90; E: 27, 33, 63, 77), compute the χ² test statistic, determine the degrees of freedom, and state your conclusion at α = 0.05. The critical value for χ² with 1 degree of freedom at α = 0.05 is 3.841.

Show answer

χ² = (40 − 27)²/27 + (20 − 33)²/33 + (50 − 63)²/63 + (90 − 77)²/77 = 169/27 + 169/33 + 169/63 + 169/77 = 6.259 + 5.121 + 2.683 + 2.195 = 16.258. Degrees of freedom: df = (2 − 1)(2 − 1) = 1. Since χ² = 16.258 far exceeds χ²_crit = 3.841, we reject H₀ at the 0.05 level. There is strong statistical evidence that handedness and eye dominance are not independent. The large positive residual in the Left/Left cell (40 vs. 27 expected) suggests that left-handed individuals are disproportionately likely to be left-eye dominant.

Question 4: APPLIED

A pharmaceutical company tests three formulations of a drug (A, B, C) for their effect on patient outcomes (Improved, No Change, Worsened). In a trial of 300 patients (100 per group), the observed data are: A: 60 Improved, 30 No Change, 10 Worsened; B: 45 Improved, 40 No Change, 15 Worsened; C: 35 Improved, 35 No Change, 30 Worsened. Conduct a full chi-square test of independence at α = 0.01. The critical value for χ² with 4 df at α = 0.01 is 13.277.

Show answer

Column totals: Improved = 140, No Change = 105, Worsened = 55. N = 300. Expected frequencies for each group (all rows have R = 100): E(Improved) = (100 × 140)/300 = 46.67, E(No Change) = (100 × 105)/300 = 35.00, E(Worsened) = (100 × 55)/300 = 18.33. χ² = (60 − 46.67)²/46.67 + (30 − 35)²/35 + (10 − 18.33)²/18.33 + (45 − 46.67)²/46.67 + (40 − 35)²/35 + (15 − 18.33)²/18.33 + (35 − 46.67)²/46.67 + (35 − 35)²/35 + (30 − 18.33)²/18.33 = 3.810 + 0.714 + 3.785 + 0.060 + 0.714 + 0.605 + 2.914 + 0 + 7.426 = 20.028. With df = (3 − 1)(3 − 1) = 4, χ² = 20.028 > 13.277. We reject H₀ at α = 0.01. There is highly significant evidence that patient outcomes differ across the three formulations. The residuals suggest Formulation A has superior outcomes while Formulation C has notably more worsened cases than expected.

Question 5: CRITICAL THINKING

A sociologist reports that a chi-square test on a 4 × 3 contingency table with N = 50,000 yielded χ² = 18.2, df = 6, and p < 0.01, concluding a 'strong association' between socioeconomic class and political affiliation. Critically evaluate this conclusion. What additional analysis would you recommend, and why might the claim of a 'strong association' be misleading?

Show answer

While the chi-square test correctly indicates statistical significance (χ² = 18.2 with df = 6 gives p ≈ 0.006), the claim of a 'strong association' is misleading because statistical significance is not synonymous with practical significance—especially with N = 50,000. With such a large sample, even trivially small departures from independence can produce significant p-values. To assess the strength of association, the sociologist should compute Cramér's V = √(χ²/(N × min(r−1, c−1))) = √(18.2/(50,000 × 2)) = √(0.000182) ≈ 0.0135. A Cramér's V of 0.014 indicates an extremely weak association, despite the significant p-value. Additionally, examining standardized residuals in each cell would reveal which specific class-affiliation combinations deviate from independence and by how much. This problem illustrates a critical lesson: always pair hypothesis tests with effect-size measures when sample sizes are large.

SUMMARY

Summary

The chi-square test of independence determines whether two categorical variables are associated in a population by comparing observed frequencies in a contingency table to the expected frequencies derived under the assumption of independence. The test statistic χ² = Σ (O − E)² / E aggregates standardized squared deviations across all cells, and is compared to a chi-square distribution with (r − 1)(c − 1) degrees of freedom to obtain a p-value.

Key requirements include random sampling, independent observations, and all expected counts ≥ 5. When the test rejects H₀, follow up with Cramér's V for effect size and standardized residuals to identify which cells drive the association. Remember: the chi-square test detects association, not causation, and statistical significance does not imply practical significance—always consider effect size alongside p-values.