Convergence in Distribution

SECTION 1

Historical Context & Motivation

The study of how sequences of random variables behave in the limit is one of the central pillars of probability theory and mathematical statistics. Long before a rigorous measure-theoretic framework existed, mathematicians observed that certain sums of independent random quantities tended toward predictable distributional shapes as the number of summands grew. The desire to formalize this observation—particularly the emergence of the Gaussian bell curve from arbitrary summands—drove the development of convergence in distribution as a precise mathematical concept. Unlike stronger modes of convergence that require the random variables to live on the same probability space, convergence in distribution concerns only the cumulative distribution functions (CDFs) and is therefore the most flexible tool in the asymptotic statistician's arsenal.

1733

De Moivre's Approximation

Abraham de Moivre showed that the binomial distribution approaches a bell-shaped curve as the number of trials increases, providing the earliest instance of what we now recognize as convergence in distribution to a normal law.

1812

Laplace's Central Limit Theorem

Pierre-Simon Laplace generalized de Moivre's result, arguing that sums of independent random variables—regardless of their original distribution—tend toward the normal distribution. Although his proof lacked modern rigor, it established the philosophical importance of distributional limits.

1920

Lévy's Continuity Theorem

Paul Lévy proved that convergence of characteristic functions to a characteristic function is equivalent to convergence in distribution, providing the most powerful tool for establishing distributional limits and making the CLT proof elegant and concise.

1933

Kolmogorov's Foundations

Andrey Kolmogorov's axiomatization of probability theory placed convergence in distribution within a rigorous measure-theoretic setting, enabling the precise hierarchy of convergence modes—almost sure, in probability, in Lᵖ, and in distribution—that graduate students study today.

1956

Portmanteau Theorem

The Portmanteau theorem consolidated multiple equivalent characterizations of convergence in distribution—via CDFs, bounded continuous functions, open sets, and closed sets—into a single unifying result, giving the concept its modern multi-faceted identity.

The central question that convergence in distribution addresses is deceptively simple: if we know that each random variable Xₙ in a sequence has a certain CDF Fₙ, under what conditions can we say that Fₙ approaches some limiting CDF F, and what does this tell us about the statistical behavior of Xₙ for large n? Answering this question rigorously requires confronting subtleties about where CDFs may jump, how characteristic functions encode distributional information, and why convergence in distribution does not imply that the random variables themselves become close in any pathwise sense.

SECTION 2

Core Principles & Definitions

Convergence in distribution—also called weak convergence or convergence in law—is fundamentally a statement about cumulative distribution functions rather than about the random variables themselves. This makes it unique among the standard modes of stochastic convergence: the random variables in the sequence need not even be defined on the same probability space. Understanding this distinction is essential for applying the concept correctly in asymptotic statistics, hypothesis testing, and the construction of confidence intervals.

CDF Pointwise Convergence

A sequence {Xₙ} converges in distribution to X if Fₙ(x) → F(x) at every continuity point x of F. Discontinuity points of the limiting CDF are deliberately excluded to avoid pathological counterexamples.

Weakest Mode of Convergence

Convergence in distribution is implied by convergence in probability, in Lᵖ, and almost sure convergence. The converse is generally false: distributional convergence alone says nothing about pathwise closeness of random variables.

Characteristic Function Criterion

By Lévy's continuity theorem, Xₙ converges in distribution to X if and only if φₙ(t) → φ(t) for every t ∈ ℝ, where φₙ and φ are the respective characteristic functions. This is often the easiest route to proving CLT-type results.

Portmanteau Equivalences

The Portmanteau theorem provides five equivalent formulations: pointwise CDF convergence at continuity points, E[g(Xₙ)] → E[g(X)] for all bounded continuous g, and inequalities involving open and closed sets, among others.

Slutsky's Theorem Extension

If Xₙ converges in distribution to X and Yₙ converges in probability to a constant c, then Xₙ + Yₙ converges in distribution to X + c and XₙYₙ converges in distribution to cX. This theorem is indispensable in applied asymptotics.

✦ KEY TAKEAWAY

Think of convergence in distribution like tuning a radio: you don't care about the exact electromagnetic waveform arriving at the antenna (the individual realizations of Xₙ); you care only that the statistical profile of the signal—its frequency distribution—stabilizes to a recognizable pattern as you refine the tuning. Two entirely different antennas on different rooftops (different probability spaces) can still converge to the same station (the same limiting distribution).

SECTION 3

Visual Explanation

CDFs Converging to the Standard Normal

The following diagram illustrates the essence of convergence in distribution. Consider the standardized sample mean Zₙ = √n(X̄ₙ − μ)/σ where each Xᵢ follows an exponential distribution. For small n, the CDF of Zₙ deviates noticeably from the standard normal CDF Φ(x), but as n increases, the CDF curves progressively hug the limiting Gaussian curve. Importantly, convergence is assessed only at the continuity points of the limiting CDF—which, for the normal distribution, means every real number.

As n increases from 2 to 30, the CDF of the standardized sample mean from an exponential population (dashed curves) converges pointwise to the standard normal CDF Φ(x) (solid cyan). The convergence is visible at every x, illustrating that convergence in distribution is assessed through the CDFs rather than individual realizations.

Observe in the diagram that for n = 2 (pink, short dashes), the CDF deviates considerably from the Gaussian curve, especially in the left tail where the exponential's inherent right-skewness is most apparent. By n = 5 (violet), the approximation improves, and by n = 30 (amber), the two curves are virtually indistinguishable. This visual progression is exactly what the formal definition captures: for every fixed x at which Φ is continuous—which is every real number—the sequence Fₙ(x) approaches Φ(x). The rate of this convergence is governed by the Berry–Esseen theorem, which provides an O(n⁻¹ᐟ²) uniform bound on the CDF difference when the third absolute moment is finite.

SECTION 4

Mathematical Framework

The formal definition of convergence in distribution requires careful attention to the continuity points of the limiting CDF. This subtlety is not merely pedantic—it prevents false conclusions when the limit involves point masses. In this section we present the definition, the characteristic function criterion, and the Portmanteau equivalences that together form the mathematical backbone of the theory.

DEFINITION OF CONVERGENCE IN DISTRIBUTION

Xₙ →ᵈ X ⟺ lim_{n→∞} Fₙ(x) = F(x) for all x ∈ C(F)

Here Fₙ is the CDF of Xₙ, F is the CDF of X, and C(F) denotes the set of continuity points of F. We write Xₙ →ᵈ X or equivalently Xₙ ⇒ X.

LÉVY CONTINUITY THEOREM

Xₙ →ᵈ X ⟺ φ_{Xₙ}(t) → φ_X(t) for all t ∈ ℝ

Where φ_{Xₙ}(t) = E[e^{itXₙ}] is the characteristic function of Xₙ. Convergence of characteristic functions pointwise to a function that is continuous at t = 0 guarantees the limit is itself a characteristic function, ensuring the existence of the limiting distribution.

PORTMANTEAU THEOREM (SELECTED EQUIVALENCES)

Xₙ →ᵈ X ⟺ E[g(Xₙ)] → E[g(X)] for all bounded continuous g : ℝ → ℝ

Additional equivalent conditions include: lim sup P(Xₙ ∈ C) ≤ P(X ∈ C) for all closed sets C, and lim inf P(Xₙ ∈ G) ≥ P(X ∈ G) for all open sets G. These set-theoretic conditions are especially useful when working in metric spaces beyond ℝ.

SLUTSKY'S THEOREM

If Xₙ →ᵈ X and Yₙ →ᵖ c, then Xₙ + Yₙ →ᵈ X + c and XₙYₙ →ᵈ cX

Slutsky's theorem is critical in applied statistics: it allows us to replace unknown parameters with consistent estimators inside asymptotic arguments without disrupting the distributional limit. Note that both Xₙ and Yₙ must converge—Xₙ in distribution and Yₙ in probability to a constant—for the result to hold.

⚠ Common Pitfall

Convergence in distribution does not imply that E[Xₙ] → E[X]. Expectations involve the entire tail behavior, while distributional convergence only constrains how probabilities accumulate. A classic counterexample: let Xₙ = n with probability 1/n and 0 otherwise. Then Xₙ →ᵈ 0, yet E[Xₙ] = 1 for all n.

SECTION 5

Convergence Hierarchy & Relationships

One cannot fully appreciate convergence in distribution without understanding where it sits in the broader hierarchy of stochastic convergence modes. There are four principal modes—almost sure convergence, convergence in Lᵖ, convergence in probability, and convergence in distribution—and the implications among them are strict. The diagram below maps these relationships, with arrows pointing from stronger to weaker modes.

The hierarchy of convergence modes. Almost sure convergence implies both convergence in Lᵖ (when dominated by an Lᵖ-integrable random variable) and convergence in probability. Both of these in turn imply convergence in distribution. The reverse implications are generally false, with one notable exception: convergence in distribution to a constant is equivalent to convergence in probability to that constant.

The exception noted at the bottom of the diagram deserves emphasis. When the limiting random variable X is degenerate—i.e., P(X = c) = 1 for some constant c—then convergence in distribution, convergence in probability, and almost sure convergence (along a subsequence) all coincide. This is because the CDF of a constant is a step function with a single jump, and pointwise convergence of CDFs to such a function forces the probability mass of Xₙ to concentrate near c. This equivalence is used frequently in proving consistency of estimators: one often shows convergence in distribution to a constant to conclude convergence in probability.

Comparison of convergence modes with respect to probability-space requirements and implication strength.

Mode	Requires Same Probability Space?	Implication Chain
Almost Sure	Yes	⇒ In probability ⇒ In distribution
In Lᵖ	Yes	⇒ In probability ⇒ In distribution
In Probability	Yes	⇒ In distribution
In Distribution	No	Weakest; to constant ⇔ in probability

SECTION 6

Worked Example

We now demonstrate how to prove convergence in distribution using the characteristic function method. This example applies the Central Limit Theorem machinery from scratch to a specific distribution, showing each algebraic step in detail.

CLT for Exponential(1) via Characteristic Functions

Step 1 — Set Up the Standardized SumLet X₁, X₂, …, Xₙ be i.i.d. Exponential(1) random variables with mean μ = 1 and variance σ² = 1. Define the standardized sample mean as Zₙ = √n(X̄ₙ − 1). We wish to show Zₙ →ᵈ N(0, 1).

Zₙ = (S_n − n) / √n where Sₙ = X₁ + ⋯ + Xₙ

Step 2 — Compute the Characteristic Function of XᵢThe characteristic function of an Exponential(1) random variable is φ_X(t) = (1 − it)⁻¹. The characteristic function of the centered variable Xᵢ − 1 is therefore φ_{X-1}(t) = e^{−it} · (1 − it)⁻¹.

φ_{X−1}(t) = e^{−it} / (1 − it)

Step 3 — Form the Characteristic Function of ZₙSince Zₙ is the sum of n independent copies of (Xᵢ − 1) scaled by 1/√n, we have φ_{Zₙ}(t) = [φ_{X−1}(t/√n)]ⁿ = [e^{−it/√n} / (1 − it/√n)]ⁿ.

φ_{Zₙ}(t) = [e^{−it/√n} · (1 − it/√n)⁻¹]ⁿ

Step 4 — Take the Logarithm and ExpandTaking the natural logarithm: log φ_{Zₙ}(t) = n[−it/√n − log(1 − it/√n)]. Using the Taylor expansion log(1 − u) = −u − u²/2 − u³/3 − ⋯ with u = it/√n, we get −log(1 − it/√n) = it/√n + (it)²/(2n) + O(n⁻³ᐟ²). Substituting: log φ_{Zₙ}(t) = n[−it/√n + it/√n + (it)²/(2n) + O(n⁻³ᐟ²)] = n[(it)²/(2n) + O(n⁻³ᐟ²)] = −t²/2 + O(n⁻¹ᐟ²).

log φ_{Zₙ}(t) = −t²/2 + O(n⁻¹ᐟ²)

Step 5 — Conclude via Lévy's Continuity TheoremAs n → ∞, log φ_{Zₙ}(t) → −t²/2, so φ_{Zₙ}(t) → e^{−t²/2}, which is the characteristic function of the standard normal distribution N(0,1). Since this limit is continuous at t = 0, Lévy's continuity theorem guarantees that Zₙ →ᵈ N(0, 1).

Zₙ = √n(X̄ₙ − 1) →ᵈ N(0, 1) ∎

💡 Why Characteristic Functions?

The characteristic function approach converts a question about distributions (which involve integrals and CDFs) into a question about pointwise limits of complex-valued functions. The logarithmic expansion in Step 4 is the universal mechanism behind the CLT: for any distribution with finite variance, the second-order term in the Taylor expansion always yields −t²/2, and higher-order terms vanish as n → ∞.

SECTION 7

Strengths, Limitations & Practical Considerations

Convergence in distribution is extraordinarily useful in theoretical and applied statistics, but its permissiveness comes with important caveats that practitioners must internalize. The following table contrasts the advantages that make it the workhorse of asymptotic statistics with the limitations that can lead to misinterpretation if ignored.

Strengths and limitations of convergence in distribution as a tool in statistical inference.

Strengths	Limitations
No common probability space needed: random variables can be defined on entirely separate experiments.	Does not imply convergence of moments: E[Xₙ] may not converge to E[X] without uniform integrability.
Directly applicable via Lévy's theorem and characteristic functions, enabling elegant CLT proofs.	Cannot be used to make pathwise statements about individual sample paths or realizations.
Preserved under continuous transformations by the Continuous Mapping Theorem.	Joint convergence in distribution of (Xₙ, Yₙ) does NOT follow from marginal convergence of Xₙ and Yₙ separately.
Slutsky's theorem allows mixing distributional and probability convergence for practical asymptotic arguments.	Rate of convergence is not specified by the definition; the Berry–Esseen theorem or delta method must be invoked separately.
Generalizes naturally to random vectors and stochastic processes (weak convergence in metric spaces).	For finite samples, the approximation quality depends heavily on the underlying distribution's skewness and kurtosis.

✦ KEY TAKEAWAY

Convergence in distribution is like reading a weather forecast that correctly predicts the probability distribution of tomorrow's temperature but tells you nothing about the specific sequence of atmospheric events that will produce that temperature. It is a statement about the ensemble of possible outcomes, not about any particular realization. This makes it ideal for designing hypothesis tests and confidence intervals (which are inherently probabilistic) but insufficient for tracking individual sample paths.

SECTION 8

Connections to Advanced Theory

Convergence in distribution is not an endpoint but a gateway to deeper results in probability theory and mathematical statistics. It connects directly to the Continuous Mapping Theorem (if Xₙ →ᵈ X and g is continuous, then g(Xₙ) →ᵈ g(X)), the Delta Method (which derives the asymptotic distribution of smooth functions of asymptotically normal estimators), and weak convergence in metric spaces (the Prokhorov/Billingsley framework for stochastic processes). The table below sketches how the basic concept generalizes.

How convergence in distribution connects to advanced probability theory.

Basic Concept	Advanced Extension	Key Implication
Xₙ →ᵈ X on ℝ	Weak convergence Pₙ ⇒ P on (S, d) (Polish spaces)	Enables Donsker's theorem (empirical process theory) and functional CLTs.
CLT: √n(X̄ − μ) →ᵈ N(0, σ²)	Delta Method: √n(g(X̄) − g(μ)) →ᵈ N(0, [g′(μ)]²σ²)	Derives asymptotic distributions of variance-stabilizing and other transforms.
Slutsky's Theorem for scalars	Cramér–Wold device for random vectors	Xₙ →ᵈ X in ℝᵈ iff a'Xₙ →ᵈ a'X for all a ∈ ℝᵈ.
Convergence of CDFs	Tightness + finite-dimensional convergence (Prokhorov's theorem)	A family of probability measures is relatively compact iff it is tight—used to prove existence of distributional limits.

Perhaps the most consequential extension is Donsker's theorem (the functional CLT), which establishes that the rescaled partial-sum process converges in distribution—in the space of càdlàg functions equipped with the Skorokhod topology—to Brownian motion. This result underlies the theory of Kolmogorov–Smirnov tests, Cramér–von Mises tests, and much of modern nonparametric inference. For students continuing into measure-theoretic probability, Billingsley's Convergence of Probability Measures remains the definitive reference on weak convergence in general metric spaces.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

Explain why convergence in distribution does not require the random variables Xₙ and X to be defined on the same probability space, whereas convergence in probability does. What property of the definition makes this possible?

PROBLEM 2 — BASIC CALCULATION

Let Xₙ ~ Uniform(0, 1 + 1/n) for each n ≥ 1. Show that Xₙ →ᵈ X where X ~ Uniform(0, 1). At which points must you verify convergence of the CDFs?

PROBLEM 3 — INTERMEDIATE

Let X₁, X₂, … be i.i.d. Bernoulli(p) with 0 < p < 1. Using the characteristic function method, prove that Zₙ = (Sₙ − np)/√(np(1−p)) →ᵈ N(0,1), where Sₙ = X₁ + ⋯ + Xₙ.

PROBLEM 4 — APPLIED

A quality engineer measures the breaking strength of n = 100 ceramic tiles. The population distribution is unknown but has mean μ = 500 N and standard deviation σ = 40 N. Using convergence in distribution (the CLT), construct an approximate 95% confidence interval for μ. Then explain why Slutsky's theorem justifies replacing σ with the sample standard deviation s in practice.

PROBLEM 5 — CRITICAL THINKING

Let Xₙ be a sequence of random variables with Xₙ = n with probability 1/n² and Xₙ = 0 with probability 1 − 1/n². (a) Does Xₙ →ᵈ 0? (b) Does E[Xₙ] → 0? (c) Does E[Xₙ²] → 0? Use this example to explain why convergence in distribution does not preserve moment convergence in general, and state a sufficient condition under which it does.

SUMMARY

Summary

Convergence in distribution (also called weak convergence) asserts that the CDFs Fₙ(x) converge pointwise to F(x) at every continuity point of the limiting CDF F. It is the weakest mode of convergence in the standard hierarchy—implied by almost sure convergence, convergence in Lᵖ, and convergence in probability—but it is by far the most commonly invoked mode in statistical practice. Its power derives from tools like Lévy's continuity theorem (characteristic functions converge pointwise if and only if distributions converge weakly) and the Portmanteau theorem (five equivalent characterizations via bounded continuous functions, open sets, closed sets, and CDFs).

The Central Limit Theorem is the most celebrated application: standardized sums of i.i.d. random variables with finite variance converge in distribution to the standard normal. Slutsky's theorem extends the utility by allowing consistent estimators to replace unknown parameters without disrupting the distributional limit. Key caveats include the fact that convergence in distribution does not guarantee convergence of moments (without uniform integrability), does not require a common probability space, and when the limit is a constant, it coincides with convergence in probability. These results form the bedrock of asymptotic inference, confidence interval construction, and hypothesis testing throughout modern statistics.