Convergence in Distribution - Statistics Graduate Level

SECTION 1

Historical Context & Motivation

The study of how sequences of random variables behave in the limit is one of the central pillars of probability theory and mathematical statistics. Long before a rigorous measure-theoretic framework existed, mathematicians observed that certain sums of independent random quantities tended toward predictable distributional shapes as the number of summands grew. The desire to formalize this observation—particularly the emergence of the Gaussian bell curve from arbitrary summands—drove the development of convergence in distribution as a precise mathematical concept. Unlike stronger modes of convergence that require the random variables to live on the same probability space, convergence in distribution concerns only the cumulative distribution functions (CDFs) and is therefore the most flexible tool in the asymptotic statistician's arsenal.

1733 • De Moivre's Approximation

Abraham de Moivre showed that the binomial distribution approaches a bell-shaped curve as the number of trials increases, providing the earliest instance of what we now recognize as convergence in distribution to a normal law.

1812 • Laplace's Central Limit Theorem

Pierre-Simon Laplace generalized de Moivre's result, arguing that sums of independent random variables—regardless of their original distribution—tend toward the normal distribution. Although his proof lacked modern rigor, it established the philosophical importance of distributional limits.

1920 • Lévy's Continuity Theorem

Paul Lévy proved that convergence of characteristic functions to a characteristic function is equivalent to convergence in distribution, providing the most powerful tool for establishing distributional limits and making the CLT proof elegant and concise.

1933 • Kolmogorov's Foundations

Andrey Kolmogorov's axiomatization of probability theory placed convergence in distribution within a rigorous measure-theoretic setting, enabling the precise hierarchy of convergence modes—almost sure, in probability, in Lᵖ, and in distribution—that graduate students study today.

1956 • Portmanteau Theorem

The Portmanteau theorem consolidated multiple equivalent characterizations of convergence in distribution—via CDFs, bounded continuous functions, open sets, and closed sets—into a single unifying result, giving the concept its modern multi-faceted identity.

The central question that convergence in distribution addresses is deceptively simple: if we know that each random variable Xₙ in a sequence has a certain CDF Fₙ, under what conditions can we say that Fₙ approaches some limiting CDF F, and what does this tell us about the statistical behavior of Xₙ for large n? Answering this question rigorously requires confronting subtleties about where CDFs may jump, how characteristic functions encode distributional information, and why convergence in distribution does not imply that the random variables themselves become close in any pathwise sense.

SECTION 2

Core Principles & Definitions

Convergence in distribution—also called weak convergence or convergence in law—is fundamentally a statement about cumulative distribution functions rather than about the random variables themselves. This makes it unique among the standard modes of stochastic convergence: the random variables in the sequence need not even be defined on the same probability space. Understanding this distinction is essential for applying the concept correctly in asymptotic statistics, hypothesis testing, and the construction of confidence intervals.

CDF Pointwise Convergence

A sequence {Xₙ} converges in distribution to X if Fₙ(x) → F(x) at every continuity point x of F. Discontinuity points of the limiting CDF are deliberately excluded to avoid pathological counterexamples.

Weakest Mode of Convergence

Convergence in distribution is implied by convergence in probability, in Lᵖ, and almost sure convergence. The converse is generally false: distributional convergence alone says nothing about pathwise closeness of random variables.

Characteristic Function Criterion

By Lévy's continuity theorem, Xₙ converges in distribution to X if and only if φₙ(t) → φ(t) for every t ∈ ℝ, where φₙ and φ are the respective characteristic functions. This is often the easiest route to proving CLT-type results.

Portmanteau Equivalences

The Portmanteau theorem provides five equivalent formulations: pointwise CDF convergence at continuity points, E[g(Xₙ)] → E[g(X)] for all bounded continuous g, and inequalities involving open and closed sets, among others.

Slutsky's Theorem Extension

If Xₙ converges in distribution to X and Yₙ converges in probability to a constant c, then Xₙ + Yₙ converges in distribution to X + c and XₙYₙ converges in distribution to cX. This theorem is indispensable in applied asymptotics.

✦

KEY TAKEAWAY

Think of convergence in distribution like tuning a radio: you don't care about the exact electromagnetic waveform arriving at the antenna (the individual realizations of Xₙ); you care only that the statistical profile of the signal—its frequency distribution—stabilizes to a recognizable pattern as you refine the tuning. Two entirely different antennas on different rooftops (different probability spaces) can still converge to the same station (the same limiting distribution).

SECTION 3

Visual Explanation

CDFs Converging to the Standard Normal

The following diagram illustrates the essence of convergence in distribution. Consider the standardized sample mean Zₙ = √n(X̄ₙ − μ)/σ where each Xᵢ follows an exponential distribution. For small n, the CDF of Zₙ deviates noticeably from the standard normal CDF Φ(x), but as n increases, the CDF curves progressively hug the limiting Gaussian curve. Importantly, convergence is assessed only at the continuity points of the limiting CDF—which, for the normal distribution, means every real number.

As n increases from 2 to 30, the CDF of the standardized sample mean from an exponential population (dashed curves) converges pointwise to the standard normal CDF Φ(x) (solid cyan). The convergence is visible at every x, illustrating that convergence in distribution is assessed through the CDFs rather than individual realizations.

Observe in the diagram that for n = 2 (pink, short dashes), the CDF deviates considerably from the Gaussian curve, especially in the left tail where the exponential's inherent right-skewness is most apparent. By n = 5 (violet), the approximation improves, and by n = 30 (amber), the two curves are virtually indistinguishable. This visual progression is exactly what the formal definition captures: for every fixed x at which Φ is continuous—which is every real number—the sequence Fₙ(x) approaches Φ(x). The rate of this convergence is governed by the Berry–Esseen theorem, which provides an O(n⁻¹ᐟ²) uniform bound on the CDF difference when the third absolute moment is finite.

SECTION 4

Mathematical Framework

The formal definition of convergence in distribution requires careful attention to the continuity points of the limiting CDF. This subtlety is not merely pedantic—it prevents false conclusions when the limit involves point masses. In this section we present the definition, the characteristic function criterion, and the Portmanteau equivalences that together form the mathematical backbone of the theory.

DEFINITION OF CONVERGENCE IN DISTRIBUTION

Xₙ →ᵈ X ⟺ lim_{n→∞} Fₙ(x) = F(x) for all x ∈ C(F)

Here Fₙ is the CDF of Xₙ, F is the CDF of X, and C(F) denotes the set of continuity points of F. We write Xₙ →ᵈ X or equivalently Xₙ ⇒ X.

LÉVY CONTINUITY THEOREM

Xₙ →ᵈ X ⟺ φ_{Xₙ}(t) → φ_X(t) for all t ∈ ℝ

Where φ_{Xₙ}(t) = E[ $e^{itXₙ}$ ] is the characteristic function of Xₙ. Convergence of characteristic functions pointwise to a function that is continuous at t = 0 guarantees the limit is itself a characteristic function, ensuring the existence of the limiting distribution.

PORTMANTEAU THEOREM (SELECTED EQUIVALENCES)

Xₙ →ᵈ X ⟺ E[g(Xₙ)] → E[g(X)] for all bounded continuous g : ℝ → ℝ

Additional equivalent conditions include: lim sup P(Xₙ ∈ C) ≤ P(X ∈ C) for all closed sets C, and lim inf P(Xₙ ∈ G) ≥ P(X ∈ G) for all open sets G. These set-theoretic conditions are especially useful when working in metric spaces beyond ℝ.

SLUTSKY'S THEOREM

If Xₙ →ᵈ X and Yₙ →ᵖ c, then Xₙ + Yₙ →ᵈ X + c and XₙYₙ →ᵈ cX

Slutsky's theorem is critical in applied statistics: it allows us to replace unknown parameters with consistent estimators inside asymptotic arguments without disrupting the distributional limit. Note that both Xₙ and Yₙ must converge—Xₙ in distribution and Yₙ in probability to a constant—for the result to hold.

⚠

Common Pitfall

Convergence in distribution does not imply that E[Xₙ] → E[X]. Expectations involve the entire tail behavior, while distributional convergence only constrains how probabilities accumulate. A classic counterexample: let Xₙ = n with probability 1/n and 0 otherwise. Then Xₙ →ᵈ 0, yet E[Xₙ] = 1 for all n.

SECTION 5

Convergence Hierarchy & Relationships

One cannot fully appreciate convergence in distribution without understanding where it sits in the broader hierarchy of stochastic convergence modes. There are four principal modes—almost sure convergence, convergence in Lᵖ, convergence in probability, and convergence in distribution—and the implications among them are strict. The diagram below maps these relationships, with arrows pointing from stronger to weaker modes.

The hierarchy of convergence modes. Almost sure convergence implies both convergence in Lᵖ (when dominated by an Lᵖ-integrable random variable) and convergence in probability. Both of these in turn imply convergence in distribution. The reverse implications are generally false, with one notable exception: convergence in distribution to a constant is equivalent to convergence in probability to that constant.

The exception noted at the bottom of the diagram deserves emphasis. When the limiting random variable X is degenerate—i.e., P(X = c) = 1 for some constant c—then convergence in distribution, convergence in probability, and almost sure convergence (along a subsequence) all coincide. This is because the CDF of a constant is a step function with a single jump, and pointwise convergence of CDFs to such a function forces the probability mass of Xₙ to concentrate near c. This equivalence is used frequently in proving consistency of estimators: one often shows convergence in distribution to a constant to conclude convergence in probability.

Mode	Requires Same Probability Space?	Implication Chain
Almost Sure	Yes	⇒ In probability ⇒ In distribution
In Lᵖ	Yes	⇒ In probability ⇒ In distribution
In Probability	Yes	⇒ In distribution
In Distribution	No	Weakest; to constant ⇔ in probability

Comparison of convergence modes with respect to probability-space requirements and implication strength.

SECTION 6

Worked Example

We now demonstrate how to prove convergence in distribution using the characteristic function method. This example applies the Central Limit Theorem machinery from scratch to a specific distribution, showing each algebraic step in detail.

CLT for Exponential(1) via Characteristic Functions

1
Step 1 — Set Up the Standardized Sum

Let X₁, X₂, …, Xₙ be i.i.d. Exponential(1) random variables with mean μ = 1 and variance σ² = 1. Define the standardized sample mean as Zₙ = √n(X̄ₙ − 1). We wish to show Zₙ →ᵈ N(0, 1).

Zₙ = ( $S_n$ − n) / √n where Sₙ = X₁ + ⋯ + Xₙ
2
Step 2 — Compute the Characteristic Function of Xᵢ

The characteristic function of an Exponential(1) random variable is φ_X(t) = (1 − it)⁻¹. The characteristic function of the centered variable Xᵢ − 1 is therefore φ_{X-1}(t) = $e^{−it}$ · (1 − it)⁻¹.

φ_{X−1}(t) = $e^{−it}$ / (1 − it)
3
Step 3 — Form the Characteristic Function of Zₙ

Since Zₙ is the sum of n independent copies of (Xᵢ − 1) scaled by 1/√n, we have φ_{Zₙ}(t) = [φ_{X−1}(t/√n)]ⁿ = [e^{−it/√n} / (1 − it/√n)]ⁿ.

φ_{Zₙ}(t) = [e^{−it/√n} · (1 − it/√n)⁻¹]ⁿ
4
Step 4 — Take the Logarithm and Expand

Taking the natural logarithm: log φ_{Zₙ}(t) = n[−it/√n − log(1 − it/√n)]. Using the Taylor expansion log(1 − u) = −u − u²/2 − u³/3 − ⋯ with u = it/√n, we get −log(1 − it/√n) = it/√n + (it)²/(2n) + O(n⁻³ᐟ²). Substituting: log φ_{Zₙ}(t) = n[−it/√n + it/√n + (it)²/(2n) + O(n⁻³ᐟ²)] = n[(it)²/(2n) + O(n⁻³ᐟ²)] = −t²/2 + O(n⁻¹ᐟ²).

log φ_{Zₙ}(t) = −t²/2 + O(n⁻¹ᐟ²)
5
Step 5 — Conclude via Lévy's Continuity Theorem

As n → ∞, log φ_{Zₙ}(t) → −t²/2, so φ_{Zₙ}(t) → $e^{−t²/2}$ , which is the characteristic function of the standard normal distribution N(0,1). Since this limit is continuous at t = 0, Lévy's continuity theorem guarantees that Zₙ →ᵈ N(0, 1).

Zₙ = √n(X̄ₙ − 1) →ᵈ N(0, 1) ∎

💡

Why Characteristic Functions?

The characteristic function approach converts a question about distributions (which involve integrals and CDFs) into a question about pointwise limits of complex-valued functions. The logarithmic expansion in Step 4 is the universal mechanism behind the CLT: for any distribution with finite variance, the second-order term in the Taylor expansion always yields −t²/2, and higher-order terms vanish as n → ∞.

SECTION 7

Strengths, Limitations & Practical Considerations

Convergence in distribution is extraordinarily useful in theoretical and applied statistics, but its permissiveness comes with important caveats that practitioners must internalize. The following table contrasts the advantages that make it the workhorse of asymptotic statistics with the limitations that can lead to misinterpretation if ignored.

Strengths	Limitations
No common probability space needed: random variables can be defined on entirely separate experiments.	Does not imply convergence of moments: E[Xₙ] may not converge to E[X] without uniform integrability.
Directly applicable via Lévy's theorem and characteristic functions, enabling elegant CLT proofs.	Cannot be used to make pathwise statements about individual sample paths or realizations.
Preserved under continuous transformations by the Continuous Mapping Theorem.	Joint convergence in distribution of (Xₙ, Yₙ) does NOT follow from marginal convergence of Xₙ and Yₙ separately.
Slutsky's theorem allows mixing distributional and probability convergence for practical asymptotic arguments.	Rate of convergence is not specified by the definition; the Berry–Esseen theorem or delta method must be invoked separately.
Generalizes naturally to random vectors and stochastic processes (weak convergence in metric spaces).	For finite samples, the approximation quality depends heavily on the underlying distribution's skewness and kurtosis.

Strengths and limitations of convergence in distribution as a tool in statistical inference.

✦

KEY TAKEAWAY

Convergence in distribution is like reading a weather forecast that correctly predicts the probability distribution of tomorrow's temperature but tells you nothing about the specific sequence of atmospheric events that will produce that temperature. It is a statement about the ensemble of possible outcomes, not about any particular realization. This makes it ideal for designing hypothesis tests and confidence intervals (which are inherently probabilistic) but insufficient for tracking individual sample paths.

SECTION 8

Connections to Advanced Theory

Convergence in distribution is not an endpoint but a gateway to deeper results in probability theory and mathematical statistics. It connects directly to the Continuous Mapping Theorem (if Xₙ →ᵈ X and g is continuous, then g(Xₙ) →ᵈ g(X)), the Delta Method (which derives the asymptotic distribution of smooth functions of asymptotically normal estimators), and weak convergence in metric spaces (the Prokhorov/Billingsley framework for stochastic processes). The table below sketches how the basic concept generalizes.

Basic Concept	Advanced Extension	Key Implication
Xₙ →ᵈ X on ℝ	Weak convergence Pₙ ⇒ P on (S, d) (Polish spaces)	Enables Donsker's theorem (empirical process theory) and functional CLTs.
CLT: √n(X̄ − μ) →ᵈ N(0, σ²)	Delta Method: √n(g(X̄) − g(μ)) →ᵈ N(0, [g′(μ)]²σ²)	Derives asymptotic distributions of variance-stabilizing and other transforms.
Slutsky's Theorem for scalars	Cramér–Wold device for random vectors	Xₙ →ᵈ X in ℝᵈ iff a'Xₙ →ᵈ a'X for all a ∈ ℝᵈ.
Convergence of CDFs	Tightness + finite-dimensional convergence (Prokhorov's theorem)	A family of probability measures is relatively compact iff it is tight—used to prove existence of distributional limits.

How convergence in distribution connects to advanced probability theory.

Perhaps the most consequential extension is Donsker's theorem (the functional CLT), which establishes that the rescaled partial-sum process converges in distribution—in the space of càdlàg functions equipped with the Skorokhod topology—to Brownian motion. This result underlies the theory of Kolmogorov–Smirnov tests, Cramér–von Mises tests, and much of modern nonparametric inference. For students continuing into measure-theoretic probability, Billingsley's Convergence of Probability Measures remains the definitive reference on weak convergence in general metric spaces.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

Explain why convergence in distribution does not require the random variables Xₙ and X to be defined on the same probability space, whereas convergence in probability does. What property of the definition makes this possible?

Show answer

Convergence in distribution is defined entirely through the CDFs: Fₙ(x) → F(x) at continuity points of F. Since CDFs are deterministic functions of a real variable—they summarize the law of the random variable without referencing the underlying sample space—the definition involves no joint events of the form {|Xₙ − X| > ε}, which would require Xₙ and X to live on the same space. In contrast, convergence in probability requires P(|Xₙ − X| > ε) → 0, which is meaningful only when Xₙ and X are defined jointly. This independence from the probability space is precisely what makes convergence in distribution the most flexible mode and why it suffices for deriving asymptotic distributions of test statistics.

Question 2: BASIC CALCULATION

Let Xₙ ~ Uniform(0, 1 + 1/n) for each n ≥ 1. Show that Xₙ →ᵈ X where X ~ Uniform(0, 1). At which points must you verify convergence of the CDFs?

Show answer

The CDF of Xₙ is Fₙ(x) = 0 for x < 0, Fₙ(x) = x/(1 + 1/n) for 0 ≤ x ≤ 1 + 1/n, and Fₙ(x) = 1 for x > 1 + 1/n. The limiting CDF is F(x) = 0 for x < 0, F(x) = x for 0 ≤ x ≤ 1, and F(x) = 1 for x > 1. The continuity points of F are all of ℝ except possibly x = 0 and x = 1; since F is actually continuous everywhere (the uniform CDF has no jumps), we check all x. For any fixed x ∈ (0, 1), Fₙ(x) = x/(1 + 1/n) → x = F(x). For x > 1, eventually 1 + 1/n < x is false only briefly, but Fₙ(x) = 1 → 1 = F(x) for sufficiently large n. For x < 0 or x = 0, both sides equal 0. Hence Fₙ(x) → F(x) at every x, confirming Xₙ →ᵈ Uniform(0,1).

Question 3: INTERMEDIATE

Let X₁, X₂, … be i.i.d. Bernoulli(p) with 0 < p < 1. Using the characteristic function method, prove that Zₙ = (Sₙ − np)/√(np(1−p)) →ᵈ N(0,1), where Sₙ = X₁ + ⋯ + Xₙ.

Show answer

The characteristic function of a single Bernoulli(p) variable is φ_X(t) = 1 − p +

pe^{it}

. The centered variable Xᵢ − p has characteristic function φ_{X−p}(t) =

e^{−ipt}

(1 − p +

pe^{it}

). Writing q = 1 − p and scaling by 1/√(npq), the characteristic function of Zₙ becomes φ_{Zₙ}(t) = [e^{−ipt/√(npq)} · (q + pe^{it/√(npq)})]ⁿ. Taking the logarithm and expanding via Taylor series: log(q +

pe^{iu}

) = log(1 + p(

e^{iu}

− 1)) ≈ p(iu) + p(iu)²/2 − p²(iu)²/2 + O(u³) where u = t/√(npq). After combining with the −ipt/√(npq) term and multiplying by n, the first-order terms cancel (by construction of the centering), the second-order term yields −t²/2, and all higher-order terms are O(n⁻¹ᐟ²). Hence log φ_{Zₙ}(t) → −t²/2, giving φ_{Zₙ}(t) →

e^{−t²/2}

, the characteristic function of N(0,1). By Lévy's continuity theorem, Zₙ →ᵈ N(0,1).

Question 4: APPLIED

A quality engineer measures the breaking strength of n = 100 ceramic tiles. The population distribution is unknown but has mean μ = 500 N and standard deviation σ = 40 N. Using convergence in distribution (the CLT), construct an approximate 95% confidence interval for μ. Then explain why Slutsky's theorem justifies replacing σ with the sample standard deviation s in practice.

Show answer

By the CLT, √n(X̄ − μ)/σ →ᵈ N(0,1). For n = 100, the approximate 95% CI is X̄ ± z₀.₀₂₅ · σ/√n = X̄ ± 1.96 × 40/10 = X̄ ± 7.84 N. In practice, σ is unknown and replaced by the sample standard deviation s. By the law of large numbers, s →ᵖ σ (s is a consistent estimator of σ). Since √n(X̄ − μ)/σ →ᵈ N(0,1) and s/σ →ᵖ 1, Slutsky's theorem gives √n(X̄ − μ)/s = [√n(X̄ − μ)/σ] · (σ/s) →ᵈ N(0,1) · 1 = N(0,1). Therefore replacing σ with s does not alter the limiting distribution, and the confidence interval X̄ ± 1.96 · s/√n remains asymptotically valid. This is the standard justification for large-sample z-intervals even when the population variance is estimated.

Question 5: CRITICAL THINKING

Let Xₙ be a sequence of random variables with Xₙ = n with probability 1/n² and Xₙ = 0 with probability 1 − 1/n². (a) Does Xₙ →ᵈ 0? (b) Does E[Xₙ] → 0? (c) Does E[Xₙ²] → 0? Use this example to explain why convergence in distribution does not preserve moment convergence in general, and state a sufficient condition under which it does.

Show answer

(a) Yes. The CDF of Xₙ satisfies Fₙ(x) = 0 for x < 0, Fₙ(x) = 1 − 1/n² for 0 ≤ x < n, and Fₙ(x) = 1 for x ≥ n. For any fixed x > 0, once n > x we have Fₙ(x) = 1 − 1/n² → 1 = F(x) where F is the CDF of the degenerate distribution at 0 (F(x) = 1 for x ≥ 0⁺). At x = 0, F has a jump so we need not check convergence there. Hence Xₙ →ᵈ 0. (b) Yes. E[Xₙ] = n · (1/n²) = 1/n → 0. So moments happen to converge here. (c) No. E[Xₙ²] = n² · (1/n²) = 1 for all n, which does not converge to E[0²] = 0. This shows that convergence in distribution does not guarantee convergence of higher moments. The sufficient condition is uniform integrability: if {|Xₙ|ᵖ} is uniformly integrable and Xₙ →ᵈ X, then E[|Xₙ|ᵖ] → E[|X|ᵖ]. Equivalently, convergence in distribution plus convergence of the pth moment implies convergence in Lᵖ.

SUMMARY

Summary

Convergence in distribution (also called weak convergence) asserts that the CDFs Fₙ(x) converge pointwise to F(x) at every continuity point of the limiting CDF F. It is the weakest mode of convergence in the standard hierarchy—implied by almost sure convergence, convergence in Lᵖ, and convergence in probability—but it is by far the most commonly invoked mode in statistical practice. Its power derives from tools like Lévy's continuity theorem (characteristic functions converge pointwise if and only if distributions converge weakly) and the Portmanteau theorem (five equivalent characterizations via bounded continuous functions, open sets, closed sets, and CDFs).

The Central Limit Theorem is the most celebrated application: standardized sums of i.i.d. random variables with finite variance converge in distribution to the standard normal. Slutsky's theorem extends the utility by allowing consistent estimators to replace unknown parameters without disrupting the distributional limit. Key caveats include the fact that convergence in distribution does not guarantee convergence of moments (without uniform integrability), does not require a common probability space, and when the limit is a constant, it coincides with convergence in probability. These results form the bedrock of asymptotic inference, confidence interval construction, and hypothesis testing throughout modern statistics.