Common Pitfalls | Varsity Tutors

SECTION 1

Historical Context & Motivation

The history of mathematical statistics is, in many ways, a history of errors corrected. From the earliest attempts to formalize probability in the seventeenth century to the rigorous measure-theoretic foundations laid by Kolmogorov, each generation of statisticians refined not only the theory but also the understanding of where reasoning could go wrong. Common pitfalls in statistical proof and derivation are not merely the mistakes of novices; even celebrated mathematicians have published flawed arguments that were later retracted or amended. Understanding these pitfalls is therefore essential to developing the critical eye necessary for rigorous graduate-level work.

The need for a systematic catalogue of proof errors became acute as statistics matured into a discipline distinct from pure mathematics. Statistical arguments blend algebraic manipulation with probabilistic reasoning, and the interface between these two modes is especially fertile ground for mistakes. An integral that converges pointwise but not uniformly, or an interchange of limit and expectation that fails without dominated convergence, can invalidate an otherwise elegant derivation. The milestones below illustrate how the field's awareness of these pitfalls evolved over centuries.

1713

Bernoulli's Ars Conjectandi

Jakob Bernoulli publishes the first rigorous proof of the Law of Large Numbers, but subsequent popularizations routinely confuse convergence in probability with almost-sure convergence—a pitfall that persists to this day.

1900

Pearson's Chi-Squared Test

Karl Pearson introduces the χ² goodness-of-fit test but miscounts the degrees of freedom when parameters are estimated from data. R. A. Fisher later corrects this, highlighting the pitfall of ignoring estimation effects on null distributions.

1933

Kolmogorov's Axioms

Kolmogorov's measure-theoretic foundations clarify when operations like conditioning on zero-probability events are well-defined, eliminating a class of paradoxes (e.g., the Borel–Kolmogorov paradox) that had plagued informal probability arguments.

1953

Stein's Paradox

Charles Stein demonstrates that the sample mean is inadmissible under squared-error loss in dimension ≥ 3, overturning the 'obvious' belief that the MLE is always best. This underscores the pitfall of uncritical reliance on maximum likelihood.

2005

Ioannidis & the Replication Crisis

John Ioannidis argues that most published research findings are false, catalyzing broad awareness that misuse of p-values and multiple testing are systemic pitfalls in applied statistics.

The central question this lesson addresses is: What are the recurring structural errors in statistical proofs and derivations, and how can we systematically guard against them? By cataloguing these pitfalls explicitly, we develop not just defensive habits but a deeper understanding of why certain mathematical conditions—such as integrability, measurability, and regularity—appear so frequently in theorem statements.

SECTION 2

Core Principles & Taxonomy of Pitfalls

Common pitfalls in statistical proof-writing can be organized into several broad categories. While the specific manifestations vary with the subject matter—whether you are deriving an asymptotic distribution, proving a Rao–Blackwell improvement, or establishing a convergence result—the underlying logical errors tend to fall into a small number of recurring patterns. The concept grid below presents five foundational categories; each subsequent section of this lesson drills deeper into specific instances within these categories.

Illicit Interchange of Limits

Swapping the order of summation, integration, differentiation, or expectation without verifying the conditions (e.g., dominated convergence, Fubini–Tonelli, uniform convergence) that justify the interchange.

Confusion of Convergence Modes

Treating convergence in probability, almost-sure convergence, convergence in distribution, and Lᵖ convergence as interchangeable. Each mode has distinct implications and different conditions under which it holds.

Misapplication of Continuous Mapping & Delta Method

Applying the continuous mapping theorem or delta method when the required function is not continuous at the relevant point, or when the first derivative vanishes and a higher-order expansion is needed.

Regularity Condition Neglect

Invoking results like the Cramér–Rao lower bound, MLE consistency, or the Fisher information identity without checking that regularity conditions—support independence of θ, differentiability under the integral sign—actually hold.

Algebraic & Notational Slips

Errors in matrix calculus, index manipulation, or notational ambiguity—such as conflating random variables with their realizations, or confusing the density of a transformation with the original density.

✦ KEY TAKEAWAY

Think of regularity conditions as the fine print on a contract. A theorem's conclusion is guaranteed only when every clause of the fine print is satisfied. Skipping the verification is like signing a contract without reading it—you may get lucky, but eventually a counterexample will enforce the overlooked clause. In statistical derivations, the 'fine print' typically involves integrability, smoothness, or support conditions, and ignoring even one can render an otherwise correct-looking proof entirely invalid.

SECTION 3

Visual Map of Pitfall Categories

The diagram below provides a visual taxonomy of the five major pitfall categories and their most common specific manifestations. Each branch represents a category from Section 2, with leaf nodes listing the concrete errors encountered in graduate-level proofs. Notice how certain pitfalls—such as failing to verify dominated convergence—appear at the intersection of multiple categories, since they involve both limit interchange and regularity condition neglect.

The tree branches from the central node into five pitfall families. Each leaf node names a specific, frequently encountered error. Note how limit interchange errors and regularity condition neglect share deep structural overlap: both involve skipping verification of technical hypotheses before applying a standard result.

When approaching any proof or derivation in a graduate statistics course, a useful habit is to mentally trace your argument through this taxonomy. At each step, ask: Am I interchanging limits? Am I invoking a named theorem? Have I verified every hypothesis of that theorem? This systematic self-audit, while initially time-consuming, rapidly becomes automatic and dramatically reduces the frequency of errors in both homework solutions and published research.

SECTION 4

Mathematical Framework — Where Proofs Break Down

Many of the most dangerous pitfalls occur at moments when a derivation invokes a standard result—differentiation under the integral sign, dominated convergence, the central limit theorem—without verifying the conditions. In this section we present the key equations and theorems alongside the conditions that are most frequently neglected, so you can see exactly where each pitfall lurks.

Pitfall 1: Differentiation Under the Integral Sign (Leibniz Rule)

LEIBNIZ INTEGRAL RULE

d/dθ ∫ f(x, θ) dμ(x) = ∫ ∂f(x, θ)/∂θ dμ(x)

This interchange is valid when there exists an integrable function g(x) such that |∂f(x, θ)/∂θ| ≤ g(x) for all θ in a neighborhood of the point of interest. Pitfall: In deriving score functions and Fisher information, students routinely move d/dθ inside the integral without establishing such a dominating function. This fails, for instance, with the Uniform(0, θ) family where the support depends on θ.

Pitfall 2: Slutsky's Theorem Misapplication

SLUTSKY'S THEOREM

If Xₙ →d X and Yₙ →p c (a constant), then XₙYₙ →d cX and Xₙ + Yₙ →d X + c

Pitfall: Slutsky requires that Yₙ converge in probability to a constant. If both Xₙ and Yₙ converge only in distribution to non-degenerate random variables, Slutsky does not apply. The joint distribution is not determined by the marginal limits without further assumptions (e.g., independence or convergence in probability of the pair).

Pitfall 3: The Delta Method When g′(θ₀) = 0

STANDARD DELTA METHOD

√n (g(θ̂ₙ) − g(θ₀)) →d N(0, [g′(θ₀)]² σ²) provided g′(θ₀) ≠ 0

Pitfall: When g′(θ₀) = 0, the limiting distribution is degenerate at zero and the standard delta method provides no useful information. One must instead use the second-order delta method: n(g(θ̂ₙ) − g(θ₀)) →d ½ g″(θ₀) σ² χ²₁, which yields a chi-squared rather than a normal limit.

Pitfall 4: Cramér–Rao Bound Without Regularity

CRAMÉR–RAO LOWER BOUND

Var(T) ≥ [b′(θ) + 1]² / I(θ) where I(θ) = E[(∂ log f / ∂θ)²]

Here T is an estimator of τ(θ) with bias b(θ), and I(θ) is the Fisher information. Pitfall: The CRLB requires (i) the support of f does not depend on θ, (ii) differentiation under the integral is valid, and (iii) I(θ) is finite and positive. For the Uniform(0, θ) distribution, condition (i) fails, and the MLE achieves a variance that is O(1/n²)—strictly below any bound the CRLB would suggest.

SECTION 5

Convergence Mode Confusion — A Detailed Breakdown

One of the most pervasive pitfalls for graduate students is the conflation of different modes of convergence. The relationships among almost-sure convergence (a.s.), convergence in probability (→p), convergence in Lᵖ, and convergence in distribution (→d) form a strict hierarchy with several one-way implications. Using the wrong mode at any step of a proof can invalidate the entire argument. The diagram below encodes these relationships and, crucially, highlights the implications that do not hold in general.

Green solid arrows indicate implications that always hold. Red dashed arrows show implications that fail without additional conditions. The classic pitfall is treating convergence in distribution as if it implied convergence in probability, which it does only when the limit is a constant.

Implication table for convergence modes. ✓ = always implies; ✗ = does not imply without additional conditions (noted in parentheses). UI = uniform integrability; subseq. = only subsequence extraction.

From \ To	Almost Sure	Lᵖ	In Probability	In Distribution
Almost Sure	—	✗ (need UI)	✓	✓
Lᵖ	✗	—	✓	✓
In Probability	✗ (subseq.)	✗ (need UI)	—	✓
In Distribution	✗	✗	✗ (unless c)	—

SECTION 6

Worked Example — Spotting and Fixing a Flawed Proof

Consider the following scenario: a student is asked to show that for a random sample X₁, …, Xₙ from Uniform(0, θ), the MLE θ̂ₙ = X₍ₙ₎ (the maximum order statistic) achieves the Cramér–Rao lower bound. This is a classic exam problem precisely because the claim is false, and the proof attempt exposes multiple pitfalls simultaneously.

Flawed Proof Attempt and Its Correction

Step 1 — The Student's Flawed SetupThe student writes the log-likelihood: ℓ(θ) = −n log θ, for x₍ₙ₎ ≤ θ. They then compute the score: ∂ℓ/∂θ = −n/θ, and the Fisher information: I(θ) = E[(−n/θ)²] = n²/θ². They proceed to invoke the CRLB.

ERROR: The support {x : 0 < x < θ} depends on θ, so the CRLB regularity conditions fail.

Step 2 — Identify the PitfallThe Cramér–Rao bound requires that the support of the density does not depend on the parameter θ. For the Uniform(0, θ) family, the indicator function 𝟙(0 < xᵢ < θ) appears in the likelihood, and its derivative with respect to θ involves a Dirac delta at the boundary—not a smooth function. Thus the interchange of differentiation and integration that underlies the identity E[∂ log f / ∂θ] = 0 is invalid.

Pitfall Category: Regularity Condition Neglect (Card 4 from Section 2).

Step 3 — Compute the Actual Variance of the MLEThe CDF of X₍ₙ₎ is F(x) = (x/θ)ⁿ for 0 < x < θ. The density is f(x) = n xⁿ⁻¹ / θⁿ. Therefore E[X₍ₙ₎] = nθ/(n+1) and Var(X₍ₙ₎) = nθ²/[(n+1)²(n+2)]. The variance is O(1/n²), not O(1/n) as the CRLB would suggest.

Var(X₍ₙ₎) = nθ² / [(n+1)²(n+2)] = O(1/n²) — superefficient relative to CRLB.

Step 4 — The Correct ConclusionThe MLE does not 'achieve' the CRLB because the CRLB does not apply. In fact, no unbiased estimator achieves a variance that matches the (inapplicable) CRLB expression. The variance O(1/n²) is strictly better than any rate achievable under standard regularity conditions, which illustrates that non-regular families can admit superefficient estimators. The student should have checked the regularity conditions before invoking the bound.

Lesson: Always verify regularity conditions before invoking the CRLB. Non-regular families can exhibit convergence rates faster than √n.

SECTION 7

Pitfall Severity & Detection Strategies

Not all pitfalls are equally damaging or equally easy to detect. Some—like a missing Jacobian determinant—produce obviously wrong answers (e.g., a density that doesn't integrate to 1) and are therefore quickly caught through sanity checks. Others—like applying the CRLB to a non-regular family—produce answers that look entirely plausible until scrutinized by an expert. The table below ranks the five pitfall categories by severity and detectability, and suggests a primary countermeasure for each.

Comparison of pitfall categories by severity, detectability, and recommended countermeasure.

Pitfall Category	Severity	Detectability	Primary Countermeasure
Illicit Limit Interchange	High — can yield entirely wrong distributions	Low — result may look correct	Cite the specific theorem (DCT, Fubini, MCT) and verify its hypotheses explicitly
Convergence Mode Confusion	High — invalidates asymptotic arguments	Medium — careful reading reveals mode switches	Annotate each convergence statement with its mode; draw the implication diagram
CMT / Delta Method Misuse	Medium — wrong variance or wrong limiting distribution	Medium — checking g′(θ₀) catches most cases	Always compute g′(θ₀) before applying delta method; verify continuity at the limit
Regularity Condition Neglect	Very High — the theorem simply does not apply	Low — result looks formally correct	Maintain a checklist: support, smoothness, integrability, identifiability
Algebraic / Notational Slips	Variable — from typo to fundamental error	High — dimensional analysis, integration checks	Verify densities integrate to 1; check limiting cases; use dimensional analysis

✦ KEY TAKEAWAY

The most dangerous pitfalls are those with high severity but low detectability—particularly illicit limit interchanges and regularity condition neglect. Think of these like structural defects in a building: the walls may look fine, but the foundation is compromised. The best defense is a systematic verification protocol, much like an engineer's checklist before a bridge is loaded. Before concluding any proof, explicitly state which theorems you invoke and verify each hypothesis rather than relying on intuition about whether the result 'should' be true.

SECTION 8

Connection to Advanced Theory & Research Practice

The pitfalls discussed in this lesson are not merely pedagogical curiosities; they connect directly to deep questions in advanced statistical theory. Understanding where standard arguments break down is often the starting point for new research directions. For instance, the failure of the CRLB for non-regular families motivated the development of local asymptotic minimax theory (Le Cam, Hájek), which provides efficiency bounds even when Fisher regularity conditions fail. Similarly, the subtleties of convergence mode selection are central to modern empirical process theory, where uniform convergence of stochastic processes replaces pointwise arguments.

How common pitfalls in graduate-level proofs connect to advanced research topics.

Pitfall in This Lesson	Advanced Theory It Motivates
CRLB fails for non-regular families	Le Cam's local asymptotic normality (LAN) and minimax theory; Hájek–Le Cam convolution theorem
Limit interchange without DCT/Fubini	Uniform integrability criteria; Vitali convergence theorem; martingale convergence in L¹
Delta method fails when g′(θ₀) = 0	Higher-order asymptotic expansions; Edgeworth expansions; bootstrap consistency results
Convergence mode confusion (a.s. vs. →p)	Empirical process theory (Glivenko–Cantelli and Donsker classes); entropy methods for uniform laws of large numbers
Slutsky misapplied to two →d sequences	Joint convergence via Cramér–Wold device; functional delta method in Banach spaces

As you move into research-level work, the pattern is consistent: a 'pitfall' at one level of abstraction becomes a 'research question' at the next. The conditions under which standard tools fail define the frontier of the theory. Developing an instinct for these failure modes is therefore not just about passing qualifying exams—it is the foundation of the critical thinking that distinguishes a competent statistician from a mechanical one.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

State precisely why Slutsky's theorem cannot be applied when both Xₙ →d X and Yₙ →d Y, where Y is a non-degenerate random variable. What additional condition would make the conclusion Xₙ + Yₙ →d X + Y valid?

PROBLEM 2 — BASIC CALCULATION

Let X₁, …, Xₙ be i.i.d. Exponential(λ). A student claims that by the delta method, √n(1/X̄ − λ) →d N(0, λ²). Verify or correct this claim. (Hint: compute g′(μ) where g(μ) = 1/μ and μ = E[X₁] = 1/λ.)

PROBLEM 3 — INTERMEDIATE

Let X₁, …, Xₙ be i.i.d. N(μ, σ²). Consider the transformation g(μ) = μ². Find the asymptotic distribution of √n((X̄)² − μ²). At what value of μ does the standard delta method fail, and what is the correct limiting distribution at that value?

PROBLEM 4 — APPLIED

A researcher models claim sizes at an insurance company as i.i.d. Pareto(α, x₀) with density f(x; α) = α x₀ᵅ x⁻⁽ᵅ⁺¹⁾ for x ≥ x₀ (x₀ known). They compute the MLE α̂ and attempt to construct a confidence interval using the Cramér–Rao bound. For α ≤ 2, E[X²] = ∞. Identify all pitfalls in this approach and suggest a valid alternative.

PROBLEM 5 — CRITICAL THINKING

Construct a concrete counterexample showing that convergence in probability does not imply convergence in L¹. Specifically, define a sequence of random variables {Xₙ} on [0, 1] with Lebesgue measure such that Xₙ →p 0 but E[|Xₙ|] → ∞. Then state the additional condition that would restore the implication and prove that it suffices.

SUMMARY

Lesson Summary

This lesson catalogued the five major families of common pitfalls in statistical proofs and derivations: illicit interchange of limits (violating DCT, Fubini, or MCT conditions), convergence mode confusion (conflating a.s., →p, Lᵖ, and →d), misapplication of the continuous mapping theorem and delta method (especially when g′(θ₀) = 0), regularity condition neglect (invoking the CRLB or MLE consistency without checking support, smoothness, and integrability), and algebraic and notational slips (missing Jacobians, confusing random variables with realizations).

The overarching lesson is one of disciplined verification: every invocation of a named theorem must be accompanied by an explicit check of its hypotheses. The Uniform(0, θ) family served as a recurring counterexample, demonstrating how a non-regular parametric family defeats the CRLB and produces superefficient estimators. The second-order delta method provides the correct limiting distribution when the first derivative vanishes. Uniform integrability is the bridge that connects convergence in probability to convergence in L¹. These pitfalls are not obstacles to avoid and forget—they are signposts pointing toward the deeper structure of statistical theory, from Le Cam's local asymptotic theory to empirical process theory.