Markov & Chebyshev Inequalities — Markov and Chebyshev inequalities

Historical Context & Motivation

Probability theory in the nineteenth century was dominated by a central question: how can one quantify the likelihood that a random variable deviates far from its expected value? Without knowing the precise form of a distribution, exact tail probabilities remain inaccessible, yet practitioners in actuarial science, physics, and engineering needed reliable worst-case guarantees. The Markov inequality and the Chebyshev inequality emerged as two of the earliest and most elegant answers: distribution-free bounds that depend only on moments of the random variable. Their influence stretches from the foundations of measure-theoretic probability to modern concentration-of-measure theory.

The fundamental question these inequalities address is deceptively simple: given only partial information about a random variable — its mean, or its mean and variance — what is the tightest universal bound on the probability that the variable exceeds a given threshold? Their answers are startlingly clean and remain indispensable tools, both as standalone estimates and as building blocks for more sophisticated concentration results.

Core Principles & Definitions

Both the Markov and Chebyshev inequalities belong to the family of moment-based tail bounds. They translate information encoded in the moments of a random variable into probabilistic guarantees about its tail behavior. Understanding these inequalities requires a firm grasp of a few foundational ideas that underpin the entire framework.

Visual Explanation — Markov's Inequality Geometry

The diagram above captures the geometric essence of Markov's inequality. For any nonnegative random variable X with finite mean μ = E[X], the probability that X exceeds a threshold a > 0 is bounded above by μ/a. Geometrically, the pink-shaded tail area cannot exceed the ratio of the mean to the threshold. This bound is intuitive: if the mean is small relative to a, then only a small fraction of the probability mass can reside beyond a. Conversely, when a is close to or smaller than μ, the bound exceeds 1 and becomes trivial — a feature that reminds us Markov's inequality is most informative in the far tail.

Mathematical Framework

Markov's Inequality — Statement and Proof

The proof is elegantly short. Define the indicator variable 𝟙{X ≥ a} which equals 1 when X ≥ a and 0 otherwise. Because X ≥ 0, the inequality a · 𝟙{X ≥ a} ≤ X holds pointwise: whenever the indicator is 1, we have a ≤ X by definition, and whenever the indicator is 0, the left side is 0 ≤ X. Taking expectations on both sides preserves the inequality (expectation is monotone), yielding a · P(X ≥ a) ≤ E[X]. Dividing by a > 0 completes the argument. This proof strategy — comparing an indicator to the original variable and taking expectations — is the template for a vast family of moment-based bounds.

Chebyshev's Inequality — Derivation from Markov

Chebyshev's inequality follows by a single application of Markov's inequality to the nonnegative random variable Y = (X − μ)². Indeed, P(|X − μ| ≥ k) = P((X − μ)² ≥ k²) ≤ E[(X − μ)²] / k² = σ² / k². The transformation from the original variable to its squared deviation is the key insight: it converts a two-sided deviation bound into a Markov problem. The resulting bound decays as 1/k², which is faster than Markov's 1/k decay but still polynomial — a reflection of the limited information (only two moments) being used.

Sharpness & Extremal Distributions

A natural question is whether the Markov and Chebyshev bounds can be improved without additional distributional assumptions. The answer is no — both inequalities are sharp, meaning that for every choice of parameters, there exists a distribution that achieves equality. Understanding these extremal (worst-case) distributions deepens intuition about when the bounds are loose and when they are informative.

For Markov's inequality, the worst-case distribution concentrates all its mass on just two points: 0 and a. By setting P(X = a) = μ/a and P(X = 0) = 1 − μ/a, we get E[X] = a · (μ/a) = μ and P(X ≥ a) = μ/a, matching the bound exactly. This tells us that Markov's inequality is tight when the distribution is maximally 'sparse' — all mass either at 0 or at the threshold. For Chebyshev's inequality, the extremal distribution is a symmetric three-point mass placing probability 1/(2k²) at each of μ − kσ and μ + kσ, and the remaining mass 1 − 1/k² at μ. In both cases, the inequality becomes loose when the true distribution is more 'spread out' than these pathological extremal cases — for instance, a normal distribution has tail probabilities that decay exponentially rather than polynomially.

Worked Examples

Strengths, Limitations & Comparative Analysis

Connections to Advanced Concentration Theory

The Markov and Chebyshev inequalities sit at the base of a rich hierarchy of concentration inequalities. By applying Markov's inequality to e^tX rather than to X itself, one derives the Chernoff bound P(X ≥ a) ≤ inf_t>0 e^−ta M_X(t), where M_X(t) = E[e^tX] is the moment generating function. This exponential optimization typically yields bounds that decay as e^−ca² — exponentially tighter than polynomial moment bounds.

In modern probability and statistics, Chebyshev's inequality also serves as the engine behind the weak law of large numbers. If X₁, X₂, …, X_n are i.i.d. with mean μ and variance σ², then the sample mean X̄_n has variance σ²/n. Applying Chebyshev's inequality gives P(|X̄_n − μ| ≥ ε) ≤ σ²/(nε²) → 0 as n → ∞, establishing convergence in probability. This elegant application demonstrates how a simple moment bound yields one of probability theory's most celebrated convergence results.

Practice Problems

Summary