Random Variables & Distributions — Random variables and distribution functions - Statistics Graduate Level

SECTION 1

Historical Context & Motivation

The notion of a random variable is so central to modern probability and statistics that it is difficult to imagine the discipline without it. Yet for centuries, probabilistic reasoning proceeded without any formal mechanism for mapping uncertain outcomes to numerical quantities. Early work by Pascal and Fermat in the seventeenth century focused on counting favorable outcomes in games of chance, treating probability as a ratio of combinatorial quantities rather than as a measure on an abstract space. The critical intellectual leap — viewing a numerical outcome of an experiment as a function defined on a sample space — took shape only gradually, driven by the demands of actuarial science, statistical mechanics, and eventually the quest for logical rigor in mathematics itself.

1713 • Bernoulli's Ars Conjectandi

Jakob Bernoulli's posthumous masterwork established the law of large numbers and implicitly used the idea of assigning numerical values to random outcomes, though without a formal definition of a random variable.

1812 • Laplace's Théorie analytique

Pierre-Simon Laplace synthesized generating functions and what we now recognize as probability distributions, using moment-generating techniques that presuppose a numerical mapping from outcomes.

1933 • Kolmogorov's Axiomatization

Andrey Kolmogorov published Grundbegriffe der Wahrscheinlichkeitsrechnung, grounding probability theory in measure theory. A random variable was formally defined as a measurable function from a probability space to the real line, unifying all earlier intuitions.

1946 • Cramér's Mathematical Methods

Harald Cramér's textbook brought Kolmogorov's framework to a broad statistical audience, making distribution functions and their properties standard tools for applied researchers and theorists alike.

1960s • Modern Measure-Theoretic Probability

Texts by Loève, Billingsley, and others cemented the measure-theoretic treatment of random variables, extending the framework to random elements in general topological spaces and paving the way for stochastic process theory.

The central question that the concept of a random variable answers is deceptively simple: how do we pass from a qualitative description of uncertainty — the set of possible outcomes of an experiment — to a quantitative, analytically tractable representation that supports calculus, integration, and the full machinery of real analysis? The answer, as Kolmogorov codified it, is to define a measurable function from the sample space to ℝ and then push the probability measure forward through that function, producing what we call a distribution function. Understanding this chain of ideas — from sample space to σ-algebra to measurable mapping to induced distribution — is the foundation upon which virtually all of probability and mathematical statistics is built.

SECTION 2

Core Principles & Definitions

Before defining random variables precisely, it is essential to recall the underlying probability space (Ω, ℱ, P). Here Ω is the sample space (the set of all possible outcomes), ℱ is a σ-algebra of events (subsets of Ω to which probability is assigned), and P is a probability measure on ℱ satisfying Kolmogorov's axioms. A random variable is the bridge that carries this abstract structure into the familiar setting of the real numbers, where we can compute expectations, variances, and perform statistical inference.

Random Variable as Measurable Function

A random variable X is a function X : Ω → ℝ such that for every Borel set B ∈ 𝓑(ℝ), the preimage X⁻¹(B) = {ω ∈ Ω : X(ω) ∈ B} belongs to ℱ. This measurability condition ensures that we can assign probabilities to statements like 'X ≤ x'.

Cumulative Distribution Function (CDF)

The CDF of X is F_X(x) = P(X ≤ x) for all x ∈ ℝ. It completely characterizes the distribution of X and is right-continuous with left limits (càdlàg), non-decreasing, with limits 0 at −∞ and 1 at +∞.

Induced (Pushforward) Measure

The pushforward measure P_X on (ℝ, 𝓑(ℝ)) is defined by P_X(B) = P(X⁻¹(B)). This is the probability distribution of X, and it is uniquely determined by the CDF.

Discrete vs. Continuous Dichotomy

A random variable is discrete if its range is countable (characterized by a PMF), and continuous if its CDF is absolutely continuous with respect to Lebesgue measure (characterized by a PDF). Mixed types also exist and are handled by the Lebesgue decomposition theorem.

σ-Algebra Generated by X

The collection σ(X) = {X⁻¹(B) : B ∈ 𝓑(ℝ)} is the smallest σ-algebra on Ω that makes X measurable. It captures precisely the information content of observing X, a concept central to conditional expectation and filtrations.

✦

KEY TAKEAWAY

Think of a random variable as a translation device. Imagine a laboratory instrument that takes a complex physical phenomenon (the sample space Ω, with all its qualitative richness) and outputs a single number on a dial. The instrument's calibration must be 'compatible' with the measurement apparatus — every readable interval on the dial must correspond to a well-defined event in the lab. That compatibility is precisely the measurability condition. The distribution function is then the instruction manual that tells you how likely each dial reading is.

SECTION 3

Visual Explanation — From Sample Space to Distribution

The diagram shows five sample points ω₁ through ω₅ in the sample space Ω (left ellipse) being mapped by the random variable X to real numbers on the vertical axis. Note that multiple outcomes can map to the same real value (ω₂ and ω₃ both map to 0), which is perfectly valid. The measurability box (right) states the key requirement: preimages of Borel sets must be events in ℱ.

The diagram above captures the essential architecture of a random variable. On the left, the sample space Ω contains all possible experimental outcomes, which need not be numerical — they could be sequences of coin flips, molecular configurations, or any abstract set. The random variable X assigns to each outcome a real number, collapsing the potentially complex structure of Ω into a one-dimensional summary. The measurability condition ensures that this collapsing is 'compatible' with the probability structure: any question of the form 'does X fall in a given Borel set?' can be translated back to a well-defined event in the original space, to which the probability measure P can be applied. When we then define F_X(x) = P(X ≤ x), we are effectively summarizing all probabilistic behavior of X in a single real-valued function — the cumulative distribution function.

SECTION 4

Mathematical Framework

We now formalize the definitions and state the fundamental properties that make random variables and their distribution functions the workhorses of probability theory. Throughout this section, (Ω, ℱ, P) denotes a probability space and 𝓑(ℝ) the Borel σ-algebra on ℝ.

RANDOM VARIABLE DEFINITION

X : (Ω, ℱ) → (ℝ, 𝓑(ℝ)) is measurable ⟺ ∀ x ∈ ℝ, {ω ∈ Ω : X(ω) ≤ x} ∈ ℱ

A function X from Ω to ℝ is a random variable if and only if for every real number x, the set {ω : X(ω) ≤ x} is an event (belongs to ℱ). Equivalently, preimages of all Borel sets lie in ℱ. The equivalence holds because sets of the form (−∞, x] generate 𝓑(ℝ).

CUMULATIVE DISTRIBUTION FUNCTION

F_X(x) = P(X ≤ x) = P({ω ∈ Ω : X(ω) ≤ x}), x ∈ ℝ

F_X satisfies three fundamental properties: (1) Non-decreasing: if a ≤ b then F_X(a) ≤ F_X(b). (2) Right-continuous: lim(x→a⁺) F_X(x) = F_X(a). (3) Boundary conditions: lim(x→−∞) F_X(x) = 0 and lim(x→+∞) F_X(x) = 1.

PROBABILITY MASS FUNCTION (DISCRETE CASE)

p_X(x_k) = P(X = x_k), k = 1, 2, 3, … with Σ_k p_X(x_k) = 1

When X takes values in a countable set {x₁, x₂, …}, the PMF gives the probability of each value. The CDF is then a step function: F_X(x) = Σ_{k : $x_k$ ≤ x} p_X(x_k).

PROBABILITY DENSITY FUNCTION (CONTINUOUS CASE)

F_X(x) = ∫_{−∞}^{x} f_X(t) dt, where f_X(t) ≥ 0 and ∫_{−∞}^{∞} f_X(t) dt = 1

When F_X is absolutely continuous, the PDF f_X is its Radon–Nikodým derivative with respect to Lebesgue measure. Note that f_X(x) is not itself a probability; it is a density, and P(X = x) = 0 for any single point.

📝

Measure-Theoretic Subtlety

By the Lebesgue decomposition theorem, any probability measure on (ℝ, 𝓑(ℝ)) can be uniquely written as the sum of a discrete part, an absolutely continuous part, and a singular continuous part. A random variable's distribution may therefore be a mixture of these three types. The CDF is the universal descriptor that handles all cases, which is why it is the primary object of study in distribution theory.

SECTION 5

Classification of Random Variables & Their Distributions

Random variables are most commonly classified by the nature of their range and the structure of their distribution function. This classification determines which analytical tools — summation, Riemann integration, Lebesgue–Stieltjes integration — are most natural for computing probabilities and expectations. The following diagram and table provide a comprehensive taxonomy.

Three panels compare CDF shapes: a discrete CDF is a right-continuous step function with jumps at each mass point; a continuous CDF is a smooth, everywhere-differentiable curve; and a mixed CDF combines smooth segments with isolated jumps. The lower box summarizes the three defining properties of any CDF and the important converse.

Type	Range of X	Characterization	CDF Structure
Discrete	Countable: {x₁, x₂, …}	PMF p(x_k) = P(X = x_k)	Step function; jumps at mass points
Absolutely Continuous	Uncountable (intervals of ℝ)	PDF f(x) = dF/dx (Radon–Nikodým derivative)	Smooth; differentiable almost everywhere
Singular Continuous	Uncountable but Lebesgue-null support	No PMF, no PDF; F' = 0 a.e. yet F increases	Continuous but not absolutely continuous (e.g., Cantor distribution)
Mixed	Combination of above	Lebesgue decomposition: μ = μ_d + μ_ac + μ_sc	Smooth intervals interspersed with jumps (and possibly singular parts)

Classification of random variables by distribution type

SECTION 6

Worked Example — Constructing and Analyzing a CDF

Consider a random variable X with a mixed distribution. Suppose X has a continuous component on [0, 1) with density f(x) = 1 (uniform) weighted by probability ¾, and a point mass at x = 1 with probability ¼. We will derive the CDF, verify its properties, and compute P(0.5 < X ≤ 1).

Mixed Distribution: CDF Construction and Probability Computation

1
Step 1 — Define the Distribution

The probability measure of X is a mixture: P_X = ¾ × Uniform[0,1) + ¼ × δ₁, where δ₁ is the Dirac point mass at 1. For the continuous part, f(x) = 1 on [0, 1) means F_cont(x) = x for x ∈ [0,1). The discrete part places all its weight at x = 1.

Mixture: ¾ · Uniform[0,1) + ¼ · δ₁
2
Step 2 — Build the CDF Piecewise

For x < 0: F_X(x) = 0 (no probability mass below 0). For 0 ≤ x < 1: F_X(x) = ¾ · x (accumulating the continuous density weighted by ¾). For x ≥ 1: F_X(x) = ¾ · 1 + ¼ = 1 (the continuous part is fully accumulated and the point mass at 1 is included).

F_X(x) = 0 for x < 0; ¾x for 0 ≤ x < 1; 1 for x ≥ 1
3
Step 3 — Verify CDF Properties

Non-decreasing: On [0,1), F_X(x) = ¾x has positive slope ¾. Right-continuity: At x = 0, lim(x→0⁺) ¾x = 0 = F_X(0). At x = 1, lim(x→1⁺) 1 = 1 = F_X(1). The left limit at x = 1 is lim(x→1⁻) ¾x = ¾ ≠ 1, confirming a jump of size ¼ at x = 1. Boundary conditions: lim(x→−∞) F_X(x) = 0 and lim(x→+∞) F_X(x) = 1. ✓

All three CDF properties verified. Jump of ¼ at x = 1.
4
Step 4 — Compute P(0.5 < X ≤ 1)

Using the CDF difference formula: P(a < X ≤ b) = F_X(b) − F_X(a). Here, P(0.5 < X ≤ 1) = F_X(1) − F_X(0.5) = 1 − ¾(0.5) = 1 − ⅜ = ⅝. This probability includes both the continuous density on (0.5, 1) — contributing ¾ × 0.5 = ⅜ — and the point mass at 1 contributing ¼. Indeed ⅜ + ¼ = ⅝. ✓

P(0.5 < X ≤ 1) = ⅝ = 0.625

SECTION 7

PMF vs. PDF vs. CDF — Strengths & Limitations

In practice, probabilists and statisticians work with whichever characterization of a distribution is most convenient for the problem at hand. Each representation — the PMF, PDF, and CDF — has distinctive strengths and limitations. The following table summarizes the trade-offs.

Feature	PMF p(x)	PDF f(x)	CDF F(x)
Applicability	Discrete random variables only	Absolutely continuous random variables only	All random variables — universal
Direct probability?	Yes: p(x) = P(X = x)	No: f(x) is a density, not a probability	Yes: F(x) = P(X ≤ x)
Computing P(a < X ≤ b)	Sum p(x) over x ∈ (a, b]	Integrate f(x) over (a, b]	F(b) − F(a) — no sum or integral needed
Uniqueness	Unique for a given discrete distribution	Unique only up to a.e. equivalence	Unique (right-continuous version)
Handling mixed distributions	Cannot represent continuous part	Cannot represent point masses	Handles all components seamlessly

Comparison of distribution characterizations

✦

KEY TAKEAWAY

Think of the CDF as the universal adapter in an electronics toolkit. A PMF is like a USB-A cable — it works perfectly for discrete devices but cannot connect to continuous ones. A PDF is like a Lightning cable — ideal for smooth, absolutely continuous distributions but useless for point masses. The CDF, by contrast, is a universal port that accepts every type of distribution. This universality is why Kolmogorov's axiomatization centers on distribution functions rather than densities: the CDF exists for any random variable whatsoever.

SECTION 8

Connection to Advanced Theory

The elementary theory of random variables and CDFs serves as the launchpad for a wide array of advanced topics. In this section, we briefly sketch how the foundational ideas connect to multivariate distributions, characteristic functions, convergence theory, and stochastic processes — topics that constitute the remainder of a rigorous probability curriculum.

Foundation Concept	Advanced Extension	Key New Idea
Univariate CDF F(x)	Joint CDF F(x₁, …, x_n)	Random vectors in ℝⁿ; marginals recovered via limits; independence as factorization F = ∏ Fᵢ
PDF / PMF	Characteristic function φ(t) = E[e^itX]	Always exists (unlike MGF); uniquely determines distribution; powerful for proving limit theorems
Single random variable X	Sequences {X_n} and convergence	Four modes: a.s., in probability, in Lᵖ, in distribution. Convergence in distribution is defined via CDFs: F_n(x) → F(x) at continuity points
CDF properties (right-continuity, limits)	Skorokhod space D[0,∞)	Càdlàg paths generalize right-continuous CDFs to stochastic process sample paths; foundational for martingale theory
Pushforward measure P_X	Conditional distributions	Regular conditional probability; disintegration theorem; conditional expectation as a random variable

From foundations to advanced probability theory

The probability integral transform is a particularly elegant theorem that underscores the power of the CDF. If X is a continuous random variable with strictly increasing CDF F, then the transformed variable U = F(X) follows a Uniform(0,1) distribution. Conversely, if U ~ Uniform(0,1), then X = F⁻¹(U) has CDF F. This result is the theoretical foundation of inverse-transform sampling in Monte Carlo simulation and connects random variable theory directly to computational statistics. More deeply, it shows that the CDF is not merely a description of a distribution but a canonical transformation that standardizes any continuous random variable to the uniform — a profound structural insight.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

Explain why the measurability condition in the definition of a random variable is necessary. Specifically, if X : Ω → ℝ were an arbitrary (non-measurable) function, what would go wrong when we attempt to define the CDF F_X(x) = P(X ≤ x)?

Show answer

The CDF is defined as F_X(x) = P({ω ∈ Ω : X(ω) ≤ x}). If X is not measurable, then the set {ω : X(ω) ≤ x} may not belong to the σ-algebra ℱ, and the probability measure P is only defined on ℱ. Consequently, P({ω : X(ω) ≤ x}) would be undefined — the expression has no meaning within the probability framework. Measurability ensures that every preimage of a Borel set is an event, so that all probabilistic queries about X are well-posed. Without it, we cannot define expectations, variances, or any distributional quantity.

Question 2: BASIC CALCULATION

Let X be a discrete random variable with PMF p(0) = 0.3, p(1) = 0.5, p(2) = 0.2. Write out the CDF F_X(x) as a piecewise function and compute P(0.5 < X ≤ 1.5).

Show answer

The CDF is a step function: F_X(x) = 0 for x < 0; F_X(x) = 0.3 for 0 ≤ x < 1; F_X(x) = 0.8 for 1 ≤ x < 2; F_X(x) = 1.0 for x ≥ 2. Then P(0.5 < X ≤ 1.5) = F_X(1.5) − F_X(0.5) = 0.8 − 0.3 = 0.5. This makes intuitive sense: the only mass point in (0.5, 1.5] is x = 1, which carries probability 0.5.

Question 3: INTERMEDIATE

A continuous random variable X has CDF F(x) = 0 for x < 0, F(x) = 1 − e^−2x for x ≥ 0. (a) Find the PDF f(x). (b) Verify that f integrates to 1. (c) Compute P(1 < X < 3) and E[X].

Show answer

(a) Differentiating: f(x) = F'(x) = 2e^−2x for x ≥ 0, and f(x) = 0 for x < 0. This is the Exponential(λ = 2) density. (b) ∫₀^∞ 2e^−2x dx = [−e^−2x]₀^∞ = 0 − (−1) = 1. ✓ (c) P(1 < X < 3) = F(3) − F(1) = (1 − e⁻⁶) − (1 − e⁻²) = e⁻² − e⁻⁶ ≈ 0.1329. E[X] = 1/λ = 1/2 = 0.5, using the known mean of the Exponential distribution.

Question 4: APPLIED

In a reliability engineering context, a device's lifetime T (in years) has CDF F(t) = 1 − (1 + t)e^−t for t ≥ 0. (a) Show that F is a valid CDF by verifying the three defining properties. (b) Find the PDF. (c) What is the probability that the device survives past t = 2 years? (d) Compute the hazard rate h(t) = f(t) / [1 − F(t)].

Show answer

(a) F(0) = 1 − 1·1 = 0 ✓. lim(t→∞) F(t) = 1 − 0 = 1 ✓. f(t) = F'(t) = −e⁻ᵗ + (1+t)e⁻ᵗ = te⁻ᵗ ≥ 0 for t ≥ 0, so F is non-decreasing ✓. Right-continuity follows since F is continuous. (b) f(t) = te⁻ᵗ for t ≥ 0, which is a Gamma(2,1) density. (c) P(T > 2) = 1 − F(2) = (1+2)e⁻² = 3e⁻² ≈ 0.4060. (d) h(t) = te⁻ᵗ / [(1+t)e⁻ᵗ] = t/(1+t). The hazard rate increases from 0 toward 1 as t → ∞, indicating an aging device — a realistic feature for many engineered systems.

Question 5: CRITICAL THINKING

Let U ~ Uniform(0,1). Define X = −(1/λ) ln(1 − U) for λ > 0. (a) Prove that X ~ Exponential(λ) by deriving its CDF from first principles. (b) Explain how this construction relates to the probability integral transform. (c) Discuss why this technique is useful in Monte Carlo simulation and identify a scenario where it fails (i.e., where inverse-transform sampling is impractical).

Show answer

(a) P(X ≤ x) = P(−(1/λ) ln(1−U) ≤ x) = P(ln(1−U) ≥ −λx) = P(1−U ≥ e⁻ˡˣ) = P(U ≤ 1−e⁻ˡˣ) = 1 − e⁻ˡˣ for x ≥ 0, which is the Exponential(λ) CDF. (b) The probability integral transform states that if X has continuous CDF F, then F(X) ~ Uniform(0,1). The construction above inverts this: starting from U ~ Uniform(0,1), we apply F⁻¹(U) = −(1/λ)ln(1−U), which is the quantile function of the Exponential distribution. (c) This technique is computationally efficient whenever F⁻¹ has a closed form. It fails in practice for distributions like the Normal, whose CDF involves the error function with no elementary inverse — in that case, methods like the Box–Muller transform or rejection sampling are preferred.

SUMMARY

Summary — Random Variables & Distribution Functions

Summary

A random variable is a measurable function X : Ω → ℝ, where the measurability condition — requiring that preimages of Borel sets belong to the σ-algebra ℱ — ensures that expressions like P(X ≤ x) are well-defined. The cumulative distribution function (CDF) F_X(x) = P(X ≤ x) is the universal descriptor of a random variable's distribution: it is non-decreasing, right-continuous, and satisfies the boundary conditions F(−∞) = 0, F(+∞) = 1. Every function with these three properties is the CDF of some random variable.

Random variables are classified as discrete (described by a PMF and a step-function CDF), absolutely continuous (described by a PDF and a smooth CDF), or mixed/singular (handled via the Lebesgue decomposition). The probability integral transform connects any continuous CDF to the Uniform(0,1) distribution, providing the theoretical basis for Monte Carlo simulation and a deep structural insight into the nature of randomness. These foundational ideas extend naturally to multivariate distributions, characteristic functions, convergence theory, and stochastic processes.