Sigma-Algebras & Probability Axioms — Sigma-algebras and probability axioms (conceptual)

For centuries, probability was guided by intuition and combinatorial reasoning—counting favorable outcomes and dividing by totals. While this approach sufficed for card games and dice, it became increasingly clear by the late nineteenth century that a rigorous, axiomatic foundation was needed. Paradoxes arose when mathematicians attempted to assign probabilities to continuous sample spaces or to infinite collections of events. The question loomed: which subsets of outcomes should be considered "events" at all, and what rules must a probability function obey? Answering these questions required importing ideas from measure theory and set algebra, ultimately culminating in one of the most influential contributions to twentieth-century mathematics.

The central insight driving this history is deceptively simple: before you can assign probabilities to events, you must first decide which collections of outcomes qualify as events. A σ-algebra provides exactly this structure, and Kolmogorov's axioms tell us how a probability function should behave on such a collection. Together, they resolve centuries of conceptual ambiguity and furnish the language in which all of modern probability is written.

The entire probabilistic framework rests on three interrelated objects: a sample space Ω, a σ-algebra ℱ of subsets of Ω, and a probability measure P defined on ℱ. The triple (Ω, ℱ, P) is called a probability space. Understanding why each component is necessary—and what constraints each must satisfy—is the core intellectual content of this lesson.

The following diagram illustrates the σ-algebra on a simple sample space Ω = {a, b, c}. The power set 2^Ω contains all 2³ = 8 subsets and is itself a valid σ-algebra (the largest possible one on this Ω). The diagram shows how each element connects to its complement and how unions close the structure.

Notice that the trivial σ-algebra {∅, Ω} is the smallest possible σ-algebra on any Ω: it contains only the certain event and the impossible event. At the other extreme, the power set 2^Ω is the largest—every subset is an event. For finite sample spaces, using the full power set is standard. However, for uncountable spaces like ℝ, the power set is too large (some subsets cannot be assigned a consistent measure), so we work with the Borel σ-algebra, generated by all open intervals.

Kolmogorov's 1933 axioms are elegant in their economy: three conditions suffice to build the entire theory of probability. Let (Ω, ℱ) be a measurable space. A function P : ℱ → ℝ is called a probability measure if and only if the following three axioms hold.

Key Consequences of the Axioms

From these three axioms alone, a rich collection of results follows by purely logical deduction. First, setting A₁ = Ω and A₂ = A₃ = … = ∅ in Axiom 3 yields P(∅) = 0. Second, because any event A and its complement Aᶜ partition Ω, we get P(Aᶜ) = 1 − P(A). Third, for any two events A and B, the inclusion–exclusion principle states P(A ∪ B) = P(A) + P(B) − P(A ∩ B). These are not additional axioms but theorems derived from the axioms, illustrating how much structure the three conditions encode.

A key concept in practice is the σ-algebra generated by a collection of sets. Given an arbitrary family 𝒞 of subsets of Ω, the generated σ-algebra σ(𝒞) is defined as the intersection of all σ-algebras containing 𝒞. Because the intersection of any family of σ-algebras is again a σ-algebra, σ(𝒞) is well-defined and is the smallest σ-algebra containing every set in 𝒞. The most important example in all of probability is the Borel σ-algebra ℬ(ℝ), generated by the open intervals of ℝ.

Consider a sample space Ω = {1, 2, 3} and the collection ℱ = {∅, {1}, {2, 3}, Ω}. We will verify that ℱ is a σ-algebra, then define a valid probability measure on it and compute several probabilities using the axioms.

One might ask: why does Kolmogorov's third axiom demand countable additivity rather than the seemingly more natural condition of finite additivity? This distinction is not pedantic—it is the fulcrum on which the entire power of modern probability rests. Finite additivity alone cannot guarantee that limits of convergent sequences of events behave well, and without well-behaved limits, the law of large numbers, the central limit theorem, and indeed all of asymptotic statistics would collapse.

A probability measure is simply a special case of a measure in the sense of Lebesgue–Carathéodory theory, namely one where the total measure is normalized to 1. Recognizing this connection opens the door to applying the full toolkit of measure theory—dominated convergence, Fubini's theorem, Radon–Nikodym derivatives—to probabilistic problems. The table below contrasts the probabilistic language with general measure-theoretic terminology.