Expected value and variance for discrete random variables |...

SECTION 1

Historical Context & Motivation

The concepts of expected value and variance arose from a deceptively simple question: when a game of chance is interrupted before completion, how should the stakes be fairly divided among the players? This question, known as the problem of points, was posed to Blaise Pascal by the Chevalier de Méré in 1654 and ignited a correspondence between Pascal and Pierre de Fermat that is widely regarded as the birth of modern probability theory. Their solution required assigning a numerical weight to each possible outcome based on its likelihood — the germ of expected value. Over the next two centuries, mathematicians extended this idea far beyond gambling, transforming it into one of the most fundamental tools in statistics, physics, economics, and computer science.

1654

Pascal–Fermat Correspondence

Blaise Pascal and Pierre de Fermat exchange letters on the problem of points, developing the first systematic approach to calculating fair stakes by weighting outcomes by their probabilities.

1657

Huygens' De Ratiociniis in Ludo Aleae

Christiaan Huygens publishes the first formal treatise on probability, introducing the concept of expectatio — the expected value of a random outcome — as a central quantity in games of chance.

1713

Bernoulli's Ars Conjectandi

Jacob Bernoulli's posthumous masterwork formalizes the law of large numbers, demonstrating that the sample mean converges to the expected value as the number of trials grows without bound.

1812

Laplace's Théorie Analytique

Pierre-Simon Laplace provides a comprehensive analytical treatment of probability, systematizing variance and higher-order moments as essential descriptors of a distribution's shape.

1933

Kolmogorov's Axiomatization

Andrey Kolmogorov publishes his axiomatic foundations of probability, placing expected value and variance on rigorous measure-theoretic footing and unifying discrete and continuous frameworks.

At its core, the question these pioneers sought to answer is still the question we face today: given a random variable whose outcomes are governed by a known probability distribution, what single number best summarizes where the distribution is centered, and how spread out are the outcomes around that center? The expected value answers the first question; the variance answers the second. Together, they form the indispensable foundation upon which all further probabilistic analysis is built.

SECTION 2

Core Principles & Definitions

Before computing expected value and variance, we need a precise vocabulary. A discrete random variable X is a function from a sample space to a countable set of real numbers; its behavior is completely specified by its probability mass function (PMF) p(x) = P(X = x). The following foundational ideas govern how we summarize such distributions numerically.

Expected Value (Mean)

The weighted average of all possible outcomes, where each outcome is weighted by its probability. Denoted E[X] or μ, it represents the long-run average of a random variable over many independent trials.

Variance

A measure of how far outcomes typically deviate from the mean. Defined as the expected value of the squared deviation, Var(X) = E[(X − μ)²], it quantifies the dispersion or spread of the distribution.

Standard Deviation

The positive square root of the variance, σ = √Var(X). Because it shares the same units as X, it provides a more interpretable scale of spread than the variance itself.

Linearity of Expectation

For any random variables X, Y and constants a, b: E[aX + bY] = aE[X] + bE[Y]. This holds regardless of dependence between X and Y, making it extraordinarily powerful.

Variance of a Linear Transform

For constants a, b: Var(aX + b) = a²Var(X). Adding a constant shifts the distribution without changing spread; scaling by a multiplies the standard deviation by |a| and the variance by a².

✦ KEY TAKEAWAY

Think of expected value as the balance point of a seesaw. If you placed weights on a number line — each weight proportional to the probability of that outcome — the expected value is exactly where you would position the fulcrum so the seesaw balances. Variance then measures how far apart those weights are from the fulcrum: a tightly clustered set of weights yields low variance, while weights flung to the extremes yield high variance. This mechanical analogy is not merely illustrative; it is mathematically exact, as E[X] is the first moment of the distribution about the origin, and Var(X) is the second central moment.

SECTION 3

Visual Explanation

The following diagram illustrates a concrete discrete random variable — the result of rolling a loaded die — showing the PMF as a bar chart together with the expected value (the vertical dashed line) and the ±1 standard deviation band. Observe how the expected value does not need to coincide with any actual outcome; it is the probability-weighted center of mass of the entire distribution.

The bar chart shows the PMF of a loaded die with probabilities 0.15, 0.20, 0.30, 0.20, 0.10, and 0.05 for outcomes 1 through 6. The pink dashed line marks the expected value E[X] = 2.95, and the violet shaded band spans one standard deviation on each side of the mean.

Notice that the expected value E[X] = 2.95 falls between outcomes 2 and 3, pulled toward 3 because that outcome carries the highest probability weight. The ±1σ band captures the bulk of the probability mass, visually demonstrating that most outcomes are reasonably close to the mean. When the distribution is more spread out — say, if outcomes 1 and 6 both had probability 0.40 — the bars would cluster at the extremes, the variance would increase dramatically, and the ±1σ band would widen to reflect greater uncertainty.

SECTION 4

Mathematical Framework

We now formalize the definitions introduced in Section 2. Let X be a discrete random variable taking values x₁, x₂, …, xₙ (or countably many values) with probability mass function p(xᵢ) = P(X = xᵢ). All sums below run over the support of X, i.e., the set of values where p(xᵢ) > 0.

EXPECTED VALUE (MEAN)

E[X] = μ = Σᵢ xᵢ · p(xᵢ)

Each outcome xᵢ is multiplied by its probability p(xᵢ) and the products are summed. The result μ is the first moment about the origin — the center of mass of the distribution.

VARIANCE (DEFINITION)

Var(X) = σ² = E[(X − μ)²] = Σᵢ (xᵢ − μ)² · p(xᵢ)

Variance measures the average squared deviation from the mean. Squaring ensures that deviations above and below the mean do not cancel, and it penalizes large deviations more heavily than small ones.

VARIANCE (COMPUTATIONAL FORMULA)

Var(X) = E[X²] − (E[X])² = Σᵢ xᵢ² · p(xᵢ) − μ²

This equivalent formula — sometimes called the raw-score formula — is often easier to compute because it avoids subtracting μ from each xᵢ. It follows directly from expanding (X − μ)² and applying linearity of expectation.

STANDARD DEVIATION

σ = √Var(X) = √(E[X²] − μ²)

Because variance is in squared units of X, the standard deviation σ restores the original units and provides a directly interpretable measure of typical deviation from the mean.

📐 Derivation of the Computational Formula

Expand the square: E[(X − μ)²] = E[X² − 2μX + μ²]. By linearity of expectation, this equals E[X²] − 2μ·E[X] + μ². Since E[X] = μ, the middle term becomes −2μ², giving E[X²] − 2μ² + μ² = E[X²] − μ². This derivation relies on linearity of expectation and the fact that μ is a constant, not a random variable.

SECTION 5

Key Properties & Comparison of Common Distributions

Expected value and variance are not merely abstract definitions; they obey a rich set of algebraic rules that make them practical computational tools. Moreover, many standard discrete distributions have clean closed-form expressions for E[X] and Var(X). The diagram below compares the PMFs of three common distributions — the Bernoulli, binomial, and geometric — annotated with their means and variances.

Three panels compare the PMFs of the Bernoulli, Binomial, and Geometric distributions, each with p = 0.4. The Bernoulli is the building block; the Binomial counts successes in n trials; the Geometric counts trials until first success. Note how the Geometric distribution's right skew produces higher variance.

Expected values and variances for common discrete distributions

Distribution	Parameters	E[X]	Var(X)
Bernoulli	p ∈ (0, 1)	p	p(1 − p)
Binomial	n ∈ ℕ, p ∈ (0, 1)	np	np(1 − p)
Geometric	p ∈ (0, 1)	1/p	(1 − p)/p²
Poisson	λ > 0	λ	λ
Discrete Uniform	a, b ∈ ℤ, a ≤ b	(a + b)/2	((b − a + 1)² − 1)/12

SECTION 6

Worked Example

Consider a carnival game where you pay $5 to spin a wheel. The wheel has four sectors labeled with the following prize amounts and probabilities:

Prize distribution for the carnival wheel

Prize ($)	Probability
0	0.40
5	0.30
10	0.20
25	0.10

Let X denote the prize won (before subtracting the cost). We will compute E[X], Var(X), and σ, then determine the expected net gain.

Carnival Wheel: Expected Value & Variance

Step 1 — Verify the PMFCheck that all probabilities sum to 1: 0.40 + 0.30 + 0.20 + 0.10 = 1.00. ✓ The PMF is valid.

Σ p(xᵢ) = 1.00 ✓

Step 2 — Compute E[X]Apply the definition: E[X] = Σ xᵢ · p(xᵢ) = (0)(0.40) + (5)(0.30) + (10)(0.20) + (25)(0.10) = 0 + 1.50 + 2.00 + 2.50.

E[X] = $6.00

Step 3 — Compute E[X²]We need the second moment for the computational variance formula: E[X²] = (0²)(0.40) + (5²)(0.30) + (10²)(0.20) + (25²)(0.10) = 0 + 7.50 + 20.00 + 62.50.

E[X²] = 90.00

Step 4 — Compute Var(X)Using the computational formula: Var(X) = E[X²] − (E[X])² = 90.00 − (6.00)² = 90.00 − 36.00.

Var(X) = 54.00 ($²)

Step 5 — Compute σ and interpretσ = √54.00 ≈ 7.35. The standard deviation tells us that on a typical spin, the prize deviates from the mean of $6.00 by about $7.35 — a large spread relative to the mean, indicating high uncertainty.

σ ≈ $7.35

Step 6 — Expected net gainLet Y = X − 5 be the net gain (prize minus cost). By linearity: E[Y] = E[X] − 5 = 6.00 − 5 = 1.00. On average, you gain $1.00 per spin. Var(Y) = Var(X − 5) = Var(X) = 54.00 (adding a constant does not change variance).

E[net gain] = $1.00 per spin; Var unchanged at 54.00

SECTION 7

Strengths, Limitations & Common Pitfalls

Expected value and variance are powerful summary statistics, but they have both strengths and inherent limitations that every practitioner should understand. The table below contrasts what these measures do well with scenarios where they may be misleading or insufficient.

Strengths vs. limitations of E[X] and Var(X)

Strengths	Limitations
Linearity of expectation applies universally — even for dependent random variables — making multi-variable computations tractable.	E[X] may not correspond to any achievable outcome (e.g., E[die roll] = 3.5), which can be misleading for discrete data.
Variance provides a single-number summary of spread that feeds directly into Chebyshev's inequality and the Central Limit Theorem.	Variance is sensitive to outliers because deviations are squared; a single extreme outcome can dominate the sum.
Closed-form expressions exist for all standard distributions, enabling rapid analysis without simulation.	Two distributions with identical mean and variance can have completely different shapes (e.g., different skewness), so these two numbers do not fully characterize a distribution.
Variance obeys Var(X + Y) = Var(X) + Var(Y) when X and Y are independent — enabling modular variance computations.	Independence is required for the additive variance property; forgetting to account for covariance is one of the most common errors in applied probability.

✦ KEY TAKEAWAY

Expected value and variance are analogous to a GPS coordinate and an uncertainty radius: the GPS pin (E[X]) tells you the center of a region, and the radius (σ) tells you how large the region is. But two cities can have the same center coordinate and the same radius yet look completely different — one might be a dense circle, the other an elongated strip along a river. Higher-order moments (skewness, kurtosis) are needed to capture those structural differences, much as terrain data supplements a flat map.

SECTION 8

Connections to Advanced Theory

The concepts of expected value and variance for discrete random variables extend naturally into several advanced areas. Understanding where these foundational ideas lead helps motivate their mastery and clarifies why they occupy such a central place in the mathematical landscape.

From discrete foundations to advanced probability and statistics

Foundational Concept	Advanced Extension	Key Difference / Addition
E[X] = Σ xᵢ p(xᵢ)	E[X] = ∫ x f(x) dx (continuous)	Summation over a countable support generalizes to Lebesgue/Riemann integration over ℝ.
Var(X + Y) for independent variables	Var(X + Y) = Var(X) + Var(Y) + 2 Cov(X, Y)	Covariance and correlation capture linear dependence; essential for portfolio theory and multivariate statistics.
E[X] as the first moment	Moment-generating function M(t) = E[e^(tX)]	MGFs encode all moments; differentiation recovers E[X], E[X²], etc., and uniquely determine a distribution.
Law of Large Numbers (LLN)	Central Limit Theorem (CLT)	The LLN guarantees convergence of sample means to E[X]; the CLT further states the distribution of the sample mean approaches N(μ, σ²/n).
Chebyshev's inequality: P(\|X − μ\| ≥ kσ) ≤ 1/k²	Concentration inequalities (Hoeffding, Chernoff)	Tighter, distribution-specific bounds replace the universal but loose Chebyshev bound; critical in algorithm analysis and machine learning.

Perhaps the most celebrated consequence of finite variance is the Central Limit Theorem: if X₁, X₂, …, Xₙ are i.i.d. with mean μ and variance σ², then the standardized sample mean (X̄ − μ)/(σ/√n) converges in distribution to the standard normal N(0, 1). This theorem is the reason the normal distribution appears throughout science, and it works precisely because the random variables possess finite expected value and variance. Without these two quantities, the CLT cannot even be stated.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

A fair six-sided die has E[X] = 3.5. Is it possible to observe the value 3.5 on any single roll? Explain what the expected value represents in this context and why it need not equal an achievable outcome.

PROBLEM 2 — BASIC CALCULATION

A discrete random variable X has the following PMF: P(X = 1) = 0.2, P(X = 3) = 0.5, P(X = 7) = 0.3. Compute E[X], E[X²], Var(X), and σ.

PROBLEM 3 — INTERMEDIATE

Let X ~ Binomial(n = 10, p = 0.3). Without listing all 11 probabilities, compute E[X] and Var(X) using the closed-form formulas. Then, if Y = 2X + 5, find E[Y] and Var(Y).

PROBLEM 4 — APPLIED

A network server receives requests according to a Poisson distribution with an average rate of λ = 8 requests per minute. Each request costs $0.03 in computing resources. Let C be the total cost per minute. Find E[C] and σ_C. Use these to determine the smallest budget B such that Chebyshev's inequality guarantees P(C > B) ≤ 0.05.

PROBLEM 5 — CRITICAL THINKING

Prove that for any discrete random variable X with finite second moment, Var(X) ≥ 0, and that Var(X) = 0 if and only if X is a constant with probability 1 (i.e., there exists c ∈ ℝ such that P(X = c) = 1).

SUMMARY

Summary

The expected value E[X] = Σ xᵢ p(xᵢ) provides the probability-weighted center of mass of a discrete distribution, while the variance Var(X) = E[X²] − (E[X])² quantifies the average squared deviation from the mean. The standard deviation σ = √Var(X) restores the original units of measurement. Together, these two summary statistics — the first and second central moments — are the most fundamental descriptors of any probability distribution.

Key operational properties include the linearity of expectation (E[aX + bY] = aE[X] + bE[Y], which holds for all random variables regardless of dependence) and the scaling rule for variance (Var(aX + b) = a²Var(X)). These properties enable efficient computation across standard distributions — Bernoulli, Binomial, Geometric, Poisson, and Discrete Uniform — each of which has clean closed-form formulas for mean and variance. Mastery of expected value and variance is prerequisite to the Law of Large Numbers, Central Limit Theorem, and essentially all further study in probability, statistics, and stochastic processes.