Discrete Random Variables & Expected Value

SECTION 1

Historical Context & Motivation

The idea that chance events could be studied systematically—rather than merely endured—emerged from a celebrated exchange of letters between two French mathematicians in 1654. Blaise Pascal and Pierre de Fermat tackled what was known as the problem of points: how should the stakes of an interrupted gambling game be fairly divided among players, given the current score? Their solution required assigning numerical values to uncertain future outcomes and weighting those values by the likelihood of their occurrence—precisely the logic that underpins the modern concept of expected value.

Over the next three centuries, the framework matured. Christiaan Huygens formalized the notion of expectation in his 1657 treatise; Jacob Bernoulli proved the first version of the law of large numbers; and Andrey Kolmogorov placed the entire enterprise on rigorous measure-theoretic foundations in 1933. The concept of a random variable—a function that maps each outcome of a random experiment to a real number—was the crucial abstraction that unified these advances and made modern probability theory possible.

1654

Pascal–Fermat Correspondence

Blaise Pascal and Pierre de Fermat exchange letters on the problem of points, laying the conceptual groundwork for expected value and probability theory.

1657

Huygens' De Ratiociniis in Ludo Aleae

Christiaan Huygens publishes the first formal treatise on probability, introducing the term expectatio (expectation) for a weighted average of possible outcomes.

1713

Bernoulli's Law of Large Numbers

Jacob Bernoulli's posthumous work Ars Conjectandi proves that observed frequencies converge to true probabilities as the number of trials grows, giving the expected value its 'long-run average' interpretation.

1933

Kolmogorov's Axioms

Andrey Kolmogorov publishes his axiomatization of probability theory, formally defining random variables as measurable functions on a probability space and placing expected value on rigorous footing.

The central question this lesson addresses is deceptively simple: if the outcome of a random experiment is numerical, what single number best summarizes what we should 'expect' in the long run? Answering this question requires two linked ideas—discrete random variables to describe the possible values the experiment can produce, and expected value to compute their probability-weighted center of gravity.

SECTION 2

Core Principles & Definitions

Before computing anything, we need a precise vocabulary. A random variable is a real-valued function defined on the sample space of a probability experiment. When the set of values the function can assume is finite or countably infinite, the variable is called discrete. In contrast, a continuous random variable can take on any value in an interval. This lesson focuses exclusively on the discrete case, where we can list each possible value alongside its probability.

Random Variable (X)

A function X : S → ℝ that assigns a real number to every outcome in the sample space S. The uppercase letter denotes the variable; a lowercase x denotes a specific value it takes.

Probability Mass Function (PMF)

The function p(x) = P(X = x) that gives the probability the random variable equals each specific value x. All probabilities are between 0 and 1, and they sum to exactly 1.

Support of X

The set of all values x for which p(x) > 0. For a fair six-sided die, the support is {1, 2, 3, 4, 5, 6}. The PMF is zero for every value outside the support.

Expected Value E[X]

The probability-weighted average of all values in the support: E[X] = Σ x · p(x). It represents the long-run mean of the random variable over infinitely many independent repetitions.

Variance Var(X)

A measure of spread around the expected value: Var(X) = E[(X − E[X])²] = Σ (x − μ)² · p(x). Its positive square root σ is the standard deviation.

✦ KEY TAKEAWAY

Think of the expected value as the balance point (center of mass) of a probability distribution. Imagine placing weights along a number line, where each weight equals the probability of the corresponding outcome. The point at which the number line would perfectly balance on a fulcrum is E[X]. Just as a physical beam balances at its center of gravity regardless of whether any mass actually sits at that point, the expected value need not be a value the random variable can actually assume—it is a theoretical average, not a guaranteed outcome.

SECTION 3

Visualizing a Discrete PMF and Its Expected Value

The most common visualization of a discrete random variable is a probability histogram (or spike plot), which places a bar or vertical line at each value in the support, with height equal to the corresponding probability. The expected value can then be visualized as a vertical line at the distribution's center of mass. The diagram below illustrates the PMF of a loaded die where the probability of rolling a 6 is twice that of any other face.

The cyan bars represent the five equally-likely faces (each with probability 1/7), while the amber bar for face 6 is twice as tall (probability 2/7). The dashed pink vertical line marks E[X] = 4, the balance point of the distribution. Notice that E[X] = 4 even though P(X = 4) = 1/7—the expected value is pulled upward by the heavier weight on 6.

Examining the diagram reveals an important nuance: the expected value E[X] = 4 lies between the most probable value (6, which has the tallest bar) and the median of the remaining outcomes. The loading of the die shifts the balance point away from the fair-die expectation of 3.5 toward the heavier side. This visual intuition—expected value as the fulcrum of the probability mass—generalizes to every discrete distribution and provides a quick sanity check whenever you compute E[X] algebraically.

SECTION 4

Mathematical Framework

We now formalize the definitions introduced in Section 2. Throughout, X denotes a discrete random variable with support {x₁, x₂, …} and probability mass function p(xᵢ) = P(X = xᵢ). All summations are taken over the support of X.

PROBABILITY MASS FUNCTION (PMF)

p(x) = P(X = x), where 0 ≤ p(x) ≤ 1 and Σ p(xᵢ) = 1

The PMF assigns a probability to each value in the support. The requirement that all probabilities sum to 1 is the discrete analogue of the total-area-equals-one condition for continuous densities.

EXPECTED VALUE (MEAN)

E[X] = μ = Σ xᵢ · p(xᵢ)

Each possible value xᵢ is multiplied by its probability and the products are summed. The result μ (mu) is the population mean of the distribution—the theoretical long-run average.

EXPECTED VALUE OF A FUNCTION OF X

E[g(X)] = Σ g(xᵢ) · p(xᵢ)

This is the Law of the Unconscious Statistician (LOTUS). To find the expected value of any function g applied to X, weight g(xᵢ) by P(X = xᵢ). No need to derive the distribution of g(X) first.

VARIANCE AND STANDARD DEVIATION

Var(X) = σ² = E[(X − μ)²] = E[X²] − (E[X])²; σ = √Var(X)

The variance measures the average squared deviation from the mean. The computational shortcut E[X²] − (E[X])² often simplifies hand calculations by avoiding the subtraction (xᵢ − μ) at every term.

⚡ Linearity of Expectation

Two fundamental properties hold for any random variables X and Y and constants a, b: E[aX + b] = a · E[X] + b and E[X + Y] = E[X] + E[Y]. The second property holds even when X and Y are dependent. Linearity of expectation is arguably the single most useful property in applied probability and is used repeatedly in deriving the expected values of named distributions (binomial, Poisson, geometric, etc.).

SECTION 5

Common Discrete Distributions & Their Expected Values

Many real-world experiments produce discrete random variables whose PMFs follow well-known patterns. Recognizing these patterns saves considerable computation because their expected values and variances have closed-form expressions derived from the general formula. The table and diagram below summarize the most frequently encountered discrete distributions at the introductory level.

Key discrete distributions with their PMFs, expected values, and variances

Distribution	PMF p(x)	Support	E[X]	Var(X)
Bernoulli(p)	p(1) = p, p(0) = 1 − p	{0, 1}	p	p(1 − p)
Binomial(n, p)	C(n,x) · p^x · (1−p)^n−x	{0, 1, …, n}	np	np(1 − p)
Geometric(p)	(1 − p)^x−1 · p	{1, 2, 3, …}	1/p	(1 − p)/p²
Poisson(λ)	e^−λ · λ^x / x!	{0, 1, 2, …}	λ	λ
Discrete Uniform(a, b)	1 / (b − a + 1)	{a, a+1, …, b}	(a + b) / 2	[(b − a + 1)² − 1] / 12

Side-by-side comparison of the Binomial(10, 0.3) and Poisson(3) PMFs. Both distributions share the same expected value E[X] = 3, yet their shapes and spreads differ. The Binomial is bounded above by n = 10, while the Poisson has no finite upper bound and exhibits slightly greater variance.

The side-by-side comparison underscores a crucial insight: the expected value alone does not fully characterize a distribution. Two distributions can share the same E[X] while differing in variance, skewness, and support. The Binomial is bounded above by n and becomes more symmetric as p approaches 0.5, whereas the Poisson extends to infinity and is always right-skewed (though the skewness decreases as λ grows). Recognizing which named distribution fits a given scenario allows you to look up E[X] and Var(X) directly rather than computing them term by term.

SECTION 6

Worked Example: Insurance Claim Payout

Suppose an insurance company offers a one-year policy with the following payout structure. The policyholder pays a premium of $500 at the start of the year. If no claim occurs (probability 0.80), the company pays nothing. If a minor claim occurs (probability 0.15), the company pays $2,000. If a major claim occurs (probability 0.05), the company pays $20,000. Let X be the company's net profit per policy (premium collected minus payout). We wish to find E[X] and Var(X).

Expected Profit on an Insurance Policy

Step 1 — Define the Random VariableThe company always collects $500 in premium. Net profit X = 500 − (payout). The possible values of X are: if no claim, X = 500 − 0 = 500; if minor claim, X = 500 − 2,000 = −1,500; if major claim, X = 500 − 20,000 = −19,500.

Support of X: {−19,500, −1,500, 500}

Step 2 — Construct the PMFAssign each value its probability: P(X = 500) = 0.80, P(X = −1,500) = 0.15, P(X = −19,500) = 0.05. Verify: 0.80 + 0.15 + 0.05 = 1.00 ✓

PMF verified: probabilities sum to 1

Step 3 — Compute E[X]Apply E[X] = Σ xᵢ · p(xᵢ): E[X] = (500)(0.80) + (−1,500)(0.15) + (−19,500)(0.05) = 400 + (−225) + (−975) = −800.

E[X] = −$800

Step 4 — Interpret E[X]On average, the company loses $800 per policy per year. This signals that the premium of $500 is insufficient to cover the expected claims. To break even in expectation, the company would need to charge a premium of at least $1,300 (since payouts average $1,300: 0.15 × 2,000 + 0.05 × 20,000 = 300 + 1,000 = 1,300).

Break-even premium ≈ $1,300

Step 5 — Compute Var(X)First find E[X²]: E[X²] = (500²)(0.80) + (−1,500²)(0.15) + (−19,500²)(0.05) = (250,000)(0.80) + (2,250,000)(0.15) + (380,250,000)(0.05) = 200,000 + 337,500 + 19,012,500 = 19,550,000. Then Var(X) = E[X²] − (E[X])² = 19,550,000 − (−800)² = 19,550,000 − 640,000 = 18,910,000. Standard deviation σ = √18,910,000 ≈ $4,348.56.

Var(X) = 18,910,000; σ ≈ $4,348.56

📊 Practical Note

The large standard deviation ($4,349) relative to the mean (−$800) tells us that individual policy outcomes are highly variable. Most policies yield a $500 profit, but the rare major claim wipes out the gains from many profitable policies. Insurers manage this risk by pooling thousands of policies: by the law of large numbers, the average profit per policy converges to E[X], and the per-policy standard deviation shrinks by a factor of 1/√n.

SECTION 7

Properties, Strengths, and Common Pitfalls

The expected value is the most widely used summary of a probability distribution, but it is not without limitations. The following table contrasts its strengths against common misconceptions and pitfalls that arise in practice.

Strengths and limitations of expected value as a distributional summary

Strengths	Limitations / Pitfalls
Linearity: E[aX + bY + c] = aE[X] + bE[Y] + c always holds, simplifying complex calculations.	Non-multiplicative in general: E[XY] ≠ E[X] · E[Y] unless X and Y are independent.
Law of large numbers guarantees convergence of sample means to E[X], giving the expected value operational meaning.	E[X] can be infinite or undefined for heavy-tailed distributions (e.g., the St. Petersburg paradox).
Basis for decision theory: choosing the action that maximizes expected payoff is a foundational rationality criterion.	Does not capture risk: two gambles with the same E[X] may differ vastly in variance and tail risk.
Decomposition via LOTUS allows E[g(X)] to be computed without finding the full distribution of g(X).	E[g(X)] ≠ g(E[X]) in general (Jensen's inequality). E.g., E[X²] ≥ (E[X])² for non-degenerate X.
Intuitive balance-point / center-of-mass interpretation aids understanding and error-checking.	E[X] need not be a possible value of X. For example, E[X] = 3.5 for a fair die, yet X ∈ {1,2,3,4,5,6}.

✦ KEY TAKEAWAY

Expected value is like the GPS-estimated arrival time for a trip: it gives you the best single prediction averaged over all possible traffic conditions, but it tells you nothing about the variance of that arrival time. Two routes might both have an expected travel time of 30 minutes, yet one could range from 25 to 35 minutes while the other could range from 10 to 60. Just as a prudent traveler considers both the average and the variability, a statistician supplements E[X] with Var(X) and higher moments to capture the full picture of uncertainty.

SECTION 8

Connection to Continuous Distributions & Moment-Generating Functions

The discrete framework developed in this lesson extends naturally to the continuous case, where the PMF is replaced by a probability density function (PDF) f(x) and summation is replaced by integration. The conceptual interpretation remains the same—expected value is a probability-weighted average—but the computational machinery shifts from Σ to ∫. Furthermore, the moment-generating function (MGF) M_X(t) = E[e^tX] provides a compact way to encode all moments of a distribution at once, with E[X] recovered as M'(0).

Discrete-to-continuous correspondence

Concept	Discrete Version	Continuous Analogue
Probability function	PMF: p(x) = P(X = x)	PDF: f(x), where P(a ≤ X ≤ b) = ∫ₐᵇ f(x) dx
Expected value	E[X] = Σ xᵢ · p(xᵢ)	E[X] = ∫ x · f(x) dx
Variance	Var(X) = Σ (xᵢ − μ)² · p(xᵢ)	Var(X) = ∫ (x − μ)² · f(x) dx
Normalization	Σ p(xᵢ) = 1	∫ f(x) dx = 1
MGF	M(t) = Σ e^(t·xᵢ) · p(xᵢ)	M(t) = ∫ e^(t·x) · f(x) dx

The transition from discrete to continuous is more than a notational shift; it opens the door to the normal, exponential, and gamma families that dominate applied statistics. However, every key idea—linearity of expectation, LOTUS, the variance shortcut formula, and the law of large numbers—carries over unchanged. Mastering the discrete case therefore builds a foundation that transfers directly to the continuous setting and, eventually, to multivariate distributions, conditional expectations, and Bayesian inference.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

A fair six-sided die has E[X] = 3.5, yet X can never actually equal 3.5. Explain why the expected value can lie outside the support of a discrete random variable. Does this undermine its usefulness? Why or why not?

PROBLEM 2 — BASIC CALCULATION

A discrete random variable X has the following PMF: P(X = 1) = 0.10, P(X = 2) = 0.25, P(X = 3) = 0.30, P(X = 4) = 0.20, P(X = 5) = 0.15. Compute E[X] and Var(X).

PROBLEM 3 — INTERMEDIATE

You play a game where you draw a card from a standard 52-card deck. If you draw an Ace, you win $20. If you draw a face card (J, Q, K), you win $5. For any other card, you lose $3. Let X be your net gain. Find E[X] and determine whether the game favors the player or the house.

PROBLEM 4 — APPLIED

A tech startup estimates that in the coming year it will generate revenue of $0 (probability 0.30), $200,000 (probability 0.40), $500,000 (probability 0.20), or $1,000,000 (probability 0.10). Fixed operating costs are $180,000. Let Y = Revenue − $180,000 be the net profit. Compute E[Y] and σ_Y. Would you advise an investor that this venture has a positive expected return?

PROBLEM 5 — CRITICAL THINKING

Prove that for any discrete random variable X, Var(X) ≥ 0, with equality if and only if X is a constant (i.e., P(X = c) = 1 for some constant c). Then use this result to derive the inequality E[X²] ≥ (E[X])², noting the conditions under which equality holds.

SUMMARY

Lesson Summary

A discrete random variable X maps outcomes of a probability experiment to a countable set of real numbers. Its behavior is fully characterized by the probability mass function (PMF) p(x) = P(X = x), which satisfies 0 ≤ p(x) ≤ 1 and Σ p(x) = 1. The expected value E[X] = Σ xᵢ · p(xᵢ) provides a probability-weighted average that represents the long-run mean of the distribution—its center of mass on the number line. Linearity of expectation (E[aX + bY + c] = aE[X] + bE[Y] + c) is the most powerful computational tool for working with expected values, and the Law of the Unconscious Statistician (LOTUS) lets us compute E[g(X)] without deriving the distribution of g(X).

The variance Var(X) = E[X²] − (E[X])² measures spread around the mean, and its square root σ is the standard deviation. Named families—Bernoulli, Binomial, Geometric, Poisson, and discrete Uniform—have closed-form expressions for E[X] and Var(X) that eliminate term-by-term computation. These discrete concepts transfer directly to the continuous setting (PDF, integrals) and underpin advanced topics including moment-generating functions, conditional expectation, and Bayesian inference.