Geometric and negative binomial distributions (intro) |...

SECTION 1

Historical Context & Motivation

Long before formal probability theory took shape, gamblers, merchants, and natural philosophers grappled with a deceptively simple question: if a certain event happens randomly with some fixed chance, how many attempts should one expect before seeing that event occur? This question lies at the heart of the geometric distribution, and its natural generalization—waiting for multiple successes—leads directly to the negative binomial distribution. These distributions are foundational in discrete probability because they model waiting-time phenomena that arise throughout science, engineering, and everyday decision-making.

The intellectual lineage of these ideas stretches back to the pioneering work on the binomial distribution, which counts how many successes occur in a fixed number of trials. By flipping the perspective—fixing the number of successes and letting the number of trials vary—mathematicians arrived at a powerful complementary framework. The development was gradual, intertwined with advances in combinatorics, generating functions, and the broader theory of stochastic processes.

1713

Bernoulli's Ars Conjectandi

Jacob Bernoulli posthumously published his treatise on probability, formalizing the concept of Bernoulli trials—independent, identical experiments with binary outcomes—which underpin all waiting-time distributions.

1738

De Moivre's Doctrine of Chances

Abraham de Moivre expanded binomial probability calculations and explored the distribution of successes across repeated trials, laying groundwork for the negative binomial concept.

1837

Poisson's Limit and Discrete Distributions

Siméon Denis Poisson published his law of rare events. His work sharpened the distinction between counting successes (binomial) and counting trials until successes (geometric/negative binomial).

1920s

Negative Binomial in Ecology & Quality Control

Statisticians such as Major Greenwood and G. U. Yule applied the negative binomial to model over-dispersed count data in biology and epidemiology, demonstrating its practical value beyond pure mathematics.

The central question these distributions address is: given a sequence of independent Bernoulli trials, each with the same probability of success p, what is the probability that the r-th success occurs on exactly the k-th trial? When r = 1, we obtain the geometric distribution; for general r, the negative binomial. Understanding this relationship illuminates the broader architecture of discrete probability.

SECTION 2

Core Principles & Definitions

Both the geometric and negative binomial distributions rest on the same probabilistic experiment: an infinite sequence of independent Bernoulli trials, each yielding "success" with probability p and "failure" with probability q = 1 − p. The distributions differ only in the stopping criterion: the geometric distribution records the trial number of the first success, while the negative binomial records the trial number of the r-th success. The following principles unify the two.

Independence of Trials

Each trial's outcome is independent of every other trial. The probability of success p remains constant across all trials, regardless of past results—there is no memory in the process.

Binary Outcomes

Every trial produces exactly one of two outcomes: success (S) or failure (F). Complex experiments must be reduced to a dichotomy before these distributions apply.

Memoryless Property (Geometric Only)

The geometric distribution is the only discrete distribution with the memoryless property: P(X > m + n | X > m) = P(X > n). Past failures do not alter the probability of future success.

Geometric as a Special Case

Setting r = 1 in the negative binomial distribution recovers the geometric distribution. Equivalently, the negative binomial is a sum of r independent geometric random variables.

Support on Positive Integers

In the "number of trials" parameterization, the random variable takes values in {r, r + 1, r + 2, …}. In the alternative "number of failures" parameterization, the support begins at 0.

✦ KEY TAKEAWAY

Think of flipping a biased coin until you see heads a certain number of times. The geometric distribution answers "how many flips until the first heads?" while the negative binomial answers "how many flips until the r-th heads?" This is analogous to quality-control sampling on a production line: the geometric models the wait for the first defect, and the negative binomial models the wait for the r-th defect. Both distributions inherit all structure from the underlying Bernoulli process.

SECTION 3

Visual Explanation — PMFs of the Geometric and Negative Binomial

The following diagram plots the probability mass functions (PMFs) for a geometric distribution with p = 0.3 alongside a negative binomial distribution with r = 3 and p = 0.3. Observe how the geometric PMF is a strictly decreasing sequence of bars—each additional failure is multiplicatively less likely—while the negative binomial PMF rises to a mode before declining, reflecting the combinatorial build-up of ways to arrange successes among trials.

The cyan bars represent the geometric PMF (r = 1): P(X = k) = (1 − p)^k−1 p, which decays exponentially from k = 1. The violet bars represent the negative binomial PMF (r = 3): the distribution is shifted right and bell-shaped, peaking near its mean of r/p = 10.

Several features are immediately visible. First, the geometric PMF is monotonically decreasing: the most likely outcome is always k = 1 (instant success on the first trial), and each successive value is multiplied by the factor q = 1 − p. Second, the negative binomial PMF for r ≥ 2 has a unimodal shape with a clear peak. The mode occurs near ⌊(r − 1)/p⌋ + 1, and the distribution becomes more symmetric and concentrated as r increases—a precursor to the normal approximation. Third, the negative binomial's support begins at k = r (you need at least r trials to observe r successes), which is why the violet bars in the diagram start at k = 3.

SECTION 4

Mathematical Framework

We now present the formal probability mass functions, expectations, and variances for both distributions. Throughout, let p ∈ (0, 1] be the success probability and q = 1 − p the failure probability. We use the "number of trials" parameterization, where the random variable X counts the trial on which the r-th success occurs.

Geometric Distribution (r = 1)

GEOMETRIC PMF

P(X = k) = (1 − p)^(k−1) · p, k = 1, 2, 3, …

Here k is the trial on which the first success occurs. The factor (1 − p)^k−1 accounts for the k − 1 failures preceding the success.

GEOMETRIC MEAN & VARIANCE

E[X] = 1/p, Var(X) = (1 − p)/p²

The expected number of trials to achieve the first success is 1/p. For p = 0.3, E[X] ≈ 3.33. The variance grows as p decreases, reflecting the long right tail when success is rare.

Negative Binomial Distribution (general r)

NEGATIVE BINOMIAL PMF

P(X = k) = C(k−1, r−1) · p^r · (1 − p)^(k−r), k = r, r+1, r+2, …

The binomial coefficient C(k−1, r−1) = (k−1)! / [(r−1)!(k−r)!] counts the number of ways to arrange r − 1 successes among the first k − 1 trials (the last trial must be a success). The term p^r covers the r successes, and (1 − p)^k−r covers the k − r failures.

NEGATIVE BINOMIAL MEAN & VARIANCE

E[X] = r/p, Var(X) = r(1 − p)/p²

Since X is the sum of r independent geometric random variables, E[X] = r × (1/p) and Var(X) = r × (1 − p)/p² by the additive properties of expectation and variance for independent variables.

⚠ Parameterization Warning

Some textbooks define Y = X − r as the number of failures before the r-th success. In that case, P(Y = j) = C(j + r − 1, r − 1) · p^r · (1 − p)^j for j = 0, 1, 2, …, with E[Y] = r(1 − p)/p and Var(Y) = r(1 − p)/p². Always check which convention your source uses before applying formulas.

SECTION 5

Derivation & Structural Insights

The negative binomial PMF can be derived by a direct counting argument. Suppose the r-th success occurs on trial k. Then among the first k − 1 trials there must be exactly r − 1 successes (and k − r failures), with the k-th trial being a success. The probability of any specific arrangement of r − 1 successes and k − r failures in k − 1 trials is p^r−1 · (1 − p)^k−r, multiplied by the probability p of succeeding on the k-th trial, giving p^r · (1 − p)^k−r. Since there are C(k − 1, r − 1) ways to choose which of the first k − 1 trials are successes, we obtain the PMF.

This diagram shows a sample sequence of 9 trials containing 3 successes (S) and 6 failures (F). The sequence is decomposed into three geometric waiting blocks (G₁, G₂, G₃), each ending with a success. The negative binomial random variable X equals their sum. The property boxes below contrast the geometric and negative binomial.

The decomposition X = G₁ + G₂ + ⋯ + G_r is not merely a convenient mnemonic—it provides the most elegant route to the negative binomial's mean and variance. Because the G_i are independent, linearity of expectation gives E[X] = r · E[G₁] = r/p, and the additivity of variance for independent random variables yields Var(X) = r · Var(G₁) = r(1 − p)/p². This structural insight also connects to moment-generating functions: since the MGF of a sum of independent variables is the product of their MGFs, M_X(t) = [pe^t / (1 − qe^t)]^r for qe^t < 1.

💡 CDF of the Geometric Distribution

The cumulative distribution function has the closed form P(X ≤ k) = 1 − (1 − p)^k for k ≥ 1. This follows from summing the geometric series, and it gives a direct way to compute tail probabilities: P(X > k) = (1 − p)^k.

SECTION 6

Worked Example

A quality-control engineer inspects circuit boards on an assembly line. Each board independently has a 5% probability of being defective. She inspects boards one at a time. What is the probability that the 3rd defective board is found on exactly the 20th inspection?

Negative Binomial — Quality Control

Step 1 — Identify the DistributionWe are counting the number of independent Bernoulli trials (inspections) until the r = 3rd success (defective board) occurs, with p = 0.05 per trial. This is the negative binomial distribution with parameters r = 3 and p = 0.05.

Step 2 — State the PMFP(X = k) = C(k − 1, r − 1) · p^r · (1 − p)^{k − r}. We need P(X = 20) with r = 3, p = 0.05, q = 0.95.

Step 3 — Compute the Binomial CoefficientC(20 − 1, 3 − 1) = C(19, 2) = 19! / (2! · 17!) = (19 × 18) / 2 = 171.

C(19, 2) = 171

Step 4 — Compute the Probability Termsp³ = (0.05)³ = 0.000125. Also, (1 − p)^{k − r} = (0.95)¹⁷ ≈ 0.4181.

p³ = 0.000125, q¹⁷ ≈ 0.4181

Step 5 — MultiplyP(X = 20) = 171 × 0.000125 × 0.4181 ≈ 171 × 5.226 × 10⁻⁵ ≈ 0.008937.

P(X = 20) ≈ 0.00894 (about 0.89%)

Step 6 — Verify with MeanAs a sanity check, E[X] = r / p = 3 / 0.05 = 60, so the expected trial for the 3rd defect is 60. The probability of seeing it as early as trial 20 should indeed be small, consistent with our result.

SECTION 7

Comparing Related Distributions

The geometric and negative binomial distributions belong to a family of Bernoulli-trial-based models. Understanding how they relate to the more familiar binomial distribution—and to each other—clarifies when to use which model. The key distinction is whether the number of trials or the number of successes is held fixed.

Comparison of Bernoulli-trial distributions

Feature	Binomial	Geometric	Negative Binomial
What is fixed?	Number of trials n	Number of successes (r = 1)	Number of successes r
Random variable	Number of successes in n trials	Trial number of 1st success	Trial number of r-th success
Support	{0, 1, …, n}	{1, 2, 3, …}	{r, r+1, r+2, …}
Mean	np	1/p	r/p
Variance	npq	q/p²	rq/p²
Memoryless?	No	Yes	No

✦ CHOOSING THE RIGHT MODEL

Think of it like planning an experiment. If you say "I will conduct exactly 50 tests and count how many pass," you are in binomial territory. If instead you say "I will keep testing until I see r failures," then you need the negative binomial (or geometric when r = 1). The random quantity shifts from the number of outcomes to the number of trials, and that shift changes which distribution applies.

SECTION 8

Connections to Advanced Theory

The geometric and negative binomial distributions serve as gateways to several deeper topics in probability and statistics. Recognizing these connections early helps situate introductory material within the larger mathematical landscape.

From introductory to advanced concepts

Introductory Concept	Advanced Extension	Key Idea
Geometric distribution (discrete)	Exponential distribution (continuous)	Both are memoryless; the exponential arises as the continuous-time limit of geometric waiting
Negative binomial (integer r)	Gamma distribution (continuous)	The gamma is the continuous analog, modeling the wait for the r-th event in a Poisson process
Negative binomial (integer r)	Negative binomial (real r > 0)	Extending r to non-integer values yields a flexible model for over-dispersed count data (e.g., ecology, insurance)
Sum of geometric r.v.s	Renewal theory	The negative binomial counts events in a discrete renewal process; generalizing leads to random walks and Markov chains
PMF via binomial coefficients	Generating functions	The probability generating function (1 − qz)⁻ʳ · pʳ encodes all moments and connects to formal power series

One particularly elegant forward-looking result is the Poisson–Gamma mixture representation of the negative binomial. If the rate parameter λ of a Poisson distribution is itself gamma-distributed, then the marginal distribution of the count is negative binomial. This Bayesian viewpoint gives the negative binomial its role as a robust alternative to the Poisson for modeling count data with extra variability (overdispersion), a scenario that arises frequently in genomics, epidemiology, and insurance claims modeling.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

Explain why the geometric distribution is memoryless but the negative binomial distribution with r ≥ 2 is not. In your explanation, state the memoryless property precisely and describe an intuitive reason why it fails for r ≥ 2.

PROBLEM 2 — BASIC CALCULATION

A fair die is rolled repeatedly. Let X be the roll number on which the first 6 appears. Find P(X = 4) and E[X].

PROBLEM 3 — INTERMEDIATE

A basketball player makes free throws with probability p = 0.7. Let X be the attempt number on which she makes her 5th free throw. Find E[X], Var(X), and P(X = 7).

PROBLEM 4 — APPLIED

A network router transmits packets, each independently failing with probability 0.02. An error-correction protocol requires retransmitting a packet until it succeeds. (a) What is the expected number of transmissions per packet? (b) If 3 packets need to be sent, what is the expected total number of transmissions and the probability that all 3 succeed on their first attempt?

PROBLEM 5 — CRITICAL THINKING

Prove that if X ~ NegBin(r, p) and Y ~ NegBin(s, p) are independent, then X + Y ~ NegBin(r + s, p). Use the moment-generating function approach and comment on the probabilistic interpretation.

SUMMARY

Lesson Summary

The geometric distribution models the number of independent Bernoulli trials needed to observe the first success, with PMF P(X = k) = (1 − p)^k−1 p, mean 1/p, and the unique memoryless property among discrete distributions. The negative binomial distribution generalizes this to the trial of the r-th success, with PMF P(X = k) = C(k − 1, r − 1) · p^r · (1 − p)^k−r, mean r/p, and variance r(1 − p)/p².

The negative binomial can be understood as a sum of r independent geometric random variables, which immediately yields its mean and variance via linearity. Its continuous counterpart is the gamma distribution, and extending the parameter r to positive reals produces a flexible family for modeling overdispersed count data. Whether you are designing quality-control procedures, modeling network retransmissions, or analyzing biological count data, these distributions provide the essential probabilistic foundation for waiting-time problems.