Law of Large Numbers - Statistics Graduate Level

1. Let $X_1,X_2,\ldots$ be independent with common mean $\mu$ and common finite variance $\sigma^2$. Consider the weighted average $W_n=\sum_{i=1}^n w_{ni}X_i$, where $w_{ni}=2i/[n(n+1)]$. Which statement is correct?
   Answer: $W_n\to\mu$ in mean square because the weights sum to one and $\sum_{i=1}^nw_{ni}^2\sim 4/(3n)\to0$. When you see a weighted average of i.i.d. random variables, your first instinct should be to check two things: do the weights sum to one, and does $\sum_i w_{ni}^2 \to 0$? These are the key conditions for mean-square (and hence probability) convergence to $\mu$. For $w_{ni} = 2i/[n(n+1)]$, verify the weights sum to one: $\sum_{i=1}^n \frac{2i}{n(n+1)} = \frac{2}{n(n+1)} \cdot \frac{n(n+1)}{2} = 1$. So $E[W_n] = \mu$ exactly for every $n$. Now compute the variance: $\operatorname{Var}(W_n) = \sigma^2 \sum_{i=1}^n w_{ni}^2 = \sigma^2 \sum_{i=1}^n \frac{4i^2}{n^2(n+1)^2}$. Using $\sum_{i=1}^n i^2 = n(n+1)(2n+1)/6$, this simplifies to $\sigma^2 \cdot \frac{4(2n+1)}{6n(n+1)} \sim \frac{4\sigma^2}{3n} \to 0$. Since both the bias is zero and the variance vanishes, $W_n \to \mu$ in mean square — confirming B. Choice A is wrong because the weights do grow with $i$, but they're normalized so the sum equals one, meaning $E[W_n] = \mu$, not $(2/3)\mu$. The heavier weighting on larger indices affects the rate of variance decay, not the limit. Choice C confuses the behavior of individual weights with the sum of squared weights — no single weight stays bounded away from zero (each $w_{nn} = 2/(n+1) \to 0$), so variance goes to zero, not $4\sigma^2/3$. Choice D is a red herring; unequal weights are perfectly compatible with LLN-type results as long as $\sum w_{ni}^2 \to 0$. Your takeaway: for any weighted average, always check $\sum w_{ni} = 1$ (unbiasedness) and $\sum w_{ni}^2 \to 0$ (variance collapse). Together, they guarantee mean-square convergence to $\mu$.
2. Let $X_1,X_2,\ldots$ be pairwise independent and identically distributed, and suppose $E|X_1|<\infty$. No mutual independence or finite variance is assumed. Which conclusion is valid?
   Answer: $\overline X_n\to E(X_1)$ almost surely; pairwise independence and integrability suffice for this strong law. When a question asks what the Law of Large Numbers guarantees under pairwise independence, your instinct might be to reach for the classical strong law, which typically assumes mutual independence. But a deeper result exists that many students overlook. The key theorem here is Etemadi's Strong Law of Large Numbers (1981): if $X_1, X_2, \ldots$ are pairwise independent and identically distributed with $E|X_1| < \infty$, then $\overline{X}_n \to E(X_1)$ almost surely. Etemadi's proof cleverly avoids relying on mutual independence or finite variance — it uses truncation arguments and the pairwise independence structure alone to establish a.s. convergence. This makes B correct: pairwise independence plus integrability is genuinely sufficient for the strong law. A is wrong because it understates what's possible. It correctly notes convergence in probability but falsely claims a.s. convergence requires mutual independence — Etemadi's theorem directly contradicts this. C is wrong for a similar reason: finite variance is a sufficient condition for the classical strong law via Kolmogorov's criterion, but it is not necessary. Assuming $E|X_1^2| < \infty$ is a much stronger requirement than $E|X_1| < \infty$, and the theorem doesn't need it. D is wrong because it conflates "no variance control" with "no conclusion." Variance-based arguments (like Chebyshev) are just one route; Etemadi's approach sidesteps them entirely. Study tip: Watch for questions that test whether you know the minimal conditions for major theorems. Examiners often use distractors that cite a sufficient condition (like finite variance or mutual independence) and present it as necessary — a classic trap on graduate-level probability exams.
3. Let $X_1,X_2,\ldots$ be iid with a distribution symmetric about zero. For $x\ge e$, suppose $P(|X_1|>x)=1/(x\log x)$, with the remaining probability assigned symmetrically on $[-e,e]$. Which statement about $\overline X_n$ is correct?
   Answer: $\overline X_n\to0$ in probability but not almost surely; the truncation conditions for the weak law are satisfied, yet $\sum_nP(|X_n|>\varepsilon n)=\infty$ forces large terms to occur infinitely often. When you encounter a question about sample means with heavy-tailed distributions, your first instinct should be to check whether the mean is finite, then separately assess conditions for the weak law (WLW) versus the strong law (SLLN). Here, $P(|X_1| > x) = 1/(x \log x)$ for $x \geq e$. Computing $E|X_1| = \int_e^\infty P(|X_1|>x)\,dx = \int_e^\infty \frac{1}{x\log x}\,dx = \Big[\log\log x\Big]_e^\infty = \infty$, so the mean is infinite. This rules out the classical SLLN immediately. However, the weak law doesn't strictly require finite mean — it requires a truncation condition. Specifically, if $n\,P(|X_1|>n)\to 0$, the WLW holds (with centering). Here $n\,P(|X_1|>n) = n/(n\log n) = 1/\log n \to 0$, so $\overline{X}_n \to 0$ in probability (symmetry kills the centering term). Yet for the SLLN, you need $\sum_n P(|X_1|> \varepsilon n) < \infty$, but $\sum_n \frac{1}{n\log n} = \infty$, meaning large values occur infinitely often almost surely — blocking a.s. convergence. This confirms D. Choice A is wrong because not every infinite-mean symmetric distribution attracts to a Cauchy law; the Cauchy domain requires a specific regular variation condition ($P(|X|>x)\sim C/x$), not $1/(x\log x)$. Choice B is wrong because symmetry alone cannot replace finite mean in the SLLN — the Etemadi/Kolmogorov SLLN genuinely requires $E|X_1|<\infty$. Choice C is wrong because it overstates what's needed for the WLW; a finite mean is sufficient but not necessary. Study tip: Always distinguish the WLW truncation condition ($n\,P(|X|>n)\to 0$) from the SLLN's summability requirement ($\sum_n P(|X|>\varepsilon n)<\infty$) — they can diverge, as this problem perfectly illustrates.
4. To what quantity does the fitted slope $\widehat\beta_n$ converge almost surely?
   Answer: $\operatorname{Cov}(X_i,Y_i)/\operatorname{Var}(X_i)$, the coefficient of the population linear projection of $Y_i$ on $X_i$. When you see OLS applied without assuming a correctly specified linear model, the key question becomes: what does OLS actually estimate in large samples? This question tests your understanding of the population linear projection, a concept distinct from conditional expectation. By the Law of Large Numbers, the OLS slope satisfies $\widehat\beta_n = \frac{\frac{1}{n}\sum(X_i - \bar{X})(Y_i - \bar{Y})}{\frac{1}{n}\sum(X_i - \bar{X})^2} \xrightarrow{a.s.} \frac{\operatorname{Cov}(X_i, Y_i)}{\operatorname{Var}(X_i)}$. This limit is the slope of the best linear predictor of $Y_i$ given $X_i$ — the population linear projection — and it holds regardless of whether $E[Y_i \mid X_i]$ is actually linear. Answer D is therefore correct. Choice A is tempting but wrong. The average derivative $E[\partial E(Y_i \mid X_i)/\partial X_i]$ equals the OLS limit only when $E[Y_i \mid X_i]$ is linear in $X_i$, which is not assumed here. In general, these two quantities differ. Choice B, $E(Y_i)/E(X_i)$, confuses the ratio of means with a regression coefficient — this would be relevant for ratio estimation, not OLS. Choice C, $E(Y_i/X_i)$, has no algebraic connection to the OLS formula and reflects a naive "pointwise slope" idea that doesn't arise from minimizing squared prediction error. The key study takeaway: always anchor OLS asymptotics to the linear projection formula $\operatorname{Cov}(X,Y)/\operatorname{Var}(X)$. This is what OLS recovers in the population, model misspecification or not.
5. Let $I_1,I_2,\ldots$ be independent Bernoulli random variables satisfying $P(I_n=1)=1/n$. Which statement best describes the convergence of $I_n$?
   Answer: $I_n\to0$ in probability but not almost surely, because successes occur infinitely often with probability one. When studying convergence of random variables, you need to keep two distinct modes of convergence carefully separated: convergence in probability and convergence almost surely. The key tools here are the Borel-Cantelli Lemmas. Since $P(I_n = 1) = 1/n$, notice that $P(I_n = 1) \to 0$, which immediately implies $I_n \to 0$ in probability — for any $\varepsilon > 0$, $P(|I_n| > \varepsilon) = 1/n \to 0$. So far so good. Now ask whether convergence is almost sure. Almost sure convergence of $I_n \to 0$ would require that, with probability one, $I_n = 1$ happens only finitely often. The Second Borel-Cantelli Lemma says: if events are independent and $\sum P(I_n = 1) = \sum 1/n = \infty$, then $P(I_n = 1 \text{ i.o.}) = 1$. Since $I_n = 1$ occurs infinitely often with probability one, $I_n$ cannot converge to 0 almost surely. This confirms A is correct. Now for the distractors. B is wrong because $p_n \to 0$ is sufficient for convergence in probability but not for almost sure convergence — you need the series of probabilities to also converge (First Borel-Cantelli), which fails here. C is wrong because divergence of $\sum p_n$ blocks almost sure convergence but does not block convergence in probability, which holds regardless. D is nonsensical — $I_n \to 1$ would require $p_n \to 1$, the opposite of what's given. Your study tip: always check both Borel-Cantelli lemmas when independence is mentioned. Divergence of $\sum p_n$ plus independence is the signature of "in probability but not almost surely."
6. Let $\{X_i\}_{i\ge 1}$ be a weakly stationary sequence with mean $\mu$ and autocovariance function $\gamma(h)$. Suppose $\sum_{h=0}^{\infty}|\gamma(h)|<\infty$. Which conclusion about $\overline X_n=n^{-1}\sum_{i=1}^n X_i$ follows directly from these assumptions?
   Answer: $\overline X_n\to\mu$ in mean square and hence in probability, but almost-sure convergence is not implied by the covariance condition alone. When you see a question combining weak stationarity with an absolute summability condition on autocovariances, your first instinct should be to compute $\text{Var}(\overline{X}_n)$ directly — this reveals exactly what mode of convergence is guaranteed. Under weak stationarity, $\text{Var}(\overline{X}_n) = n^{-2}\sum_{i,j}\gamma(i-j) = n^{-1}\sum_{h=-(n-1)}^{n-1}\left(1-\frac{|h|}{n}\right)\gamma(h)$. Since $\sum_{h=0}^{\infty}|\gamma(h)|<\infty$, the Cesàro-weighted sum is bounded, so $\text{Var}(\overline{X}_n) = O(n^{-1}) \to 0$. This means $\overline{X}_n \to \mu$ in mean square, and since mean-square convergence implies convergence in probability, both follow. This confirms A is correct. B is wrong because absolute summability of autocovariances is a second-moment (covariance) condition. Almost-sure convergence requires stronger tools — typically independence, mixing rates, or moment conditions beyond what $\gamma(h)$ alone provides. You cannot promote an $L^2$ result to an a.s. result for free. C contains a subtle arithmetic error. $\text{Var}(\overline{X}_n) = O(n^{-1})$, not $O(n^{-2})$, so $\text{Var}(\sqrt{n}(\overline{X}_n - \mu)) = O(1)$ — it stays bounded, not vanishing. Hence $\sqrt{n}(\overline{X}_n - \mu)$ does not go to zero in probability. D is wrong because absolute summability of $\gamma(h)$ is precisely the condition that controls long-range correlations enough to ensure convergence. The correlations decay sufficiently fast. Study tip: Always anchor stationarity-and-averaging questions to the variance calculation of $\overline{X}_n$. The mode of convergence you can claim is determined entirely by what controls that variance — $L^2$ conditions give $L^2$/probability convergence, not almost-sure convergence.
7. Suppose $X_i$ are independent Bernoulli random variables with $P(X_i=1)=i^{-1/2}$, and let $\overline X_n=n^{-1}\sum_{i=1}^nX_i$. Which limiting statement is correct?
   Answer: $\overline X_n\to0$ almost surely, because the average success probability vanishes and the centered averages satisfy a strong law. When dealing with the strong law for non-identically distributed independent random variables, your first instinct should be to check Kolmogorov's strong law: if $\sum_{i=1}^n \text{Var}(X_i)/i^2 < \infty$, then $\overline{X}_n - \overline{\mu}_n \to 0$ almost surely, where $\overline{\mu}_n = n^{-1}\sum_{i=1}^n \mu_i$. Here, $\mu_i = E[X_i] = i^{-1/2}$, so $\overline{\mu}_n = n^{-1}\sum_{i=1}^n i^{-1/2}$. Since $\sum_{i=1}^n i^{-1/2} \sim 2\sqrt{n}$, we get $\overline{\mu}_n \sim 2n^{-1}\sqrt{n} = 2n^{-1/2} \to 0$. For the variance condition: $\text{Var}(X_i) \leq 1$, so $\sum_{i=1}^\infty \text{Var}(X_i)/i^2 \leq \sum_{i=1}^\infty i^{-2} < \infty$. Kolmogorov's criterion is satisfied, confirming $\overline{X}_n \to 0$ almost surely. Answer A is correct. Answer B is wrong because it conflates $i^{-1/2}$ with harmonic-type behavior. The harmonic series $\sum i^{-1}$ diverges at rate $\log n$, but $\sum i^{-1/2} \sim 2\sqrt{n}$, so $\overline{\mu}_n \to 0$, not $1/2$. Answer C confuses the divergence of $\sum \mu_i = \sum i^{-1/2} = \infty$ with failure of the average to converge — but it's the average that matters, and averages can still converge even when totals diverge. Answer D incorrectly claims that non-identical distributions block almost sure convergence; the SLLN generalizes well beyond the i.i.d. setting via Kolmogorov's criterion. Your study tip: whenever variables are independent but not identically distributed, immediately check Kolmogorov's variance condition $\sum \text{Var}(X_i)/i^2 < \infty$ and separately compute $\overline{\mu}_n$. These two pieces together fully determine almost sure behavior of $\overline{X}_n$.
8. Let $X_1,X_2,\ldots$ be iid with $E|X_1|<\infty$ and mean $\mu$. For each $n$, let $N_n$ be a positive integer-valued random variable independent of the entire sequence, with $N_n\to\infty$ in probability. Define $T_n=N_n^{-1}\sum_{i=1}^{N_n}X_i$. What can be concluded under the stated assumptions?
   Answer: $T_n\to\mu$ in probability, while almost-sure convergence need not follow from convergence of $N_n$ in probability. When a sample size is itself random, you need the random index law of large numbers: if $N_n \to \infty$ in probability and $N_n$ is independent of the $X_i$'s, then $T_n = N_n^{-1}\sum_{i=1}^{N_n} X_i \to \mu$ in probability. The key tool is a standard epsilon-delta argument conditioning on $N_n$. For any $\varepsilon > 0$, the event $\{|T_n - \mu| > \varepsilon\}$ has small probability whenever $N_n$ is large (by the ordinary LLN applied at each fixed sample size), and since $N_n \to \infty$ in probability, you can make that probability arbitrarily small. This confirms C is correct. Almost-sure convergence, however, is a strictly stronger mode of convergence and requires more structure. Knowing only that $N_n \to \infty$ in probability is insufficient to upgrade from "in probability" to "almost surely" — you would need $N_n \to \infty$ a.s. (or a summability condition on the tails). So C correctly identifies the precise strength of the conclusion. A is wrong because convergence in probability of $T_n$ does not require $N_n/n \to 1$; it only requires $N_n \to \infty$ in probability, which is given. B overclaims: almost-sure convergence of $T_n$ does not follow automatically from the ordinary SLLN when the index is only assumed to diverge in probability. D is wrong because the partial sum $\sum_{i=1}^{N_n} X_i$ grows like $\mu N_n$, so dividing by $N_n$ recovers $\mu$, not zero. The study tip: always distinguish carefully between convergence modes. In probability and almost surely are not interchangeable — know what each assumption about $N_n$ buys you.
9. For each $n$, let $X_{n1},\ldots,X_{nn}$ be independent within row, with $X_{ni}=n$ with probability $1/n$ and $X_{ni}=0$ otherwise. Define $\overline X_n=n^{-1}\sum_{i=1}^nX_{ni}$. Although $E(\overline X_n)=1$ for every $n$, what is the limiting behavior of $\overline X_n$?
   Answer: $\overline X_n\Rightarrow\operatorname{Poisson}(1)$, so it does not converge in probability to its expectation. Whenever you see a triangular array where individual entries grow with $n$, you should immediately think about Poisson convergence, not classical LLN behavior. The key diagnostic: as $n \to \infty$, each $X_{ni}$ is nonzero with probability $1/n \to 0$, but there are $n$ such terms — a classic rare-event accumulation setup. To see why C is correct, count how many of the $n$ variables actually equal $n$. Let $S_n = \#\{i : X_{ni} = n\}$. Each indicator is Bernoulli$(1/n)$, so $S_n \sim \text{Binomial}(n, 1/n)$, which converges in distribution to $\text{Poisson}(1)$. Now notice that $\overline{X}_n = n^{-1}\sum X_{ni} = n^{-1} \cdot n \cdot S_n = S_n$. So $\overline{X}_n \stackrel{d}{=} S_n \Rightarrow \text{Poisson}(1)$. The mean of $\text{Poisson}(1)$ is 1, consistent with $E(\overline{X}_n)=1$, but the distribution is not degenerate at 1 — so convergence in probability to the mean fails entirely. Choice A is the core trap: it invokes the LLN, but the LLN's i.i.d. version requires a fixed distribution. Here the distribution of $X_{ni}$ changes with $n$, so classical LLN does not apply. Choice B confuses individual behavior with aggregate behavior — yes, each term is zero with high probability, but the aggregate $\overline{X}_n$ still fluctuates because rare large values dominate. Choice D misapplies the CLT; with variance $\text{Var}(\overline{X}_n) = 1$ not shrinking to zero, there is no normal concentration around the mean. Your study tip: whenever individual probabilities vanish like $1/n$ but the number of trials grows like $n$, Poisson limits govern the sum — not Gaussian CLT or LLN concentration.
10. Suppose $X_1,X_2,\ldots$ are independent and identically distributed with $P(X_i>x)=x^{-3/2}$ for $x\ge1$. Which statement about the sample mean is correct?
    Answer: $\overline X_n\to3$ almost surely; the mean is finite (equal to 3) even though $\operatorname{Var}(X_i)=\infty$, and the strong law requires only $E|X_i|<\infty$. When you see a question involving a heavy-tailed distribution and the sample mean, your first instinct should be to carefully compute whether $E[X_i]$ is finite — because the Strong Law of Large Numbers (SLLN) requires only $E|X_i| < \infty$, nothing more. Here, $P(X_i > x) = x^{-3/2}$ for $x \geq 1$ defines a Pareto-type distribution with density $f(x) = \frac{3}{2}x^{-5/2}$. Computing the mean directly: $E[X_i] = \int_1^\infty x \cdot \frac{3}{2}x^{-5/2}\,dx = \frac{3}{2}\int_1^\infty x^{-3/2}\,dx = \frac{3}{2}\cdot\left[\frac{x^{-1/2}}{-1/2}\right]_1^\infty = \frac{3}{2}\cdot 2 = 3.$ Since $E[X_i] = 3 < \infty$, the SLLN guarantees $\overline{X}_n \to 3$ almost surely. The variance being infinite is irrelevant — the SLLN only cares about the first moment. This confirms A is correct. B is wrong because it conflates the SLLN with the Central Limit Theorem. Infinite variance disables normal CLT approximations, but it does not downgrade the SLLN from almost-sure to in-probability convergence. C is wrong because it misapplies the "tail exponent" rule. The mean fails to exist only when the tail exponent satisfies $\alpha \leq 1$ (i.e., $P(X>x) \sim x^{-\alpha}$ with $\alpha \leq 1$). Here $\alpha = 3/2 > 1$, so the mean is finite. D is a fabrication — the support's lower endpoint has no special role in determining the long-run average. Study tip: Always check finiteness of $E|X_i|$ before worrying about variance. The SLLN is robust — infinite variance is a CLT problem, not an LLN problem.

Start this quiz