Common Pitfalls - Statistics Graduate Level

1. Which conclusion correctly accounts for the fact that $g'(0)=0$?
   Answer: $n\,g(\widehat\theta_n)\Rightarrow\sigma^2\chi_1^2$ by applying the continuous mapping theorem to the squared limit. When the delta method's first derivative vanishes at the true parameter value, you need a second-order (quadratic) delta method instead. The ordinary delta method says $\sqrt{n}(g(\hat\theta_n) - g(\theta)) \Rightarrow N(0, [g'(\theta)]^2\sigma^2)$, but when $g'(0) = 0$, this collapses to a degenerate point mass at zero — useless. Instead, expand to second order: $g(\hat\theta_n) = g(0) + \frac{1}{2}g''(0)\hat\theta_n^2 + \cdots$, giving $n\,g(\hat\theta_n) \approx \frac{1}{2}g''(0)\cdot(\sqrt{n}\,\hat\theta_n)^2$. Here $g(\theta)=\theta^2$, so $g''(0)=2$, and $\sqrt{n}\,\hat\theta_n \Rightarrow N(0,\sigma^2)$. By the continuous mapping theorem, $n\,g(\hat\theta_n) \Rightarrow \sigma^2 \chi_1^2$, since the square of a standard normal times $\sigma^2$ is exactly $\sigma^2\chi_1^2$. That confirms C is correct. Choice A fails because it blindly applies the ordinary delta method with $g'(0)=2\cdot 0=0$, producing a degenerate limit — and even if you substituted the wrong derivative, $4\sigma^2$ is not the correct variance. Choice B gets the rescaling rate right ($n$ instead of $\sqrt{n}$) but claims a normal limit, ignoring that squaring a normal yields a chi-squared distribution, not another normal. Choice D is the most tempting distractor — it correctly identifies that the first derivative vanishes, but wrongly concludes no nondegenerate limit exists; the second-order expansion rescues the situation. The key study pattern: whenever you see $g'(\theta_0)=0$, immediately switch to the second-order delta method, expect a rate of $n$ (not $\sqrt{n}$), and look for a chi-squared limit rather than a normal one.
2. What is the asymptotic covariance matrix of $\sqrt n(\widehat\beta-\beta^*)$ under these assumptions?
   Answer: $Q^{-1}\Omega Q^{-1}$, because the projection condition alone does not imply homoskedasticity. When you see a question about OLS asymptotics without a homoskedasticity assumption, your immediate instinct should be: "sandwich estimator." The classical OLS variance formula only simplifies when $E(u^2 \mid X)$ is constant — without that, you need the full robust (sandwich) form. Here's the core derivation. The OLS estimator satisfies $\sqrt{n}(\hat{\beta} - \beta^*) = \left(\frac{1}{n}\sum X_i X_i^\top\right)^{-1} \frac{1}{\sqrt{n}}\sum X_i u_i$. By the law of large numbers, $\frac{1}{n}\sum X_i X_i^\top \xrightarrow{p} Q$. By the CLT, $\frac{1}{\sqrt{n}}\sum X_i u_i \xrightarrow{d} N(0, \Omega)$ where $\Omega = E(u^2 X X^\top)$. Applying the delta method (Slutsky), the asymptotic covariance is $Q^{-1}\Omega Q^{-1}$ — the sandwich form. This is answer A, and it's correct precisely because heteroskedasticity breaks the simplification that would otherwise collapse the sandwich. Answer B is wrong because $E(u^2)Q^{-1}$ is only valid under homoskedasticity, where $\Omega = E(u^2)Q$. Orthogonality of residuals and regressors is a first-order condition, not a variance result. Answer C is wrong because minimizing expected squared error determines where $\beta^*$ is, not the shape of the variance matrix — nothing "normalizes" residual variance to produce $Q^{-1}$ alone. Answer D inverts the roles of $Q$ and $\Omega$, which is a structural error; the outer matrices must be $Q^{-1}$, not $\Omega^{-1}$. Study tip: Memorize the sandwich as $Q^{-1}\Omega Q^{-1}$. Anytime an exam omits homoskedasticity, that's your formula — the simpler forms are just special cases.
3. Can the analyst use the two resulting marginal likelihoods to obtain a uniquely defined Bayes factor?
   Answer: No, because arbitrary prior constants enter the two marginal likelihoods and need not cancel across models. Whenever you see a question involving Bayes factors with improper priors across different models, your first instinct should be to ask: are the arbitrary normalizing constants truly shared, or do they float independently? The Bayes factor is defined as the ratio of marginal likelihoods, $BF_{10} = m_1(\mathbf{x})/m_0(\mathbf{x})$, where each marginal likelihood integrates the likelihood against its prior. With improper priors, each prior is only defined up to an arbitrary positive constant: $\pi_0(\sigma) = c_0/\sigma$ and $\pi_1(\mu,\sigma) = c_1/\sigma$. These constants propagate directly into their respective marginal likelihoods, giving $m_0(\mathbf{x}) \propto c_0 \cdot (\text{finite integral})$ and $m_1(\mathbf{x}) \propto c_1 \cdot (\text{finite integral})$. The ratio $BF_{10}$ then contains the factor $c_1/c_0$, which is completely arbitrary. Since $M_0$ and $M_1$ are separate models with separately specified priors, there is no reason these constants must agree or cancel. The Bayes factor is therefore not uniquely defined — confirming that B is correct. Choice A is wrong because posterior propriety tells you only that the integral is finite; it does not pin down the value of the normalizing constant for an improper prior — those are entirely separate issues. Choice C is the most tempting distractor: the symbolic form $1/\sigma$ looks the same in both priors, but the constants multiplying them are independently arbitrary and live in different parameter spaces, so nothing forces cancellation across models. Choice D states a false general rule; an improper prior can absolutely yield a proper posterior (as the passage explicitly confirms). A useful rule of thumb: improper priors are "safe" within a single model for posterior inference, but they are never safe for cross-model comparison via Bayes factors unless the constants are demonstrably tied together.
4. Why does applying optional stopping to conclude $E(S_\tau)=E(S_0)=0$ produce an incorrect result?
   Answer: Almost-sure finiteness is insufficient; $E(\tau)=\infty$ and the stopped martingale lacks the needed uniform integrability. Whenever you encounter a question about the Optional Stopping Theorem (OST), your first instinct should be to check the conditions under which it applies — not just whether a martingale exists. The OST guarantees $E(S_\tau) = E(S_0)$ only when additional regularity conditions hold beyond $P(\tau < \infty) = 1$. Here, $S_n$ is indeed a martingale (mean-zero, symmetric increments), and $\tau$ is almost surely finite. But the classical OST requires something stronger: either $E(\tau) < \infty$, or the stopped process $\{S_{n \wedge \tau}\}$ is uniformly integrable. For the simple symmetric random walk hitting level 1, it is a classical result that $E(\tau) = \infty$. Without finite expected stopping time, the martingale can "drift" over an arbitrarily long horizon before stopping, and the stopped process fails to be uniformly integrable. Consequently, you cannot exchange the limit and expectation, and $E(S_\tau) = 1 \neq 0 = E(S_0)$. This makes C correct. A is wrong because the OST applies perfectly well to discrete distributions — the symmetric random walk is a textbook martingale regardless of its discrete increments. B is wrong because $\tau$ is a standard stopping time and is measurable with respect to the natural filtration; the issue is integrability, not measurability. D is wrong and essentially fabricated — OST places no restriction on whether the terminal value is fixed or random. Your study tip: whenever OST is invoked, immediately ask three questions — Is it a martingale? Is $E(\tau) < \infty$? Is the stopped process uniformly integrable? Failing any one condition invalidates the conclusion.
5. Why does the usual identity $E_\theta[\partial\log L(\theta)/\partial\theta]=0$ fail in this model?
   Answer: The support depends on $\theta$, and differentiating the likelihood integral introduces a boundary contribution. Whenever you encounter a question about regularity conditions for maximum likelihood theory, your first instinct should be to check whether the support of the distribution depends on the parameter. This is the central issue here. The standard identity $E_\theta\!\left[\frac{\partial \log L}{\partial \theta}\right] = 0$ is derived by differentiating under the integral sign: $\frac{\partial}{\partial \theta}\int f(x;\theta)\,dx = \int \frac{\partial}{\partial \theta} f(x;\theta)\,dx = 0.$ This interchange is only valid when the limits of integration are fixed (i.e., do not depend on $\theta$). For $\text{Uniform}(0,\theta)$, the density is $f(x;\theta) = \theta^{-1}\mathbf{1}_{0 \le x \le \theta}$, and the upper integration limit is $\theta$ itself. Differentiating the integral properly requires a boundary term via Leibniz's rule, which is nonzero. That boundary contribution is precisely why the score $-n/\theta$ has a nonzero expectation — the regularity condition breaks down, not the score itself. This confirms B is correct. Choice A is a red herring. Sufficiency of the maximum is true, but it has nothing to do with whether the score's expectation equals zero — these are unrelated properties. Choice C confuses the issue: the score here actually has finite variance, and infinite variance would be a separate regularity failure. The problem is not about variance at all. Choice D is partly true but misidentifies the consequence. The MLE does lie at a boundary, but likelihood derivatives are still well-defined almost everywhere — the issue is whether the derivative-expectation interchange is valid. Study tip: Whenever you see a uniform or truncated distribution with a parameter-dependent bound, immediately flag it as violating the Leibniz interchange condition — this single check resolves most regularity-condition questions on graduate exams.
6. Which statement about the convergence of $X_n$ is correct?
   Answer: $X_n\to 0$ almost surely and in distribution, but not in $L^1$. When you see a question mixing almost sure convergence, $L^1$ convergence, and convergence in probability, your first move should be to analyze each mode independently — they don't always agree, and this sequence is a classic counterexample designed to expose exactly those gaps. Start with almost sure convergence. Fix any $\omega \in (0,1)$. For large enough $n$, we have $U(\omega) > 1/n$, so $X_n(\omega) = 0$ for all sufficiently large $n$. This holds for every $\omega \in (0,1)$ (a set of probability 1), so $X_n \to 0$ almost surely. Since almost sure convergence implies convergence in distribution, $X_n \to 0$ in distribution as well. Now check **$L^1$ convergence**. Compute $E[X_n] = n \cdot P(U \leq 1/n) = n \cdot (1/n) = 1$ for every $n$. Since the expectations never approach 0, $E|X_n - 0| = 1 \not\to 0$, so $X_n$ does not converge to 0 in $L^1$. This makes C the correct answer — almost sure convergence does not imply $L^1$ convergence without uniform integrability, and this sequence fails that condition. A is wrong because $X_n \to 0$ does hold almost surely (stronger than in probability), and $E|X_n| = 1 \not\to 0$. B claims $L^1$ convergence, which fails since $E|X_n| = 1$ stays constant. D incorrectly asserts $L^1$ convergence; almost sure convergence alone does not force $E[X_n] \to 0$. Study tip: Always check whether a sequence is uniformly integrable before concluding that almost sure convergence implies $L^1$ convergence — the two modes are independent without it.
7. For two distinct units $i$ and $j$, which expression gives $\operatorname{Cov}(W_i,W_j)$ under the randomization distribution?
   Answer: $-\dfrac{p(1-p)}{N-1}$, because conditioning on exactly $m$ treatments induces negative dependence. Whenever you encounter randomization distributions in experimental design, the key insight is that assignment indicators are not independent — they are constrained by the fixed total $\sum_i W_i = m$. This constraint is what drives the covariance calculation. To derive $\operatorname{Cov}(W_i, W_j)$, use the identity $\operatorname{Var}\!\left(\sum_i W_i\right) = \sum_i \operatorname{Var}(W_i) + \sum_{i \neq j} \operatorname{Cov}(W_i, W_j)$. Since $\sum_i W_i = m$ is fixed, its variance is exactly 0. Each $W_i$ is Bernoulli with mean $p$, so $\operatorname{Var}(W_i) = p(1-p)$. Substituting: $0 = N \cdot p(1-p) + N(N-1)\operatorname{Cov}(W_i, W_j)$. Solving gives $\operatorname{Cov}(W_i, W_j) = -\dfrac{p(1-p)}{N-1}$, confirming answer B. The negative sign is intuitive: knowing unit $i$ is treated slightly reduces the probability that unit $j$ is also treated, since only $m$ slots are available. A is wrong because it confuses identical marginal distributions with independence. Equal marginal probabilities do not imply zero covariance when a global constraint links the variables. C overstates the dependence. The expression $-p(1-p)$ would require each treated unit to exclude exactly one other, but treatment of unit $i$ reduces the probability for all remaining $N-1$ units, spread evenly — hence the $N-1$ denominator matters. D gets the sign backwards. The fixed-total constraint forces negative, not positive, dependence between any two indicators. As a study tip: whenever assignments are made under a fixed-total constraint, always expect negative pairwise covariance and use the zero-variance trick to compute it exactly.
8. Which statement best diagnoses whether the ordinary bootstrap consistently estimates the limiting distribution of the scaled estimation error?
   Answer: It is inconsistent because the bootstrap error has an asymptotic atom at zero absent from the target limit. When a question asks about bootstrap consistency for an estimator near a boundary, your first instinct should be to examine the support of the bootstrap resample relative to the original parameter space — specifically, what the maximum of a resample can and cannot see. Here, the key fact is that the scaled error $n(\theta - M_n)$ converges in distribution to an $\text{Exponential}(1/\theta)$ random variable — a continuous limit with no point mass anywhere. Now consider what happens when you bootstrap. The bootstrap resample is drawn from the empirical distribution, so $M_n^* \leq M_n$ always. This means $n(M_n - M_n^*) \geq 0$ always, and crucially, whenever any observation equals $M_n$ is resampled, $M_n^* = M_n$ exactly, giving a contribution at zero. Because each resample includes $M_n$ with probability $1 - (1 - 1/n)^n \to 1 - e^{-1} > 0$, the bootstrap error distribution has a genuine atom at zero with positive probability in the limit. The true limiting distribution is continuous and has no such atom. This is exactly what D describes — the distributions differ structurally, so the bootstrap is inconsistent. A is wrong because convergence rates matching does not imply distributional consistency; the shapes of the limiting distributions must also match. B is wrong because studentization does not resolve a boundary problem — the issue is that $\theta$ is the boundary of the support, and resampling cannot exceed $M_n$. C is wrong because the atom at zero arises generically from resampling, not only when ties exist in the original sample. The takeaway: whenever the MLE sits at a boundary of the parameter support (like a uniform maximum), bootstrap resamples are confined below that boundary, injecting a probability atom that the true limiting distribution lacks — a reliable signal of bootstrap inconsistency.
9. Although both $X_n$ and $Y_n$ converge in probability to zero, what is the correct conclusion about $X_n/Y_n$?
   Answer: Its distribution is standard Cauchy for every $n$, so it does not converge in probability to a constant. When a question involves ratios of random variables that are each converging to zero, you cannot simply apply limit rules as if they were deterministic sequences. The key question is: what does the ratio's distribution look like at every finite $n$? Notice that $X_n/Y_n = (Z_1/\sqrt{n})/(Z_2/\sqrt{n}) = Z_1/Z_2$. The $\sqrt{n}$ terms cancel exactly, leaving the ratio of two independent standard normals — which is the definition of a standard Cauchy distribution. This holds for every $n$, not just in the limit. Since the distribution of $X_n/Y_n$ is exactly Cauchy for all $n$, it cannot converge in probability to any constant. A Cauchy random variable has such heavy tails that it doesn't even have a finite mean, and its spread never collapses. Answer A is correct. Answer B is wrong because sharing the same limiting distribution (both converge to 0) tells you nothing about the behavior of their ratio. Convergence in probability is not preserved under division without additional conditions. Answer C commits an even more tempting error: you cannot conclude $X_n/Y_n \to 0$ just because the numerator $X_n \to 0$, because the denominator $Y_n \to 0$ at the same rate, and the rates cancel. Answer D is also incorrect — the ratio doesn't diverge; it stays Cauchy-distributed at every $n$. As a study rule: when you see a ratio where both numerator and denominator shrink at the same rate, always check whether they cancel before concluding anything about convergence. Rates matter more than limits alone.
10. Which statement correctly explains why one cannot conclude that $\int_0^1 f_\theta(x)\,dx\to\int_0^1 f_0(x)\,dx$?
    Answer: Pointwise convergence is insufficient because no common integrable dominating function controls the moving spike. Whenever you encounter a question about exchanging limits and integrals, your first instinct should be to ask: which convergence theorem applies, and are its hypotheses satisfied? The classic theorems — Dominated Convergence, Monotone Convergence, Bounded Convergence — each require conditions beyond mere pointwise convergence. Here, each $f_\theta$ is a "moving spike": a rectangle of height $\theta^{-1}$ and width $\theta$, so $\int_0^1 f_\theta(x)\,dx = 1$ for every $\theta > 0$. Yet pointwise, $f_\theta(x) \to 0$ for every fixed $x > 0$, since eventually $x > \theta$. So $\int f_\theta \to 1 \neq 0 = \int f_0$. The culprit is the lack of a dominating function. Any integrable $g$ that dominates this family would need $g(x) \geq \theta^{-1}$ for $x \in (0,\theta)$ for all small $\theta$, which forces $g$ to be non-integrable near zero. Without such a dominator, the Dominated Convergence Theorem fails, and you cannot pass the limit inside the integral. This is precisely what D captures. A is wrong because the functions do converge pointwise almost everywhere (everywhere on $(0,1)$ in fact) — the issue is not a failure of a.e. convergence. B is wrong because the Monotone Convergence Theorem requires a monotone increasing sequence of functions, and $f_\theta$ is not monotone as $\theta \downarrow 0$. C is wrong because the Bounded Convergence Theorem requires a uniform bound across all functions; here the spike height $\theta^{-1} \to \infty$, so no such uniform bound exists. As a study rule: moving-spike examples are the canonical illustration that pointwise convergence alone never justifies swapping limits and integrals — always verify a dominating function exists.

Start this quiz