Asymptotic Normality of MLE - Statistics Graduate Level

1. Which statement best describes the asymptotic behavior of the MLE and the regularity issue responsible for it?
   Answer: $n(\theta_0-\widehat\theta_n)/\theta_0\Rightarrow \operatorname{Exp}(1)$ because the parameter determines the support of the distribution. When the parameter governs the support boundary of a distribution rather than appearing inside a smooth density, classical $\sqrt{n}$-asymptotics and Fisher information break down entirely. This is the core regularity violation being tested here. For $X_i \sim \text{Uniform}(0,\theta_0)$, the MLE is $\widehat{\theta}_n = \max_i X_i$. To find its limit law, compute $P(n(\theta_0 - \widehat{\theta}_n)/\theta_0 > t)$ for $t > 0$. This equals $P(\widehat{\theta}_n < \theta_0 - t\theta_0/n) = \left(1 - t/n\right)^n \to e^{-t}$, which is exactly the survival function of an $\text{Exp}(1)$ random variable. So $n(\theta_0 - \widehat{\theta}_n)/\theta_0 \Rightarrow \text{Exp}(1)$ — answer B is correct. The $n$-rate (not $\sqrt{n}$) arises because the maximum concentrates on the boundary at speed $1/n$, far faster than interior estimators. Answer A is wrong on two levels: the $\sqrt{n}$ normalization is too slow, and there is no finite Fisher information here because the density's support depends on $\theta$, violating the regularity conditions required for the Cramér-Rao bound. Answer C flips the sign: since $\widehat{\theta}_n \leq \theta_0$ always (you can never observe a value exceeding $\theta_0$), the quantity $\widehat{\theta}_n - \theta_0$ is non-positive, making a positive exponential limit impossible. Answer D imposes a $\sqrt{n}$ rate and a half-normal limit, neither of which applies; the one-sided nature of the estimator produces an exponential limit, not a folded Gaussian. Study tip: Whenever the support of a density depends on the parameter, expect $n$-rate convergence to an exponential (or related) limit — this is a signature of non-regular models, and it appears frequently on qualifying exams.
2. Which asymptotic statement is correct, and what regularity feature explains it?
   Answer: $n^{1/6}\widehat\theta_n\Rightarrow\sqrt[3]{Z}$ for $Z\sim N(0,1)$ because the Fisher information vanishes at $\theta_0=0$, producing a slower rate and a non-normal limit. When the Fisher information vanishes at the true parameter, the classical MLE theory breaks down — this is the core concept being tested. Regular asymptotic theory requires that the Fisher information $I(\theta_0)$ be strictly positive and finite, so whenever you see a problem where this fails, expect a non-standard rate and a non-normal limit. Here, $X_i \sim N(\theta^3, 1)$, so the log-likelihood score has the form involving $\frac{d}{d\theta}\theta^3 = 3\theta^2$. At $\theta_0 = 0$, this derivative vanishes, meaning $I(0) = (3\theta_0^2)^2 \cdot 1 = 0$. The MLE is $\widehat{\theta}_n = \overline{X}_n^{1/3}$. Since $\overline{X}_n \sim N(0, 1/n)$ at $\theta_0 = 0$, we can write $\widehat{\theta}_n = (n^{-1/2} Z_n)^{1/3} = n^{-1/6} Z_n^{1/3}$ where $Z_n \xrightarrow{d} Z \sim N(0,1)$. Therefore $n^{1/6}\widehat{\theta}_n \Rightarrow Z^{1/3}$, a non-normal limit — confirming D is correct. Choice A is wrong because identifiability alone never guarantees asymptotic normality; you need regularity conditions including $I(\theta_0) > 0$. Choice B gets the rate right ($n^{1/6}$) but fatally misapplies the delta method: the delta method requires $g'(\mu_0) \neq 0$, yet $g'(\mu) = \frac{1}{3}\mu^{-2/3}$ diverges at $\mu_0 = 0$, so the standard delta method is inapplicable. Choice C invents a cube-root rate with a normal limit, confusing this problem with Chernoff-type cube-root asymptotics, which arise from a different mechanism entirely. Your study tip: whenever $I(\theta_0) = 0$, immediately abandon standard $\sqrt{n}$-normality and instead directly track the distribution of the estimator through its explicit formula.
3. Which statement correctly describes the asymptotic behavior of $\widehat\psi_n$?
   Answer: $n(\widehat\psi_n-\psi_0)\Rightarrow\chi_1^2$ because the first derivative of the transformation vanishes at $\mu_0$. When the delta method's first derivative vanishes at the point of evaluation, the standard $\sqrt{n}$ central limit theorem result breaks down — and that's exactly what's being tested here. The ordinary delta method says that if $\sqrt{n}(\hat{\mu}_n - \mu_0) \Rightarrow N(0,1)$, then $\sqrt{n}(g(\hat{\mu}_n) - g(\mu_0)) \Rightarrow N(0, [g'(\mu_0)]^2)$. For $g(\mu) = \mu^2$, we get $g'(\mu) = 2\mu$, so at $\mu_0 = 0$, $g'(\mu_0) = 0$. The leading linear term vanishes entirely, meaning you must go to the second-order delta method. Writing $g(\hat{\mu}_n) - g(\mu_0) \approx \frac{1}{2}g''(\mu_0)(\hat{\mu}_n - \mu_0)^2$, and since $g''(\mu) = 2$, we get $\hat{\psi}_n - 0 = \hat{\mu}_n^2 \approx (\hat{\mu}_n - 0)^2$. Therefore $n\hat{\psi}_n = n\hat{\mu}_n^2 = (\sqrt{n}\,\hat{\mu}_n)^2 \Rightarrow Z^2 \sim \chi_1^2$. The rate accelerates to $n$ (not $\sqrt{n}$), confirming B is correct. Choice A fails because it applies the first-order delta method without checking whether $g'(\mu_0) = 0$ — a critical prerequisite. Choice C has the right distributional form ($\chi_1^2$) but the wrong scaling: using $\sqrt{n}$ instead of $n$ is precisely the error the second-order analysis corrects. Choice D gets the rate right ($n$) but claims a normal limit, ignoring that squaring a normal variable produces a chi-squared, not another normal. Your strategy: **always check $g'(\mu_0) = 0$ before applying the delta method**. When it vanishes, shift to the second-order expansion, expect a faster convergence rate of $n$, and anticipate a chi-squared (not normal) limiting distribution.
4. What is the appropriate asymptotic distribution of the estimator?
   Answer: $\sqrt n(\widehat\theta_n-\theta^*)\Rightarrow N(0,2)$ using the sandwich variance $A^{-1}BA^{-1}$. Whenever you encounter M-estimation under potential misspecification, your default tool is the sandwich variance estimator — not the naive inverse-Hessian. The key insight is that misspecification breaks the information matrix equality ($A = B$ under correct specification), so the two quantities must be treated separately. The asymptotic distribution of a misspecified M-estimator follows the sandwich formula: $\sqrt{n}(\hat{\theta}_n - \theta^*) \Rightarrow N(0, A^{-1}BA^{-1})$, where $A = -E\{\ell_i''(\theta^*)\}$ captures curvature (how sharply the likelihood peaks) and $B = \operatorname{Var}\{\ell_i'(\theta^*)\}$ captures score variability. Plugging in $A = 2$ and $B = 8$: $A^{-1}BA^{-1} = \frac{1}{2} \cdot 8 \cdot \frac{1}{2} = 2$. This confirms C is correct. A is wrong because it uses only $A^{-1} = 1/2$, which would be valid if the model were correctly specified (invoking the information matrix equality to write $A^{-1}BA^{-1} = A^{-1}$). Under misspecification, you cannot collapse the sandwich. B computes $B/A = 8/2 = 4$, which confuses the sandwich $A^{-1}BA^{-1}$ with a simple ratio. The outer $A^{-1}$ factors appear on both sides, not just once. D discards curvature entirely, using only $B = 8$. Score variance alone never determines the asymptotic variance — the Hessian always enters through the $A^{-1}$ wrapping. Study tip: Memorize the sandwich as "bread–filling–bread": $A^{-1} \cdot B \cdot A^{-1}$. Under correct specification, $A = B$ simplifies this to $A^{-1}$, but whenever misspecification is mentioned, always keep all three pieces.
5. Which conclusion is best supported by these assumptions?
   Answer: $\sqrt n(\widehat\beta_n-\beta_0)\Rightarrow N(0,J^{-1})$ under the corresponding Lindeberg and local Hessian conditions. Whenever you encounter asymptotic normality questions for MLEs in non-i.i.d. settings, your instinct should be to reach for the Lindeberg-Feller CLT framework for triangular arrays rather than the classical i.i.d. CLT. The key ingredients are: a consistent MLE, a well-behaved Fisher information limit, and a negligibility condition ensuring no single observation dominates. Here, the score function for a logistic model is $S_n(\beta) = \sum_i x_{ni}(Y_{ni} - p_{ni}(\beta))$, and the Fisher information is $I_n(\beta_0) = \sum_i x_{ni}^2 p_{ni}(1-p_{ni})$. The assumption $n^{-1}I_n(\beta_0) \to J > 0$ gives a stable, nondegenerate information limit. The "no single observation dominates" condition is precisely the Lindeberg negligibility condition. Under these, a standard Taylor expansion of the score around $\beta_0$ yields $\sqrt{n}(\hat\beta_n - \beta_0) \Rightarrow N(0, J^{-1})$, confirming B is correct. A is wrong because non-i.i.d. structure alone does not preclude asymptotic normality — the Lindeberg-Feller CLT was designed exactly for this triangular array setting. The i.i.d. assumption is sufficient but not necessary. C is wrong due to a variance miscalculation. The sandwich formula for the MLE gives variance $I^{-1}$, not $I^{-2}$. The score and Hessian don't each contribute an independent information factor; via the information equality, they cancel to leave a single $J^{-1}$. D is wrong because differing success probabilities do not produce a mixture distribution in the limit — they are averaged out through the CLT, producing a single Gaussian. Your study tip: memorize that for MLEs in triangular arrays, the asymptotic variance is $J^{-1}$ (one inverse information), and that Lindeberg's condition replaces i.i.d. — not eliminates normality.
6. What is the limiting distribution of the properly scaled MLE?
   Answer: $\sqrt n\widehat\theta_n\Rightarrow \max(0,Z)$, where $Z\sim N(0,1)$, so the limit has an atom at zero. When the MLE is constrained to a boundary of the parameter space, standard asymptotic normality breaks down — this question tests whether you recognize that boundary behavior fundamentally changes the limiting distribution. Here's the key insight: since $\widehat\theta_n = \max(0, \overline X_n)$, scaling gives $\sqrt{n}\,\widehat\theta_n = \max(0, \sqrt{n}\,\overline X_n)$. Because $\theta_0 = 0$, we know $\sqrt{n}\,\overline X_n \Rightarrow Z$ where $Z \sim N(0,1)$. The continuous mapping theorem then immediately delivers $\sqrt{n}\,\widehat\theta_n \Rightarrow \max(0, Z)$. This distribution is a mixture: it places probability $1/2$ at zero (when $Z \leq 0$) and follows a half-normal on $(0,\infty)$. The atom at zero is the signature of a boundary-constrained estimator, confirming C. A is wrong because simply invoking asymptotic normality of $\overline X_n$ ignores the truncation at zero. The max operator is nonlinear and shifts mass onto the boundary. B is wrong because $|Z|$ and $\max(0,Z)$ are different objects — $|Z|$ folds the entire distribution onto $(0,\infty)$ with no point mass, whereas $\max(0,Z)$ keeps only the positive part and piles the negative part at zero. D is wrong because squaring $\widehat\theta_n$ does not recover a $\chi^2_1$ — that would require the unrestricted normal, and the boundary creates an asymmetric distribution whose square is not chi-squared. Study tip: Whenever the true parameter sits on the boundary of $\Theta$, apply the continuous mapping theorem to $\max(0, \sqrt{n}\,\overline X_n)$ directly. The resulting half-normal-plus-atom structure appears frequently in constrained inference problems on graduate exams.
7. Why does the standard fixed-dimensional MLE asymptotic-normality theorem not yield a normal limit centered at the true $\sigma^2$?
   Answer: The variance MLE converges to $\sigma^2/2$ because the number of nuisance parameters grows with the number of groups. Whenever you see MLEs in a setting where the number of parameters grows with the sample size, the classical Cramér-Rao / asymptotic normality theorem is in danger — that theorem assumes a fixed parameter dimension as $n \to \infty$. Here, as $m \to \infty$, you are estimating $m$ group means plus $\sigma^2$, so the parameter count grows alongside the data. The MLE of $\sigma^2$ is $\hat{\sigma}^2 = \frac{1}{2m}\sum_{i=1}^{m}\sum_{j=1}^{2}(X_{ij}-\bar{X}_{i\cdot})^2$. Within each group of size 2, the within-group sum of squares uses 1 degree of freedom (not 2), because one df is consumed estimating $\mu_i$. So $E[\hat{\sigma}^2] = \frac{1}{2}\sigma^2$, and as $m\to\infty$ the MLE converges to $\sigma^2/2$, not $\sigma^2$. This is exactly the Neyman–Scott phenomenon: estimating an ever-growing collection of incidental nuisance parameters biases the common-parameter MLE in the limit. Answer A correctly identifies this mechanism. B is wrong because the model is identifiable — two observations per group uniquely determine both $\mu_i$ and contribute to estimating $\sigma^2$; the issue is bias, not non-identifiability. C is wrong on two counts: the MLE is not consistent (it converges to the wrong value), and boundary issues are irrelevant here since the means are unconstrained. D gets the direction of the bias backwards. The MLE converges to $\sigma^2/2$, not $2\sigma^2$; losing a degree of freedom understates the variance, it does not double it. As a study rule: whenever parameters grow proportionally to observations (finite group size, many groups), immediately suspect the Neyman–Scott problem — MLEs of common parameters can be inconsistent despite standard regularity conditions appearing satisfied.
8. What is the asymptotic distribution of the joint MLE's component $\widehat\theta_n$?
   Answer: $\sqrt n(\widehat\theta_n-\theta_0)\Rightarrow N(0,3/8)$ because nuisance-parameter estimation reduces efficient information. Whenever you see a question involving MLE asymptotics with a nuisance parameter, your instinct should be to reach for the inverse of the full Fisher information matrix — not just individual entries of $I(\eta_0)$. By standard MLE theory, $\sqrt{n}(\widehat{\eta}_n - \eta_0) \Rightarrow N(0, I(\eta_0)^{-1})$. The asymptotic variance of $\widehat{\theta}_n$ is the **(1,1) entry of $I(\eta_0)^{-1}$**, not $1/I_{11}$. For a 2×2 matrix $\begin{pmatrix}a&b\\b&d\end{pmatrix}$, the inverse has (1,1) entry $\frac{d}{ad-b^2}$. Here, $\frac{3}{(4)(3)-(2)^2} = \frac{3}{8}$. So $\sqrt{n}(\widehat{\theta}_n - \theta_0) \Rightarrow N(0, 3/8)$, confirming answer B. A is wrong because $1/I_{11} = 1/4$ would be the correct variance only if $\lambda$ were known — it ignores the statistical cost of estimating the nuisance parameter, which inflates uncertainty about $\theta$. C is wrong in a subtle way: the Schur complement $I_{11} - I_{12}I_{22}^{-1}I_{21} = 4 - 4/3 = 8/3$ is the nuisance-adjusted information for $\theta$, and its reciprocal $3/8$ is indeed the correct variance — but C mistakenly uses $4/3$ (the Schur complement itself) as the variance rather than its reciprocal. D has no statistical basis; there is no principle that "counts the off-diagonal once." Study tip: Always invert the full information matrix first, then read off the diagonal entry — never invert individual entries of $I(\eta_0)$ when nuisance parameters are present.
9. Which additional collection of conditions most directly justifies a nonsingular $\sqrt n$-normal limit through a Taylor expansion of the score?
   Answer: A score central limit theorem at $\theta_0$, uniform local convergence of $-n^{-1}\nabla^2\ell_n(\theta)$ to a positive-definite matrix, and negligible Taylor remainder. When proving asymptotic normality of an MLE, the standard route is a mean-value Taylor expansion of the score around $\theta_0$. Writing $0 = n^{-1/2}\nabla\ell_n(\widehat\theta_n) \approx n^{-1/2}\nabla\ell_n(\theta_0) + \left[-n^{-1}\nabla^2\ell_n(\bar\theta)\right]\sqrt{n}(\widehat\theta_n - \theta_0)$, you can solve for $\sqrt{n}(\widehat\theta_n - \theta_0)$ only if three ingredients are present: (1) the score term $n^{-1/2}\nabla\ell_n(\theta_0)$ converges in distribution to a normal (score CLT), (2) the Hessian matrix $-n^{-1}\nabla^2\ell_n(\bar\theta)$ converges to a nonsingular positive-definite limit uniformly in a neighborhood (so the intermediate point $\bar\theta$ is handled), and (3) the remainder from the expansion is $o_p(1)$. This is precisely option A, confirming it as the correct answer. Option B fails because unbiasedness of $\widehat\theta_n$ is neither required nor sufficient, and pointwise (rather than uniform) convergence of the normalized likelihood does not control the Hessian at intermediate points needed in the expansion. Option C confuses finite-sample uniqueness and continuity with the asymptotic structural conditions. Positive definiteness of the observed Hessian at $\widehat\theta_n$ doesn't guarantee the uniform convergence to a population quantity required to invert and pass to a limit. Option D is off-track entirely: asymptotic normality of individual likelihood contributions is stronger than necessary and non-standard, while boundedness of the parameter space and convergence of the maximized value are irrelevant to the score-expansion argument. As a study strategy, memorize the three-part "score CLT + uniform Hessian convergence + negligible remainder" checklist — every rigorous MLE normality proof ultimately reduces to these three roles in the Taylor expansion.
10. Which conclusion best reflects the role of differentiability in the asymptotic normality of this MLE?
    Answer: $\sqrt n(\widehat\theta_n-\theta_0)\Rightarrow N(0,1)$ by quantile asymptotics, despite failure of classical smooth-likelihood conditions. Whenever you see an MLE question where the log-likelihood lacks classical smoothness, resist the reflex to declare asymptotic normality dead — instead, ask whether an alternative asymptotic argument applies. For the Laplace location model, the log-likelihood is $\ell(\theta) = -\sum_{i=1}^n |X_i - \theta|$, which is piecewise linear and non-differentiable at each observation. Classical Cramér-type proofs require twice differentiability of the log-likelihood, so they don't directly apply. However, asymptotic normality can still be established via quantile asymptotics. The MLE is the sample median $\widehat{\theta}_n = X_{((n+1)/2)}$, and a standard result states that for a density $f$ with $f(\theta_0) > 0$, the sample median satisfies $\sqrt{n}(\widehat{\theta}_n - \theta_0) \Rightarrow N\!\left(0, \frac{1}{4f(\theta_0)^2}\right)$. Since $f(\theta_0) = \tfrac{1}{2}$, the asymptotic variance is $\frac{1}{4(1/2)^2} = 1$. The Fisher information for this model is also $I(\theta) = 1$, confirming the median achieves the Cramér-Rao bound. Answer D is correct. A is wrong because twice differentiability of the log-likelihood is sufficient, not necessary, for asymptotic normality — quantile theory provides an independent route. B is wrong on two counts: the correct rate is $\sqrt{n}$, not $n$, and the limit is Gaussian, not Laplace. C is wrong about the variance: plugging $f(\theta_0)=\tfrac{1}{2}$ into the quantile formula gives variance 1, not 2; the mean and median of the Laplace distribution are not asymptotically equivalent in variance. As a study habit, always distinguish sufficient conditions from necessary conditions in asymptotic theory — many exam distractors exploit this confusion by treating sufficiency as necessity.

Start this quiz