Likelihood Ratio Tests & Wilks' Theorem - Statistics Graduate Level

1. What is the likelihood ratio test result for the three added predictors?
   Answer: The statistic is $8.5$ on $3$ degrees of freedom, giving a p-value between $0.025$ and $0.05$. When comparing nested generalized linear models, your go-to tool is the likelihood ratio test (LRT). The key insight: deviance is defined as $-2\log\mathcal{L}$, so the difference in deviances between a reduced and full model directly gives the LRT statistic, which follows a $\chi^2$ distribution under the null hypothesis that the added predictors have zero effect. Here, the test statistic is simply $132.6 - 124.1 = 8.5$. The degrees of freedom equal the number of additional parameters in the full model, which is 3 (one for each added slope coefficient). Consulting a $\chi^2_3$ table, the critical values are approximately $9.35$ ($p = 0.025$) and $7.81$ ($p = 0.05$). Since $8.5$ falls between these, the p-value lies between $0.025$ and $0.05$, confirming answer A is correct. Answer B misidentifies the degrees of freedom as 1 instead of 3. This would apply only if a single parameter were added — confusing the number of constraints with something else. Answer C halves the statistic to $4.25$, which would only make sense if you were converting to an $F$-statistic or mistakenly divided by something; on 3 df, $4.25$ yields $p > 0.10$, a completely different conclusion. Answer D appears to confuse the LRT statistic with an exponentiated likelihood ratio (i.e., computing $e^{8.5/2}$ or similar), producing a nonsensical value of $256.7$. Study tip: Always remember the LRT statistic = difference in deviances, and df = difference in number of parameters. Memorize a few $\chi^2$ critical values (3 df: $7.81$ at $0.05$, $11.34$ at $0.01$) to quickly bracket p-values on exams.
2. Which statement best describes the large-sample behavior of the likelihood ratio statistic?
   Answer: Under local alternatives it converges to a noncentral $\chi^2_r$, while under fixed alternatives it typically diverges. Whenever you encounter questions about the likelihood ratio test (LRT), anchor your thinking to two regimes: local alternatives (shrinking toward the null at rate $n^{-1/2}$) and fixed alternatives (bounded away from the null regardless of $n$). Under the null, the classical result gives $2\log\Lambda_n \xrightarrow{d} \chi^2_r$. The deeper insight is what happens when you shift away. Under local alternatives of the form $\theta_n = \theta_0 + \delta/\sqrt{n}$, a Le Cam-style contiguity argument shows the LRT statistic converges in distribution to a noncentral $\chi^2_r(\lambda)$, where the noncentrality parameter $\lambda$ depends on $\delta$ and the Fisher information. The statistic stays stochastically bounded but shifts its mass rightward. Under fixed alternatives, the log-likelihood ratio grows at rate $n$ (each observation contributes a fixed positive expected log-likelihood difference), so the statistic diverges to infinity — which is exactly why the LRT is consistent against fixed alternatives. This confirms B as correct. A is wrong on both counts: local alternatives do not preserve the central $\chi^2_r$, and fixed alternatives produce divergence, not a bounded noncentral distribution. C is wrong because local alternatives produce a noncentral $\chi^2_r$, not a normal distribution (the normality applies to the score statistic, not the LRT itself). D is wrong because degrees of freedom equal $r$ (the number of restrictions), not the full parameter dimension, and the statistic diverges rather than converges to zero under fixed alternatives. Study tip: Pair each asymptotic regime with its outcome — null $\to$ central $\chi^2_r$, local alternative $\to$ noncentral $\chi^2_r$, fixed alternative $\to$ divergence. This three-way mapping appears repeatedly in large-sample theory.
3. Using Wilks' theorem, which conclusion is correct at significance level $0.05$?
   Answer: The statistic is about $4.46$; compare it with $\chi^2_1$ and reject $H_0$. When you see a likelihood ratio test (LRT) for a normal distribution with unknown parameters, your first instinct should be to identify the degrees of freedom carefully — this is where most errors occur. Under $H_0: \mu = 0$, you estimate only $\sigma^2$ (1 free parameter). Under $H_1$, you estimate both $\mu$ and $\sigma^2$ (2 free parameters). By Wilks' theorem, the LRT statistic $-2\log\Lambda$ follows a $\chi^2$ distribution with degrees of freedom equal to the difference in parameters: $2 - 1 = 1$. Now compute the MLEs. Under $H_1$: $\hat\mu = \bar X$ and $\hat\sigma^2_1 = \frac{1}{n}\sum(X_i - \bar X)^2 = \frac{40}{20} = 2$. Under $H_0$ ($\mu = 0$): $\hat\sigma^2_0 = \frac{1}{n}\sum X_i^2 = \frac{50}{20} = 2.5$. The LRT statistic is: $-2\log\Lambda = n\log\!\left(\frac{\hat\sigma^2_0}{\hat\sigma^2_1}\right) = 20\log\!\left(\frac{2.5}{2}\right) \approx 20(0.2231) \approx 4.46$ This exceeds the $\chi^2_1$ critical value of $3.841$ at $\alpha = 0.05$, so you **reject $H_0$ — confirming C** is correct. Choice A doubles the statistic to $8.93$, which has no valid derivation here. Choice B gets the statistic right but uses $\chi^2_2$, incorrectly counting the total parameters under $H_1$ rather than the difference in parameter counts. Choice D produces a negative statistic, which is impossible since $-2\log\Lambda \geq 0$ always. Study tip: Always remember that the degrees of freedom for Wilks' theorem equals the difference in the number of free parameters between the two hypotheses — not the total number estimated under either model alone.
4. Which assessment of the proposed calibration is most accurate?
   Answer: It is generally invalid because mixture parameters are unidentified under the null and regular Wilks conditions fail. Whenever you see a likelihood ratio test (LRT) applied to mixture models, your first instinct should be to check whether Wilks' theorem conditions are satisfied — because mixture models are a classic setting where they silently fail. Wilks' theorem guarantees that $2(\ell_1 - \ell_0) \xrightarrow{d} \chi^2_k$ under the null when the null hypothesis corresponds to an interior point of the parameter space and the model is identifiable under the null. In a one-vs-two-component Gaussian test, both conditions collapse simultaneously. Under $H_0$ (one component), the mixing proportion $\pi$ is unidentified — you can set $\pi = 0$ or $\pi = 1$, or make the two means equal, all achieving the same likelihood. This means the null lies on the boundary of the parameter space, and multiple parameter configurations are observationally equivalent. The regularity conditions Wilks requires simply do not hold, so the $\chi^2$ reference distribution is incorrect, and the test is miscalibrated in general. C is correct for exactly this reason: the unidentifiability of mixture parameters under the null is a fundamental violation, not a fixable technicality. A is wrong because nesting alone is insufficient — Wilks' theorem requires regularity conditions beyond nesting, which fail here. B is wrong because adjusting degrees of freedom is a red herring; the problem isn't parameter counting, it's that the asymptotic distribution is not chi-square at all. D is wrong because the invalidity is structural and general, not confined to the edge case where estimated means coincide numerically. As a study strategy: memorize that mixture model order testing is a canonical counterexample to naive LRT application — expect exam questions to probe whether you know the chi-square calibration fails and why.
5. Which statistic and asymptotic degrees of freedom give the multinomial likelihood ratio test?
   Answer: Use $2\sum_{j=1}^6 O_j\log(O_j/E_j)$ with a $\chi^2_4$ reference distribution. Whenever you see a likelihood ratio test in a multinomial setting, you need two things exactly right: the formula and the degrees of freedom. The multinomial log-likelihood ratio statistic (also called $G^2$) compares observed counts to expected counts under the null. Twice the log-likelihood ratio between the saturated model and the null model reduces to: $G^2 = 2\sum_{j=1}^6 O_j \log\!\left(\frac{O_j}{E_j}\right)$ Note carefully that the observed counts $O_j$ appear in front of the logarithm — this comes directly from the log-likelihood of the multinomial, where each cell contributes $O_j \log p_j$. The ratio inside the log compares the saturated model's fitted probabilities ($O_j/n$) to the null model's fitted probabilities ($E_j/n$). For degrees of freedom: with 6 cells, the saturated model has 5 free parameters (since probabilities sum to 1). The null model estimates one scalar parameter $\theta$, leaving only 4 free parameters after fitting. The asymptotic $\chi^2$ degrees of freedom equal the difference: $5 - 1 = 4$. So the correct answer is D. A is wrong because it swaps $O_j$ and $E_j$ in both the coefficient and the log argument — that expression has no standard likelihood interpretation. B has the correct formula for $G^2$ but uses $\chi^2_5$, ignoring that one parameter was estimated from the data. C is Pearson's chi-squared statistic, not the likelihood ratio test, and furthermore uses $\chi^2_4$ — mixing the wrong statistic with the right degrees of freedom. A useful rule of thumb: degrees of freedom = (cells $- 1$) $-$ (parameters estimated). Never forget to subtract estimated parameters, or you'll be tempted by the $\chi^2_5$ trap in B.
6. Which likelihood ratio statistic and Wilks reference distribution are appropriate?
   Answer: The statistic is $7.4$ and is compared with a $\chi^2_2$ distribution. Whenever you see a likelihood ratio test question, anchor yourself to two things: the formula for the test statistic and the degrees of freedom, which equals the number of restrictions imposed by the null hypothesis. The likelihood ratio (LR) statistic is defined as $\Lambda = -2(\ell_0 - \ell_1)$, where $\ell_0$ is the maximized log-likelihood under the null and $\ell_1$ is the maximized log-likelihood under the unrestricted model. Here, $\ell_1 = -121.7$ and $\ell_0 = -125.4$, so: $\Lambda = -2(-125.4 - (-121.7)) = -2(-3.7) = 7.4$ By Wilks' theorem, this statistic follows a $\chi^2$ distribution with degrees of freedom equal to the number of restrictions. The null hypothesis sets both slopes to zero — that's exactly 2 restrictions. The intercept and variance are free under both models, so they don't contribute. Therefore, the reference distribution is $\chi^2_2$, confirming answer A is correct. Answer B uses $3.7$, which is the raw log-likelihood difference $\ell_1 - \ell_0$ — it forgets the factor of $-2$ that scales the statistic to match the chi-squared distribution. Answer C correctly computes $7.4$ but uses $\chi^2_4$, mistakenly counting all four parameters in the unrestricted model rather than only the two parameters being constrained. Answer D uses a negative value, which is impossible — because $\ell_1 \geq \ell_0$ always (the unrestricted model can only do at least as well), the statistic is always non-negative. Your memory anchor: **LR statistic = $-2 \times \Delta\ell$**, and degrees of freedom = number of constraints, not total parameters.
7. How should their likelihood ratio tests compare?
   Answer: They have identical likelihood ratio statistics and the same asymptotic $\chi^2_1$ calibration. When you see a question about likelihood ratio tests under different parameterizations, the key insight to reach for is invariance: the likelihood function itself is defined in terms of the data and the model, not the labeling of parameters. The likelihood ratio statistic is $\Lambda = 2[\ell(\hat{\theta}) - \ell(\hat{\theta}_0)]$, where you subtract the maximized log-likelihood under $H_0$ from the unrestricted maximum. Notice that this quantity depends entirely on likelihood values at specific parameter points — not on how you name those points. If $\phi = \log\sigma$, then $\sigma = 1$ and $\phi = 0$ describe the exact same null constraint. The MLE under the alternative satisfies $\hat{\phi} = \log\hat{\sigma}$, so both analysts evaluate the likelihood at the same model configurations. The numerical value of $\Lambda$ is therefore identical — not approximately, but exactly. Furthermore, both are testing a single scalar constraint in a regular model, so by Wilks' theorem both statistics follow an asymptotic $\chi^2_1$ distribution. This confirms D is correct. A is wrong because it introduces a false finite-sample dependence on parameterization. The LRT statistic is algebraically identical regardless of parameterization at every sample size. B is wrong for the same reason — claiming the statistics differ contradicts the invariance property. C correctly notes the statistics are identical but then incorrectly inflates the degrees of freedom to $\chi^2_2$; the calibration depends on the number of constrained dimensions (one here), not on any feature of the reparameterization. As a general study tip: the LRT is parameterization-invariant by construction — anytime a question hints otherwise, that's the distractor. Score degrees of freedom by counting independent constraints, not by counting transformations applied.
8. What is the asymptotic null distribution of the likelihood ratio statistic?
   Answer: It is $\chi^2_2$ because the restriction Jacobian has rank $2$. Whenever you see a Wilks' theorem question involving multiple constraints, your first job is to determine the effective degrees of freedom — not by counting equations, but by computing the rank of the constraint Jacobian evaluated at the null. Wilks' theorem states that the likelihood ratio statistic is asymptotically $\chi^2_r$, where $r$ is the rank of the matrix of partial derivatives of the constraints with respect to $\theta$, evaluated at the true null value. Here the three constraints are $g_1 = \theta_1+\theta_2$, $g_2 = 2\theta_1+2\theta_2$, and $g_3 = \theta_3\theta_4-1$. The Jacobian is: $J = \begin{pmatrix} 1 & 1 & 0 & 0 \\ 2 & 2 & 0 & 0 \\ 0 & 0 & \theta_4 & \theta_3 \end{pmatrix}$ Row 2 is exactly twice row 1, so rows 1 and 2 are linearly dependent — they encode the same restriction. Row 3 is independent (since $\theta_3, \theta_4 \neq 0$). The rank is therefore 2, giving an asymptotic $\chi^2_2$ distribution, confirming answer B. Choice C is the classic trap: naively counting three written equations. Redundant constraints don't add degrees of freedom. Choice A is wrong because the linear restriction $\theta_1+\theta_2=0$ does contribute one independent restriction — it's not rendered irrelevant by the nonlinear one. Choice D is wrong because Wilks' theorem extends to smooth nonlinear constraints, provided the Jacobian has full rank locally and $\theta_3, \theta_4 \neq 0$ ensures smoothness here. The key study habit: always compute the Jacobian rank, not the equation count. Redundancy collapses degrees of freedom silently.
9. Which asymptotic $100(1-\alpha)\%$ likelihood ratio confidence region for $\psi$ is appropriate?
   Answer: Include values satisfying $2[\ell_p(\widehat\psi)-\ell_p(\psi)]\le\chi^2_{2,1-\alpha}$. Whenever you see a likelihood ratio confidence region, the key question is: how many parameters are you making inference about? Wilks' theorem tells us that twice the log-likelihood ratio statistic converges in distribution to a chi-squared random variable whose degrees of freedom equal the dimension of the parameter of interest — not the nuisance parameter. Here, $\psi = (\psi_1, \psi_2)$ is two-dimensional, so the profile likelihood ratio statistic $2[\ell_p(\hat{\psi}) - \ell_p(\psi)]$ is asymptotically $\chi^2_2$ under the true $\psi$. The correct confidence region therefore includes all $\psi$ satisfying $2[\ell_p(\hat{\psi}) - \ell_p(\psi)] \le \chi^2_{2,1-\alpha}$, which is answer A. Answer B uses $\chi^2_3$, which corresponds to the dimension of the nuisance parameter $\lambda$. This is a classic trap: the nuisance parameter is already profiled out and plays no role in the degrees of freedom of the resulting pivot. Answer C uses $\chi^2_5$, apparently adding the dimensions of $\psi$ and $\lambda$ together (2 + 3 = 5). This double-counts the nuisance parameter entirely — once it is maximized out to form the profile likelihood, it no longer contributes degrees of freedom. Answer D divides by 2, using $\ell_p(\hat{\psi}) - \ell_p(\psi) \le \chi^2_{1,1-\alpha}$ without the factor of 2. The factor of 2 is essential to the Wilks result, and using $\chi^2_1$ ignores the two-dimensional nature of $\psi$ entirely. Study tip: Always anchor degrees of freedom to the dimension of the parameter of interest. Nuisance parameters that are profiled out disappear from the chi-squared degrees of freedom entirely.
10. Which null distribution and approximate level-$0.05$ critical value should be used for the likelihood ratio statistic?
    Answer: Use $\tfrac12\chi^2_0+\tfrac12\chi^2_1$ with critical value approximately $2.71$. Whenever you test a variance component like $\tau^2 \geq 0$, the null hypothesis places the parameter on the boundary of its admissible space (variance cannot be negative). Standard likelihood ratio theory assumes the null value lies in the interior of the parameter space, so the usual $\chi^2_1$ result breaks down here. The correct framework comes from the one-boundary-parameter result: when the true parameter is on the boundary, the likelihood ratio statistic under $H_0$ converges in distribution to an equal mixture of a point mass at zero and a $\chi^2_1$, written as $\tfrac{1}{2}\chi^2_0 + \tfrac{1}{2}\chi^2_1$. Intuitively, in half of large samples the restricted and unrestricted MLEs coincide (giving LR = 0), and in the other half the unrestricted estimate moves into the interior (giving a $\chi^2_1$-like contribution). For a level-$0.05$ test, you need the 95th percentile of this mixture. Since $P(\chi^2_1 > c) = 0.05$ gives $c \approx 3.84$, the mixture satisfies $P(\text{LR} > c) = \tfrac{1}{2}P(\chi^2_1 > c) = 0.05$ when $c \approx 2.71$ (the 90th percentile of $\chi^2_1$). This confirms answer B. Choice A uses $\chi^2_1$ with critical value $3.84$, ignoring the boundary constraint — this is the most common trap and yields a conservative test. Choice C uses $\chi^2_2$, which would apply if two interior parameters were being tested simultaneously — incorrect here. Choice D uses $\tfrac{1}{2}\chi^2_1 + \tfrac{1}{2}\chi^2_2$, which arises when testing two boundary parameters, not one. As a study tip: anytime you test $H_0: \sigma^2 = 0$ or any variance component, immediately flag it as a boundary-parameter problem and shift from $\chi^2_k$ to the $\tfrac{1}{2}\chi^2_{k-1} + \tfrac{1}{2}\chi^2_k$ mixture distribution.

Start this quiz