Likelihood-Based CIs - Statistics Graduate Level

1. To which value should the ordinary profile likelihood-ratio statistic $2\{\ell_p(\widehat\psi)-\ell_p(\psi)\}$ be equated at the upper endpoint?
   Answer: $2.71=(1.645)^2$ Whenever you see a question mixing profile likelihood-ratio statistics with confidence bounds, the key is carefully tracking the relationship between the signed root statistic and its square. The signed root profile likelihood-ratio statistic is $r_p(\psi) = \text{sgn}(\widehat{\psi}-\psi)\sqrt{2\{\ell_p(\widehat\psi)-\ell_p(\psi)\}}$, which is approximately $N(0,1)$. For a one-sided 95% upper bound, you want the value $\psi_U$ such that $P(\widehat\psi \leq \psi_U) = 0.95$. On the upper side of $\widehat\psi$, the signed root is negative (since $\widehat\psi - \psi < 0$), so you set $r_p(\psi_U) = -1.645$. Squaring both sides gives $2\{\ell_p(\widehat\psi)-\ell_p(\psi_U)\} = (1.645)^2 = 2.71$. That confirms B is correct. Choice A is the trap most students fall into. The value $3.84 = (1.960)^2$ corresponds to a two-sided 95% interval, where you split the 5% across both tails, each carrying $z_{0.025}=1.960$. For a one-sided bound, you use the full 5% in one tail, giving $z_{0.05}=1.645$, not $1.960$. Choice C correctly identifies $1.645$ but forgets the squaring step — the profile likelihood-ratio statistic equals $r_p^2$, not $r_p$ itself. Choice D, $5.99$, comes from a chi-squared distribution with two degrees of freedom, which is irrelevant here since $\psi$ is a scalar (one degree of freedom). A useful pattern: one-sided 95% bounds always square the one-tailed $z_{0.05}=1.645$, giving $2.71$, while two-sided 95% intervals square $z_{0.025}=1.960$, giving $3.84$. Memorize this pairing — confusing them is among the most common errors on likelihood-based inference questions.
2. Without recomputing the likelihood optimization, which interval is the corresponding profile-likelihood interval for $\phi$?
   Answer: $[0.20,\ 0.50]$ Whenever you encounter profile-likelihood inference under a parameter transformation, the key principle is equivariance: the profile-likelihood confidence interval transforms exactly as the parameter does, with no re-optimization needed. Here's why this works. The profile likelihood for $\psi$ defines a set of "plausible" values $\{\psi : \ell_p(\psi) \geq \ell_p(\hat\psi) - c\}$, which produces the interval $[2, 5]$. Because $\phi = 1/\psi$ is a monotone (strictly decreasing) function, the set of plausible $\phi$ values is simply the image of $[2, 5]$ under this map. You apply $\phi = 1/\psi$ to each endpoint: $1/5 = 0.20$ and $1/2 = 0.50$. Since the transformation is decreasing, the endpoints swap, giving $[0.20, 0.50]$. That's answer D. Now, why are the others wrong? A simply copies $[2, 5]$ unchanged — a clear error, since $\phi$ and $\psi$ live on different scales and the transformation was never applied. B applies $1/\psi$ to the MLE ($1/3 \approx 0.33$) and seems to confuse endpoint mapping, yielding $[0.17, 0.50]$ — the lower bound $0.17$ doesn't correspond to either endpoint of the original interval. C gives $[0.20, 0.33]$, which uses $1/5$ correctly but replaces the upper bound with the transformed MLE $1/3$ instead of $1/2$ — a tempting trap if you conflate the MLE with a confidence boundary. Study tip: For any monotone transformation $g$, just apply $g$ to both endpoints and re-order if necessary. Never substitute the MLE into an interval endpoint — the MLE transforms correctly on its own, but it is not a boundary of the confidence interval.
3. After expressing $\lambda_1=\psi\lambda_2$ and profiling over $\lambda_2$, which expression is the profile log-likelihood for $\psi$, up to an additive constant independent of $\psi$?
   Answer: $\ell_p(\psi)=12\log\psi-20\log(1+\psi)$ When you encounter a profile likelihood problem, your goal is to eliminate nuisance parameters by substituting their MLEs (expressed as functions of the parameter of interest) back into the full log-likelihood. Start with the joint log-likelihood. Since $X \sim \text{Poisson}(\lambda_1)$ and $Y \sim \text{Poisson}(\lambda_2)$ independently, the full log-likelihood (dropping constants not involving parameters) is $\ell(\lambda_1, \lambda_2) = x\log\lambda_1 - \lambda_1 + y\log\lambda_2 - \lambda_2.$ Substituting $\lambda_1 = \psi\lambda_2$ gives $\ell(\psi, \lambda_2) = x\log(\psi\lambda_2) - \psi\lambda_2 + y\log\lambda_2 - \lambda_2 = x\log\psi + (x+y)\log\lambda_2 - (1+\psi)\lambda_2.$ To profile, maximize over $\lambda_2$: setting the derivative to zero yields $\hat{\lambda}_2(\psi) = \frac{x+y}{1+\psi}.$ Substituting back and using $x+y = 20$, $x = 12$: $\ell_p(\psi) = 12\log\psi + 20\log\!\left(\frac{20}{1+\psi}\right) - 20 = 12\log\psi - 20\log(1+\psi) + \text{const},$ confirming answer A. Answer B uses $y = 8$ instead of $x = 12$ in the first term — this would arise from confusing which count multiplies $\log\psi$. Answer C correctly identifies the $12\log\psi$ term but uses $x$ alone (12) instead of $x+y$ (20) in the second term, forgetting that both observations contribute to the total count constraining $\lambda_2$. Answer D replaces $x\log\psi$ with $(x+y)\log\psi$, incorrectly attributing the full count to $\psi$ rather than recognizing that only $x$ directly links to $\psi\lambda_2$. A reliable strategy: always track which terms carry $\psi$ versus $\lambda_2$ before profiling. The coefficient of $\log\psi$ comes solely from $x\log\lambda_1 = x\log\psi + x\log\lambda_2$, while $x + y$ appears only after maximizing over $\lambda_2$.
4. Which candidate values belong to the confidence set?
   Answer: Only $\psi_1$ and $\psi_2$ belong to the set. Whenever you see a question involving profile likelihood confidence sets, your core task is converting the relative profile likelihood into a log-likelihood ratio statistic and comparing it to a chi-squared cutoff. The profile likelihood confidence set at level 95% includes all values of $\psi$ satisfying: $-2\log R_p(\psi) \leq \chi^2_{1,0.95} = 3.84$ Since $R_p(\psi) = L_p(\psi)/L_p(\hat{\psi})$, you compute $-2\log R_p(\psi)$ for each candidate: - $\psi_1$: $-2\log(0.20) = -2(-1.609) = 3.22 \leq 3.84$ ✓ - $\psi_2$: $-2\log(0.15) = -2(-1.897) = 3.79 \leq 3.84$ ✓ - $\psi_3$: $-2\log(0.14) = -2(-1.966) = 3.93 > 3.84$ ✗ - $\psi_4$: $-2\log(0.02) = -2(-3.912) = 7.82 > 3.84$ ✗ So only $\psi_1$ and $\psi_2$ fall inside the confidence set, confirming answer B. Answer A incorrectly includes $\psi_3$, which fails the cutoff by a small but decisive margin (3.93 > 3.84). This is the most tempting trap — $\psi_3$'s relative likelihood of 0.14 looks close to $\psi_2$'s 0.15, but the log transformation amplifies small differences near the boundary. Answer C excludes $\psi_2$, likely from failing to perform the log conversion at all and instead using some raw threshold on $R_p$. Answer D includes all four values, ignoring the test entirely. Study tip: Always convert $R_p(\psi)$ to the log-likelihood ratio scale before comparing to the chi-squared cutoff — never apply the 3.84 threshold directly to the relative likelihood values themselves.
5. Which approach is most defensible for calibrating a likelihood-based confidence set for $a$?
   Answer: Calibrate the profiled statistic under the null, such as by parametric bootstrap. When a nuisance parameter is unidentifiable under the null hypothesis, standard asymptotic theory for likelihood-ratio statistics breaks down entirely. This is the classic Davies problem: because $\omega$ disappears from the likelihood when $a=0$, the profile likelihood-ratio statistic no longer follows the usual $\chi^2$ reference distribution, regardless of how many parameters you count. Recognizing this setup should immediately signal that off-the-shelf calibration is unreliable. The most defensible approach is D — calibrating the profiled statistic empirically under the null, for example via parametric bootstrap. You simulate data under $a=0$, compute the profiled statistic (maximizing over $\omega$) on each simulated dataset, and build the reference distribution directly. This respects the actual null distribution, which can have heavier tails than any $\chi^2$. Choice A applies standard Wilks' theorem for a one-dimensional parameter of interest, but Wilks' theorem requires the model to be regular at the null — identifiability of all parameters is part of that regularity. It fails here. Choice B tries to account for the extra optimization over $\omega$ by adding a degree of freedom, but this is equally unfounded: the $\chi^2_2$ approximation also relies on regularity conditions that are violated. Adjusting the degrees of freedom does not fix a fundamentally non-standard distribution. Choice C eliminates the optimization by conditioning on the alternative's $\hat{\omega}$, but this ignores the uncertainty in $\omega$ and produces a statistic whose null distribution is still non-standard and poorly characterized. As a study strategy: whenever you see a nuisance parameter that vanishes under $H_0$, immediately set aside Wilks' theorem and think simulation-based calibration.
6. Which interval is obtained for $\psi$?
   Answer: $[0.61,\ 3.39]$ Whenever you see a profile likelihood problem, your goal is to eliminate the nuisance parameter $\lambda$ by maximizing the log-likelihood over it for each fixed $\psi$, then find where the profile log-likelihood drops by half the critical value. Finding the profile likelihood. For fixed $\psi$, maximize over $\lambda$. Taking the derivative of $\ell(\psi,\lambda)$ with respect to $(\lambda-1)$ and setting it to zero: $-\frac{1}{2}\left[4(\psi-2) + 4(\lambda-1)\right] = 0 \implies \lambda-1 = -(\psi-2).$ Substituting back, the profile log-likelihood becomes: $\ell_p(\psi) = \ell(2,1) - \frac{1}{2}\left\{4(\psi-2)^2 + 4(\psi-2)(-(\psi-2)) + 2(\psi-2)^2\right\} = \ell(2,1) - (\psi-2)^2.$ Constructing the interval. The profile likelihood ratio statistic is $2[\ell(2,1) - \ell_p(\psi)] = 2(\psi-2)^2$. Setting this equal to $3.84$: $2(\psi-2)^2 = 3.84 \implies (\psi-2)^2 = 1.92 \implies |\psi-2| = \sqrt{1.92} \approx 1.386.$ This gives the interval $[2 - 1.386,\ 2 + 1.386] \approx [0.61,\ 3.39]$, confirming A. The wrong answers reflect common mistakes. B results from forgetting to profile out $\lambda$ entirely and using the observed curvature in $\psi$ directly ($4(\psi-2)^2 = 3.84$). C comes from using the full $\chi^2_2$ cutoff instead of $\chi^2_1$. D arises from an algebraic error during the substitution step that leaves a residual cross-term. Study tip: Always complete the profiling step fully before applying the cutoff — the most common trap is using the raw curvature in $\psi$ without accounting for how $\lambda$ adjusts, which artificially narrows the interval.
7. Which approximate critical value should be used to obtain a 95% likelihood-ratio confidence set by inversion at this boundary?
   Answer: $2.71$, the 90th percentile of $\chi^2_1$ Whenever you see a likelihood-ratio test at a boundary of the parameter space, the standard $\chi^2_1$ critical value no longer applies — you must account for the non-standard asymptotic distribution. Here, because $\tau \geq 0$ and $\widehat{\tau} = 0$, the likelihood-ratio statistic has the asymptotic null distribution $\frac{1}{2}\chi^2_0 + \frac{1}{2}\chi^2_1$. To invert this test and form a 95% confidence set, you need the 95th percentile of this mixture. The mixture CDF satisfies $P(T \leq t) = \frac{1}{2}\cdot\mathbf{1}(t \geq 0) + \frac{1}{2}\cdot F_{\chi^2_1}(t)$. Setting this equal to 0.95 gives $\frac{1}{2} + \frac{1}{2}F_{\chi^2_1}(t) = 0.95$, so $F_{\chi^2_1}(t) = 0.90$. This means you need the **90th percentile of $\chi^2_1$**, which is $2.71$ — confirming answer C. Answer A (5.99) is the 95th percentile of $\chi^2_2$, which would apply if you were testing two unconstrained parameters simultaneously — a completely different setting. Answer B (3.84) is the standard 95th percentile of $\chi^2_1$, which you would use if $\tau$ were an interior parameter with no boundary constraint — ignoring the point mass at zero entirely. Answer D (1.64) conflates the likelihood-ratio statistic with a normal-based test statistic; the LR statistic is already squared, so you never take an unsquared normal percentile here. The key study tip: boundary parameters cut the asymptotic distribution in half, effectively shifting you one quantile level — a 95% confidence set uses the **90th percentile of $\chi^2_1$**, not the 95th.
8. Which construction gives an asymptotic 95% joint profile-likelihood confidence region for $\boldsymbol{\beta}$?
   Answer: $2\{\ell(\widehat{\boldsymbol{\beta}},\widehat{\boldsymbol{\gamma}})-\ell_p(\boldsymbol{\beta})\}\leq\chi^2_{2,0.95}$ Whenever you see a question about profile likelihood regions, anchor yourself to Wilks' theorem: the likelihood ratio statistic $2\{\ell(\hat{\boldsymbol{\beta}}, \hat{\boldsymbol{\gamma}}) - \ell(\boldsymbol{\beta}, \boldsymbol{\gamma})\}$ converges in distribution to a $\chi^2$ whose degrees of freedom equal the dimension of the parameter being tested, not the total parameter count. The profile likelihood $\ell_p(\boldsymbol{\beta}) = \sup_{\boldsymbol{\gamma}} \ell(\boldsymbol{\beta}, \boldsymbol{\gamma})$ plugs in the nuisance MLE at each fixed $\boldsymbol{\beta}$, so the statistic $2\{\ell(\hat{\boldsymbol{\beta}}, \hat{\boldsymbol{\gamma}}) - \ell_p(\boldsymbol{\beta})\}$ is a proper profile likelihood ratio. Because $\boldsymbol{\beta}$ is two-dimensional, Wilks' theorem gives asymptotic distribution $\chi^2_2$, and the 95% region is exactly $2\{\ell(\hat{\boldsymbol{\beta}}, \hat{\boldsymbol{\gamma}}) - \ell_p(\boldsymbol{\beta})\} \leq \chi^2_{2, 0.95}$. That is answer A. B uses $\chi^2_5$, which would correspond to testing all five parameters simultaneously — a miscount that ignores how profiling eliminates the nuisance dimensions from the asymptotic distribution. C substitutes $\ell(\boldsymbol{\beta}, \hat{\boldsymbol{\gamma}})$ instead of $\ell_p(\boldsymbol{\beta})$; this fixes the nuisance estimates at their overall MLE values rather than re-optimizing them at each $\boldsymbol{\beta}$, so it is not a profile likelihood ratio and loses its clean $\chi^2_2$ limit. D uses $\chi^2_1$, appropriate only for a scalar parameter of interest. The key study tip: degrees of freedom in a likelihood ratio test always count the dimension of the subspace you're testing, and profiling over nuisance parameters does not add degrees of freedom — it removes those dimensions from the problem.
9. Which interval results from profiling out $\sigma^2$?
   Answer: $[4.08,\ 5.92]$ When you see a likelihood-ratio (LR) interval formed by profiling out a nuisance parameter, your goal is to find all values of the parameter of interest where the profile log-likelihood ratio statistic stays within the critical threshold. Here, the full log-likelihood for $(\mu, \sigma^2)$ under normality yields a profile log-likelihood for $\mu$ alone by plugging in the MLE of $\sigma^2$ at each fixed $\mu$. That profile MLE is $\hat{\sigma}^2(\mu) = \frac{1}{n}\left[\sum(x_i - \bar{x})^2 + n(\bar{x}-\mu)^2\right]$. The profile LR statistic simplifies to: $-2\log R(\mu) = n\log\left(1 + \frac{n(\bar{x}-\mu)^2}{\sum(x_i-\bar{x})^2}\right) \leq 3.84$ With $n=10$, $\bar{x}=5$, and $\sum(x_i-\bar{x})^2=18$, set the statistic equal to 3.84: $10\log\left(1 + \frac{10(\mu-5)^2}{18}\right) = 3.84$ $1 + \frac{10(\mu-5)^2}{18} = e^{0.384} \approx 1.4685$ $\frac{10(\mu-5)^2}{18} = 0.4685 \implies (\mu-5)^2 = 0.8433 \implies |\mu-5| \approx 0.918$ This gives the interval $[4.08,\ 5.92]$, confirming A. Choice B ($[4.17, 5.83]$) results from incorrectly using a $t$-based pivot without proper LR construction. Choice C ($[3.99, 6.01]$) over-widens the interval, likely from misapplying a chi-squared quantile directly to $(\bar{x}-\mu)^2$. Choice D ($[4.38, 5.62]$) is too narrow, corresponding to an error where $\sigma^2$ is treated as known or the sample size is mishandled. Your study tip: always remember that profiling replaces $\sigma^2$ with its conditional MLE at each $\mu$, converting the LR statistic into a log-ratio of observed variances — the algebra collapses neatly into the logarithmic form above.
10. How should this likelihood-ratio confidence set ordinarily be reported?
    Answer: Report $[-2,-1]\cup[0.5,1.5]$ as a disconnected confidence set. When a profile log-likelihood is nonconcave, the likelihood-ratio confidence set is constructed mechanically: collect all parameter values whose profile log-likelihood falls within the specified cutoff of the global maximum. If that set happens to be disconnected, you report it as-is — the math dictates the shape, not your aesthetic preference. Here, the cutoff captures two disjoint intervals, $[-2,-1]$ and $[0.5,1.5]$, because both regions satisfy the likelihood-ratio criterion. The correct answer is C: report $[-2,-1]\cup[0.5,1.5]$ as a disconnected confidence set. This is statistically honest — it communicates that the data are consistent with two separate regions of plausible parameter values, a meaningful inferential message about the likelihood surface. A is wrong because discarding $[-2,-1]$ arbitrarily ignores values that legitimately satisfy the confidence criterion. The global mode being at $\widehat\psi=1$ doesn't disqualify the secondary region if it clears the cutoff — the rule is about the threshold, not uniqueness of the mode. B is wrong because filling the gap with $[-2,1.5]$ includes values in $(−1,0.5)$ that fail the likelihood-ratio criterion. Merging intervals to appear "cleaner" inflates the confidence set and misrepresents the likelihood surface — this is a conservative distortion, not a valid simplification. D is wrong for the mirror reason as A: keeping only the secondary interval while dropping the global-mode interval $[0.5,1.5]$ is indefensible. Study tip: On any question involving nonconcave likelihoods, remember that the confidence set is defined by a threshold condition applied globally — shape follows function. Never prune or pad a valid confidence set for cosmetic reasons.

Start this quiz