Deviance & Goodness-of-Fit - Statistics Graduate Level

1. What is the likelihood-ratio deviance statistic for testing the two added coefficients, and what is its usual asymptotic reference distribution under the null hypothesis?
   Answer: $9.12$, compared with a $χ^2_2$ distribution When comparing nested Gaussian models fitted by maximum likelihood, the likelihood-ratio test (LRT) statistic is not simply the difference in RSS. Because ML estimates of $\sigma^2$ use $n$ in the denominator (not $n-p$), the log-likelihood for a Gaussian model is: $\ell = -\frac{n}{2}\ln\hat{\sigma}^2 + \text{const} = -\frac{n}{2}\ln\!\left(\frac{\text{RSS}}{n}\right) + \text{const}$ The deviance ($-2\Delta\ell$) statistic becomes: $D = -2\left(\ell_{\text{reduced}} - \ell_{\text{full}}\right) = n\ln\!\left(\frac{\text{RSS}_{\text{reduced}}}{\text{RSS}_{\text{full}}}\right) = 50\ln\!\left(\frac{120}{100}\right) = 50\ln(1.2) \approx 50 \times 0.1823 \approx 9.12$ Under the null, this statistic follows an asymptotic $\chi^2_2$ distribution, where the degrees of freedom equal the number of added parameters (two coefficients). So B is correct. A is wrong because $10.00$ comes from naively halving the RSS difference $(120-100)/2 = 10$, which corresponds to an F-statistic numerator scaled differently — not the LRT deviance. C incorrectly uses $df=1$ instead of $df=2$ (two coefficients were added), and the value $20.00$ is simply the raw RSS difference with no log transformation. D references $df=47$, which would be the residual degrees of freedom of the full model — irrelevant to the chi-squared reference distribution for this test. A useful memory aid: whenever ML (not REML or OLS) is used in nested Gaussian models, compute the LRT as $n\ln(\text{RSS}_0/\text{RSS}_1)$, not as a simple difference. The $\chi^2$ degrees of freedom always equal the number of additional parameters, not residual degrees of freedom.
2. What is the combined contribution of these observations to the residual deviance?
   Answer: Approximately $2.55$, including a contribution of $1.00$ from the zero count. When working with Poisson GLMs, residual deviance measures how far observed counts stray from their fitted values. The deviance contribution for a single observation is $2\left[y\ln\!\left(\frac{y}{\mu}\right) - (y - \mu)\right]$, with the convention that $y\ln(y/\mu) = 0$ whenever $y = 0$. Apply this to each observation. For the first observation, $y_1 = 0$ and $\mu_1 = 0.5$: the log term vanishes by convention, leaving $2[0 - (0 - 0.5)] = 2(0.5) = 1.00$. Notice that a zero count still contributes deviance through the $(y - \mu)$ term — it is not automatically zero. For the second observation, $y_2 = 4$ and $\mu_2 = 2$: the contribution is $2\left[4\ln(4/2) - (4-2)\right] = 2[4\ln 2 - 2] = 2[2.773 - 2] = 2(0.773) \approx 1.55$. The combined deviance is $1.00 + 1.55 = 2.55$, confirming B. Choice A makes the common mistake of assuming a zero observed count contributes nothing to deviance. While the logarithmic term disappears, the fitted-mean term $-2(y - \mu) = 1.00$ remains and must be included. Choice C drops the $(y-\mu)$ correction term entirely, keeping only the log component; this gives roughly $1.55 + 2.77 \approx 3.77$ but ignores half the deviance formula. Choice D applies an erroneous extra factor of 2 to the already-doubled formula, inflating the result to around $5.09$. A reliable study tip: always write out the full deviance formula $2[y\ln(y/\mu) - (y-\mu)]$ before plugging in numbers. The $(y-\mu)$ piece is easy to overlook, especially when $y=0$.
3. Which statement about their deviance residuals is correct?
   Answer: The residuals are $1,-2,3$, and their squared sum equals the total deviance $14$. Whenever you see a question about deviance residuals in a GLM, anchor yourself to one key definition: the deviance residual for observation $i$ is $d_i = \text{sign}(y_i - \hat{\mu}_i)\sqrt{D_i}$, where $D_i$ is the individual deviance contribution. The sign matches whether the observed value sits above or below its fitted mean, and the magnitude is the square root of the deviance contribution. The total deviance equals $\sum d_i^2$ — the sum of squared deviance residuals. For this problem, the three deviance contributions are $1, 4, 9$, so the magnitudes of the deviance residuals are $\sqrt{1}=1$, $\sqrt{4}=2$, and $\sqrt{9}=3$. The signs come from the direction of each residual: observation 1 is above its mean $(+)$, observation 2 is below $(-)$, and observation 3 is above $(+)$. This gives deviance residuals $1, -2, 3$. Their squared sum is $1+4+9=14$, which equals the total deviance. That confirms D is correct. A is wrong because it assigns all positive signs, ignoring that observation 2 falls below its mean. It also incorrectly claims the ordinary sum equals the square root of total deviance — that's not a standard property. B is wrong on two counts: deviance residuals use square roots of deviance contributions, not the raw contributions themselves, and the sum of unsquared residuals does not generally equal the total deviance. C gets the signs right but wrong: observation 1 is above its mean, so its residual should be $+1$, not $-1$; also, squared deviance residuals sum to total deviance, not necessarily to the residual degrees of freedom. Your study tip: always remember the formula $d_i = \text{sign}(y_i - \hat{\mu}_i)\sqrt{D_i}$, and that $\sum d_i^2 = D_{\text{total}}$ — not the unsquared sum.
4. At the five-percent level, which conclusion is best supported by these results?
   Answer: The added predictors are significant, but the full model does not show significant lack of fit. When analyzing Poisson log-linear models, you typically need to run two separate chi-squared tests: one to assess whether the added predictors improve fit, and another to assess whether the full model itself fits the data adequately. Test 1 — Significance of added predictors (likelihood ratio test): Compare the two models by taking the difference in deviances: $132 - 118 = 14$ on $100 - 97 = 3$ degrees of freedom. You compare this to a $\chi^2_3$ distribution. The critical value at $\alpha = 0.05$ is approximately $7.81$. Since $14 > 7.81$, the three added predictors are statistically significant. Test 2 — Goodness-of-fit of the full model: Use the full model's residual deviance directly: $118$ on $97$ degrees of freedom. The critical value for $\chi^2_{97}$ at $\alpha = 0.05$ is approximately $120.99$. Since $118 < 120.99$, you fail to reject the null hypothesis of adequate fit — the full model does not show significant lack of fit. This confirms answer A: the predictors are significant, but the full model fits adequately. Answer B is wrong because it reverses both conclusions — the predictors are significant and the full model does not show lack of fit. Answer C correctly identifies the predictors as significant but wrongly concludes the full model shows lack of fit ($118$ does not exceed the critical value). Answer D misses the significant drop in deviance from adding the predictors. Study tip: Always remember these are two distinct tests — never assume that a significant improvement automatically implies the full model fails goodness-of-fit, or vice versa.
5. Which bootstrap procedure most appropriately accounts for estimation of the regression parameters?
   Answer: Simulate from the fitted model, refit the model to each simulated data set, and compare its deviance with the observed deviance. When you encounter a parametric bootstrap goodness-of-fit test, the central question is always: what sampling distribution are you trying to approximate? Residual deviance measures how much worse the fitted model is than the saturated model. Under the null hypothesis that your model is correct, you want to know the distribution of that deviance statistic — but this distribution depends on the fact that parameters are estimated, not known. Ignoring estimation uncertainty produces an overly optimistic (narrow) reference distribution. Option C is correct because it faithfully replicates the entire modeling pipeline. You simulate data from the fitted Poisson model, refit the model to each simulated dataset, and compute each simulated deviance from its own newly estimated parameters. This generates a reference distribution that correctly accounts for the variability introduced by parameter estimation, giving you a valid p-value even when fitted means are too small for the chi-square approximation to be reliable. Option A is wrong on two fronts: simulating from the saturated model generates data that doesn't reflect your fitted model's structure, and comparing raw counts bypasses the deviance statistic entirely. Option B makes the critical error of fixing coefficient estimates at their original values across all simulated datasets. This ignores estimation variability, producing a reference distribution that is too narrow and inflating the apparent significance of any departure. Option D abandons the parametric framework altogether — resampling deviance residuals without replacement is finite-population sampling, not a bootstrap, and assuming normality of the reference distribution defeats the entire motivation of avoiding asymptotic approximations. As a study rule: in any bootstrap test involving estimated parameters, ask whether the procedure refits the model at each iteration. If not, it's almost certainly underestimating variability.
6. Which statement gives the most appropriate conventional deviance goodness-of-fit assessment?
   Answer: Compare the grouped residual deviance with $χ^2_9$ because there are $12$ binomial groups and $3$ fitted parameters. Whenever you see a deviance goodness-of-fit question for logistic regression, your first instinct should be to identify whether the data are grouped (binomial counts) or ungrouped (individual Bernoulli trials), because this determines which chi-squared approximation is valid. Here, you have $g = 12$ distinct covariate patterns, each with $n_i = 20$ trials, and the model has $p = 3$ parameters (intercept + two slopes). The residual deviance for grouped binomial data follows an approximate $\chi^2$ distribution with degrees of freedom equal to the number of groups minus the number of fitted parameters: $df = g - p = 12 - 3 = 9$. The passage explicitly tells you that fitted expected successes and failures are reasonably large within every pattern, satisfying the condition needed for this chi-squared approximation to work. That confirms A is correct. B is wrong because when data consist of individual Bernoulli trials (binary 0/1 outcomes), the deviance does not follow a chi-squared distribution in any useful approximating sense — the expected cell counts are simply too small (each is one observation). Comparing individual-trial deviance to $\chi^2_{237}$ is a well-known invalid approach. C incorrectly sets $df = 12$, ignoring that the 3 fitted parameters must be subtracted. Degrees of freedom are always residual — you lose one for each estimated parameter. D invents an extra degree-of-freedom penalty for the intercept. The intercept is already counted among the $p = 3$ parameters, so subtracting it again is double-counting. Key takeaway: For grouped logistic regression, use $df = \text{(number of groups)} - \text{(total parameters estimated)}$, and remember that the individual-trial deviance chi-squared approximation is invalid regardless of sample size.
7. Which assessment is most defensible?
   Answer: The nominal reference is $χ^2_6$, but a conditional or bootstrap calibration is preferable because the table is sparse. Whenever you fit an independence model to a contingency table, two issues arise simultaneously: the degrees of freedom and the validity of the chi-squared approximation. Keep both in mind when the data are sparse. For a $3 \times 4$ table, the independence model has $(3-1)(4-1) = 6$ degrees of freedom, so the nominal reference distribution is $\chi^2_6$. Under regular conditions, the likelihood-ratio statistic $G^2 = 13.2$ would be compared to this distribution and would exceed the critical value near $12.59$, suggesting rejection. However, the chi-squared approximation for $G^2$ relies on large expected counts — roughly, all expected counts should exceed 1 and most should exceed 5. Here, five of the twelve expected counts fall below 1, meaning the table is sparse. In sparse settings, the $\chi^2$ approximation can be unreliable in either direction, so the nominal p-value is untrustworthy. The most defensible path is to acknowledge the $\chi^2_6$ reference but use a conditional exact test or bootstrap/permutation calibration to obtain valid p-values. That reasoning makes D correct. Choice A is wrong on two counts: the degrees of freedom are correctly 6, but sparsity does not systematically make the approximation conservative — the direction of the bias is unpredictable. Choice B is wrong because it confuses the saturated model's parameter count with degrees of freedom; the correct df is 6, not 11. Choice C is wrong because it falsely claims sparsity is irrelevant — sparse expected counts are precisely when the $G^2$ approximation breaks down most severely. As a study rule: whenever you see small expected counts (especially below 1), always flag the chi-squared approximation as suspect and consider exact or resampling methods, regardless of which test statistic is used.
8. Which interpretation of the deviance is most appropriate under this dispersion adjustment?
   Answer: The estimated dispersion is $2.00$, so the adjusted deviance is $105$ and does not indicate excess lack of fit. When you encounter quasi-Poisson regression questions, focus on two things: how dispersion is estimated, and what that estimate does to your inference tools. In quasi-Poisson models, the dispersion parameter $\hat{\phi}$ is typically estimated as the Pearson statistic divided by the residual degrees of freedom. Here, that gives $\hat{\phi} = 240/120 = 2.00$. Once you have this estimate, the residual deviance is rescaled by dividing by $\hat{\phi}$: the adjusted deviance becomes $210/2.00 = 105$. This adjusted statistic is then compared to a $\chi^2_{120}$ distribution. Since $105 < 120$ (well below the mean of the reference distribution), there is no evidence of excess lack of fit — the model fits adequately. This confirms B as correct. A is wrong on two counts: it uses the deviance ($210/120 = 1.75$) rather than the Pearson statistic to estimate dispersion, and then incorrectly claims the adjusted deviance equals exactly $120$, implying "exact fit" — a nonsensical standard. C makes a critical error by suggesting you compare the unadjusted deviance against $\chi^2_{120}$. Under quasi-Poisson, overdispersion inflates the deviance, so comparing the raw deviance without scaling by $\hat{\phi}$ would spuriously signal lack of fit. D inverts the dispersion estimate entirely, computing $\hat{\phi} = 120/240 = 0.50$ — which would imply underdispersion and an inflated adjusted deviance of $420$. Dispersion is always Pearson statistic divided by df, never the other way around. A reliable rule: in quasi models, dispersion comes from the Pearson statistic ÷ df, and you always divide the deviance by $\hat{\phi}$ to adjust it.
9. Which comparison is justified from the reported deviances?
   Answer: Model A has an AIC lower by $6$, but the deviance difference has no standard nested-model chi-square reference. When comparing non-nested models, you need to think carefully about which tools are valid. The likelihood-ratio test (LRT) requires nested models — one must be a special case of the other. AIC, however, works across any models fit to the same data, making it the appropriate comparison tool here. AIC is defined as $\text{AIC} = \text{Residual Deviance} + 2p$, where $p$ is the number of fitted parameters. Since both models have $p = 4$, the $2p$ penalty is identical ($2 \times 4 = 8$ for each). The AIC difference is therefore driven entirely by the deviance difference: Model A's AIC is $84 + 8 = 92$ and Model B's is $90 + 8 = 98$, so Model A is lower by exactly $6$. Crucially, because the models are non-nested, you cannot use a chi-square reference distribution to interpret the deviance difference as a formal hypothesis test. This makes A correct. B is wrong on two counts: the LRT requires nested models, and even if it applied, the degrees of freedom would be the difference in parameters (here $0$, not $6$). C has the likelihood direction exactly backwards — a lower residual deviance corresponds to a higher maximized log-likelihood, so Model A, not B, would be preferred. D is tempting but wrong: equal numbers of parameters means equal penalty terms, but AIC still differs because the deviance terms differ. Your takeaway: memorize that LRT $\Rightarrow$ nested models only, while AIC is the go-to for non-nested comparisons. Equal penalty terms mean deviance differences translate directly into AIC differences.
10. Which conclusion best explains this behavior?
    Answer: The likelihood supremum occurs at an infinite coefficient, so near-zero deviance does not establish regular model adequacy. Whenever you see a question involving logistic regression where a predictor perfectly separates the two outcome groups, you should immediately think about complete separation and its consequences for maximum likelihood estimation. In logistic regression, the log-likelihood for a binary outcome involves terms like $\log(\hat{p}_i)$ and $\log(1 - \hat{p}_i)$. When one predictor perfectly separates outcomes, increasing the coefficient $\beta$ toward $+\infty$ drives predicted probabilities toward 1 for all $Y=1$ cases and toward 0 for all $Y=0$ cases, pushing the log-likelihood toward its theoretical maximum of 0. Critically, this maximum is never actually achieved at any finite value of $\beta$ — the likelihood has a supremum but no maximum. The MLE does not exist in the conventional sense. The iterative algorithm keeps inflating $\hat{\beta}$ without converging, and the residual deviance approaches 0 simply because the algorithm is chasing an asymptote. This is exactly what C describes: the likelihood supremum occurs at infinity, so a near-zero deviance does not signal a well-fitted, well-identified model. A is wrong because standard chi-square goodness-of-fit inference requires a proper, finite MLE — that regularity condition fails here. B is wrong because the saturated model is not misspecified; the problem is with the fitted model's parameter space, not the reference model. D is wrong because this phenomenon has nothing to do with response variance or a Gaussian likelihood; it is a structural identifiability problem specific to complete separation. As a study tip: on graduate-level stats exams, questions about near-zero deviance paired with diverging coefficients almost always test your understanding of complete separation and MLE non-existence — not model fit quality.

Start this quiz