Regression Diagnostics - Statistics Graduate Level

1. Which diagnostic strategy most directly addresses the phenomenon described?
   Answer: Compute joint or subset-deletion diagnostics for the two observations because individual deletion may be affected by masking When multiple observations share unusual predictor values, they can mask each other's influence — each looks harmless in isolation, but together they distort the fitted model. This is the core phenomenon being tested: masking in influence diagnostics. The passage describes a classic masking scenario. Two high-leverage points with small individual residuals each produce modest Cook's distances when deleted alone, yet jointly removing them shifts a slope substantially. This happens because when one influential point remains in the data while the other is deleted, it anchors the fit, concealing how much both observations together are driving the estimates. The diagnostic that directly addresses this is joint or subset-deletion, where you remove both observations simultaneously and measure the aggregate impact — making D the correct answer. Choice A is a red herring. Residual normality plots assess distributional assumptions, not influence. Masked influential points can produce perfectly well-behaved residuals precisely because they're pulling the surface toward themselves, so normality provides no protection against masking. Choice B misidentifies the mechanism. Masking is not solely a function of underestimated error variance — it arises from the geometry of leverage. Externally studentized residuals correct for one observation's effect on variance, but they still evaluate observations individually and cannot detect joint influence. Choice C contains a subtle but important error: high leverage is necessary but not sufficient for influence. A high-leverage point that happens to lie near the true regression surface may not be influential. Leverage alone does not replace deletion diagnostics. Study tip: On regression diagnostics questions, whenever you see multiple outliers or clustered unusual observations, immediately think about masking — individual diagnostics will systematically understate influence, and only subset-deletion methods reveal the true joint effect.
2. What is the value of the contrast after deleting observation $i$?
   Answer: $1.22$, because deletion subtracts the case's leverage-adjusted contribution from the full-data estimate When working with case-deletion diagnostics in OLS, the key formula to internalize is how removing a single observation shifts a contrast estimate. The exact update formula states: $c^\mathsf{T}\hat\beta_{(i)} = c^\mathsf{T}\hat\beta - \frac{c^\mathsf{T}(X^\mathsf{T}X)^{-1}x_i \cdot e_i}{1 - h_{ii}}$ This tells you that deleting observation $i$ adjusts the full-data estimate by subtracting a leverage-weighted residual contribution. Plugging in the given values — $c^\mathsf{T}\hat\beta = 1.70$, $c^\mathsf{T}(X^\mathsf{T}X)^{-1}x_i = 0.12$, $e_i = 3$, and $h_{ii} = 0.25$ — you get: $c^\mathsf{T}\hat\beta_{(i)} = 1.70 - \frac{(0.12)(3)}{1 - 0.25} = 1.70 - \frac{0.36}{0.75} = 1.70 - 0.48 = 1.22$ This confirms D is correct: deletion subtracts the leverage-adjusted contribution. Choice A is wrong because it incorrectly reverses the sign of the adjustment and ignores the $(1 - h_{ii})$ denominator entirely, treating deletion as an addition. Choice B arrives at 2.18 by adding the adjustment rather than subtracting it — a sign error that would describe what happens if you included an extra influential observation, not removed one. Choice C omits the leverage correction, computing $1.70 - (0.12)(3) = 1.34$; this forgets that high-leverage points have their contributions amplified by the $(1 - h_{ii})$ denominator. A useful memory anchor: case deletion always divides by $(1 - h_{ii})$, which inflates the correction for high-leverage points — the closer $h_{ii}$ is to 1, the more sensitive the estimate is to that observation's removal.
3. Under a quasi-binomial analysis using the Pearson or deviance ratio as a dispersion estimate, which adjustment is most appropriate?
   Answer: Retain the coefficient estimates and multiply their model-based standard errors by approximately $\sqrt{2.8}\approx1.67$ Whenever you encounter overdispersion in a GLM, your first instinct should be to separate two questions: what happens to the coefficients? and what happens to the standard errors? These are answered independently under quasi-likelihood theory. Here, the estimated dispersion parameter is $\hat{\phi} = \frac{\text{Residual Deviance}}{\text{Residual df}} = \frac{252}{90} = 2.8$. Under a quasi-binomial model, the mean structure — and therefore the coefficient estimates themselves — remains exactly as fitted. What changes is the variance: quasi-likelihood inflates the variance of each coefficient by a factor of $\hat{\phi}$, so standard errors are multiplied by $\sqrt{\hat{\phi}} = \sqrt{2.8} \approx 1.67$. This is choice B, and it is the correct adjustment. Wider standard errors reflect that the data are more variable than a pure binomial model assumes, making inference more conservative. Choice A gets the direction exactly backwards — dividing standard errors would narrow them, falsely increasing precision when you actually have excess variation. This is a trap for students who confuse "correcting downward" with "correcting for overdispersion." Choice C is doubly wrong: it incorrectly adjusts the coefficient estimates (which are unaffected under quasi-likelihood) and applies a linear rather than square-root scaling to the standard errors. Choice D confuses a diagnostic signal with model invalidity. A dispersion ratio above 1 does not invalidate the mean model — it simply signals that variance needs rescaling, which quasi-likelihood handles without abandoning the binomial mean structure entirely. Study tip: Remember the quasi-likelihood mantra — coefficients stay, standard errors scale by $\sqrt{\hat{\phi}}$. Overdispersion is a variance problem, not a bias problem.
4. Which response is most appropriate if the stated variance relationship is credible?
   Answer: Fit feasible weighted least squares using weights proportional to $1/\hat\mu_i^2$ and reassess the weighted residuals Whenever you see heteroskedasticity with a known variance structure, your goal is to exploit that structure to recover efficiency — not just protect against it. The key principle: weighted least squares (WLS) downweights observations with high variance, assigning weight $w_i \propto 1/\operatorname{Var}(Y_i \mid X_i)$. Here, since $\operatorname{Var}(Y_i \mid X_i) = \sigma^2 \mu_i^2$, the optimal weights are $w_i \propto 1/\mu_i^2$. Because $\mu_i$ is unknown, you substitute $\hat{\mu}_i$ from an initial OLS fit, yielding feasible WLS. After fitting, you must check weighted residuals to verify the variance model was appropriate — if the model is right, those residuals should appear homoskedastic. This is exactly what A prescribes, making it correct. B inverts the logic fatally. Weighting by $\hat{\mu}_i^2$ gives more influence to high-variance observations, which is precisely backwards — it amplifies the problem rather than correcting it. C is a common conceptual trap. Robust (sandwich) standard errors correct inference under heteroskedasticity but do not restore estimator efficiency. OLS with robust SEs remains inefficient when the variance structure is known and exploitable; only WLS achieves the Gauss-Markov efficiency bound in this setting. D is tempting but wrong. A log-transform changes the mean model itself — $E[\log Y_i \mid X_i] \neq \log E[Y_i \mid X_i]$ by Jensen's inequality — so the original conditional-mean parameters are not preserved. Study tip: On problems involving known heteroskedasticity, always set up weights as $w_i = 1/\operatorname{Var}(Y_i)$ and distinguish between correcting inference (robust SEs) versus correcting efficiency (WLS). Only WLS actually improves estimation.
5. Based on Cook's distance, which comparison of the two observations is correct?
   Answer: Observation $i$ is more influential because its Cook's distance is approximately $1.00$, versus $0.25$ for observation $j$. Cook's distance measures how much all fitted values change when observation $i$ is deleted, capturing both leverage (how unusual the predictor values are) and residual magnitude (how poorly the model fits). The formula is: $D_i = \frac{e_i^2}{p \cdot s^2} \cdot \frac{h_{ii}}{(1 - h_{ii})^2}$ where $p$ is the number of fitted coefficients, $s^2$ is the residual mean square, $e_i$ is the residual, and $h_{ii}$ is the leverage. For observation $i$: $D_i = \frac{(2)^2}{4 \cdot 2} \cdot \frac{0.50}{(0.50)^2} = \frac{4}{8} \cdot \frac{0.50}{0.25} = 0.5 \cdot 2 = 1.00$ For observation $j$: $D_j = \frac{(4)^2}{4 \cdot 2} \cdot \frac{0.10}{(0.90)^2} = \frac{16}{8} \cdot \frac{0.10}{0.81} \approx 2 \cdot 0.123 = 0.247$ So observation $i$ has the larger Cook's distance (~1.00 vs ~0.25), confirming A is correct. The high leverage of $h_{ii} = 0.50$ creates an enormous multiplier through the $\frac{h}{(1-h)^2}$ term, more than compensating for observation $j$'s larger raw residual. Choice B incorrectly computes Cook's distance, likely omitting the leverage ratio or miscalculating the $(1-h)^2$ denominator. Choice C is tempting but wrong — the two quantities don't simply "offset"; leverage enters nonlinearly through $(1-h_{ii})^2$, which amplifies influence dramatically near $h = 0.5$. Choice D is fundamentally incorrect; Cook's distance explicitly depends on leverage, not just residuals. The key study tip: always remember that leverage enters Cook's distance as $\frac{h}{(1-h)^2}$, which grows explosively as $h \to 1$. A moderate residual with high leverage can easily dominate a large residual with low leverage.
6. Which value and interpretation are correct?
   Answer: $D\approx0.67$, providing evidence of positive serial correlation in adjacent residuals The Durbin–Watson statistic tests for serial correlation in regression residuals, and its value always falls between 0 and 4: values near 0 signal strong positive autocorrelation, near 2 signal no autocorrelation, and near 4 signal strong negative autocorrelation. When you see residuals that follow a smooth wave pattern — rising then falling, or vice versa — your instinct should immediately be "positive autocorrelation." Start by computing the numerator: the sum of squared successive differences. The consecutive differences $(e_t - e_{t-1})$ are $(2-1), (1-2), (-1-1), (-2-(-1)), (-1-(-2)) = 1, -1, -2, -1, 1$. Squaring and summing gives $1+1+4+1+1 = 8$. The denominator is $\sum e_t^2 = 1+4+1+1+4+1 = 12$. Therefore $D = 8/12 \approx 0.67$. Since $0.67 \ll 2$, this indicates positive serial correlation — adjacent residuals move together rather than alternating. The residual pattern $1, 2, 1, -1, -2, -1$ visually confirms this: values drift smoothly before reversing, a hallmark of positive autocorrelation. Choice A is doubly wrong: the computed value $2.67$ is incorrect (that would be $32/12$), and even if it were right, $D > 2$ would suggest negative, not general, correlation. Choice B gets the direction of interpretation right (positive autocorrelation) but miscalculates $D$ as $1.33$ — likely from dividing 8 by 6 instead of 12. Choice D correctly computes $D \approx 0.67$ but misidentifies the direction, confusing "small D" with negative correlation. A reliable memory anchor: think of $D \approx 2$ as your neutral baseline. Anything below 2 means residuals are too similar (positive autocorrelation); anything above 2 means they alternate too sharply (negative autocorrelation).
7. Which modification is most directly supported by this residual structure?
   Answer: Add a positive quadratic term in the predictor to represent curvature in the conditional mean Whenever you see a residual plot described in terms of where residuals are positive or negative, your first instinct should be to ask: "What pattern in the mean function does this reveal?" The key diagnostic here is the sign structure — positive residuals at both extremes and negative residuals in the middle — not the spread or any individual point's influence. This sign pattern is the classic fingerprint of a concave-down (∩-shaped) relationship between the response and predictor. When you fit a straight line through a curve that bends downward, the line overshoots in the middle (hence negative residuals there) and undershoots at both tails (hence positive residuals at the extremes). The fix is to let the mean function itself bend — specifically, by adding a term $\beta_2 x^2$ with an expected negative coefficient. This is exactly what C proposes with a positive quadratic term (positive in the sense of adding the squared predictor; the coefficient's sign is estimated from data). C directly addresses the misspecified conditional mean. A is wrong because residual sign does not indicate influence or leverage. Cook's distance — which the passage explicitly tells you is unremarkable — measures influence. Deleting extreme observations based solely on residual sign is a methodological error. B is wrong because heteroskedasticity means non-constant spread. The passage explicitly states that residual spread is roughly constant, so inverse-variance weighting addresses a problem that doesn't exist here. D is wrong for a subtle but important reason: heteroskedasticity-consistent (HC) standard errors correct inference under non-constant variance, but they do nothing to fix a misspecified mean function. You'd still have biased estimates of the mean. A useful rule of thumb: mean misspecification (curved residuals) calls for a better model structure; variance misspecification (fan-shaped spread) calls for weighted least squares or HC standard errors. Don't confuse the two.
8. Which diagnostic comparison is most accurate?
   Answer: Observation $A$ has lower leverage but a far larger Pearson residual, so it may still be more influential than observation $B$ When diagnosing influential observations in logistic regression, you need to track two separate quantities — leverage and residual size — and understand that influence typically depends on both. Neither alone tells the whole story. Start by computing the leverages. For observation $A$: $h_A = 0.001 \times 40 = 0.04$. For observation $B$: $h_B = 0.25 \times 0.8 = 0.20$. So $B$ has higher leverage. Now compute the Pearson residuals, defined as $r_i = (y_i - \hat{p}_i)/\sqrt{\hat{p}_i(1-\hat{p}_i)}$. For $A$ (outcome 1, $\hat{p}_A = 0.001$): $r_A = (1 - 0.001)/\sqrt{0.001 \times 0.999} \approx 0.999/0.0316 \approx 31.6$. For $B$: $r_B = (1 - 0.50)/\sqrt{0.25} = 0.5/0.5 = 1.0$. Observation $A$ has dramatically larger residual despite lower leverage, meaning it could still exert substantial influence — making B the most accurate statement. Choice A is wrong because it concludes that lower leverage necessarily implies less influence, ignoring that a massive residual can compensate and drive influence metrics like Cook's distance upward. Choice C incorrectly claims $B$ "dominates every influence diagnostic" — with its residual near 1 and moderate leverage, $B$ is actually fairly unremarkable compared to $A$'s enormous residual. Choice D is simply false; the products $0.04$ and $0.20$ are not equal. The key study tip: influence = leverage × residual (conceptually). An observation in an unusual predictor region and poorly fit by the model is the dangerous combination — never dismiss a point just because one component looks small.
9. Which statement best describes the observation's current and potential influence?
   Answer: Its case-deletion coefficient change and Cook's distance are zero, but a small response perturbation could make it influential because leverage is high Influence in regression depends on two separate ingredients: leverage (how unusual an observation's predictors are) and the residual (how far its response deviates from the fitted value). High leverage alone is not sufficient for influence — the residual must also be nonzero. This distinction is precisely what this question tests. The standard formulas make the logic clear. Cook's distance is proportional to $h_{ii} \cdot e_i^2$, and the case-deletion coefficient change (DFBETAS) also scales with the residual. When $e_i = 0$, both quantities equal zero regardless of how large $h_{ii}$ is. So right now, this observation — despite sitting at $h_{ii} = 0.80$ — exerts no actual influence on the estimated coefficients. Answer A captures this perfectly: current influence is zero, but the high leverage means the observation sits in a powerful position. If its response were nudged even slightly, that nonzero residual would be amplified by the large leverage factor $h_{ii}/(1-h_{ii})$, potentially producing a dramatic shift in the fitted model. A is correct. Answer B is wrong because leverage alone does not determine influence — the residual is an equally essential factor, not an afterthought. Answer C misreads the Cook's distance formula: the $1/(1-h_{ii})^2$ term appears in the denominator of the variance, but it multiplies $e_i^2$, which is zero here, so Cook's distance is exactly zero, not infinite. Answer D inverts the logic entirely — zero residual does not imply low leverage, and high leverage means the observation can be influential if perturbed. Study tip: Memorize that influence = leverage × residual. Neither alone is sufficient. A high-leverage, zero-residual point is a "loaded gun" — harmless now, dangerous if the response shifts.
10. Using the full-sample residual mean square, which pair gives the internally studentized residual and the deleted prediction residual for observation $i$, respectively?
    Answer: $1.29$ and $3.33$, respectively When you see a question involving residual diagnostics in OLS regression, keep two formulas straight: the internally studentized residual and the deleted prediction residual (also called the PRESS residual). The internally studentized residual is $r_i = \frac{e_i}{\sqrt{s^2(1 - h_{ii})}}$. Plugging in: $r_i = \frac{2}{\sqrt{4(1 - 0.40)}} = \frac{2}{\sqrt{4 \times 0.60}} = \frac{2}{\sqrt{2.4}} = \frac{2}{1.549} \approx 1.29$. The deleted prediction residual is $d_i = \frac{e_i}{1 - h_{ii}}$. Plugging in: $d_i = \frac{2}{1 - 0.40} = \frac{2}{0.60} \approx 3.33$. This confirms answer A is correct: $1.29$ and $3.33$, respectively. Now for the distractors. Answer B (1.67 and 2.58) results from omitting $1 - h_{ii}$ inside the square root — computing $2/\sqrt{4} = 1.0$ is wrong, but 1.67 suggests dividing by $s$ alone without the leverage correction; 2.58 is $2/\sqrt{0.6}$ rather than $2/0.6$, confusing the deleted residual formula with another variance adjustment. Answer C (0.77 and 3.33) gets the deleted residual right but inverts the studentized formula — using $\sqrt{s^2/(1-h_{ii})}$ in the denominator instead of $\sqrt{s^2(1-h_{ii})}$. Answer D (1.29 and 1.20) correctly computes the studentized residual but then applies an extra square root to the deleted residual: $\sqrt{3.33} \approx 1.83$, or confuses the formula entirely. A reliable study habit: memorize that the deleted residual inflates $e_i$ by dividing by $(1-h_{ii})$, while the studentized residual deflates it by multiplying $s^2$ by the same factor before taking the square root — opposite operations, easy to mix up under pressure.

Start this quiz