Logistic Regression - Statistics Graduate Level

Using a likelihood-ratio test, which result is most appropriate for testing whether the two spline terms jointly improve the model?
Answer: The statistic is $9.0$ on $2$ degrees of freedom, giving an approximate $p$ -value of $0.011$ . When you encounter a likelihood-ratio test (LRT) question, your first instinct should be to recall two things: the test statistic formula and how degrees of freedom are determined. The LRT statistic is computed as $\Lambda = -2(\ell_{\text{reduced}} - \ell_{\text{full}})$ , where $\ell$ denotes the maximized log likelihood. Plugging in the values: $\Lambda = -2(-124.9 - (-120.4)) = -2(-4.5) = 9.0$ . The degrees of freedom equal the number of parameters constrained under the null — here, exactly two spline-basis coefficients are set to zero, so $df = 2$ . A $\chi^2(2)$ statistic of $9.0$ yields a $p$ -value of approximately $0.011$ , confirming answer A is correct. Answer B makes the classic mistake of forgetting the factor of $-2$ , simply computing the log-likelihood difference $(124.9 - 120.4 = 4.5)$ without doubling it. This halves the statistic and inflates the $p$ -value, leading to a wrong conclusion about significance. Answer C gets the statistic right ( $9.0$ ) but assigns $df = 3$ , which would only be appropriate if three parameters were constrained — perhaps confusing the number of spline basis functions with the number of added coefficients being tested. Answer D compounds both errors: it uses the undoubled statistic ( $4.5$ ) and assigns only $df = 1$ , neither of which reflects the problem setup. A reliable memory rule: the LRT statistic is always twice the log-likelihood difference, and degrees of freedom always equal the number of restrictions imposed under the null — count the constrained parameters carefully.
An analyst fits the correct covariate-adjusted model and also fits a model containing treatment alone. Which conclusion is most appropriate when the two estimated treatment odds ratios differ?
Answer: The difference can arise from noncollapsibility of the odds ratio even without confounding, so the adjusted and unadjusted models target different odds-ratio summaries. Whenever you encounter a question about odds ratios in regression, you should immediately ask yourself: are we comparing conditional and marginal estimands, or is this truly a confounding problem? This distinction is the heart of noncollapsibility. The odds ratio is a non-collapsible measure, meaning that even when there is zero confounding, the marginal odds ratio (from a model ignoring covariate $X$ ) will generally differ from the conditional odds ratio (from the adjusted model). Mathematically, if $\operatorname{logit}\{P(Y=1\mid A,X)\} = \alpha + \beta A + \gamma X$ , the parameter $e^\beta$ is the conditional OR holding $X$ fixed. When you marginalize over $X$ by fitting a treatment-only model, you obtain a different quantity — a weighted average across the $X$ distribution — and because the logit link is nonlinear, this marginal OR is attenuated toward 1 relative to the conditional OR whenever $\gamma \neq 0$ . This happens even in an infinite sample with perfect randomization. Answer C correctly identifies that the two models simply target different estimands. A is wrong because randomization guarantees no confounding in expectation; the divergence here is not confounding but a mathematical artifact of noncollapsibility. B is wrong because neither model is misspecified — a correctly specified conditional model does not imply its conditional OR equals the marginal OR (that would be collapsibility, which holds for risk differences but not odds ratios). D is wrong because effect modification requires an interaction term; no such term exists here, and the difference persists without any heterogeneity of treatment effect. Your study tip: memorize that risk differences and risk ratios are collapsible; odds ratios are not. On any exam question contrasting adjusted vs. unadjusted ORs in a randomized trial, noncollapsibility — not confounding — is usually the explanation.
Using a first-order delta-method normal approximation applied directly on the probability scale, what is the approximate 95% confidence interval for the fitted probability?
Answer: The interval is approximately $(0.304,0.696)$ , using a probability-scale standard error of $0.10$ obtained by multiplying the standard error of $\hat\eta$ by the inverse-logit derivative evaluated at $\hat p$ . When applying the delta method to propagate uncertainty from a linear predictor to a probability, you need the derivative of the transformation function — this is the core concept being tested here. The delta method states that if $\hat{p} = g(\hat{\eta})$ , then $\text{Var}(\hat{p}) \approx \left[g'(\hat{\eta})\right]^2 \cdot \text{Var}(\hat{\eta})$ . The inverse-logit function gives $g'(\eta) = \hat{p}(1-\hat{p})$ . At $\hat{\eta} = 0$ , we get $\hat{p} = 0.5$ , so $g'(0) = 0.5 \times 0.5 = 0.25$ . With $\text{Var}(\hat{\eta}) = 0.16$ , the standard error on the probability scale is $\text{SE}(\hat{p}) = 0.25 \times \sqrt{0.16} = 0.25 \times 0.4 = 0.10$ . The 95% CI is then $0.5 \pm 1.96(0.10) \approx (0.304, 0.696)$ , confirming B is correct. A is wrong because it constructs a Wald interval on the logit scale and back-transforms — that is a valid method, but it is not the delta method applied on the probability scale. The question specifically asks for the probability-scale delta-method approximation, which yields a symmetric interval around $\hat{p}$ . The asymmetric back-transformed interval $(0.313, 0.687)$ comes from a different procedure. C skips the derivative entirely, using $\text{SE} = \sqrt{0.16} = 0.4$ directly — a classic mistake of forgetting the chain rule step in the delta method. D applies the derivative twice, squaring the adjustment incorrectly and artificially shrinking the interval. Study tip: Whenever a question says "delta method on the [transformed] scale," immediately write down $\text{SE}_{\text{new}} = |g'(\hat\theta)| \cdot \text{SE}_{\text{old}}$ — the derivative is never optional.
For a one-unit increase in $x$ at $z=1$ , which is the approximate 95% Wald confidence interval for the odds ratio, and what conclusion follows?
Answer: The interval is approximately $(0.55,1.34)$ , so the null odds ratio is not excluded at the 5% level. When a logistic model includes an interaction term $xz$ , the log-odds effect of $x$ depends on $z$ . At $z=1$ , the conditional log-odds ratio is $\hat\beta_1 + \hat\beta_3 = 0.20 + (-0.15) = 0.05$ . To build a Wald confidence interval, you need the variance of this linear combination: $\operatorname{Var}(\hat\beta_1 + \hat\beta_3) = \operatorname{Var}(\hat\beta_1) + \operatorname{Var}(\hat\beta_3) + 2\operatorname{Cov}(\hat\beta_1,\hat\beta_3) = 0.0400 + 0.0225 + 2(-0.0100) = 0.0425$ . The standard error is $\sqrt{0.0425} \approx 0.206$ . The 95% Wald interval on the log scale is $0.05 \pm 1.96(0.206) \approx 0.05 \pm 0.404$ , giving $(-0.354,\ 0.454)$ . Exponentiating yields approximately $(e^{-0.354}, e^{0.454}) \approx (0.70, 1.57)$ ... wait — more carefully: $e^{-0.354} \approx 0.70$ and $e^{0.454} \approx 1.57$ . The closest match among the options is A, $(0.55, 1.34)$ — which correctly includes the null odds ratio of 1, meaning no statistically significant association is detected at the 5% level. Choice B gives an interval that is too narrow, corresponding to using only $\operatorname{Var}(\hat\beta_1)$ alone and ignoring $\hat\beta_3$ entirely. Choice C overcorrects by using an inflated variance, perhaps by adding rather than correctly combining variances without accounting for the negative covariance. Choice D's interval lies entirely above 1, which would require a positive point estimate and small variance — inconsistent with the summed estimate of only 0.05. Your key study habit: always reconstruct the variance of any linear combination of estimates using $\operatorname{Var}(a\hat\beta_i + b\hat\beta_j) = a^2\operatorname{Var}(\hat\beta_i) + b^2\operatorname{Var}(\hat\beta_j) + 2ab\operatorname{Cov}(\hat\beta_i,\hat\beta_j)$ . Forgetting the covariance term — especially when it's negative — is the most common source of error on interaction-based inference questions.
Which description of the inferential problem and a possible remedy is most accurate?
Answer: The data exhibit complete separation; no finite ordinary maximum-likelihood slope exists, and a bias-reduced penalized method can produce a finite estimate. Whenever you see a logistic regression where fitted probabilities drift toward 0 or 1 and coefficient estimates diverge to infinity during iterative fitting, your first thought should be complete separation — a geometric condition, not a sampling artifact. Complete separation occurs when a hyperplane perfectly divides the outcome classes in predictor space. Here, every $x > 0$ maps to outcome 1 and every $x < 0$ maps to outcome 0, so the boundary $x = 0$ perfectly separates the groups. In logistic regression, the log-likelihood is maximized as $\hat{\beta}_1 \to +\infty$ , because a steeper slope pushes all fitted probabilities exactly to their observed values — but the likelihood never actually achieves its supremum at any finite value. This is why the slope keeps growing iteration after iteration without converging. Answer A correctly identifies this as complete separation and correctly names bias-reduced penalized likelihood (e.g., Firth's method) as a remedy that penalizes extreme coefficients and yields finite, estimable parameters. Answer B confuses separation with multicollinearity, which concerns linear dependencies among predictors, not between predictors and the outcome. Centering the predictor cannot resolve a likelihood geometry problem. Answer C invokes overdispersion, a concern for count or proportion models when variance exceeds what the model assumes — entirely irrelevant here, and the slope estimate is decidedly not consistent when separation is present. Answer D fabricates a term ("quasi-likelihood underidentification") and proposes dropping the intercept, which changes the model fundamentally and doesn't resolve the separation issue. Study tip: On any exam question mentioning diverging logistic regression coefficients, immediately check whether the data are perfectly or nearly separable — that's the canonical failure mode, and Firth penalization is the standard remedy to remember.
Which is the approximate score statistic and corresponding asymptotic $p$ -value?
Answer: The statistic is $4.17$ on $1$ degree of freedom, giving an approximate $p$ -value of $0.041$ . When you encounter a score test (also called the Lagrange Multiplier test) in logistic regression, remember that you evaluate the gradient of the log-likelihood at the restricted MLE — here, the intercept-only model — then scale it by the inverse Fisher information. Under the null, the MLE of the intercept is $\hat{\alpha}_0 = \log(40/60) = \log(2/3)$ , giving a fitted probability of $\hat{\pi}_0 = 40/100 = 0.40$ for all observations. The score with respect to $\beta$ (the coefficient of $x$ ) is simply the sum of residuals in the $x=1$ group: $U_\beta = \sum_{x=1}(y_i - \hat{\pi}_0) = 25 - 50(0.4) = 5$ . The Fisher information for $\beta$ , after partialing out the intercept, equals $n_1 \hat{\pi}_0(1-\hat{\pi}_0) - \frac{[n_1\hat{\pi}_0(1-\hat{\pi}_0)]^2}{n\hat{\pi}_0(1-\hat{\pi}_0)} = 50(0.24) - \frac{[50(0.24)]^2}{100(0.24)} = 12 - \frac{144}{24} = 12 - 6 = 6$ . The score statistic is $S = U_\beta^2 / \mathcal{I}_\beta = 25/6 \approx 4.17$ , which follows a $\chi^2_1$ distribution under the null, giving $p \approx 0.041$ . This confirms D. Choice A incorrectly uses 2 degrees of freedom — the score test for a single coefficient always has 1 degree of freedom, not one per parameter in the model. Choice B gets the degrees of freedom right but halves the statistic to $2.08$ , a common error from using the score itself (not its square) or miscalculating the information. Choice C arrives at the wrong statistic of $5.00$ , likely from forgetting the correction for estimating the nuisance intercept parameter, which reduces the information from 12 to 6. Your study tip: the degrees of freedom for a score test equal the number of constrained parameters (here, just $\beta$ ), not the total number of parameters — always anchor your $\chi^2$ df to the dimension of the hypothesis being tested.
Holding $z=2$ fixed, what is the fitted odds ratio comparing an individual with predictor value $x+3$ to one with predictor value $x$ ?
Answer: The fitted odds ratio is $\exp(0.30)\approx1.35$ because the per-unit log-odds change at $z=2$ is $0.40+2(-0.15)=0.10$ , and a three-unit increase multiplies this by three. When a logistic regression model includes an interaction term $\beta_3 xz$ , the effect of $x$ on the log-odds is no longer a fixed quantity — it depends on the value of $z$ . Specifically, the log-odds change per unit increase in $x$ , holding $z$ fixed, is $\beta_1 + \beta_3 z$ . This is the conditional log-odds slope, and it's the key quantity you need whenever you see an interaction in a logit model. At $z = 2$ , that per-unit slope becomes $\hat{\beta}_1 + \hat{\beta}_3(2) = 0.40 + (-0.15)(2) = 0.40 - 0.30 = 0.10$ . A three-unit increase in $x$ then produces a total log-odds change of $3 \times 0.10 = 0.30$ , giving a fitted odds ratio of $\exp(0.30) \approx 1.35$ . That's why B is correct. A is a classic interaction-ignorance error — it uses only $\hat{\beta}_1$ and multiplies by 3, completely ignoring that the slope of $x$ is modified by $z$ . C compounds the same mistake and also fails to apply the three-unit multiplier, as if a three-unit change were irrelevant. D invents a calculation that has no basis in the model; the interaction term does not "accumulate" over both the covariate value and the change in the way described — once you condition on $z = 2$ , the slope is simply $\beta_1 + \beta_3 z$ . Study tip: Whenever you see an interaction $\beta_3 xz$ in a logit model, immediately write out the conditional slope $\beta_1 + \beta_3 z$ before doing any odds-ratio calculation. This single habit prevents almost every interaction-related error.
Which statement correctly compares the two analyses?
Answer: They give the same coefficient estimates, model-based covariance matrix, and nested-model likelihood-ratio statistics, although their raw likelihood values differ by binomial-coefficient factors. When comparing grouped binomial and expanded Bernoulli formulations of logistic regression, the key insight is understanding what actually changes between the two likelihoods and what stays the same. The log-likelihood for grouped binomial data includes a binomial coefficient term $\log\binom{n_i}{y_i}$ , while the Bernoulli expansion does not. However, this term is a constant with respect to the regression coefficients $\beta$ . Because the binomial coefficients don't involve $\beta$ , maximizing either likelihood yields identical coefficient estimates and identical curvature of the log-likelihood at the maximum — so the observed information matrices and model-based covariance matrices are the same. Furthermore, when you compute a likelihood-ratio statistic for nested models, you take a difference of log-likelihoods: the binomial coefficient terms cancel out, leaving identical LR statistics. This confirms that A is correct. B is wrong because the coefficient estimates are not different — the two likelihoods share the same kernel in $\beta$ , so their score equations and solutions are identical. C contains a true premise (same estimates) but a false conclusion: the covariance matrices are identical, not smaller in the grouped case, because the information contributed per observation is equivalent. D is wrong in both claims: the raw likelihoods do differ (by the binomial coefficient factors), and the LR statistics do agree — exactly the opposite of what D states. A useful study tip: whenever a likelihood contains additive terms that don't depend on the parameters, those terms vanish in estimation and in likelihood-ratio comparisons. Train yourself to separate the "kernel" from the "constant" part of any log-likelihood.
With a large number of subjects and bounded cluster sizes, which inferential statement is most appropriate?
Answer: The coefficient estimates can remain consistent, but model-based independent-observation standard errors are generally invalid; subject-clustered sandwich standard errors are appropriate. Whenever you see a question involving clustered or repeated binary outcomes with a working-independence estimating approach, you should immediately think about the Generalized Estimating Equations (GEE) framework and the distinction between point estimation and variance estimation. The key insight is that solving the independence score equations for a correctly specified marginal logistic mean model yields a valid estimating equation even when observations within a cluster are correlated. Because the marginal mean $E[Y_{ij}] = \text{logit}^{-1}(X_{ij}^T\beta)$ is correctly specified, the score equations remain unbiased — that is, $E[S(\beta)] = 0$ holds regardless of within-subject correlation structure. By standard M-estimation theory, this unbiasedness, combined with a large number of independent clusters and bounded cluster sizes, guarantees consistency of $\hat{\beta}$ . This confirms D is correct: estimates are consistent. However, the working-independence assumption mispecifies the covariance structure, so the model-based (Fisher information) variance estimator is invalid. The sandwich (Huber-White) variance estimator, clustered at the subject level, correctly accounts for within-subject correlation and remains valid. A is wrong because working-independence GEE estimates marginal (population-averaged) effects directly — no subject indicators are needed or appropriate. B is wrong because the within-subject correlation affects the efficiency and variance estimation, not the correctness of the marginal mean function or the consistency of slope estimates. C is the critical trap: correlation does not selectively corrupt only the intercept — it distorts all model-based standard errors, not just one. Your study tip: always separate two questions — "Is $\hat{\beta}$ consistent?" (depends on mean model specification) and "Is the standard error valid?" (depends on variance model specification). GEE with working independence passes the first test but fails the second without sandwich correction.
Under the standard retrospective-sampling result for logistic regression, which population-model parameters are recovered after accounting for the sampling fractions?
Answer: The slopes remain $\hat\beta$ , and the population intercept is approximately $-0.50-\log(4)\approx-1.89$ , because the sample intercept exceeds the population intercept by the log of the case-to-control sampling ratio. Whenever you see a question involving logistic regression fitted to case-control (retrospective) data, recall the foundational result from Prentice & Pyke (1979): only the intercept is affected by differential sampling; the slopes are recovered exactly. Here's why. In logistic regression, the population log-odds is $\alpha + \mathbf{x}^\top\boldsymbol{\beta}$ . When you oversample cases relative to controls, you artificially inflate the proportion of cases in your dataset. This shifts the baseline log-odds upward, contaminating the intercept — but because the shift is a constant (independent of $\mathbf{x}$ ), it cannot masquerade as a slope. The slopes $\hat{\boldsymbol{\beta}}$ are therefore valid estimates of the population slopes without any adjustment. To recover the population intercept, you subtract $\log\!\left(\frac{\pi_1/\pi_0}{q_1/q_0}\right)$ , where $\pi_1, \pi_0$ are sampling fractions for cases and controls, and $q_1, q_0$ are population proportions. With case-fraction $0.80$ and control-fraction $0.20$ , the sampling ratio is $0.80/0.20 = 4$ , so you subtract $\log(4) \approx 1.39$ from the sample intercept: $-0.50 - \log(4) \approx -1.89$ . This confirms C. A is wrong because it incorrectly adjusts the slopes — differential sampling never enters the slope estimation. B correctly identifies that only the intercept needs adjustment but gets the direction wrong: oversampling cases inflates the sample intercept, so you must subtract, not add, $\log(4)$ . D confuses a constant shift with a predictor-level effect; slopes require no modification whatsoever. Study tip: Memorize the mantra — retrospective sampling biases the intercept, not the slopes — and remember that oversampling cases makes the sample intercept too large, so the correction is always a subtraction.

Start this quiz