Exponential Family & Link Functions - Statistics Graduate Level

1. Conditional on the observed total $N$, which description of the joint distribution of the counts is correct?
   Answer: It is multinomial with probabilities proportional to $t_i\exp(x_i^{\mathsf T}\beta)$, and the intercept $\alpha$ cancels. Whenever you see Poisson regression conditioned on the total count, think about the classic result that ties Poisson models to the multinomial distribution. This connection is what the question tests. Start from the joint probability of $(Y_1, \ldots, Y_n)$ given $N = \sum_i Y_i$. Because the $Y_i$ are independent Poisson, the conditional distribution is multinomial with cell probabilities $\pi_i = \mu_i / \sum_j \mu_j$. Now substitute $\mu_i = t_i \exp(\alpha + x_i^\mathsf{T}\beta)$. In the ratio, $\exp(\alpha)$ appears in every numerator and denominator, so it cancels completely. What remains is $\pi_i = \frac{t_i \exp(x_i^\mathsf{T}\beta)}{\sum_j t_j \exp(x_j^\mathsf{T}\beta)}.$ The exposure $t_i$ survives because it can differ across units, while $\alpha$ (a common multiplicative factor) drops out. This confirms D as correct. Choice A is wrong because conditioning on $N$ yields a multinomial, not a product of Bernoullis — those arise in binary logistic regression, a different model entirely. Choice B is tempting but incorrect: it drops $t_i$ from the probabilities, which is only valid if all exposures are equal. In general, $t_i$ is unit-specific and cannot be absorbed into the intercept, so it must remain. Choice C is wrong because conditioning on $N$ destroys independence among the $Y_i$; the counts are no longer marginally Poisson once their sum is fixed. A useful study tip: whenever you condition Poisson counts on their sum, immediately write out the multinomial cell probability as a ratio of means — cancellations (like the intercept here) become obvious, and you can see exactly which terms survive.
2. What is the fitted conditional mean of the response at this covariate value?
   Answer: $\exp(0.7)$, because the inverse link directly models the conditional mean. Whenever you see a GLM question specifying a non-canonical link, your first instinct should be to carefully distinguish between the linear predictor $\eta_i$, the link function $g$, and the conditional mean $\mu_i$. The GLM framework always defines $g(\mu_i) = \eta_i$, which means the fitted conditional mean is recovered by applying the inverse link: $\mu_i = g^{-1}(\eta_i)$. This holds regardless of which family or link you choose — the inverse link directly gives you $E[Y_i \mid x_i]$. Here, the log link gives $\log(\mu_i) = 0.7$, so the fitted conditional mean is simply $\mu_i = \exp(0.7)$. That confirms C as correct. A confuses the canonical link with the actual link being used. The canonical link for a Gaussian GLM is the identity link (not the log link), under which $\mu_i = \eta_i = 0.7$ would be correct. But because the problem explicitly specifies a log link, this reasoning doesn't apply here. B applies a lognormal retransformation correction (the Duan smearing or lognormal bias correction $\exp(\eta + \sigma^2/2)$). This correction is needed when you fit a linear model on log(Y) — i.e., when the response itself is logged and you want to back-transform to the original scale. In a GLM, you are modeling $E[Y]$ directly through the link; no such correction is needed. D is a fabricated formula with no statistical basis — it conflates variance with a penalty on the mean in a way that appears in no standard result. Study tip: Always remember the GLM mantra — the inverse link gives $E[Y \mid x]$ directly. Retransformation corrections only arise outside the GLM framework, when you model a transformed response as the dependent variable in a classical regression.
3. If category $2$ is instead chosen as the baseline, what are the two new canonical linear predictors for categories $1$ and $3$?
   Answer: $\log(p_1/p_2)=\eta_1-\eta_2$ and $\log(p_3/p_2)=-\eta_2$ When working with baseline-category logit models, the key insight is that log-odds are transitive: any log-odds ratio can be derived by arithmetic combinations of the original ones. This is the core skill being tested here. Starting from the original specification with category 3 as baseline, you have $\log(p_1/p_3) = \eta_1$ and $\log(p_2/p_3) = \eta_2$. To re-baseline on category 2, you need $\log(p_1/p_2)$ and $\log(p_3/p_2)$. Use the log-odds decomposition: $\log(p_1/p_2) = \log(p_1/p_3) - \log(p_2/p_3) = \eta_1 - \eta_2$ $\log(p_3/p_2) = -\log(p_2/p_3) = -\eta_2$ This confirms answer D as correct. Answer A incorrectly keeps $\eta_1$ unchanged for $\log(p_1/p_2)$, which ignores that switching baselines requires subtracting the new baseline's log-odds. It also assigns $\eta_1 - \eta_2$ to $\log(p_3/p_2)$, confusing which ratio is being inverted. Answer B adds rather than subtracts $\eta_2$, reflecting a sign error when transitioning baselines — a common algebraic trap when students forget that removing category 2 from the denominator means subtracting $\eta_2$. Answer C negates the wrong term, producing $\eta_2 - \eta_1$ instead of $\eta_1 - \eta_2$ — effectively reversing the direction of the first log-odds ratio. A reliable strategy: always write the target log-odds as a difference of two known log-odds, both relative to the original baseline. This telescoping property holds for any reparameterization, and mastering it will serve you across multinomial, conditional logit, and IIA-related questions.
4. An exponential-dispersion family has variance function $V(\mu)=\mu^3$. Choosing the integration constant to be zero, which function of $\mu$ is its canonical parameter?
   Answer: $\theta(\mu)=-\frac{1}{2\mu^2}$ When working with exponential-dispersion families, the canonical parameter $\theta(\mu)$ is recovered from the variance function through a two-step integration process. The key relationship is: $\frac{d\theta}{d\mu} = \frac{1}{V(\mu)}$ This comes directly from the structure of the natural exponential family, where $V(\mu) = b''(\theta)$ and $\mu = b'(\theta)$, so the chain rule gives $\frac{d\theta}{d\mu} = [b''(\theta)]^{-1} = \frac{1}{V(\mu)}$. With $V(\mu) = \mu^3$, you integrate: $\theta(\mu) = \int \frac{1}{\mu^3}\, d\mu = \int \mu^{-3}\, d\mu = \frac{\mu^{-2}}{-2} + C = -\frac{1}{2\mu^2} + C$ Setting the integration constant $C = 0$ gives $\theta(\mu) = -\frac{1}{2\mu^2}$, which is answer A — the correct choice. Answer B, $\frac{1}{2\mu^2}$, is the most tempting distractor: it has the right magnitude but drops the negative sign. A simple sign error in the power rule (forgetting that integrating $\mu^{-3}$ produces a negative coefficient) leads you here. Answer C, $-\frac{1}{\mu}$, would correspond to $V(\mu) = \mu^2$ — the inverse Gaussian family's variance function — not $\mu^3$. This is a common family to confuse with the inverse Gaussian. Answer D, $\log(\mu)$, arises from $V(\mu) = \mu$, which is the Poisson variance function, not $\mu^3$. Study tip: Memorize the big three variance functions and their canonical links — Poisson ($\mu$, log link), gamma ($\mu^2$, reciprocal link), and inverse Gaussian ($\mu^3$, $-1/2\mu^2$ link). Exam questions often test whether you can move fluently between $V(\mu)$ and $\theta(\mu)$ via integration.
5. What is this observation's contribution to the score for $\beta$ at the current parameter value?
   Answer: $4\varphi(0)\approx 1.596$ When working with probit GLMs, the key is carefully deriving the score function using the chain rule through the GLM likelihood. The log-likelihood contribution for a Bernoulli observation is $\ell_i = y_i \log \Phi(\eta_i) + (1-y_i)\log(1-\Phi(\eta_i))$. The score with respect to $\beta$ is $\frac{\partial \ell_i}{\partial \beta} = \frac{\partial \ell_i}{\partial \eta_i} \cdot x_i$. For $y_i = 1$, the derivative with respect to $\eta_i$ is $\frac{\varphi(\eta_i)}{\Phi(\eta_i)}$. Evaluated at $\eta_i = 0$: $\Phi(0) = 0.5$, so this equals $\frac{\varphi(0)}{0.5} = 2\varphi(0)$. Multiplying by $x_i = 2$ gives the full score contribution $4\varphi(0) \approx 4(0.3989) \approx 1.596$, confirming A. Choice B, $2\varphi(0)$, forgets to multiply by $x_i = 2$. This is a classic mistake — students compute $\partial \ell_i / \partial \eta_i$ correctly but confuse it with $\partial \ell_i / \partial \beta$, neglecting the design vector. Choice C, $2(1-\Phi(0)) = 1$, would apply if you incorrectly used the residual $(y_i - \mu_i)$ without the probit adjustment factor $\varphi/[\Phi(1-\Phi)]$ — this resembles the logistic score structure, not the probit. Choice D, $\varphi(0)$, ignores both the division by $\Phi(0)$ and the multiplication by $x_i$, leaving only the raw density evaluated at zero. A useful habit: always write the score as a product of three pieces — $\frac{\partial \ell}{\partial \mu} \cdot \frac{d\mu}{d\eta} \cdot x_i$ — and track each factor separately. Probit and logit differ in that middle term, so keeping them explicit prevents link-function mix-ups under exam pressure.
6. Relative to an otherwise identical observation, by what factor does the fitted expected count change if exposure is tripled and the covariate increases by $2$ units?
   Answer: The expected count is multiplied by $3(1.5)^2=6.75$. When working with Poisson GLMs that include an offset, you need to think carefully about how each component of the linear predictor contributes multiplicatively to the fitted mean. The model specifies $\mu_i = t_i \cdot e^{\beta_0 + \beta_1 x_i}$, so the expected count scales linearly with exposure and exponentially with the covariate. To find the multiplicative change, form the ratio of the new fitted mean to the original. If exposure triples ($t \to 3t$) and the covariate increases by 2 ($x \to x + 2$), the new mean is $\mu^* = 3t_i \cdot e^{\beta_0 + \beta_1(x_i+2)}$. Dividing by the original $\mu_i = t_i \cdot e^{\beta_0 + \beta_1 x_i}$ gives: $\frac{\mu^*}{\mu_i} = 3 \cdot e^{2\beta_1} = 3 \cdot (1.5)^2 = 3 \cdot 2.25 = 6.75$ This confirms A is correct. Choice B, $3(1.5) = 4.50$, uses only one power of $e^{\beta_1}$ instead of two — it treats a 2-unit increase as a 1-unit increase, missing the second factor of 1.5. Choice C, $1.5^2 = 2.25$, correctly handles the covariate change but ignores the tripled exposure entirely, forgetting that the offset enters multiplicatively just like any other factor. Choice D, $3 + 2(1.5) = 6.00$, confuses a multiplicative model with an additive one — you never add the exposure and covariate effects in a log-linear model; you multiply them. The key habit to build: always exponentiate the entire change in the linear predictor (including the offset) and multiply the resulting factors together. Additive changes on the log scale become multiplicative on the mean scale.
7. A gamma response is written in exponential-dispersion form as $f(y;\theta,\phi)=\exp\left\{\frac{y\theta-b(\theta)}{\phi}+c(y,\phi)\right\}$, with conditional mean $\mu>0$ and variance $\phi\mu^2$. Which statement correctly identifies the canonical parameter and an important consequence of using the canonical link?
   Answer: $\theta=-1/\mu$, and the canonical linear predictor must remain negative to give a positive mean. When working with generalized linear models in exponential-dispersion form, your first task is always to identify the canonical parameter by matching the distribution's natural sufficient statistic to the $\frac{y\theta}{\phi}$ term in the exponent. For the Gamma distribution, the log-likelihood structure reveals that the natural sufficient statistic is $y$ itself, and the canonical parameter is $\theta = -1/\mu$. You can verify this by recalling that for a Gamma with mean $\mu$, the standard exponential family form requires $b(\theta) = -\log(-\theta)$, which satisfies $b'(\theta) = \mu$, confirming $\theta = -1/\mu$. Notice that $\mu > 0$ forces $\theta < 0$ always. This is the critical consequence: the canonical link function is $g(\mu) = -1/\mu$, so the linear predictor $\eta = \mathbf{x}^T\boldsymbol{\beta}$ must satisfy $\eta < 0$ to ensure $\mu = -1/\eta > 0$. This makes B correct. Choice A gets the sign wrong. $\theta = 1/\mu$ would allow positive $\theta$, which is inconsistent with the Gamma's parameter space, and claiming an unrestricted predictor gives positive means ignores the constraint entirely. Choice C describes the log link ($\theta = \log\mu$), which is actually the more commonly used link for Gamma regression precisely because it allows an unrestricted linear predictor — but it is not the canonical link. Choice D inverts the log relationship and misidentifies the constraint direction. A useful study pattern: canonical links for common GLMs often introduce parameter-space constraints (Gamma and inverse Gaussian both have negative canonical parameters), which is exactly why practitioners frequently prefer non-canonical links like log for Gamma models despite losing the mathematical elegance of canonicality.
8. Which assessment of this proposal is most accurate?
   Answer: It is not generally a Bernoulli likelihood; overdispersion requires a quasi-likelihood or an alternative probability model. When you see a question about dispersion parameters in generalized linear models, ask yourself: does the proposed modification still correspond to a valid probability distribution? The Bernoulli distribution has no free dispersion parameter. In the exponential family representation, the variance function for Bernoulli outcomes is $V(\mu) = \mu(1-\mu)$, and the dispersion is fixed at $\phi = 1$ by the structure of the distribution itself. This isn't a modeling choice — it's a mathematical fact. If you allow $\phi$ to float freely, you are no longer writing down a Bernoulli likelihood at all. Binary data are either 0 or 1; there is no mechanism by which a single trial can exhibit "extra" variance beyond $\mu(1-\mu)$. This is why B is correct: introducing an unrestricted $\phi$ abandons the Bernoulli framework, and handling overdispersion properly requires either a quasi-likelihood approach (which gives valid estimating equations without claiming a full likelihood) or a genuine alternative model like beta-binomial regression. A is wrong because replication of covariate patterns doesn't rescue the proposal — even with replication you can compute a Pearson dispersion estimate, but that doesn't make $\phi \neq 1$ consistent with Bernoulli probability. C is wrong for a similar reason: restricting $\phi \geq 1$ doesn't restore a valid Bernoulli likelihood; it just further confirms you've left that distributional family. D is wrong because a random intercept model induces a specific correlation structure (beta-binomial marginal), which is not equivalent to simply freeing $\phi$ in the likelihood. As a study strategy, remember: for binary (0/1) outcomes, overdispersion cannot exist at the individual trial level — it only appears when modeling counts of successes across grouped trials. That distinction is frequently tested.
9. A covariate change increases the linear predictor by $\log 2$. What is the new fitted event probability?
   Answer: $1-(1-0.20)^2=0.36$ When working with the complementary log-log (cloglog) link, your first instinct should be to translate everything back through the link function rather than guessing at a transformation pattern. The cloglog link is $\eta = \log[-\log(1-p)]$, so the inverse is $p = 1 - e^{-e^\eta}$. If the original probability is $p_0 = 0.20$, then $1 - p_0 = 0.80 = e^{-e^{\eta_0}}$, meaning $e^{-e^{\eta_0}} = 0.80$. Now increase the linear predictor by $\log 2$: the new predictor is $\eta_1 = \eta_0 + \log 2$, so $e^{\eta_1} = 2e^{\eta_0}$. The new survival probability is $e^{-e^{\eta_1}} = e^{-2e^{\eta_0}} = (e^{-e^{\eta_0}})^2 = (0.80)^2 = 0.64$. Therefore the new event probability is $1 - 0.64 = 1 - (0.80)^2 = 1-(1-0.20)^2 = \mathbf{0.36}$, confirming A is correct. Choice B resembles an odds-based manipulation — dividing by $1 + p$ rather than working through the cloglog inverse — which has no grounding in this link function. Choice C looks like it doubles the odds ($p/(1-p)$) then converts back, confusing the logit link's multiplicative-odds property with the cloglog. Choice D applies a square-root to $(1-p)$ instead of squaring it, essentially reversing the exponent direction — the kind of sign error that arises from misremembering the inverse. A reliable strategy: for any GLM link question, write out $\mu = g^{-1}(\eta)$ explicitly, substitute the new $\eta$, and simplify algebraically. Never pattern-match to logit rules when the link is cloglog.
10. In Fisher scoring or IRLS, how do the working weights depend on the fitted means?
    Answer: They are constant in $\mu_i$ because the derivative and variance factors cancel. When you encounter IRLS/Fisher scoring questions in GLMs, the key is to carefully track how the working weight formula combines the variance function and the link derivative — they don't always reinforce each other. The working weight for observation $i$ is defined as $w_i = \frac{1}{V(\mu_i)\left(\frac{d\eta_i}{d\mu_i}\right)^2}$. For the gamma GLM, the variance function is $V(\mu_i) = \mu_i^2$ (since $\operatorname{Var}(Y_i \mid x_i) = \phi\mu_i^2$ and the dispersion $\phi$ is absorbed separately). With the log link $\eta_i = \log(\mu_i)$, the derivative is $\frac{d\eta_i}{d\mu_i} = \frac{1}{\mu_i}$, so $\left(\frac{d\eta_i}{d\mu_i}\right)^2 = \frac{1}{\mu_i^2}$. Plugging in: $w_i = \frac{1}{\mu_i^2 \cdot \frac{1}{\mu_i^2}} = \frac{1}{1} = 1$. The two $\mu_i^2$ factors cancel exactly, making the weights constant — confirming answer C. Answer A is wrong because it confuses the variance function alone with the full weight formula, ignoring the link derivative contribution. Answer B commits the opposite error: it correctly inverts something but applies only the variance function's reciprocal without recognizing that the link derivative also contributes a $\mu_i^{-2}$ term that cancels it. Answer D describes a partial cancellation that would arise from mismatching powers — for example, mixing the gamma variance with a different link — but doesn't apply here. A useful pattern to memorize: the canonical link for the gamma is the reciprocal link $\eta = 1/\mu$, and the log link is actually a non-canonical choice. Whenever the log link pairs with a variance function of $V(\mu) \propto \mu^2$, the weights collapse to constants — this specific combination appears frequently on exam problems precisely because the cancellation surprises students.

Start this quiz