Statistics Graduate Level Quiz: Exponential Family And Link Functions

10 questions · exam conditions

0:00

Exponential Family And Link FunctionsQuestion 1 of 10

An exponential-dispersion family has variance function $V(\mu)=\mu^3$ . Choosing the integration constant to be zero, which function of $\mu$ is its canonical parameter?

\theta(\mu)=-\frac{1}{2\mu^2}

\theta(\mu)=\frac{1}{2\mu^2}

\theta(\mu)=-\frac{1}{\mu}

\theta(\mu)=\log(\mu)

← Back to quizzes

Statistics Graduate Level Quiz

Statistics Graduate Level Quiz: Exponential Family And Link Functions

Practice Exponential Family And Link Functions in Statistics Graduate Level with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Exponential Family And Link Functions, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics Graduate Level.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

An exponential-dispersion family has variance function $V(\mu)=\mu^3$ . Choosing the integration constant to be zero, which function of $\mu$ is its canonical parameter?

$\theta(\mu)=-\frac{1}{2\mu^2}$ (correct answer)
$\theta(\mu)=\frac{1}{2\mu^2}$
$\theta(\mu)=-\frac{1}{\mu}$
$\theta(\mu)=\log(\mu)$

Explanation: When working with exponential-dispersion families, the canonical parameter

\theta(\mu)

is recovered from the variance function through a two-step integration process. The key relationship is:

\frac{d\theta}{d\mu} = \frac{1}{V(\mu)}

This comes directly from the structure of the natural exponential family, where

V(\mu) = b''(\theta)

and

\mu = b'(\theta)

, so the chain rule gives

\frac{d\theta}{d\mu} = [b''(\theta)]^{-1} = \frac{1}{V(\mu)}

. With

V(\mu) = \mu^3

, you integrate:

\theta(\mu) = \int \frac{1}{\mu^3}\, d\mu = \int \mu^{-3}\, d\mu = \frac{\mu^{-2}}{-2} + C = -\frac{1}{2\mu^2} + C

Setting the integration constant

C = 0

gives

\theta(\mu) = -\frac{1}{2\mu^2}

, which is answer A — the correct choice. Answer B,

\frac{1}{2\mu^2}

, is the most tempting distractor: it has the right magnitude but drops the negative sign. A simple sign error in the power rule (forgetting that integrating

\mu^{-3}

produces a negative coefficient) leads you here. Answer C,

-\frac{1}{\mu}

, would correspond to

V(\mu) = \mu^2

— the inverse Gaussian family's variance function — not

\mu^3

. This is a common family to confuse with the inverse Gaussian. Answer D,

\log(\mu)

, arises from

V(\mu) = \mu

, which is the Poisson variance function, not

\mu^3

. Study tip: Memorize the big three variance functions and their canonical links — Poisson (

\mu

, log link), gamma (

\mu^2

, reciprocal link), and inverse Gaussian (

\mu^3

-1/2\mu^2

link). Exam questions often test whether you can move fluently between

V(\mu)

and

\theta(\mu)

via integration.

Question 2

A gamma response is written in exponential-dispersion form as $f(y;\theta,\phi)=\exp\left\{\frac{y\theta-b(\theta)}{\phi}+c(y,\phi)\right\}$ , with conditional mean $\mu>0$ and variance $\phi\mu^2$ . Which statement correctly identifies the canonical parameter and an important consequence of using the canonical link?

$\theta=1/\mu$ , and an unrestricted linear predictor automatically gives a positive mean.
$\theta=-1/\mu$ , and the canonical linear predictor must remain negative to give a positive mean. (correct answer)
$\theta=\log(\mu)$ , and the canonical linear predictor may range over the entire real line.
$\theta=-\log(\mu)$ , and the canonical linear predictor must remain positive to give a positive mean.

Explanation: When working with generalized linear models in exponential-dispersion form, your first task is always to identify the canonical parameter by matching the distribution's natural sufficient statistic to the

\frac{y\theta}{\phi}

term in the exponent. For the Gamma distribution, the log-likelihood structure reveals that the natural sufficient statistic is

y

itself, and the canonical parameter is

\theta = -1/\mu

. You can verify this by recalling that for a Gamma with mean

\mu

, the standard exponential family form requires

b(\theta) = -\log(-\theta)

, which satisfies

b'(\theta) = \mu

, confirming

\theta = -1/\mu

. Notice that

\mu > 0

forces

\theta < 0

always. This is the critical consequence: the canonical link function is

g(\mu) = -1/\mu

, so the linear predictor

\eta = \mathbf{x}^T\boldsymbol{\beta}

must satisfy

\eta < 0

to ensure

\mu = -1/\eta > 0

. This makes B correct. Choice A gets the sign wrong.

\theta = 1/\mu

would allow positive

\theta

, which is inconsistent with the Gamma's parameter space, and claiming an unrestricted predictor gives positive means ignores the constraint entirely. Choice C describes the log link (

\theta = \log\mu

), which is actually the more commonly used link for Gamma regression precisely because it allows an unrestricted linear predictor — but it is not the canonical link. Choice D inverts the log relationship and misidentifies the constraint direction. A useful study pattern: canonical links for common GLMs often introduce parameter-space constraints (Gamma and inverse Gaussian both have negative canonical parameters), which is exactly why practitioners frequently prefer non-canonical links like log for Gamma models despite losing the mathematical elegance of canonicality.

Question 3

A gamma GLM has variance $\operatorname{Var}(Y_i\mid x_i)=\phi\mu_i^2$ and uses the log link $\eta_i=\log(\mu_i)$ . Assume equal prior weights for all observations.

In Fisher scoring or IRLS, how do the working weights depend on the fitted means?

They are proportional to $\mu_i^2$ , so larger fitted means receive greater weight.
They are proportional to $1/\mu_i^2$ , so smaller fitted means receive greater weight.
They are constant in $\mu_i$ because the derivative and variance factors cancel. (correct answer)
They are proportional to $1/\mu_i$ because only one power of the mean cancels.

Explanation: When you encounter IRLS/Fisher scoring questions in GLMs, the key is to carefully track how the working weight formula combines the variance function and the link derivative — they don't always reinforce each other. The working weight for observation

i

is defined as

w_i = \frac{1}{V(\mu_i)\left(\frac{d\eta_i}{d\mu_i}\right)^2}

. For the gamma GLM, the variance function is

V(\mu_i) = \mu_i^2

(since

\operatorname{Var}(Y_i \mid x_i) = \phi\mu_i^2

and the dispersion

\phi

is absorbed separately). With the log link

\eta_i = \log(\mu_i)

, the derivative is

\frac{d\eta_i}{d\mu_i} = \frac{1}{\mu_i}

, so

\left(\frac{d\eta_i}{d\mu_i}\right)^2 = \frac{1}{\mu_i^2}

. Plugging in:

w_i = \frac{1}{\mu_i^2 \cdot \frac{1}{\mu_i^2}} = \frac{1}{1} = 1

. The two

\mu_i^2

factors cancel exactly, making the weights constant — confirming answer C. Answer A is wrong because it confuses the variance function alone with the full weight formula, ignoring the link derivative contribution. Answer B commits the opposite error: it correctly inverts something but applies only the variance function's reciprocal without recognizing that the link derivative also contributes a

\mu_i^{-2}

term that cancels it. Answer D describes a partial cancellation that would arise from mismatching powers — for example, mixing the gamma variance with a different link — but doesn't apply here. A useful pattern to memorize: the canonical link for the gamma is the reciprocal link

\eta = 1/\mu

, and the log link is actually a non-canonical choice. Whenever the log link pairs with a variance function of

V(\mu) \propto \mu^2

, the weights collapse to constants — this specific combination appears frequently on exam problems precisely because the cancellation surprises students.

Question 4

A Gaussian GLM specifies $Y_i\mid x_i\sim N(\mu_i,\sigma^2)$ and uses the noncanonical log link $\log(\mu_i)=x_i^{\mathsf T}\beta$ . At a particular covariate value, the fitted linear predictor is $0.7$ and the estimated variance is $0.4$ .

What is the fitted conditional mean of the response at this covariate value?

$0.7$ , because a Gaussian family's canonical mean is its linear predictor.
$\exp(0.7+0.4/2)$ , because a lognormal retransformation correction is required.
$\exp(0.7)$ , because the inverse link directly models the conditional mean. (correct answer)
$\exp(0.7)-0.4/2$ , because Gaussian variation reduces the inverse-link prediction.

Explanation: Whenever you see a GLM question specifying a non-canonical link, your first instinct should be to carefully distinguish between the linear predictor

\eta_i

, the link function

g

, and the conditional mean

\mu_i

. The GLM framework always defines

g(\mu_i) = \eta_i

, which means the fitted conditional mean is recovered by applying the inverse link:

\mu_i = g^{-1}(\eta_i)

. This holds regardless of which family or link you choose — the inverse link directly gives you

E[Y_i \mid x_i]

. Here, the log link gives

\log(\mu_i) = 0.7

, so the fitted conditional mean is simply

\mu_i = \exp(0.7)

. That confirms C as correct. A confuses the canonical link with the actual link being used. The canonical link for a Gaussian GLM is the identity link (not the log link), under which

\mu_i = \eta_i = 0.7

would be correct. But because the problem explicitly specifies a log link, this reasoning doesn't apply here. B applies a lognormal retransformation correction (the Duan smearing or lognormal bias correction

\exp(\eta + \sigma^2/2)

). This correction is needed when you fit a linear model on log(Y) — i.e., when the response itself is logged and you want to back-transform to the original scale. In a GLM, you are modeling

E[Y]

directly through the link; no such correction is needed. D is a fabricated formula with no statistical basis — it conflates variance with a penalty on the mean in a way that appears in no standard result. Study tip: Always remember the GLM mantra — the inverse link gives

E[Y \mid x]

directly. Retransformation corrections only arise outside the GLM framework, when you model a transformed response as the dependent variable in a classical regression.

Question 5

Independent counts satisfy $Y_i\sim\operatorname{Poisson}(\mu_i)$ with $\log(\mu_i)=\log(t_i)+\alpha+x_i^{\mathsf T}\beta$ , where $t_i$ is a known exposure. Let $N=\sum_iY_i$ .

Conditional on the observed total $N$ , which description of the joint distribution of the counts is correct?

It becomes a product of independent Bernoulli distributions with logistic probabilities involving $\alpha+x_i^{\mathsf T}\beta$ .
It is multinomial with probabilities proportional to $\exp(x_i^{\mathsf T}\beta)$ , and both $t_i$ and $\alpha$ cancel.
It remains a product of independent Poisson distributions with means proportional to $t_i\exp(\alpha+x_i^{\mathsf T}\beta)$ .
It is multinomial with probabilities proportional to $t_i\exp(x_i^{\mathsf T}\beta)$ , and the intercept $\alpha$ cancels. (correct answer)

Explanation: Whenever you see Poisson regression conditioned on the total count, think about the classic result that ties Poisson models to the multinomial distribution. This connection is what the question tests. Start from the joint probability of

(Y_1, \ldots, Y_n)

given

N = \sum_i Y_i

. Because the

Y_i

are independent Poisson, the conditional distribution is multinomial with cell probabilities

\pi_i = \mu_i / \sum_j \mu_j

. Now substitute

\mu_i = t_i \exp(\alpha + x_i^\mathsf{T}\beta)

. In the ratio,

\exp(\alpha)

appears in every numerator and denominator, so it cancels completely. What remains is

\pi_i = \frac{t_i \exp(x_i^\mathsf{T}\beta)}{\sum_j t_j \exp(x_j^\mathsf{T}\beta)}.

The exposure

t_i

survives because it can differ across units, while

\alpha

(a common multiplicative factor) drops out. This confirms D as correct. Choice A is wrong because conditioning on

N

yields a multinomial, not a product of Bernoullis — those arise in binary logistic regression, a different model entirely. Choice B is tempting but incorrect: it drops

t_i

from the probabilities, which is only valid if all exposures are equal. In general,

t_i

is unit-specific and cannot be absorbed into the intercept, so it must remain. Choice C is wrong because conditioning on

N

destroys independence among the

Y_i

; the counts are no longer marginally Poisson once their sum is fixed. A useful study tip: whenever you condition Poisson counts on their sum, immediately write out the multinomial cell probability as a ratio of means — cancellations (like the intercept here) become obvious, and you can see exactly which terms survive.

Question 6

For a response with three unordered categories, a baseline-category multinomial logit model uses category $3$ as the baseline and specifies $\log(p_1/p_3)=\eta_1$ and $\log(p_2/p_3)=\eta_2$ .

If category $2$ is instead chosen as the baseline, what are the two new canonical linear predictors for categories $1$ and $3$ ?

$\log(p_1/p_2)=\eta_1$ and $\log(p_3/p_2)=\eta_1-\eta_2$
$\log(p_1/p_2)=\eta_1+\eta_2$ and $\log(p_3/p_2)=\eta_2$
$\log(p_1/p_2)=\eta_2-\eta_1$ and $\log(p_3/p_2)=-\eta_1$
$\log(p_1/p_2)=\eta_1-\eta_2$ and $\log(p_3/p_2)=-\eta_2$ (correct answer)

Explanation: When working with baseline-category logit models, the key insight is that log-odds are transitive: any log-odds ratio can be derived by arithmetic combinations of the original ones. This is the core skill being tested here. Starting from the original specification with category 3 as baseline, you have

\log(p_1/p_3) = \eta_1

and

\log(p_2/p_3) = \eta_2

. To re-baseline on category 2, you need

\log(p_1/p_2)

and

\log(p_3/p_2)

. Use the log-odds decomposition:

\log(p_1/p_2) = \log(p_1/p_3) - \log(p_2/p_3) = \eta_1 - \eta_2

\log(p_3/p_2) = -\log(p_2/p_3) = -\eta_2

This confirms answer D as correct. Answer A incorrectly keeps

\eta_1

unchanged for

\log(p_1/p_2)

, which ignores that switching baselines requires subtracting the new baseline's log-odds. It also assigns

\eta_1 - \eta_2

\log(p_3/p_2)

, confusing which ratio is being inverted. Answer B adds rather than subtracts

\eta_2

, reflecting a sign error when transitioning baselines — a common algebraic trap when students forget that removing category 2 from the denominator means subtracting

\eta_2

. Answer C negates the wrong term, producing

\eta_2 - \eta_1

instead of

\eta_1 - \eta_2

— effectively reversing the direction of the first log-odds ratio. A reliable strategy: always write the target log-odds as a difference of two known log-odds, both relative to the original baseline. This telescoping property holds for any reparameterization, and mastering it will serve you across multinomial, conditional logit, and IIA-related questions.

Question 7

An analyst fits a logistic regression to individual Bernoulli outcomes but proposes replacing the fixed Bernoulli dispersion by an unrestricted parameter $\phi$ in the exponential-family likelihood. The analyst plans to estimate $\phi$ jointly with the regression coefficients.

Which assessment of this proposal is most accurate?

It gives a valid Bernoulli likelihood whenever replicated covariate patterns are available to estimate $\phi$ .
It is not generally a Bernoulli likelihood; overdispersion requires a quasi-likelihood or an alternative probability model. (correct answer)
It gives a valid Bernoulli likelihood provided the fitted dispersion satisfies the restriction $\phi\geq1$ .
It is exactly equivalent to introducing a normally distributed random intercept into the logistic regression.

Explanation: When you see a question about dispersion parameters in generalized linear models, ask yourself: does the proposed modification still correspond to a valid probability distribution? The Bernoulli distribution has no free dispersion parameter. In the exponential family representation, the variance function for Bernoulli outcomes is

V(\mu) = \mu(1-\mu)

, and the dispersion is fixed at

\phi = 1

by the structure of the distribution itself. This isn't a modeling choice — it's a mathematical fact. If you allow

\phi

to float freely, you are no longer writing down a Bernoulli likelihood at all. Binary data are either 0 or 1; there is no mechanism by which a single trial can exhibit "extra" variance beyond

\mu(1-\mu)

. This is why B is correct: introducing an unrestricted

\phi

abandons the Bernoulli framework, and handling overdispersion properly requires either a quasi-likelihood approach (which gives valid estimating equations without claiming a full likelihood) or a genuine alternative model like beta-binomial regression. A is wrong because replication of covariate patterns doesn't rescue the proposal — even with replication you can compute a Pearson dispersion estimate, but that doesn't make

\phi \neq 1

consistent with Bernoulli probability. C is wrong for a similar reason: restricting

\phi \geq 1

doesn't restore a valid Bernoulli likelihood; it just further confirms you've left that distributional family. D is wrong because a random intercept model induces a specific correlation structure (beta-binomial marginal), which is not equivalent to simply freeing

\phi

in the likelihood. As a study strategy, remember: for binary (0/1) outcomes, overdispersion cannot exist at the individual trial level — it only appears when modeling counts of successes across grouped trials. That distinction is frequently tested.

Question 8

Counts are modeled by a Poisson GLM with exposure $t_i$ and mean specification $\log(\mu_i)=\log(t_i)+\beta_0+\beta_1x_i$ . The fitted slope satisfies $\exp(\beta_1)=1.5$ .

Relative to an otherwise identical observation, by what factor does the fitted expected count change if exposure is tripled and the covariate increases by $2$ units?

The expected count is multiplied by $3(1.5)^2=6.75$ . (correct answer)
The expected count is multiplied by $3(1.5)=4.50$ .
The expected count is multiplied by $1.5^2=2.25$ .
The expected count is multiplied by $3+2(1.5)=6.00$ .

Explanation: When working with Poisson GLMs that include an offset, you need to think carefully about how each component of the linear predictor contributes multiplicatively to the fitted mean. The model specifies

\mu_i = t_i \cdot e^{\beta_0 + \beta_1 x_i}

, so the expected count scales linearly with exposure and exponentially with the covariate. To find the multiplicative change, form the ratio of the new fitted mean to the original. If exposure triples (

t \to 3t

) and the covariate increases by 2 (

x \to x + 2

), the new mean is

\mu^* = 3t_i \cdot e^{\beta_0 + \beta_1(x_i+2)}

. Dividing by the original

\mu_i = t_i \cdot e^{\beta_0 + \beta_1 x_i}

gives:

\frac{\mu^*}{\mu_i} = 3 \cdot e^{2\beta_1} = 3 \cdot (1.5)^2 = 3 \cdot 2.25 = 6.75

This confirms A is correct. Choice B,

3(1.5) = 4.50

, uses only one power of

e^{\beta_1}

instead of two — it treats a 2-unit increase as a 1-unit increase, missing the second factor of 1.5. Choice C,

1.5^2 = 2.25

, correctly handles the covariate change but ignores the tripled exposure entirely, forgetting that the offset enters multiplicatively just like any other factor. Choice D,

3 + 2(1.5) = 6.00

, confuses a multiplicative model with an additive one — you never add the exposure and covariate effects in a log-linear model; you multiply them. The key habit to build: always exponentiate the entire change in the linear predictor (including the offset) and multiply the resulting factors together. Additive changes on the log scale become multiplicative on the mean scale.

Question 9

A binary-response GLM uses the complementary log-log link $\log[-\log(1-p)]=\eta$ . For a subject with linear predictor $\eta$ , the fitted event probability is $p=0.20$ .

A covariate change increases the linear predictor by $\log 2$ . What is the new fitted event probability?

$1-(1-0.20)^2=0.36$ (correct answer)
$\frac{2(0.20)}{1+0.20}=\frac{1}{3}$
$\frac{2(0.20)}{1-0.20}=0.50$
$1-\sqrt{1-0.20}\approx0.106$

Explanation: When working with the complementary log-log (cloglog) link, your first instinct should be to translate everything back through the link function rather than guessing at a transformation pattern. The cloglog link is

\eta = \log[-\log(1-p)]

, so the inverse is

p = 1 - e^{-e^\eta}

. If the original probability is

p_0 = 0.20

, then

1 - p_0 = 0.80 = e^{-e^{\eta_0}}

, meaning

e^{-e^{\eta_0}} = 0.80

. Now increase the linear predictor by

\log 2

: the new predictor is

\eta_1 = \eta_0 + \log 2

, so

e^{\eta_1} = 2e^{\eta_0}

. The new survival probability is

e^{-e^{\eta_1}} = e^{-2e^{\eta_0}} = (e^{-e^{\eta_0}})^2 = (0.80)^2 = 0.64

. Therefore the new event probability is

1 - 0.64 = 1 - (0.80)^2 = 1-(1-0.20)^2 = \mathbf{0.36}

, confirming A is correct. Choice B resembles an odds-based manipulation — dividing by

1 + p

rather than working through the cloglog inverse — which has no grounding in this link function. Choice C looks like it doubles the odds (

p/(1-p)

) then converts back, confusing the logit link's multiplicative-odds property with the cloglog. Choice D applies a square-root to

(1-p)

instead of squaring it, essentially reversing the exponent direction — the kind of sign error that arises from misremembering the inverse. A reliable strategy: for any GLM link question, write out

\mu = g^{-1}(\eta)

explicitly, substitute the new

\eta

, and simplify algebraically. Never pattern-match to logit rules when the link is cloglog.

Question 10

A Bernoulli GLM uses the probit link $\Phi^{-1}(\mu_i)=\eta_i=x_i\beta$ , where $\Phi$ and $\varphi$ are the standard normal distribution and density functions. For one observation, $x_i=2$ , $y_i=1$ , and the current linear predictor is $\eta_i=0$ .

What is this observation's contribution to the score for $\beta$ at the current parameter value?

$4\varphi(0)\approx 1.596$ (correct answer)
$2\varphi(0)\approx 0.798$
$2(1-\Phi(0))=1.000$
$\varphi(0)\approx 0.399$

Explanation: When working with probit GLMs, the key is carefully deriving the score function using the chain rule through the GLM likelihood. The log-likelihood contribution for a Bernoulli observation is

\ell_i = y_i \log \Phi(\eta_i) + (1-y_i)\log(1-\Phi(\eta_i))

. The score with respect to

\beta

\frac{\partial \ell_i}{\partial \beta} = \frac{\partial \ell_i}{\partial \eta_i} \cdot x_i

. For

y_i = 1

, the derivative with respect to

\eta_i

\frac{\varphi(\eta_i)}{\Phi(\eta_i)}

. Evaluated at

\eta_i = 0

\Phi(0) = 0.5

, so this equals

\frac{\varphi(0)}{0.5} = 2\varphi(0)

. Multiplying by

x_i = 2

gives the full score contribution

4\varphi(0) \approx 4(0.3989) \approx 1.596

, confirming A. Choice B,

2\varphi(0)

, forgets to multiply by

x_i = 2

. This is a classic mistake — students compute

\partial \ell_i / \partial \eta_i

correctly but confuse it with

\partial \ell_i / \partial \beta

, neglecting the design vector. Choice C,

2(1-\Phi(0)) = 1

, would apply if you incorrectly used the residual

(y_i - \mu_i)

without the probit adjustment factor

\varphi/[\Phi(1-\Phi)]

— this resembles the logistic score structure, not the probit. Choice D,

\varphi(0)

, ignores both the division by

\Phi(0)

and the multiplication by

x_i

, leaving only the raw density evaluated at zero. A useful habit: always write the score as a product of three pieces —

\frac{\partial \ell}{\partial \mu} \cdot \frac{d\mu}{d\eta} \cdot x_i

— and track each factor separately. Probit and logit differ in that middle term, so keeping them explicit prevents link-function mix-ups under exam pressure.