Statistics Graduate Level Quiz: Exponential Families

10 questions · exam conditions

0:00

Exponential FamiliesQuestion 1 of 10

A normal model has mean $\mu>0$ and variance constrained by $\sigma^2=\mu^2$ . Viewed as a subfamily of the two-parameter normal exponential family with natural parameters $\eta_1=\mu/\sigma^2$ and $\eta_2=-1/(2\sigma^2)$ , which statement is correct?

It is a full one-parameter family with

\eta_2=-\eta_1/2

and natural parameter space

\eta_1>0

It is a curved one-parameter family with

\eta_2=-\eta_1^2/2

and natural parameter space

\eta_1>0

It is a curved one-parameter family with

\eta_2=-1/(2\eta_1^2)

and natural parameter space

\eta_1<0

It is a full two-parameter family because both

\eta_1

and

\eta_2

vary as

\mu

varies.

← Back to quizzes

Statistics Graduate Level Quiz

Statistics Graduate Level Quiz: Exponential Families

Practice Exponential Families in Statistics Graduate Level with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Exponential Families, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics Graduate Level.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

It is a full one-parameter family with $\eta_2=-\eta_1/2$ and natural parameter space $\eta_1>0$ .
It is a curved one-parameter family with $\eta_2=-\eta_1^2/2$ and natural parameter space $\eta_1>0$ . (correct answer)
It is a curved one-parameter family with $\eta_2=-1/(2\eta_1^2)$ and natural parameter space $\eta_1<0$ .
It is a full two-parameter family because both $\eta_1$ and $\eta_2$ vary as $\mu$ varies.

Explanation: When a question asks you to embed a constrained subfamily into an exponential family, your first task is to express the natural parameters purely in terms of the free parameter, then classify the resulting curve in natural parameter space. Here, the two-parameter normal exponential family has natural parameters

\eta_1 = \mu/\sigma^2

and

\eta_2 = -1/(2\sigma^2)

. The constraint

\sigma^2 = \mu^2

(with

\mu > 0

) means

\sigma = \mu

. Substituting:

\eta_1 = \mu/\mu^2 = 1/\mu

and

\eta_2 = -1/(2\mu^2)

. Since

\eta_1 = 1/\mu

, we get

\mu = 1/\eta_1

, so

\eta_2 = -1/(2(1/\eta_1)^2) = -\eta_1^2/2

. Because

\mu > 0

, we have

\eta_1 > 0

. This relationship

\eta_2 = -\eta_1^2/2

traces a curved (nonlinear) path through the two-dimensional natural parameter space — the hallmark of a curved exponential family. This confirms B is correct. A is wrong because it claims

\eta_2 = -\eta_1/2

, a linear relationship, which would make this a full (not curved) subfamily. The algebra simply doesn't support linearity here. C has the right functional form

\eta_2 = -1/(2\eta_1^2)

, but this expression equals

-\mu^2/2

, not

-\eta_1^2/2

. It also incorrectly states

\eta_1 < 0

, when

\mu > 0

forces

\eta_1 > 0

. D is a conceptual error. Variation of both natural parameters with a single underlying parameter doesn't make a family "full" — fullness requires the natural parameter space to contain an open set in

\mathbb{R}^2

, which a one-dimensional curve never does. Remember: curved = nonlinear constraint in natural parameter space; full = open set in natural parameter space. Always substitute and simplify before classifying.

Question 2

For one observation from the uniform model $f(x\mid\theta)=\theta^{-1}I\{0<x<\theta\}$ with $\theta>0$ , a researcher proposes treating $-\log\theta$ as a natural parameter. Which assessment is most accurate?

The model is a regular one-parameter exponential family because its log density contains $-\log\theta$ .
The model is not a regular fixed-support exponential family because the support depends on $\theta$ . (correct answer)
The model is a curved exponential family because the indicator is a nonlinear canonical statistic.
The model becomes a regular exponential family after using the parameter $\eta=1/\theta$ .

Explanation: When classifying a statistical model as an exponential family, the most critical requirement is that the support of the distribution must not depend on the parameter. This is the fixed-carrier condition, and it's the first thing you should check before attempting any reparameterization. For the uniform model

f(x\mid\theta) = \theta^{-1}I\{0 < x < \theta\}

, the support

\{x : 0 < x < \theta\}

moves with

\theta

. This immediately disqualifies the model from being any form of regular exponential family, because the exponential family form

h(x)\exp\{\eta(\theta)T(x) - A(\theta)\}

requires

h(x)

to be free of

\theta

—but the indicator function

I\{0 < x < \theta\}

cannot be factored into a

\theta

-free piece times a

\theta

-only piece. That's why B is correct. Answer A is tempting because

\log f = -\log\theta

does appear in the log density, making it look like the log density has the right shape. But this reasoning ignores the indicator function entirely—the support problem persists regardless of how you write the density's main term. Answer D proposes the reparameterization

\eta = 1/\theta

. While valid reparameterizations can simplify exponential family representations, they cannot fix a varying-support problem. The support still depends on

\theta

, and no algebraic trick removes that dependence. Answer C misuses the term "curved exponential family." Curved exponential families are subfamilies of regular exponential families with constraints on the natural parameter space—they don't arise from indicator-function complications. Study tip: Before attempting any exponential family classification, always check the support first. If the support depends on

\theta

, the conversation ends there—no reparameterization can rescue it.

Question 3

Independent binary responses satisfy $P(Y_i=1)=\Phi(x_i^{\mathsf T}\beta)$ , where $\Phi$ is the standard normal distribution function. There are $n>p$ observations, and the design matrix has rank $p$ .

Viewed inside the full exponential family of independent Bernoulli distributions, how should this probit regression model generally be classified?

It is not an exponential family because the probit link is not the Bernoulli canonical link.
It is a full regular family with natural parameter vector $X\beta$ .
It is a curved family with coordinates $\eta_i=\logit\{\Phi(x_i^{\mathsf T}\beta)\}$ . (correct answer)
It is a full family because every component $\eta_i$ ranges over all real numbers.

Explanation: When working with exponential families, a crucial distinction is between the full family (where the natural parameter vector ranges freely over an open set with dimension equal to the sufficient statistic) and a curved exponential family (where the parameter of interest lives on a lower-dimensional submanifold). This question asks you to place probit regression within that taxonomy. The full Bernoulli exponential family has natural parameters

\eta_i = \log\frac{p_i}{1-p_i}

for each observation, living freely in

\mathbb{R}^n

. In probit regression, however, these

n

natural parameters are constrained:

\eta_i = \text{logit}\{\Phi(x_i^\mathsf{T}\beta)\}

, which traces out a

p

-dimensional curved submanifold inside

\mathbb{R}^n

\beta

varies. Because the model is parameterized by

\beta \in \mathbb{R}^p

with

p < n

, it sits on a curved, lower-dimensional surface within the full family — making C the correct classification. Answer A is wrong because membership in an exponential family doesn't require the canonical link. Probit regression is still a valid statistical model; the question is about its position within the full Bernoulli family, not whether it "qualifies" as exponential. Answer B fails because

X\beta

is not the natural parameter vector —

\text{logit}\{\Phi(x_i^\mathsf{T}\beta)\}

is, and the model does not fill the full

n

-dimensional natural parameter space. Answer D confuses the range of individual components with the dimension of the parameter manifold; even though each

\eta_i

can range over all reals, the

n

components are jointly constrained to a

p

-dimensional surface. A useful rule of thumb: whenever a model imposes

p < n

constraints on an

n

-dimensional natural parameter, think curved exponential family.

Question 4

An exponential-family representation uses the statistic vector $T(X)=(X,2X+1)^{\mathsf T}$ and natural parameter vector $\eta=(\eta_1,\eta_2)^{\mathsf T}$ .

Which statement correctly identifies the nonidentifiability in this representation?

Two parameter vectors are equivalent when their changes satisfy $\Delta\eta_1+2\Delta\eta_2=0$ . (correct answer)
Two parameter vectors are equivalent when their changes satisfy $2\Delta\eta_1+\Delta\eta_2=0$ .
The representation is identifiable because the two components of $T(X)$ are numerically different.
Only changes satisfying $\Delta\eta_1=\Delta\eta_2=0$ can produce the same distribution.

Explanation: When a statistic vector has linearly dependent components, the exponential family representation becomes nonidentifiable — multiple parameter vectors produce the identical distribution. Your job is to find which parameter changes leave the inner product

\eta^{\mathsf T} T(X)

unchanged for all values of

X

. The key quantity is the linear combination in the exponent:

\eta^{\mathsf T} T(X) = \eta_1 \cdot X + \eta_2 \cdot (2X+1)

. Expanding gives

(\eta_1 + 2\eta_2)X + \eta_2

. Two parameter vectors

\eta

and

\eta + \Delta\eta

yield the same distribution if and only if their exponents agree for all

X

, meaning both the coefficient of

X

and the constant term are unchanged. The coefficient of

X

changes by

\Delta\eta_1 + 2\Delta\eta_2

, so equivalence requires

\Delta\eta_1 + 2\Delta\eta_2 = 0

. This confirms A is correct. B reverses the coefficients — it would be correct if

T(X) = (2X, X+1)^{\mathsf T}

or some other arrangement, but it misreads which component multiplies which parameter. C is a classic trap: the two components of

T(X)

are numerically different for any given

X

, but that's irrelevant — what matters is whether they are linearly independent as functions, and

(X)

and

(2X+1)

are not (they span only a one-dimensional function space). D would be the answer only if

T(X)

had linearly independent components; since it doesn't, a whole line of parameter shifts produces the same distribution. Study tip: Whenever you see an exponential family, immediately check whether the components of

T(X)

are linearly independent as functions of

X

. If not, set

\Delta\eta^{\mathsf T} T(X) = 0

and solve — the resulting constraint equation directly reveals the nonidentifiability structure.

Question 5

A minimal exponential family is written as $f(x\mid\eta)=h(x)\exp\{\eta^{\mathsf T}T(x)-A(\eta)\}$ . A new statistic is defined by $T^*(x)=MT(x)+b$ , where $M$ is nonsingular and $b$ is fixed.

Which natural parameter and log-partition function produce an equivalent representation using $T^*$ ?

$\eta^*=M^{\mathsf T}\eta$ and $A^*(\eta^*)=A(\eta)-\eta^{\mathsf T}M^{-1}b$
$\eta^*=M\eta$ and $A^*(\eta^*)=A(\eta)-\eta^{\mathsf T}b$
$\eta^*=M^{-1}\eta$ and $A^*(\eta^*)=A(\eta)+b^{\mathsf T}b$
$\eta^*=M^{-\mathsf T}\eta$ and $A^*(\eta^*)=A(\eta)+\eta^{\mathsf T}M^{-1}b$ (correct answer)

Explanation: When reparametrizing an exponential family, your goal is to rewrite

\eta^{\mathsf T}T(x)

in terms of the new sufficient statistic

T^*(x) = MT(x) + b

, preserving the density exactly. This requires finding

\eta^*

such that

(\eta^*)^{\mathsf T}T^*(x)

reproduces the original inner product, then adjusting the log-partition function to absorb any leftover constants. Start by solving: you need

(\eta^*)^{\mathsf T}T^*(x) = \eta^{\mathsf T}T(x)

. Substituting

T^*(x) = MT(x) + b

, expand the left side as

(\eta^*)^{\mathsf T}MT(x) + (\eta^*)^{\mathsf T}b

. For the

T(x)

terms to match, you need

(\eta^*)^{\mathsf T}M = \eta^{\mathsf T}

, which gives

\eta^* = M^{-\mathsf T}\eta

. This confirms D is correct. The residual constant term

(\eta^*)^{\mathsf T}b = \eta^{\mathsf T}M^{-1}b

must be absorbed into the new log-partition function:

A^*(\eta^*) = A(\eta) + \eta^{\mathsf T}M^{-1}b

, exactly as in D. A uses

M^{\mathsf T}\eta

instead of

M^{-\mathsf T}\eta

—a common transpose/inverse mix-up—and gets the sign on the correction term wrong. B applies

M

directly to

\eta

without transposing or inverting, ignoring the structure of the inner product algebra entirely. C inverts without transposing (

M^{-1}\eta

instead of

M^{-\mathsf T}\eta

) and replaces the correction term with the unrelated quantity

b^{\mathsf T}b

, which has no basis in the derivation. A useful rule of thumb: when you see a linear transformation of a sufficient statistic, always derive

\eta^*

by matching inner products term-by-term, and track every leftover scalar into

A^*

. Confusing transpose with inverse is the most common trap here.

Question 6

Independent event counts satisfy $Y_i\sim\operatorname{Poisson}(e_i\lambda)$ , where the exposures $e_i>0$ are known and $\lambda>0$ is common to all observations.

Using $\eta=\log\lambda$ , which pair gives the canonical statistic and log-partition function for the joint model, up to terms independent of $\eta$ ?

$T=\sum_iY_i$ and $A(\eta)=e^\eta\sum_ie_i$ (correct answer)
$T=\sum_ie_iY_i$ and $A(\eta)=ne^\eta$
$T=\sum_iY_i$ and $A(\eta)=ne^\eta\sum_ie_i$
$T=\sum_iY_i/e_i$ and $A(\eta)=e^\eta\sum_ie_i$

Explanation: When you encounter exponential family questions, your job is to factor the joint log-likelihood into the canonical form

\ell(\eta) = \eta \cdot T(\mathbf{Y}) - A(\eta) + C(\mathbf{Y})

, identifying the sufficient statistic

T

and log-partition function

A(\eta)

by inspection. Start with the joint log-likelihood. Since the

Y_i

are independent Poisson

(e_i\lambda)

, the log-likelihood is:

\ell(\lambda) = \sum_i \left[ Y_i \log(e_i\lambda) - e_i\lambda - \log(Y_i!) \right]

Substituting

\lambda = e^\eta

\ell(\eta) = \sum_i Y_i(\log e_i + \eta) - e^\eta\sum_i e_i - \sum_i\log(Y_i!)

Collecting terms involving

\eta

\ell(\eta) = \eta\underbrace{\sum_i Y_i}_{T} - \underbrace{e^\eta \sum_i e_i}_{A(\eta)} + \text{(terms free of } \eta\text{)}

This confirms Answer A:

T = \sum_i Y_i

and

A(\eta) = e^\eta \sum_i e_i

. Answer B uses

T = \sum_i e_i Y_i

, which would arise if

e_i

multiplied

Y_i

inside the canonical term — but the log-likelihood shows

\eta

pairs with

\sum Y_i

, not

\sum e_i Y_i

. The exposures belong in the log-partition function, not the sufficient statistic. Answer C incorrectly multiplies

n

and

\sum e_i

together in

A(\eta)

, confusing the structural role of the exposures. Answer D divides

Y_i

e_i

, which has no basis in the log-likelihood factorization. A reliable strategy: always write out the full joint log-likelihood first, then isolate every term containing your natural parameter. Whatever multiplies

\eta

is your canonical statistic

T

; whatever you subtract is

A(\eta)

Question 7

A single multinomial observation has three categories with probabilities $p_1,p_2,p_3>0$ . Using category 3 as the baseline gives natural parameters $\eta_1=\log(p_1/p_3)$ and $\eta_2=\log(p_2/p_3)$ .

Suppose the model is restricted by $p_1p_3=p_2^2$ . Which description of the restricted model is correct?

It is a one-parameter full exponential family satisfying $\eta_1=2\eta_2$ . (correct answer)
It is a curved exponential family satisfying $\eta_1\eta_2=2$ .
It is a one-parameter full exponential family satisfying $2\eta_1=\eta_2$ .
It is not an exponential family because the restriction is nonlinear in the probabilities.

Explanation: When a multinomial model is restricted, your first move should always be to translate the constraint into the natural parameter space — that's where exponential family structure becomes transparent. The natural parameters here are

\eta_1 = \log(p_1/p_3)

and

\eta_2 = \log(p_2/p_3)

. Now apply the constraint

p_1 p_3 = p_2^2

. Taking logarithms of both sides gives

\log p_1 + \log p_3 = 2\log p_2

. Rewriting in terms of the natural parameters:

\log(p_1/p_3) = 2\log(p_2/p_3)

, which is exactly

\eta_1 = 2\eta_2

. This is a linear constraint on the natural parameters, so the restricted model is a one-parameter exponential family obtained by substituting

\eta_1 = 2\eta_2

and letting

\eta_2 = \theta

vary freely. Because the natural parameter space is an open interval and the parameterization is linear, this is a full exponential family — confirming answer A is correct. Answer B claims

\eta_1 \eta_2 = 2

, which would be a nonlinear (curved) constraint in the natural parameter space, making it a curved exponential family. But the log transformation turns the multiplicative constraint into an additive one, so this is the wrong algebraic form entirely. Answer C gets the direction backwards: it says

2\eta_1 = \eta_2

, but the derivation clearly gives

\eta_1 = 2\eta_2

, not its reciprocal. Answer D is a conceptual trap — nonlinearity in the probabilities does not disqualify a model from being an exponential family; what matters is linearity in the natural parameters after the log transformation. Your takeaway: always convert restrictions to log-odds (natural parameter) form before classifying the family. A constraint that looks curved in probability space can be perfectly linear — and thus "full" — in natural parameter space.

Question 8

For a scalar canonical exponential family, $f(x\mid\eta)=h(x)\exp\{\eta T(x)-A(\eta)\}$ . A conjugate prior has kernel $\pi(\eta)\propto\exp\{\xi\eta-\nu A(\eta)\}$ . After observing $n$ independent observations, assume the posterior mode is interior and unique.

Which equation characterizes the posterior mode $\widehat\eta$ ?

$A''(\widehat\eta)=(\xi+\sum_iT(X_i))/(\nu+n)$
$A'(\widehat\eta)=(\nu+\sum_iT(X_i))/(\xi+n)$
$A'(\widehat\eta)=(\xi+n)/(\nu+\sum_iT(X_i))$
$A'(\widehat\eta)=(\xi+\sum_iT(X_i))/(\nu+n)$ (correct answer)

Explanation: When working with exponential families and conjugate priors, your instinct should be to write out the log-posterior and differentiate. The posterior combines the conjugate prior kernel with the likelihood, giving:

\log \pi(\eta \mid \mathbf{x}) \propto \xi\eta - \nu A(\eta) + \eta\sum_i T(x_i) - nA(\eta) = \left(\xi + \sum_i T(x_i)\right)\eta - (\nu + n)A(\eta)

Taking the derivative with respect to

\eta

and setting it to zero at the mode

\widehat{\eta}

\left(\xi + \sum_i T(x_i)\right) - (\nu + n)A'(\widehat{\eta}) = 0

Solving directly yields

A'(\widehat{\eta}) = \dfrac{\xi + \sum_i T(x_i)}{\nu + n}

, which is D. This makes intuitive sense: in an exponential family,

A'(\eta) = \mathbb{E}_\eta[T(X)]

, so the posterior mode sets the mean-value parameter equal to a precision-weighted average of the prior pseudo-observations and data sufficient statistics. A is wrong because it involves

A''(\widehat{\eta})

, the second derivative (which governs curvature, not the mode condition). Confusing

A'

with

A''

is a common careless error. B flips the roles of

\xi

and

\nu

\xi

counts pseudo-observations on the sufficient statistic side, while

\nu

acts as a pseudo-sample-size scaling

A(\eta)

, so swapping them mixes up the prior's two hyperparameters. C inverts the entire fraction, which would arise from accidentally dividing by the wrong term. As a study tip: always distinguish the two roles of the conjugate prior hyperparameters — one multiplies

\eta

(sufficient statistic side) and one multiplies

A(\eta)

(log-normalizer side). Keeping that structure clear prevents both B-type and C-type errors.

Question 9

A gamma random variable has density $f(x\mid\alpha,\beta)=\beta^\alpha x^{\alpha-1}e^{-\beta x}/\Gamma(\alpha)$ for $x>0$ , with $\alpha>0$ and $\beta>0$ . Use natural parameters $\eta_1=\alpha-1$ and $\eta_2=-\beta$ and canonical statistics $T_1(X)=\log X$ and $T_2(X)=X$ .

What is $\operatorname{Cov}(\log X,X)$ under this model?

$\alpha/\beta$
$-1/\beta$
$1/\beta$ (correct answer)
$1/(\alpha\beta)$

Explanation: Whenever you see a question about covariances of sufficient statistics in an exponential family, reach for the information matrix identity: the covariance matrix of the canonical sufficient statistics equals the matrix of second-order partial derivatives of the log-partition function

A(\boldsymbol{\eta})

with respect to the natural parameters. For the gamma family with natural parameters

\eta_1 = \alpha - 1

and

\eta_2 = -\beta

, the log-partition function is:

A(\eta_1, \eta_2) = \log\Gamma(\eta_1+1) - (\eta_1+1)\log(-\eta_2)

The cross-covariance is given by the mixed partial derivative:

\operatorname{Cov}(T_1, T_2) = \frac{\partial^2 A}{\partial \eta_1 \,\partial \eta_2}

Taking

\partial A/\partial \eta_1 = \psi(\eta_1+1) - \log(-\eta_2)

, where

\psi

is the digamma function, and then differentiating with respect to

\eta_2

\frac{\partial^2 A}{\partial \eta_1\,\partial \eta_2} = -\frac{1}{\eta_2} = \frac{1}{\beta}

\operatorname{Cov}(\log X, X) = 1/\beta

, confirming C. Choice A (

\alpha/\beta

) corresponds to

\operatorname{Var}(X) = \alpha/\beta^2

confused with the covariance — it mixes up variance of

X

and scales incorrectly. Choice B (

-1/\beta

) is a sign error: students often forget that

\eta_2 = -\beta < 0

, so

-1/\eta_2 = +1/\beta

, not

-1/\beta

. Choice D (

1/(\alpha\beta)

) has no grounding in the derivatives and likely reflects a dimensional-analysis guess. Study tip: Memorize that in any exponential family,

\operatorname{Cov}(T_i, T_j) = \partial^2 A/\partial\eta_i\,\partial\eta_j

. This single identity replaces lengthy moment calculations and is a recurring tool on graduate-level theory exams.

Question 10

A scalar natural exponential family has log-partition function $A(\eta)=-r\log(-\eta)$ for $\eta<0$ , where $r>0$ is known. Let the mean be parameterized as $m=-r/\eta$ .

What is the Fisher information in one observation when the model is parameterized by $m$ ?

$I_m(m)=r/m^2$ (correct answer)
$I_m(m)=m^2/r$
$I_m(m)=r^2/m^2$
$I_m(m)=1/(rm^2)$

Explanation: When you see Fisher information asked under a reparameterization, your first instinct should be the reparameterization formula: if you change from the natural parameter

\eta

to a new parameter

m

, the Fisher information transforms as

I_m(m) = I_\eta(\eta)\left(\frac{d\eta}{dm}\right)^2

. Start by finding

I_\eta(\eta)

. In a natural exponential family, the Fisher information in the natural parameter equals the variance, which equals

A''(\eta)

. Differentiating

A(\eta) = -r\log(-\eta)

twice:

A'(\eta) = -r/\eta

and

A''(\eta) = r/\eta^2

. So

I_\eta(\eta) = r/\eta^2

. Next, find

d\eta/dm

. From

m = -r/\eta

, solve to get

\eta = -r/m

, so

d\eta/dm = r/m^2

. Applying the chain rule formula:

I_m(m) = \frac{r}{\eta^2}\cdot\left(\frac{r}{m^2}\right)^2

. But substitute

\eta = -r/m

, giving

\eta^2 = r^2/m^2

, so

I_\eta = r/(r^2/m^2) = m^2/r

. Then

I_m(m) = \frac{m^2}{r}\cdot\frac{r^2}{m^4} = \frac{r}{m^2}

. This confirms answer A. Answer B,

m^2/r

, is actually

I_\eta

expressed in terms of

m

— it forgets to apply the Jacobian factor. Answer C,

r^2/m^2

, likely arises from squaring

r/m

without correctly assembling the transformation. Answer D,

1/(rm^2)

, flips the role of

r

in the numerator and denominator. Study tip: Always track which parameterization you're working in and apply the squared-derivative Jacobian explicitly — confusing

I_\eta

with

I_m

is one of the most common errors on reparameterization problems.