Statistics Graduate Level Quiz: Identifiability And Parameterization

9 questions · exam conditions

0:00

Identifiability And ParameterizationQuestion 1 of 9

A random-intercept model is proposed for one response from each of many clusters: $Y_i=\mu+b_i+\varepsilon_i$ , where $b_i\sim N(0,\sigma_b^2)$ and $\varepsilon_i\sim N(0,\sigma_e^2)$ independently. Each cluster contributes exactly one observation.

Which modification most directly resolves the variance-component nonidentifiability in this model?

Increase the number of clusters indefinitely while retaining one independent response from each cluster.

Center all observed responses at their sample mean before fitting both variance components.

Obtain at least two conditionally independent responses in some clusters, creating observable within-cluster covariance.

Replace the Gaussian assumptions by two distinct symmetric distributions with unknown scale parameters.

← Back to quizzes

Statistics Graduate Level Quiz

Statistics Graduate Level Quiz: Identifiability And Parameterization

Practice Identifiability And Parameterization in Statistics Graduate Level with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Identifiability And Parameterization, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics Graduate Level.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

Which modification most directly resolves the variance-component nonidentifiability in this model?

Increase the number of clusters indefinitely while retaining one independent response from each cluster.
Center all observed responses at their sample mean before fitting both variance components.
Obtain at least two conditionally independent responses in some clusters, creating observable within-cluster covariance. (correct answer)
Replace the Gaussian assumptions by two distinct symmetric distributions with unknown scale parameters.

Explanation: Whenever you see a question about mixed models with random effects, ask yourself: what observed information would let me separate the two sources of variance? That's the heart of identifiability. In this model,

\text{Var}(Y_i) = \sigma_b^2 + \sigma_e^2

. With only one observation per cluster, you observe a single marginal variance — one equation, two unknowns. There is no way to disentangle

\sigma_b^2

from

\sigma_e^2

because they enter the likelihood only through their sum. This is a classic nonidentifiability problem. C is correct because obtaining at least two conditionally independent observations within some clusters creates observable within-cluster covariance. For observations

j \neq k

in cluster

i

\text{Cov}(Y_{ij}, Y_{ik}) = \sigma_b^2

, since they share

b_i

but have independent

\varepsilon

's. This covariance is estimable from the data, giving you a second equation and allowing both components to be identified separately. More replication within clusters is the structural fix. A is wrong because adding more clusters only improves estimation of the marginal variance

\sigma_b^2 + \sigma_e^2

— you get more data on the same unidentifiable sum, not a way to split it. B is wrong because centering is a location transformation. It has no effect on variance estimation or identifiability, which is purely a property of the variance structure. D is wrong because changing distributional families doesn't resolve the structural confounding — two unknown scales summing into one observed variance remains nonidentifiable regardless of shape. Study tip: Identifiability problems are resolved by structural changes to the data design, not by transformations or larger samples of the same structure. Always ask what the data can and cannot distinguish.

Question 2

Suppose $X_1,\ldots,X_n$ are independent with distribution $N(\theta^2,1)$ , where initially $\theta\in\mathbb{R}$ . For one observed sample, the sample mean is $\bar X=0.49$ .

Which statement best distinguishes the sample-specific likelihood behavior from identifiability of the parameter?

The parameter $\theta$ is identifiable because the likelihood has exactly two isolated maximizers, whereas $\theta^2$ is not identifiable.
The parameter $\theta$ is not identifiable, but $\theta^2$ is identifiable; restricting the parameter space to $\theta\geq 0$ makes $\theta$ identifiable. (correct answer)
The parameter $\theta$ is identifiable for this sample because the positive maximizer $0.7$ can be selected by convention without changing the model.
Neither $\theta$ nor $\theta^2$ is identifiable, because the likelihood is unchanged when $\theta$ is replaced by $-\theta$ .

Explanation: Whenever you see a question mixing a nonlinear parameter transformation with likelihood analysis, your first instinct should be to separate two distinct concepts: identifiability (a property of the model/parameter space) versus likelihood behavior for a specific sample (a data-dependent phenomenon). A parameter

\theta

is identifiable if distinct values of

\theta

produce distinct distributions. Here,

N(\theta^2, 1)

depends on

\theta

only through

\theta^2

, so

\theta = 0.7

and

\theta = -0.7

generate the exact same distribution for every possible dataset. This means

\theta

itself is not identifiable — no amount of data can distinguish

+\theta

from

-\theta

. However,

\theta^2

maps each distribution to a unique value, so

\theta^2

is identifiable. Restricting to

\theta \geq 0

creates a one-to-one correspondence between parameter values and distributions, restoring identifiability for

\theta

. This is exactly what answer B captures. Answer A is wrong because it inverts the identifiability claims —

\theta

is the non-identifiable parameter, not

\theta^2

. Saying

\theta^2

is not identifiable is precisely backwards. Answer C conflates a convention (choosing the positive root) with actual identifiability. Identifiability is a model-level property, not something rescued by selecting a convenient maximizer after observing data. The model itself remains symmetric in

\theta

. Answer D correctly notes the symmetry

\theta \to -\theta

but wrongly concludes that

\theta^2

is also non-identifiable, which it isn't —

\theta^2

uniquely parameterizes the family. Study tip: Always check identifiability at the model level before analyzing the likelihood. If

f(x;\theta) = f(x;-\theta)

for all

x

, the parameter is not identifiable — and restricting the parameter space, not data-driven conventions, is the proper fix.

Question 3

A Gaussian factor model is specified as $Y=\Lambda F+\varepsilon$ , where $F\sim N(0,I_r)$ , $\varepsilon\sim N(0,\Psi)$ , and $\Psi$ is known. No structural restrictions are placed on the loading matrix $\Lambda$ .

Assuming the marginal covariance of $Y$ is known exactly, which object is identifiable without further constraints?

Each column of $\Lambda$ is identifiable up to an independent sign change, but arbitrary rotations are distinguishable.
The matrix $\Lambda\Lambda^{\mathsf T}$ is identifiable, but $\Lambda$ is unchanged observationally under right multiplication by an orthogonal matrix. (correct answer)
The matrix $\Lambda^{\mathsf T}\Lambda$ is identifiable, but the marginal covariance cannot determine the column space of $\Lambda$ .
The loading matrix $\Lambda$ is fully identifiable whenever its columns are linearly independent and $\Psi$ is known.

Explanation: When working with factor models, your first instinct should be to examine what the observed data actually constrains. The marginal covariance of

Y

\Sigma = \Lambda\Lambda^{\mathsf{T}} + \Psi

. Since

\Psi

is known, knowing

\Sigma

exactly pins down

\Lambda\Lambda^{\mathsf{T}}

uniquely — this product is identifiable. However, for any orthogonal matrix

Q

(so

QQ^{\mathsf{T}} = I

), substituting

\Lambda^* = \Lambda Q

gives

\Lambda^*(\Lambda^*)^{\mathsf{T}} = \Lambda QQ^{\mathsf{T}}\Lambda^{\mathsf{T}} = \Lambda\Lambda^{\mathsf{T}}

, leaving the covariance unchanged. This means

\Lambda

and

\Lambda Q

are observationally equivalent for every orthogonal

Q

— you can never distinguish them from the marginal distribution alone. This is exactly what B describes, making it correct. A is wrong because the rotational indeterminacy is far broader than independent sign flips. Sign changes correspond to diagonal orthogonal matrices (diagonal entries

\pm 1

), a tiny subgroup; the full rotation group

O(r)

is the actual source of non-identifiability. C is wrong on two counts:

\Lambda^{\mathsf{T}}\Lambda

is generally not identifiable (it changes under arbitrary rotations), and the column space of

\Lambda

actually is determined once

\Lambda\Lambda^{\mathsf{T}}

is known (it equals the column space of that product). D is wrong because linear independence of the columns is insufficient to resolve rotational indeterminacy. You need additional structural constraints — such as requiring

\Lambda^{\mathsf{T}}\Psi^{-1}\Lambda

to be diagonal with decreasing entries, or sparsity assumptions — to fully identify

\Lambda

. The key study takeaway: in any factor model question, immediately write out the covariance decomposition and ask which transformations of

\Lambda

leave it invariant. That invariance group defines precisely what remains unidentified.

Question 4

A proportional-hazards model is written as $h(t\mid x)=h_0(t)\exp(\alpha+\beta x)$ , where the baseline hazard $h_0(t)$ is completely unspecified and positive. Exact event and censoring times are observed under independent censoring.

Which statement correctly characterizes the parameterization?

The intercept $\alpha$ is not separately identifiable from $h_0(t)$ , while $\beta$ and the product $e^\alpha h_0(t)$ are identifiable under standard conditions. (correct answer)
The intercept $\alpha$ is identifiable from event times, but $\beta$ is not identifiable unless the baseline hazard is parametrically specified.
Both $\alpha$ and $h_0(t)$ are identifiable because one is constant in time and the other is a function of time.
Neither $\alpha$ nor $\beta$ is identifiable because the partial likelihood eliminates every multiplicative component of the hazard.

Explanation: When you encounter identifiability questions in survival models, your first instinct should be to ask: can two different parameter configurations produce identical likelihoods? If yes, those parameters are not separately identifiable. Here, the hazard is

h(t\mid x) = h_0(t)\exp(\alpha + \beta x) = [e^\alpha h_0(t)]\exp(\beta x)

. Notice that

\alpha

and

h_0(t)

always appear together as the product

e^\alpha h_0(t)

. You can shift

\alpha

by any constant

c

and absorb

e^c

into

h_0(t)

without changing the hazard at any time or covariate value — the model is observationally identical. Therefore,

\alpha

and

h_0(t)

individually have no unique values the data can pin down. However, their product

e^\alpha h_0(t)

is identifiable, and so is

\beta

, because

\beta

governs ratios of hazards across covariate values, which the partial likelihood recovers without ever estimating

h_0(t)

. This makes A correct. B is wrong because it reverses the truth:

\beta

is identifiable (via the partial likelihood) without specifying

h_0(t)

, and

\alpha

is never separately identifiable regardless of what you know about event times. C is wrong because the argument "constant vs. function of time" is irrelevant to identifiability. A constant multiplied into an unspecified function simply redefines that function — the time-varying nature of

h_0(t)

offers no leverage on

\alpha

alone. D is wrong because the partial likelihood eliminates

h_0(t)

but preserves

\beta

— that is precisely its purpose and power. Study tip: On any survival analysis question, remember that Cox's partial likelihood is specifically designed to make

\beta

identifiable while treating

h_0(t)

(absorbing any intercept) as a nuisance.

Question 5

For a multinomial logistic regression with categories $1,\ldots,K$ , suppose $\Pr(Y=k\mid x)=\exp(x^{\mathsf T}\beta_k)/\sum_{j=1}^K\exp(x^{\mathsf T}\beta_j)$ . The design matrix has full column rank, and all category probabilities are positive.

Which action yields an identifiable parameterization without changing the modeled probability distributions?

Constrain every coefficient vector to have unit norm, because only the direction of each $\beta_k$ affects category probabilities.
Set one category's coefficient vector to zero, because adding the same vector to every $\beta_k$ leaves all probabilities unchanged. (correct answer)
Require the coefficient vectors to be pairwise orthogonal, because correlated category effects cause the likelihood invariance.
Add a strictly convex ridge penalty, because a unique penalized optimizer proves the original likelihood model is identifiable.

Explanation: Whenever you see a question about identifiability in multinomial logistic regression, your first instinct should be to examine the model's invariances — specifically, what transformations of the parameters leave all predicted probabilities unchanged. The core issue here is that the softmax probability

\Pr(Y=k\mid x)=\exp(x^\mathsf{T}\beta_k)/\sum_j\exp(x^\mathsf{T}\beta_j)

is invariant to adding any common vector

\mathbf{c}

to every

\beta_k

. You can verify this directly: replacing each

\beta_k

with

\beta_k+\mathbf{c}

multiplies every numerator and denominator by

\exp(x^\mathsf{T}\mathbf{c})

, which cancels exactly. This means the likelihood has infinitely many maximizers — a classic non-identifiability. B is correct because fixing one category's coefficient vector to zero (typically the reference category) breaks this shift symmetry by anchoring the parameter space, producing a unique, identifiable parameterization while leaving all modeled probabilities unchanged. A is wrong because unit-norm constraints don't resolve the invariance — the issue is translational (adding a shared vector), not about scale or direction of individual

\beta_k

's. Projecting onto the unit sphere doesn't eliminate the degeneracy. C is wrong because pairwise orthogonality among coefficient vectors is unrelated to the source of non-identifiability. The shift invariance exists regardless of how the

\beta_k

's relate to each other geometrically. D is a subtle trap: a ridge penalty does produce a unique optimizer of the penalized objective, but this doesn't make the original likelihood model identifiable — it merely regularizes estimation. Identifiability is a property of the model, not the optimization procedure. Study tip: Always distinguish between fixing a model's identifiability (structural) and regularizing its estimation (computational). On exam questions about identifiability, ask yourself what symmetry is causing the problem, then find the constraint that breaks exactly that symmetry.

Question 6

In a source population, disease status satisfies the logistic model $\Pr(D=1\mid X=x)=\operatorname{expit}(\alpha+\beta x)$ . Investigators use case-control sampling, selecting subjects with probabilities depending on disease status but not otherwise on $X$ . The population disease prevalence and the case and control sampling fractions are not known.

Under this sampling scheme, which conclusion about $\alpha$ and $\beta$ is correct?

Both parameters remain identifiable because conditioning on the observed numbers of cases and controls removes only nuisance sampling probabilities.
Only $\alpha$ remains identifiable because case-control sampling preserves disease prevalence but distorts the exposure odds ratio.
Neither parameter is identifiable because retrospective sampling changes the entire conditional distribution of disease given exposure.
The slope $\beta$ is identifiable, but the population intercept $\alpha$ is not identifiable without prevalence or sampling-fraction information. (correct answer)

Explanation: Whenever you encounter a question about logistic regression under case-control sampling, your anchor concept should be the prospective vs. retrospective likelihood distinction and what each sampling scheme preserves. In a case-control study, subjects are sampled based on disease status. The observed data follow a retrospective distribution, not the population distribution. However, a fundamental result due to Prentice and Pyke (1979) shows that the retrospective likelihood for case-control data is proportional to the prospective logistic likelihood — but only up to a shift in the intercept. Specifically, if the true model is

\Pr(D=1\mid X=x)=\operatorname{expit}(\alpha+\beta x)

, then what you can estimate from case-control data is

\operatorname{expit}(\alpha^*+\beta x)

, where

\alpha^* = \alpha + \log\!\left(\frac{\pi_1}{\pi_0}\right)

encodes unknown sampling fractions

\pi_1, \pi_0

for cases and controls. Because these fractions are unknown (and prevalence is unknown),

\alpha

is not recoverable — but

\beta

, the log-odds-ratio parameter, cancels out of the sampling adjustment entirely and remains identifiable. This confirms D as correct. Choice A is wrong because while conditioning on case-control counts is one framing, it does not recover

\alpha

— the intercept is still confounded with unknown sampling probabilities. Choice B has it exactly backwards: case-control sampling distorts the intercept (prevalence-related), not the exposure odds ratio, so

\beta

— not

\alpha

— is the identifiable parameter. Choice C overstates the damage; the entire slope structure is preserved, so the claim that "neither" is identifiable is false. A useful memory anchor: "case-control kills the intercept, not the slope." On any exam question mixing logistic regression with retrospective sampling, immediately ask yourself which parameter depends on prevalence — that one is lost without additional external information.

Question 7

A zero-inflated Poisson model has probabilities $\Pr(Y=0)=\pi+(1-\pi)e^{-\lambda}$ and $\Pr(Y=k)=(1-\pi)e^{-\lambda}\lambda^k/k!$ for positive integers $k$ . The initial parameter space is $0\leq\pi\leq1$ and $\lambda\geq0$ .

Which restriction is sufficient to remove the boundary nonidentifiability while retaining ordinary Poisson models as special cases?

Require $0<\pi<1$ while allowing $\lambda=0$ , because a positive inflation probability labels the structural-zero component and ensures distinct distributions.
Require $\pi\leq1/2$ and $\lambda\geq0$ , because bounding the mixing weight below one-half prevents equivalent representations of the zero-count distribution.
Require $\pi>0$ and $\lambda>0$ , thereby excluding both the ordinary Poisson model ( $\pi=0$ ) and the degenerate point-mass case.
Require $0\leq\pi<1$ and $\lambda>0$ , so positive-count ratios identify $\lambda$ and their total mass relative to zero-count probability identifies $\pi$ . (correct answer)

Explanation: Identifiability in mixture models is the core concept here. A model is nonidentifiable at a boundary when two different parameter values produce the same distribution. In the zero-inflated Poisson (ZIP), ask yourself: when do distinct

(\pi, \lambda)

pairs collapse into the same probability distribution? The key insight is that when

\lambda > 0

, the positive-count probabilities

\Pr(Y=k) = (1-\pi)e^{-\lambda}\lambda^k/k!

for

k \geq 1

uniquely determine

\lambda

through their ratios (

\Pr(Y=k+1)/\Pr(Y=k) = \lambda/(k+1)

), and once

\lambda

is pinned down, the total mass on positive counts identifies

(1-\pi)

, which identifies

\pi

. The ordinary Poisson is recovered at

\pi = 0

, which is allowed under D. Requiring

\lambda > 0

excludes the degenerate case

\lambda = 0

, where every observation is zero regardless of

\pi

, making

\pi

completely unidentifiable. Requiring

\pi < 1

excludes the pure point-mass at zero, where

\lambda

becomes irrelevant. Answer D threads this needle precisely. Answer A fails because allowing

\lambda = 0

reintroduces the boundary where any

\pi

produces an all-zero distribution — the very nonidentifiability you want to remove. Answer B is wrong because the threshold

\pi \leq 1/2

is arbitrary and has no probabilistic justification; the identifiability problem stems from boundary values of

\pi

, not large ones. Answer C excludes

\pi = 0

, which means ordinary Poisson models are no longer special cases — violating the explicit requirement stated in the question. When you encounter mixture model identifiability questions, always check two things: which boundary collapses the parameter space, and which restriction fixes that collapse without discarding scientifically meaningful special cases.

Question 8

In each arm of a randomized trial, a binary outcome is measured with nondifferential sensitivity $Se$ and specificity $Sp$ . Let the true event probabilities be $p_1$ and $p_0$ and the observed positive probabilities be $q_1$ and $q_0$ . The values of $Se$ and $Sp$ are unknown, but investigators know that $Se+Sp>1$ .

What aspect of the true treatment effect is identifiable from $q_1$ and $q_0$ under these assumptions?

The exact risk difference $p_1-p_0$ is identifiable because randomization eliminates uncertainty from outcome misclassification.
The risk ratio $p_1/p_0$ is identifiable, although the two arm-specific event probabilities are not separately identifiable.
The sign and nullity of $p_1-p_0$ are identifiable, but its magnitude is not identifiable without more misclassification information. (correct answer)
No feature of the treatment contrast is identifiable, because unknown sensitivity can reverse every observed difference.

Explanation: When outcome misclassification is nondifferential, the observed probability in each arm follows the classic reclassification formula:

q_i = Se \cdot p_i + (1 - Sp)(1 - p_i)

. You can rewrite this as

q_i = (Se + Sp - 1)p_i + (1 - Sp)

. This is a linear transformation of

p_i

with slope

\lambda = Se + Sp - 1

and intercept

1 - Sp

. The key insight is that since

Se + Sp > 1

, we know

\lambda > 0

— the slope is strictly positive. This is why C is correct. Taking the difference:

q_1 - q_0 = \lambda(p_1 - p_0)

. Because

\lambda > 0

, the sign of

q_1 - q_0

equals the sign of

p_1 - p_0

, and

q_1 - q_0 = 0

if and only if

p_1 - p_0 = 0

. So you can determine the direction and nullity of the true risk difference. However, without knowing

\lambda

exactly, you cannot recover the magnitude — the attenuation factor is unknown. A is wrong because randomization addresses confounding, not measurement error. Misclassification bias persists regardless of how treatment was assigned, and the true risk difference is still shrunk by the unknown factor

\lambda

. B is wrong because the risk ratio is not preserved under this transformation. The additive intercept term

(1-Sp)

means

q_1/q_0 \neq p_1/p_0

in general, so the ratio is not recoverable. D is wrong because

Se + Sp > 1

guarantees

\lambda > 0

, which preserves the sign. Only if

\lambda

could be negative would reversal be possible. As a study habit, always ask: does a monotone transformation preserve the sign of a difference even when it distorts the magnitude? That distinction frequently separates what is identifiable from what is not.

Question 9

Consider the two-component normal mixture model $f(y)=\pi\phi(y;\mu_1,1)+(1-\pi)\phi(y;\mu_2,1)$ , where $0<\pi<1$ and $\phi(\cdot;\mu,1)$ denotes a normal density with mean $\mu$ and variance $1$ .

Which statement about imposing the constraint $\mu_1<\mu_2$ is most accurate?

It removes label switching when the component means are distinct, but it excludes the coincident-component case in which the mixing weight is not identifiable. (correct answer)
It makes the model globally identifiable even when the component means coincide, because the mixing proportion then distinguishes the components.
It is insufficient to remove label switching unless the additional restriction $\pi<1/2$ is imposed simultaneously.
It changes the set of mixture distributions whenever the means are distinct, because ordered components represent fewer observable densities.

Explanation: Mixture model identifiability questions hinge on two distinct problems: label switching and parameter identifiability. When you see a question like this, separate them carefully — a constraint can solve one while leaving the other untouched. In a two-component normal mixture, "label switching" means the parameterizations

(\pi, \mu_1, \mu_2)

and

(1-\pi, \mu_2, \mu_1)

produce the exact same density, so the likelihood has symmetric modes. Imposing

\mu_1 < \mu_2

selects one labeling from each symmetric pair, effectively eliminating this redundancy — but only when $\mu_1 \neq \mu_2$ . When the means coincide (

\mu_1 = \mu_2 = \mu

), the density collapses to

\phi(y;\mu,1)

regardless of

\pi

, so

\pi

becomes completely unidentifiable. The ordering constraint does nothing here because there's no pair to order. This is precisely what answer A captures, making it correct. Answer B is wrong because when the means coincide,

\pi

does not distinguish components — the entire mixing structure vanishes from the observable density, leaving

\pi

unidentifiable rather than newly identifiable. Answer C is wrong because adding

\pi < 1/2

is unnecessary;

\mu_1 < \mu_2

alone is sufficient to remove label switching when means are distinct. The additional restriction would only create ambiguity when

\pi = 1/2

. Answer D is wrong because ordering constraints on parameters do not reduce the set of representable densities — every mixture distribution with distinct means is still representable; the constraint merely eliminates duplicate parameterizations. Study tip: Always ask two separate questions about any identifying constraint: Does it remove parameter redundancy (label switching)? And does it handle degenerate boundary cases (component collapse)? These often have different answers.