Identifiability & Parameterization - Statistics Graduate Level

1. Assuming the marginal covariance of $Y$ is known exactly, which object is identifiable without further constraints?
   Answer: The matrix $\Lambda\Lambda^{\mathsf T}$ is identifiable, but $\Lambda$ is unchanged observationally under right multiplication by an orthogonal matrix. When working with factor models, your first instinct should be to examine what the observed data actually constrains. The marginal covariance of $Y$ is $\Sigma = \Lambda\Lambda^{\mathsf{T}} + \Psi$. Since $\Psi$ is known, knowing $\Sigma$ exactly pins down $\Lambda\Lambda^{\mathsf{T}}$ uniquely — this product is identifiable. However, for any orthogonal matrix $Q$ (so $QQ^{\mathsf{T}} = I$), substituting $\Lambda^* = \Lambda Q$ gives $\Lambda^*(\Lambda^*)^{\mathsf{T}} = \Lambda QQ^{\mathsf{T}}\Lambda^{\mathsf{T}} = \Lambda\Lambda^{\mathsf{T}}$, leaving the covariance unchanged. This means $\Lambda$ and $\Lambda Q$ are observationally equivalent for every orthogonal $Q$ — you can never distinguish them from the marginal distribution alone. This is exactly what B describes, making it correct. A is wrong because the rotational indeterminacy is far broader than independent sign flips. Sign changes correspond to diagonal orthogonal matrices (diagonal entries $\pm 1$), a tiny subgroup; the full rotation group $O(r)$ is the actual source of non-identifiability. C is wrong on two counts: $\Lambda^{\mathsf{T}}\Lambda$ is generally not identifiable (it changes under arbitrary rotations), and the column space of $\Lambda$ actually is determined once $\Lambda\Lambda^{\mathsf{T}}$ is known (it equals the column space of that product). D is wrong because linear independence of the columns is insufficient to resolve rotational indeterminacy. You need additional structural constraints — such as requiring $\Lambda^{\mathsf{T}}\Psi^{-1}\Lambda$ to be diagonal with decreasing entries, or sparsity assumptions — to fully identify $\Lambda$. The key study takeaway: in any factor model question, immediately write out the covariance decomposition and ask which transformations of $\Lambda$ leave it invariant. That invariance group defines precisely what remains unidentified.
2. What aspect of the true treatment effect is identifiable from $q_1$ and $q_0$ under these assumptions?
   Answer: The sign and nullity of $p_1-p_0$ are identifiable, but its magnitude is not identifiable without more misclassification information. When outcome misclassification is nondifferential, the observed probability in each arm follows the classic reclassification formula: $q_i = Se \cdot p_i + (1 - Sp)(1 - p_i)$. You can rewrite this as $q_i = (Se + Sp - 1)p_i + (1 - Sp)$. This is a linear transformation of $p_i$ with slope $\lambda = Se + Sp - 1$ and intercept $1 - Sp$. The key insight is that since $Se + Sp > 1$, we know $\lambda > 0$ — the slope is strictly positive. This is why C is correct. Taking the difference: $q_1 - q_0 = \lambda(p_1 - p_0)$. Because $\lambda > 0$, the sign of $q_1 - q_0$ equals the sign of $p_1 - p_0$, and $q_1 - q_0 = 0$ if and only if $p_1 - p_0 = 0$. So you can determine the direction and nullity of the true risk difference. However, without knowing $\lambda$ exactly, you cannot recover the magnitude — the attenuation factor is unknown. A is wrong because randomization addresses confounding, not measurement error. Misclassification bias persists regardless of how treatment was assigned, and the true risk difference is still shrunk by the unknown factor $\lambda$. B is wrong because the risk ratio is not preserved under this transformation. The additive intercept term $(1-Sp)$ means $q_1/q_0 \neq p_1/p_0$ in general, so the ratio is not recoverable. D is wrong because $Se + Sp > 1$ guarantees $\lambda > 0$, which preserves the sign. Only if $\lambda$ could be negative would reversal be possible. As a study habit, always ask: does a monotone transformation preserve the sign of a difference even when it distorts the magnitude? That distinction frequently separates what is identifiable from what is not.
3. Which statement correctly characterizes the parameterization?
   Answer: The intercept $\alpha$ is not separately identifiable from $h_0(t)$, while $\beta$ and the product $e^\alpha h_0(t)$ are identifiable under standard conditions. When you encounter identifiability questions in survival models, your first instinct should be to ask: can two different parameter configurations produce identical likelihoods? If yes, those parameters are not separately identifiable. Here, the hazard is $h(t\mid x) = h_0(t)\exp(\alpha + \beta x) = [e^\alpha h_0(t)]\exp(\beta x)$. Notice that $\alpha$ and $h_0(t)$ always appear together as the product $e^\alpha h_0(t)$. You can shift $\alpha$ by any constant $c$ and absorb $e^c$ into $h_0(t)$ without changing the hazard at any time or covariate value — the model is observationally identical. Therefore, $\alpha$ and $h_0(t)$ individually have no unique values the data can pin down. However, their product $e^\alpha h_0(t)$ is identifiable, and so is $\beta$, because $\beta$ governs ratios of hazards across covariate values, which the partial likelihood recovers without ever estimating $h_0(t)$. This makes A correct. B is wrong because it reverses the truth: $\beta$ is identifiable (via the partial likelihood) without specifying $h_0(t)$, and $\alpha$ is never separately identifiable regardless of what you know about event times. C is wrong because the argument "constant vs. function of time" is irrelevant to identifiability. A constant multiplied into an unspecified function simply redefines that function — the time-varying nature of $h_0(t)$ offers no leverage on $\alpha$ alone. D is wrong because the partial likelihood eliminates $h_0(t)$ but preserves $\beta$ — that is precisely its purpose and power. Study tip: On any survival analysis question, remember that Cox's partial likelihood is specifically designed to make $\beta$ identifiable while treating $h_0(t)$ (absorbing any intercept) as a nuisance.
4. Which statement about imposing the constraint $\mu_1<\mu_2$ is most accurate?
   Answer: It removes label switching when the component means are distinct, but it excludes the coincident-component case in which the mixing weight is not identifiable. Mixture model identifiability questions hinge on two distinct problems: label switching and parameter identifiability. When you see a question like this, separate them carefully — a constraint can solve one while leaving the other untouched. In a two-component normal mixture, "label switching" means the parameterizations $(\pi, \mu_1, \mu_2)$ and $(1-\pi, \mu_2, \mu_1)$ produce the exact same density, so the likelihood has symmetric modes. Imposing $\mu_1 < \mu_2$ selects one labeling from each symmetric pair, effectively eliminating this redundancy — **but only when $\mu_1 \neq \mu_2$**. When the means coincide ($\mu_1 = \mu_2 = \mu$), the density collapses to $\phi(y;\mu,1)$ regardless of $\pi$, so $\pi$ becomes completely unidentifiable. The ordering constraint does nothing here because there's no pair to order. This is precisely what answer A captures, making it correct. Answer B is wrong because when the means coincide, $\pi$ does not distinguish components — the entire mixing structure vanishes from the observable density, leaving $\pi$ unidentifiable rather than newly identifiable. Answer C is wrong because adding $\pi < 1/2$ is unnecessary; $\mu_1 < \mu_2$ alone is sufficient to remove label switching when means are distinct. The additional restriction would only create ambiguity when $\pi = 1/2$. Answer D is wrong because ordering constraints on parameters do not reduce the set of representable densities — every mixture distribution with distinct means is still representable; the constraint merely eliminates duplicate parameterizations. Study tip: Always ask two separate questions about any identifying constraint: Does it remove parameter redundancy (label switching)? And does it handle degenerate boundary cases (component collapse)? These often have different answers.
5. Which restriction is sufficient to remove the boundary nonidentifiability while retaining ordinary Poisson models as special cases?
   Answer: Require $0\leq\pi<1$ and $\lambda>0$, so positive-count ratios identify $\lambda$ and their total mass relative to zero-count probability identifies $\pi$. Identifiability in mixture models is the core concept here. A model is nonidentifiable at a boundary when two different parameter values produce the same distribution. In the zero-inflated Poisson (ZIP), ask yourself: when do distinct $(\pi, \lambda)$ pairs collapse into the same probability distribution? The key insight is that when $\lambda > 0$, the positive-count probabilities $\Pr(Y=k) = (1-\pi)e^{-\lambda}\lambda^k/k!$ for $k \geq 1$ uniquely determine $\lambda$ through their ratios ($\Pr(Y=k+1)/\Pr(Y=k) = \lambda/(k+1)$), and once $\lambda$ is pinned down, the total mass on positive counts identifies $(1-\pi)$, which identifies $\pi$. The ordinary Poisson is recovered at $\pi = 0$, which is allowed under D. Requiring $\lambda > 0$ excludes the degenerate case $\lambda = 0$, where every observation is zero regardless of $\pi$, making $\pi$ completely unidentifiable. Requiring $\pi < 1$ excludes the pure point-mass at zero, where $\lambda$ becomes irrelevant. Answer D threads this needle precisely. Answer A fails because allowing $\lambda = 0$ reintroduces the boundary where any $\pi$ produces an all-zero distribution — the very nonidentifiability you want to remove. Answer B is wrong because the threshold $\pi \leq 1/2$ is arbitrary and has no probabilistic justification; the identifiability problem stems from boundary values of $\pi$, not large ones. Answer C excludes $\pi = 0$, which means ordinary Poisson models are no longer special cases — violating the explicit requirement stated in the question. When you encounter mixture model identifiability questions, always check two things: which boundary collapses the parameter space, and which restriction fixes that collapse without discarding scientifically meaningful special cases.
6. Which modification most directly resolves the variance-component nonidentifiability in this model?
   Answer: Obtain at least two conditionally independent responses in some clusters, creating observable within-cluster covariance. Whenever you see a question about mixed models with random effects, ask yourself: what observed information would let me separate the two sources of variance? That's the heart of identifiability. In this model, $\text{Var}(Y_i) = \sigma_b^2 + \sigma_e^2$. With only one observation per cluster, you observe a single marginal variance — one equation, two unknowns. There is no way to disentangle $\sigma_b^2$ from $\sigma_e^2$ because they enter the likelihood only through their sum. This is a classic nonidentifiability problem. C is correct because obtaining at least two conditionally independent observations within some clusters creates observable within-cluster covariance. For observations $j \neq k$ in cluster $i$, $\text{Cov}(Y_{ij}, Y_{ik}) = \sigma_b^2$, since they share $b_i$ but have independent $\varepsilon$'s. This covariance is estimable from the data, giving you a second equation and allowing both components to be identified separately. More replication within clusters is the structural fix. A is wrong because adding more clusters only improves estimation of the marginal variance $\sigma_b^2 + \sigma_e^2$ — you get more data on the same unidentifiable sum, not a way to split it. B is wrong because centering is a location transformation. It has no effect on variance estimation or identifiability, which is purely a property of the variance structure. D is wrong because changing distributional families doesn't resolve the structural confounding — two unknown scales summing into one observed variance remains nonidentifiable regardless of shape. Study tip: Identifiability problems are resolved by structural changes to the data design, not by transformations or larger samples of the same structure. Always ask what the data can and cannot distinguish.
7. Which action yields an identifiable parameterization without changing the modeled probability distributions?
   Answer: Set one category's coefficient vector to zero, because adding the same vector to every $\beta_k$ leaves all probabilities unchanged. Whenever you see a question about identifiability in multinomial logistic regression, your first instinct should be to examine the model's invariances — specifically, what transformations of the parameters leave all predicted probabilities unchanged. The core issue here is that the softmax probability $\Pr(Y=k\mid x)=\exp(x^\mathsf{T}\beta_k)/\sum_j\exp(x^\mathsf{T}\beta_j)$ is invariant to adding any common vector $\mathbf{c}$ to every $\beta_k$. You can verify this directly: replacing each $\beta_k$ with $\beta_k+\mathbf{c}$ multiplies every numerator and denominator by $\exp(x^\mathsf{T}\mathbf{c})$, which cancels exactly. This means the likelihood has infinitely many maximizers — a classic non-identifiability. B is correct because fixing one category's coefficient vector to zero (typically the reference category) breaks this shift symmetry by anchoring the parameter space, producing a unique, identifiable parameterization while leaving all modeled probabilities unchanged. A is wrong because unit-norm constraints don't resolve the invariance — the issue is translational (adding a shared vector), not about scale or direction of individual $\beta_k$'s. Projecting onto the unit sphere doesn't eliminate the degeneracy. C is wrong because pairwise orthogonality among coefficient vectors is unrelated to the source of non-identifiability. The shift invariance exists regardless of how the $\beta_k$'s relate to each other geometrically. D is a subtle trap: a ridge penalty does produce a unique optimizer of the penalized objective, but this doesn't make the original likelihood model identifiable — it merely regularizes estimation. Identifiability is a property of the model, not the optimization procedure. Study tip: Always distinguish between fixing a model's identifiability (structural) and regularizing its estimation (computational). On exam questions about identifiability, ask yourself what symmetry is causing the problem, then find the constraint that breaks exactly that symmetry.
8. Which statement best distinguishes the sample-specific likelihood behavior from identifiability of the parameter?
   Answer: The parameter $\theta$ is not identifiable, but $\theta^2$ is identifiable; restricting the parameter space to $\theta\geq 0$ makes $\theta$ identifiable. Whenever you see a question mixing a nonlinear parameter transformation with likelihood analysis, your first instinct should be to separate two distinct concepts: identifiability (a property of the model/parameter space) versus likelihood behavior for a specific sample (a data-dependent phenomenon). A parameter $\theta$ is identifiable if distinct values of $\theta$ produce distinct distributions. Here, $N(\theta^2, 1)$ depends on $\theta$ only through $\theta^2$, so $\theta = 0.7$ and $\theta = -0.7$ generate the exact same distribution for every possible dataset. This means $\theta$ itself is not identifiable — no amount of data can distinguish $+\theta$ from $-\theta$. However, $\theta^2$ maps each distribution to a unique value, so $\theta^2$ is identifiable. Restricting to $\theta \geq 0$ creates a one-to-one correspondence between parameter values and distributions, restoring identifiability for $\theta$. This is exactly what answer B captures. Answer A is wrong because it inverts the identifiability claims — $\theta$ is the non-identifiable parameter, not $\theta^2$. Saying $\theta^2$ is not identifiable is precisely backwards. Answer C conflates a convention (choosing the positive root) with actual identifiability. Identifiability is a model-level property, not something rescued by selecting a convenient maximizer after observing data. The model itself remains symmetric in $\theta$. Answer D correctly notes the symmetry $\theta \to -\theta$ but wrongly concludes that $\theta^2$ is also non-identifiable, which it isn't — $\theta^2$ uniquely parameterizes the family. Study tip: Always check identifiability at the model level before analyzing the likelihood. If $f(x;\theta) = f(x;-\theta)$ for all $x$, the parameter is not identifiable — and restricting the parameter space, not data-driven conventions, is the proper fix.
9. Under this sampling scheme, which conclusion about $\alpha$ and $\beta$ is correct?
   Answer: The slope $\beta$ is identifiable, but the population intercept $\alpha$ is not identifiable without prevalence or sampling-fraction information. Whenever you encounter a question about logistic regression under case-control sampling, your anchor concept should be the prospective vs. retrospective likelihood distinction and what each sampling scheme preserves. In a case-control study, subjects are sampled based on disease status. The observed data follow a retrospective distribution, not the population distribution. However, a fundamental result due to Prentice and Pyke (1979) shows that the retrospective likelihood for case-control data is proportional to the prospective logistic likelihood — but only up to a shift in the intercept. Specifically, if the true model is $\Pr(D=1\mid X=x)=\operatorname{expit}(\alpha+\beta x)$, then what you can estimate from case-control data is $\operatorname{expit}(\alpha^*+\beta x)$, where $\alpha^* = \alpha + \log\!\left(\frac{\pi_1}{\pi_0}\right)$ encodes unknown sampling fractions $\pi_1, \pi_0$ for cases and controls. Because these fractions are unknown (and prevalence is unknown), $\alpha$ is not recoverable — but $\beta$, the log-odds-ratio parameter, cancels out of the sampling adjustment entirely and remains identifiable. This confirms D as correct. Choice A is wrong because while conditioning on case-control counts is one framing, it does not recover $\alpha$ — the intercept is still confounded with unknown sampling probabilities. Choice B has it exactly backwards: case-control sampling distorts the intercept (prevalence-related), not the exposure odds ratio, so $\beta$ — not $\alpha$ — is the identifiable parameter. Choice C overstates the damage; the entire slope structure is preserved, so the claim that "neither" is identifiable is false. A useful memory anchor: "case-control kills the intercept, not the slope." On any exam question mixing logistic regression with retrospective sampling, immediately ask yourself which parameter depends on prevalence — that one is lost without additional external information.

Start this quiz