Likelihood Manipulation - Statistics Graduate Level

1. A multinomial model has three cell probabilities $p_1=\theta$, $p_2=\theta(1-\theta)$, and $p_3=(1-\theta)^2$, where $0<\theta<1$. The observed cell counts are $n_1=12$, $n_2=8$, and $n_3=10$. What is the maximum likelihood estimate of $\theta$?
   Answer: $\widehat\theta=\frac{n_1+n_2}{n_1+2n_2+2n_3}=\frac{5}{12}$ When working with multinomial MLE problems, your first instinct should be to write down the log-likelihood and differentiate. The multinomial log-likelihood (ignoring constants) is: $\ell(\theta) = n_1\ln p_1 + n_2\ln p_2 + n_3\ln p_3$ Substituting the cell probabilities gives: $\ell(\theta) = n_1\ln\theta + n_2[\ln\theta + \ln(1-\theta)] + n_3[2\ln(1-\theta)]$ $= (n_1+n_2)\ln\theta + (n_2+2n_3)\ln(1-\theta)$ Differentiating with respect to $\theta$ and setting equal to zero: $\frac{d\ell}{d\theta} = \frac{n_1+n_2}{\theta} - \frac{n_2+2n_3}{1-\theta} = 0$ Cross-multiplying and solving: $(n_1+n_2)(1-\theta) = (n_2+2n_3)\theta$, which gives: $\widehat\theta = \frac{n_1+n_2}{n_1+2n_2+2n_3} = \frac{20}{48} = \frac{5}{12}$ This confirms D is correct. Choice A doubles $n_2$ in the numerator, which would arise if you incorrectly counted each appearance of $\theta$ in $p_2$ twice rather than tracking contributions to the $\theta$ and $(1-\theta)$ terms separately. Choice B forgets that $p_2$ also contributes a factor of $(1-\theta)$, losing the $n_2$ term in the denominator's coefficient. Choice C naively uses only $n_1$, as if $\theta$ appeared only in $p_1$, ignoring that $p_2$ also contains $\theta$. Your study tip: always expand each $\ln p_i$ fully before differentiating. Collecting coefficients on $\ln\theta$ versus $\ln(1-\theta)$ separately prevents the algebra errors that generate the wrong distractors here.
2. Suppose $X_1,\ldots,X_6$ are independent normal observations with unknown variance $\sigma^2$ and mean constrained by $\mu\ge 0$. The data satisfy $\bar X=-1$ and $\sum_{i=1}^6(X_i-\bar X)^2=18$. What is the maximum likelihood estimate of $\sigma^2$ under the constraint?
   Answer: $\widehat{\sigma}^2=4$ When you encounter constrained MLE problems, your first instinct should be to find the unconstrained MLE, then check whether it satisfies the constraint. If it doesn't, the constrained MLE lives on the boundary of the parameter space. Here, the unconstrained MLEs are $\hat{\mu} = \bar{X} = -1$ and $\hat{\sigma}^2 = \frac{1}{n}\sum(X_i - \bar{X})^2 = \frac{18}{6} = 3$. But $\hat{\mu} = -1$ violates $\mu \geq 0$, so we must impose the boundary condition $\mu = 0$. With $\mu$ fixed at 0, the log-likelihood is maximized over $\sigma^2$ alone, giving the constrained MLE: $\widehat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^6 X_i^2 = \frac{1}{6}\left[\sum(X_i - \bar{X})^2 + n\bar{X}^2\right] = \frac{18 + 6(1)}{6} = \frac{24}{6} = 4$ This confirms the correct answer is C. Choice A ($\widehat{\sigma}^2 = 3$) is the unconstrained MLE of $\sigma^2$, which pairs with an infeasible $\hat{\mu} = -1$. You cannot use this estimate because the joint MLE violates the constraint. Choice B ($18/5$) suggests dividing by $n-1 = 5$, which is the unbiased estimator of $\sigma^2$, not the MLE — a classic confusion. Choice D ($24/5$) similarly divides the constrained sum of squares by 5 instead of 6, mixing unbiased estimation logic with the constrained setup. The key study tip: when a constraint forces you to the boundary, recompute the MLE of all remaining free parameters from scratch using that boundary value — don't just patch the unconstrained answer.
3. A sample comes from a Pareto distribution with density $f(x;\alpha,\theta)=\alpha\theta^\alpha x^{-(\alpha+1)}$ for $x\ge\theta$, where both $\alpha>0$ and $\theta>0$ are unknown. Which pair is the joint maximum likelihood estimator?
   Answer: $\widehat\theta=X_{(1)},\qquad \widehat\alpha=\frac{n}{\sum_i\log(X_i/X_{(1)})}$ When both parameters of a distribution are unknown and appear in the support, you must be especially careful: the likelihood is not always differentiable with respect to every parameter, so blind calculus can mislead you. For this Pareto distribution, the log-likelihood is $\ell(\alpha,\theta) = n\log\alpha + n\alpha\log\theta - (\alpha+1)\sum_i\log x_i,$ but this is only valid when $\theta \le x_{(1)}$ (the sample minimum), since every observation must satisfy $x_i \ge \theta$. For fixed $\alpha > 0$, the term $n\alpha\log\theta$ is strictly increasing in $\theta$, so you push $\theta$ as high as possible without violating the constraint — giving $\widehat{\theta} = X_{(1)}$. With $\theta$ fixed at $X_{(1)}$, differentiate with respect to $\alpha$ and set equal to zero: $\frac{\partial\ell}{\partial\alpha} = \frac{n}{\alpha} + n\log X_{(1)} - \sum_i\log X_i = 0,$ which yields $\widehat{\alpha} = \frac{n}{\sum_i\log(X_i/X_{(1)})}.$ This is Choice C, the correct answer. Choice A uses $\sum_i \log X_i$ in the denominator without subtracting $n\log X_{(1)}$, which is what you'd get if you incorrectly treated $\theta$ as known or equal to 1. Choice B sets $\widehat{\theta} = X_{(n)}$, the sample maximum — this violates the support constraint since $X_{(n)} \ge x_i$ for all $i$ is impossible unless all observations are equal. Choice D uses a geometric-mean formula for $\widehat{\theta}$, which has no MLE justification here and also fails to respect the boundary constraint. The key study tip: whenever a parameter appears in the boundary of the support, recognize that the MLE for that parameter is found by a boundary argument, not calculus. Then condition on that estimate before differentiating for the remaining parameters.
4. Independent counts originally follow a Poisson distribution with mean $\lambda$, but observations equal to zero are not recorded. For the resulting zero-truncated sample, let $\bar X$ denote the sample mean. Which equation characterizes an interior maximum likelihood estimate of $\lambda$?
   Answer: $\bar X=\frac{\lambda}{1-e^{-\lambda}}$ When sampling from a truncated distribution, you must re-normalize the likelihood to reflect only the observable outcomes. Here, zeros are excluded, so the zero-truncated Poisson assigns probability $P(X=k \mid X>0) = \frac{e^{-\lambda}\lambda^k/k!}{1-e^{-\lambda}}$ for $k=1,2,3,\ldots$ The log-likelihood for a sample of size $n$ is $\ell(\lambda) = n\bar{X}\ln\lambda - n\lambda - n\ln(1-e^{-\lambda}) + \text{const}$. Setting the derivative with respect to $\lambda$ equal to zero gives $\frac{\bar{X}}{\lambda} - 1 - \frac{e^{-\lambda}}{1-e^{-\lambda}} = 0$. Rearranging, $\bar{X} = \lambda\left(1 + \frac{e^{-\lambda}}{1-e^{-\lambda}}\right) = \frac{\lambda}{1-e^{-\lambda}}$, confirming answer B. This equation also has a beautiful intuitive interpretation: $\frac{\lambda}{1-e^{-\lambda}}$ is exactly the mean of the zero-truncated Poisson distribution, so the MLE simply equates the sample mean to the truncated population mean — a standard MLE moment-matching result. A sets $\bar{X}=\lambda$, which would be correct for an untruncated Poisson, ignoring the re-normalization entirely. C inverts the relationship incorrectly, placing the denominator $1-e^{-\lambda}$ in the numerator instead. D rewrites $\frac{\lambda}{e^\lambda - 1}$, which equals $\frac{\lambda e^{-\lambda}}{1-e^{-\lambda}}$ — a structurally plausible but incorrect expression that confuses the truncation adjustment. As a study strategy: whenever you encounter truncated or censored distributions on exams, always re-derive the likelihood from scratch using the conditional probability. The MLE scoring equation will differ from the standard case precisely because of the normalizing denominator.
5. A random sample is drawn from a uniform distribution on $[\theta,\theta+1]$. The observed sample minimum is $0.42$ and the observed sample maximum is $1.11$. Which value is the largest maximum likelihood estimator of $\theta$?
   Answer: $0.42$ When working with MLE for a uniform distribution on $[\theta, \theta+1]$, your first instinct should be to write down the likelihood function explicitly. Since the density is $f(x|\theta) = 1$ for $x \in [\theta, \theta+1]$ and 0 otherwise, the likelihood for a sample of size $n$ is simply $L(\theta) = 1$ whenever all observations fall within $[\theta, \theta+1]$, and 0 otherwise. The MLE is any $\theta$ that keeps all data inside the interval — so you need to find the range of valid $\theta$ values, then identify the largest one. For every observation $x_i$ to satisfy $\theta \leq x_i \leq \theta + 1$, you need two constraints simultaneously: $\theta \leq x_{(1)}$ (the minimum) and $\theta \geq x_{(n)} - 1$ (the maximum). With $x_{(1)} = 0.42$ and $x_{(n)} = 1.11$, the valid range is $1.11 - 1 \leq \theta \leq 0.42$, i.e., $0.11 \leq \theta \leq 0.42$. The likelihood equals 1 throughout this entire interval, so every value in $[0.11, 0.42]$ is an MLE. The question asks for the largest MLE, which is $\mathbf{0.42}$ — answer B. Answer A (0.11) is the smallest MLE, not the largest — it's the lower bound of the valid interval. Answer C (0.765) is simply the midpoint of the two observations, which corresponds to the method-of-moments estimator, not the MLE. Answer D (1.11) violates the upper constraint: if $\theta = 1.11$, then $x_{(1)} = 0.42 < 1.11 = \theta$, placing data outside the support. The key study tip: for uniform distribution MLEs, always write the constraint region explicitly. The likelihood is flat (not peaked), so the MLE is a set, and exam questions will test whether you can identify specific boundary values within it.
6. Independent subjects have exponential event times with hazard $\lambda$. For subject $i$, one observes $t_i=\min(T_i,C_i)$ and $\delta_i=I(T_i\le C_i)$, where censoring is independent and its distribution does not involve $\lambda$. Let $D=\sum_{i=1}^n\delta_i$ and $R=\sum_{i=1}^n t_i$. Assuming $D>0$, which estimator maximizes the likelihood based on the observed data?
   Answer: $\widehat{\lambda}=\frac{D}{R}$ When you encounter survival analysis questions involving exponential event times and censoring, your first instinct should be to write down the likelihood function carefully — the MLE falls out naturally from there. For exponential survival times with hazard $\lambda$, each subject contributes $\lambda e^{-\lambda t_i}$ if their event is observed ($\delta_i = 1$) and $e^{-\lambda t_i}$ if censored ($\delta_i = 0$). The full likelihood is therefore: $L(\lambda) = \prod_{i=1}^n \lambda^{\delta_i} e^{-\lambda t_i} = \lambda^D e^{-\lambda R}$ Taking the log: $\ell(\lambda) = D\log\lambda - \lambda R$. Differentiating and setting equal to zero gives $\frac{d\ell}{d\lambda} = \frac{D}{\lambda} - R = 0$, which immediately yields $\widehat{\lambda} = D/R$. This is answer A — the number of observed events divided by the total accumulated time (observed plus censored), which is the empirical event rate per unit of exposure. Answer B uses $n$ instead of $D$, treating all subjects as having experienced the event regardless of censoring — this ignores the censoring indicator entirely and will overestimate $\lambda$. Answer C uses only the event times of uncensored subjects in the denominator ($\sum \delta_i t_i$), discarding the valuable exposure time contributed by censored individuals, which is a fundamental misuse of the survival likelihood. Answer D inverts the correct formula, giving an estimate of mean survival time rather than the hazard rate. A useful memory anchor: in survival analysis, censored observations still contribute information through their risk exposure. The MLE always reflects total person-time at risk ($R$) in the denominator, not just event times.
7. Suppose independent counts satisfy $Y_i\sim\operatorname{Poisson}(b_i+\theta e_i)$, where $b_i>0$ and $e_i>0$ are known and $\theta\ge 0$ is an unknown signal strength. Which statement correctly characterizes the maximum likelihood estimator?
   Answer: It is zero when $\sum_i e_i(Y_i/b_i-1)\le 0$; otherwise it is the positive root of $\sum_i e_i\{Y_i/(b_i+\theta e_i)-1\}=0$. When maximizing a likelihood over a constrained parameter space like $\theta \ge 0$, you must check whether the unconstrained optimum respects the boundary. Here, the log-likelihood is $\ell(\theta) = \sum_i [Y_i \log(b_i + \theta e_i) - (b_i + \theta e_i)]$, and its derivative is $\ell'(\theta) = \sum_i e_i\{Y_i/(b_i + \theta e_i) - 1\}$. Because $\ell$ is concave in $\theta$, the MLE is $\hat\theta = 0$ when $\ell'(0) \le 0$, and otherwise it's the unique positive root of $\ell'(\theta) = 0$. Evaluating at $\theta = 0$ gives $\ell'(0) = \sum_i e_i(Y_i/b_i - 1)$. So the MLE is zero precisely when this quantity is non-positive — which is exactly what A states, making it correct. B flips the inequality sign, claiming the MLE is zero when $\ell'(0) \ge 0$. This is backwards: a non-negative derivative at zero means the likelihood is still increasing, so the optimum lies in the interior, not at the boundary. C proposes a simple closed-form formula analogous to a homogeneous Poisson model. This would only be valid if all $e_i$ were equal and $b_i$ were a common constant — in general, the score equation doesn't collapse to a single balance of totals. D uses an unweighted sum $\sum_i(Y_i - b_i)$ for the boundary condition, omitting the essential $e_i$ weights. It also drops the $e_i$ weights inside the score equation itself, making neither condition match the true derivative. Study tip: For constrained MLEs, always evaluate the score function at the boundary — the sign tells you whether to report the boundary value or solve the interior equation.
8. Observations are independent from a two-component mixture with density $g(x;\theta)=\theta f_1(x)+(1-\theta)f_0(x)$, where $f_0$ and $f_1$ are known and $0<\theta<1$. Define $r_i(\theta)=\theta f_1(x_i)/g(x_i;\theta)$. Which equation must any interior maximum likelihood estimate satisfy?
   Answer: $\theta=\frac{1}{n}\sum_{i=1}^n r_i(\theta)$ Mixture model MLE questions hinge on one core move: differentiating the log-likelihood and setting the score to zero. The log-likelihood is $\ell(\theta) = \sum_{i=1}^n \log[\theta f_1(x_i) + (1-\theta)f_0(x_i)]$. Differentiating with respect to $\theta$ gives $\frac{d\ell}{d\theta} = \sum_{i=1}^n \frac{f_1(x_i) - f_0(x_i)}{g(x_i;\theta)}$. Setting this equal to zero and rearranging: $\sum_{i=1}^n \frac{f_1(x_i)}{g(x_i;\theta)} = \sum_{i=1}^n \frac{f_0(x_i)}{g(x_i;\theta)}$. Now notice that $r_i(\theta) = \theta f_1(x_i)/g(x_i;\theta)$, so $f_1(x_i)/g(x_i;\theta) = r_i(\theta)/\theta$. Substituting and multiplying both sides by $\theta/n$ yields $\theta = \frac{1}{n}\sum_{i=1}^n r_i(\theta)$, confirming D is the correct fixed-point equation for any interior MLE. Choice A resembles a geometric-mean likelihood-ratio condition — it has no derivation from the score equation and conflates the MLE with a simple ratio test statistic. Choice B is tempting because it looks like a posterior mean, but it replaces $g(x_i;\theta)$ in the denominator with $f_1(x_i)+f_0(x_i)$, which ignores $\theta$ entirely and destroys the self-consistency structure. Choice C introduces an extra $f_1(x_i)$ factor in both numerator and denominator with no basis in the score equation — it's a plausible-looking distortion of D. A useful mental anchor: the MLE fixed-point equation for a mixture always has the form "parameter equals average posterior weight," since $r_i(\theta)$ is exactly the posterior probability that observation $i$ came from component $f_1$. Recognizing this EM-style self-consistency structure will help you immediately identify the correct score equation on mixture problems.
9. Independent responses satisfy $Y_i\sim N(\mu,\sigma^2/w_i)$, where all weights $w_i>0$ are known and both $\mu$ and $\sigma^2$ are unknown. Which pair gives their joint maximum likelihood estimators?
   Answer: $\widehat\mu=\frac{\sum_iw_iY_i}{\sum_iw_i},\qquad \widehat\sigma^2=\frac{1}{n}\sum_iw_i(Y_i-\widehat\mu)^2$ When you see a weighted normal model like this, your instinct should be to derive the MLEs directly from the log-likelihood rather than memorizing formulas — the structure of the variance tells you everything. Since $Y_i \sim N(\mu, \sigma^2/w_i)$, the log-likelihood is: $\ell(\mu,\sigma^2) = -\frac{n}{2}\log\sigma^2 - \frac{1}{2\sigma^2}\sum_i w_i(Y_i - \mu)^2 + \text{const}$ Notice that $w_i$ appears because the precision of each observation is $w_i/\sigma^2$. Setting $\partial\ell/\partial\mu = 0$ gives $\sum_i w_i(Y_i - \mu) = 0$, which yields the weighted mean $\widehat\mu = \frac{\sum_i w_i Y_i}{\sum_i w_i}$. Then setting $\partial\ell/\partial\sigma^2 = 0$ gives $\widehat\sigma^2 = \frac{1}{n}\sum_i w_i(Y_i - \widehat\mu)^2$. This is exactly answer A, confirming it as correct. B uses the unweighted sample mean for $\widehat\mu$, ignoring that observations with larger $w_i$ are more precise and deserve more influence — a fundamental misread of the model. C uses the correct $\widehat\mu$ but divides by $n-1$ instead of $n$. That divisor appears in unbiased variance estimation (REML-style), not MLE — MLEs are known to be biased for $\sigma^2$. D inverts the weighting scheme entirely, treating higher-weight observations as less informative, which contradicts the model where $\text{Var}(Y_i) = \sigma^2/w_i$. Study tip: Always write out the log-likelihood explicitly. The weight $w_i$ divides the variance, so it multiplies the squared residual in the exponent — that asymmetry is what separates A from D every time.
10. Event times have a gamma distribution with known shape $2$ and unknown rate $\theta$, with density $f(x;\theta)=\theta^2xe^{-\theta x}$ for $x>0$. Only event times exceeding a fixed threshold $c>0$ are observed, and the sample consists of $n$ independent times $x_i>c$. Which likelihood equation must an interior maximum likelihood estimate satisfy?
    Answer: $\frac{2n}{\theta}-\sum_{i=1}^n(x_i-c)-\frac{nc}{1+\theta c}=0$ When dealing with truncated data, the key insight is that you're not working with the ordinary likelihood — you must condition on the event that each observation exceeds the threshold $c$. This means the conditional density for each observed $x_i > c$ is $f(x;\theta)/P(X > c;\theta)$, making the log-likelihood $\ell(\theta) = \sum \log f(x_i;\theta) - n\log P(X>c;\theta)$. For the Gamma(2, $\theta$) density $f(x;\theta) = \theta^2 x e^{-\theta x}$, the log contribution per observation is $2\log\theta + \log x_i - \theta x_i$. The survival function is $P(X>c;\theta) = e^{-\theta c}(1+\theta c)$, so $\log P(X>c;\theta) = -\theta c + \log(1+\theta c)$. Differentiating the full log-likelihood with respect to $\theta$: $\frac{d\ell}{d\theta} = \frac{2n}{\theta} - \sum x_i - n\left(-c + \frac{c}{1+\theta c}\right) = \frac{2n}{\theta} - \sum x_i + nc - \frac{nc}{1+\theta c}$ Rewriting $-\sum x_i + nc = -\sum(x_i - c)$ confirms the likelihood equation is $\frac{2n}{\theta} - \sum(x_i - c) - \frac{nc}{1+\theta c} = 0$, which is answer A. Answer B incorrectly uses $\sum x_i$ instead of $\sum(x_i - c)$, forgetting to incorporate the $+nc$ term from differentiating $-n(-\theta c)$. Answer C has the wrong sign on the last term, which would arise from differentiating incorrectly. Answer D uses $n/\theta$ instead of $2n/\theta$, reflecting a shape parameter of 1 rather than 2. Your strategy: always write out $\log P(X>c;\theta)$ explicitly and differentiate it carefully — sign errors and forgotten terms from the survival function account for most mistakes on truncated-likelihood problems.

Start this quiz