Likelihood & Score Function - Statistics Graduate Level

1. After correctly normalizing the density, which score and maximum likelihood estimate result?
   Answer: $U(\theta)=\frac{3}{\theta+1}-4,\qquad \widehat\theta=-\frac14$ When you encounter a density "proportional to" some function, your first move must always be to find the normalizing constant — this is what separates options B and C from each other and from the rest. The density is $f(x|\theta) = c \cdot x^\theta$ on $(0,1)$. Integrating: $\int_0^1 x^\theta\,dx = \frac{1}{\theta+1}$, so $c = \theta+1$, giving the proper density $f(x|\theta) = (\theta+1)x^\theta$. With three independent observations, the log-likelihood is: $\ell(\theta) = 3\log(\theta+1) + \theta\sum_{i=1}^3 \log x_i = 3\log(\theta+1) + \theta(-4).$ Differentiating with respect to $\theta$ yields the score function: $U(\theta) = \frac{3}{\theta+1} - 4.$ Setting $U(\hat\theta)=0$: $\frac{3}{\theta+1}=4 \Rightarrow \theta+1=\frac{3}{4} \Rightarrow \hat\theta = -\frac{1}{4}.$ Since $-\frac{1}{4} > -1$, this is a valid interior MLE. This confirms answer B. Answer A ignores the normalizing constant entirely, keeping only the $\theta \sum \log x_i$ term and finding no score equation with a solution — a critical oversight. Answer C writes $\frac{3}{\theta}$ in the score, which would arise if the density were $x^{\theta-1}$ (a power-law with exponent $\theta$, not $\theta+1$ in the denominator) — a normalization error. Answer D gets the score partially right but with the wrong sign on the $-4$ term, producing an equation with no finite solution. Study tip: On likelihood problems, always normalize the density first before writing the log-likelihood — forgetting the normalizing constant is one of the most common errors on graduate-level MLE questions, and it cascades into every subsequent calculation.
2. Which statement correctly gives the effects of these two transformations on the score?
   Answer: $U_\theta^Y(y)=U_\theta(g^{-1}(y))$ and $U_\phi=U_\theta\,d\theta/d\phi$. When you see a question combining data transformations and reparameterizations, tackle each piece independently using the chain rule on log-likelihoods. Data transformation (Y = g(X)): The log-likelihood under Y uses the change-of-variables formula: $\log f_Y(y;\theta) = \log f(g^{-1}(y);\theta) + \log|dg^{-1}/dy|$. Taking $\partial/\partial\theta$, the Jacobian term $\log|dg^{-1}/dy|$ drops out because it contains no $\theta$. So $U_\theta^Y(y) = U_\theta(g^{-1}(y))$ — the score simply evaluates at the back-transformed point. **Reparameterization ($\phi = h(\theta)$):** The log-likelihood is unchanged as a function of the data; only the differentiation variable changes. By the chain rule: $U_\phi = \frac{\partial \log f}{\partial \phi} = \frac{\partial \log f}{\partial \theta}\cdot\frac{d\theta}{d\phi} = U_\theta \cdot \frac{d\theta}{d\phi}$. This confirms C as correct. Why the distractors fail: Choice A gets the data transformation right but inverts the chain rule factor for reparameterization, writing $d\phi/d\theta$ instead of $d\theta/d\phi$ — a classic inversion error. Choice B correctly handles the reparameterization side but wrongly retains the Jacobian term in the score, forgetting that $\log|dg^{-1}/dy|$ is $\theta$-free and vanishes upon differentiation. Choice D compounds errors on both sides: it mishandles the Jacobian sign/form and claims the score is invariant to reparameterization, which contradicts the chain rule. A reliable memory anchor: the score under reparameterization scales by $d\theta/d\phi$ (derivative of the old parameter with respect to the new one), while data transformations leave the score structurally intact because Jacobians carry no parameter information.
3. Which statement correctly describes the constrained maximum likelihood estimate and the score at that estimate?
   Answer: $\widehat p=0.4$ and $U(\widehat p)=25/3>0$, consistent with an upper-bound maximum. When the parameter space is constrained, the MLE lives at whichever point in the feasible region maximizes the likelihood — and that point need not satisfy the score equation $U(p) = 0$. Here, the unconstrained MLE for a binomial is $\widehat{p} = y/n = 6/10 = 0.6$, but $0.6$ violates the constraint $p \le 0.4$. Because the log-likelihood is strictly increasing toward $p = 0.6$, it is still increasing at the boundary $p = 0.4$ within the feasible region. So the constrained MLE is $\widehat{p} = 0.4$ — the closest feasible point to the unconstrained maximizer. Now compute the score at that boundary. The binomial log-likelihood gives: $U(p) = \frac{y}{p} - \frac{n - y}{1 - p} = \frac{6}{0.4} - \frac{4}{0.6} = 15 - \frac{20}{3} = \frac{25}{3} > 0.$ A positive score at the upper bound is geometrically correct: the likelihood is still "pushing upward" at $p = 0.4$, confirming we've been stopped by the constraint, not by a true interior maximum. This confirms B. A is wrong because it assumes all MLEs solve the score equation — that only holds for unconstrained interior maxima. C has the right estimate but the wrong sign; a negative score would indicate the likelihood is decreasing at the boundary, which would mean $p = 0.4$ isn't even the constrained maximum. D simply ignores the constraint entirely. Study tip: At a binding upper-bound constraint, expect a positive score (likelihood still wants to increase); at a binding lower-bound constraint, expect a negative score. The sign tells you which boundary is active.
4. Which pair gives the score function and the maximum likelihood estimate of $\theta$?
   Answer: $U(\theta)=\frac{3}{\theta}-10,\qquad \widehat\theta=\frac{3}{10}$ When you see survival data with exponential lifetimes and right censoring, your first move is to write the likelihood contribution for each subject: $\theta e^{-\theta t_i}$ if the event occurred ($\delta_i=1$), and $e^{-\theta t_i}$ if censored ($\delta_i=0$). Combining these, the log-likelihood is: $\ell(\theta) = \left(\sum_i \delta_i\right)\log\theta - \theta\sum_i t_i = 3\log\theta - 10\theta$ The score function is the derivative with respect to $\theta$: $U(\theta) = \frac{d\ell}{d\theta} = \frac{3}{\theta} - 10$ Setting $U(\theta)=0$ gives $\widehat\theta = \frac{3}{10}$, confirming answer A is correct. Answer B makes the error of substituting $\sum_i t_i - \sum_i \delta_i = 10 - 3 = 7$ as the total exposure, perhaps confusing total observed time with some adjusted quantity — but all observed times, censored or not, contribute to the exposure term. Answer C swaps the roles of $\sum \delta_i$ and $\sum t_i$, placing 10 in the numerator and 3 in the denominator. This would arise from differentiating a backwards log-likelihood and produces an MLE greater than 1, which should immediately signal something is wrong for a rate parameter in this context. Answer D reports the log-likelihood itself ($3\log\theta - 10\theta$) as if it were the score function, then correctly solves for $\widehat\theta$. The MLE is right, but the score function is wrong — don't confuse $\ell(\theta)$ with $U(\theta) = \ell'(\theta)$. Study tip: Always distinguish the log-likelihood from its derivative. On survival analysis questions, remember that every subject's observed time $t_i$ enters the exponent of the likelihood, regardless of censoring status.
5. What are the efficient score for $\psi$ after adjustment for $\lambda$ and its efficient information, respectively?
   Answer: $U_{\psi\cdot\lambda}=2,\qquad I_{\psi\cdot\lambda}=\frac72$ When a model has a nuisance parameter, you need the efficient score and efficient information for the parameter of interest — these are the building blocks of the score test after profiling out nuisance parameters. Partition the information matrix as $I_{\psi\psi}=4$, $I_{\psi\lambda}=I_{\lambda\psi}=1$, and $I_{\lambda\lambda}=2$. The efficient (adjusted) score is obtained by removing from $U_\psi$ the component explained by $U_\lambda$: $U_{\psi\cdot\lambda} = U_\psi - \frac{I_{\psi\lambda}}{I_{\lambda\lambda}}U_\lambda = 3 - \frac{1}{2}(2) = 3 - 1 = 2.$ The efficient information is the Schur complement of $I_{\lambda\lambda}$ in $I$: $I_{\psi\cdot\lambda} = I_{\psi\psi} - \frac{I_{\psi\lambda}^2}{I_{\lambda\lambda}} = 4 - \frac{1}{2} = \frac{7}{2}.$ This confirms answer C. Answer A gets the efficient score right ($U_{\psi\cdot\lambda}=2$) but miscalculates the efficient information as $9/2$ — a common error from adding rather than subtracting the adjustment term ($4 + 1/2$ instead of $4 - 1/2$). Answer B makes the opposite score error: computing $3 + 1 = 4$ instead of $3 - 1 = 2$, while coincidentally arriving at the correct efficient information formula but with a wrong numerator. Answer D simply takes the raw score and raw diagonal information with no adjustment at all, ignoring the nuisance parameter entirely. Study tip: Memorize the two-step pattern — the efficient score subtracts a regression-type correction from $U_\psi$, and the efficient information is the Schur complement. Both formulas share the same ratio $I_{\psi\lambda}/I_{\lambda\lambda}$, so compute it once and reuse it.
6. What is the profile score $d\ell_p(\mu)/d\mu$ at $\mu=1.5$?
   Answer: $\frac{16}{5}$, using the likelihood variance estimate with divisor $8$. When you encounter a profile likelihood question, your first move is to derive the profile log-likelihood by substituting the MLE of the nuisance parameter—here $\sigma^2$—as a function of the parameter of interest $\mu$. For normal data, the MLE of $\sigma^2$ at fixed $\mu$ is $\hat{\sigma}^2(\mu) = \frac{1}{n}\sum_{i=1}^8(X_i - \mu)^2$. Substituting this into the log-likelihood and simplifying gives the profile log-likelihood: $\ell_p(\mu) = -\frac{n}{2}\log\!\left(\frac{1}{n}\sum_{i=1}^8(X_i-\mu)^2\right) + \text{const}$ Differentiating with respect to $\mu$: $\frac{d\ell_p}{d\mu} = \frac{n \cdot \sum_{i=1}^8(X_i-\mu)}{\sum_{i=1}^8(X_i-\mu)^2}$ At $\mu = 1.5$: the numerator is $8 \cdot \sum(X_i - 1.5) = 8 \cdot n(\bar{X}-\mu) = 8 \cdot 8(2-1.5) = 8 \cdot 4 = 32$, and the denominator is $\sum(X_i-1.5)^2 = 10$, giving $32/10 = \mathbf{16/5}$. This confirms answer A. Answer B is wrong because dividing by 7 (the unbiased estimator) is never used in maximum likelihood—the MLE of $\sigma^2$ always uses divisor $n$, and this propagates into the profile score. Answer C is wrong because it uses only $(\bar{X}-\mu) = 0.5$ divided by something, ignoring that the score involves the full sum $n(\bar{X}-\mu)$, not just the mean residual alone. Answer D is wrong because the sign is positive: the score $\propto \sum(X_i - \mu)$, which is positive when $\bar{X} > \mu$, as it is here ($2 > 1.5$). Study tip: Always derive the profile score by differentiating after substituting the MLE of the nuisance parameter—never mix in unbiased estimators or forget the factor of $n$ in the numerator.
7. Which comparison of the likelihoods for the two observed outcomes is correct?
   Answer: They differ only by a factor independent of $p$, so both have score $4/p-6/(1-p)$ and MLE $0.4$. Whenever you see a question involving stopping rules and likelihood, your first instinct should be to ask: *does the stopping rule affect the likelihood function as a function of $p$?* This is the heart of the likelihood principle. In the fixed-sample experiment, the likelihood is $L_1(p) = \binom{10}{4}p^4(1-p)^6$. In the negative binomial experiment (stop at the 4th success), the last trial must be a success, so you're arranging the first 3 successes among trials 1–9, giving $L_2(p) = \binom{9}{3}p^4(1-p)^6$. Both likelihoods share the same kernel $p^4(1-p)^6$; they differ only by the constants $\binom{10}{4} = 210$ and $\binom{9}{3} = 84$, which do not depend on $p$. Since inference is based on the likelihood up to a multiplicative constant, these two outcomes yield identical inferential content. The score (log-likelihood derivative) is $\frac{d}{dp}\ln[p^4(1-p)^6] = \frac{4}{p} - \frac{6}{1-p}$, and setting this to zero gives MLE $\hat{p} = 0.4$ in both cases. This confirms D. Choice A is wrong because it claims the score loses a success term due to the fixed last trial — but the last success still contributes to the likelihood kernel, so the full $4/p$ term remains. Choice B is wrong because it claims the combinatorial factors are identical, when in fact they differ ($\binom{10}{4} \neq \binom{9}{3}$); the point is they're both irrelevant constants. Choice C is wrong because stopping rules never change the MLE when the likelihood kernels match — the MLE is still $0.4$. Remember: the likelihood principle states that two likelihoods proportional in $p$ carry identical statistical evidence. Stopping rules only affect the normalizing constant, not inference.
8. What is the score vector for $(\beta_0,\beta_1)$ at $(\beta_0,\beta_1)=(0,0)$?
   Answer: $U(0,0)=\left(\frac12,\frac12\right)^{\mathsf T}$, using residuals $y_i-p_i$. When you see a question about the score vector in logistic regression, think: the score is the gradient of the log-likelihood, and for logistic regression it takes the beautifully clean form $U(\boldsymbol{\beta}) = \mathbf{X}^\mathsf{T}(\mathbf{y} - \mathbf{p})$, where $\mathbf{p}$ contains the fitted probabilities and $\mathbf{y}$ the observed responses. At $(\beta_0, \beta_1) = (0,0)$, every linear predictor equals zero, so every fitted probability is $\text{logit}^{-1}(0) = \frac{1}{2}$. The residuals $y_i - p_i$ for the three observations are: $1 - \tfrac{1}{2} = \tfrac{1}{2}$, $0 - \tfrac{1}{2} = -\tfrac{1}{2}$, and $1 - \tfrac{1}{2} = \tfrac{1}{2}$. The design matrix is $\mathbf{X} = \begin{pmatrix}1 & -1 \\ 1 & 0 \\ 1 & 2\end{pmatrix}$. Multiplying $\mathbf{X}^\mathsf{T}(\mathbf{y}-\mathbf{p})$ gives $U_{\beta_0} = \tfrac{1}{2} - \tfrac{1}{2} + \tfrac{1}{2} = \tfrac{1}{2}$ and $U_{\beta_1} = (-1)(\tfrac{1}{2}) + (0)(-\tfrac{1}{2}) + (2)(\tfrac{1}{2}) = \tfrac{1}{2}$. This confirms D is correct. Choice A ignores the fitted probabilities entirely, summing raw $y_i$ values — this confuses the score with a simple count statistic. Choice B flips the sign, computing $p_i - y_i$ instead of $y_i - p_i$; the score uses observed minus fitted, not the reverse. Choice C somehow drops the negative covariate contribution from observation one, an arithmetic error that produces the wrong $U_{\beta_1}$. Your study tip: memorize $U(\boldsymbol{\beta}) = \mathbf{X}^\mathsf{T}(\mathbf{y} - \mathbf{p})$ as a single formula. Sign errors and forgetting to compute $\mathbf{p}$ are the two most common traps on score-vector questions.
9. What is the score with respect to $\eta$, evaluated at $\eta_0$?
   Answer: $U_\eta(\eta_0)=-1$, after applying the derivative of the logistic transformation. Whenever you encounter a score function question involving a reparameterization, your first instinct should be the chain rule: the score with respect to a transformed parameter equals the score on the original scale multiplied by the derivative of the transformation. The log-likelihood for a binomial in terms of $p$ is $\ell(p) = y\log p + (n-y)\log(1-p)$, giving score $U_p = \frac{y}{p} - \frac{n-y}{1-p} = \frac{y - np}{p(1-p)}$. At $p_0 = 0.4$, this equals $\frac{7 - 20(0.4)}{0.4 \cdot 0.6} = \frac{-1}{0.24} = -\frac{25}{6}$. Now apply the chain rule: $U_\eta = U_p \cdot \frac{dp}{d\eta}$. For the logistic link, $\frac{dp}{d\eta} = p(1-p)$, which at $p_0$ equals $0.4 \times 0.6 = 0.24 = \frac{6}{25}$. Therefore, $U_\eta = -\frac{25}{6} \cdot \frac{6}{25} = -1$. Answer A is correct. Answer B gives $-25/6$, which is actually $U_p$, the score on the probability scale — it skips the crucial chain rule multiplication step. Answer C claims $+1$, simply reversing the sign of the correct answer, reflecting a common error in setting up observed-minus-expected (it should be $y - np$, not $np - y$). Answer D gives $-6/25$, which is the derivative $dp/d\eta$ alone — it confuses the Jacobian factor with the score itself. Your key takeaway: reparameterized scores always require the chain rule. A useful check is that $U_\eta = (y - np)$, a beautifully clean result for logistic models that's worth memorizing.

Start this quiz