Statistics Graduate Level Quiz: Likelihood And Score Function

9 questions · exam conditions

0:00

Likelihood And Score FunctionQuestion 1 of 9

Independent subjects have exponential event times with rate $\theta>0$ and are subject to independent right censoring. For each subject, the observed time is $t_i$ and the event indicator is $\delta_i$ . In one study, $\sum_i \delta_i=3$ and $\sum_i t_i=10$ .

Which pair gives the score function and the maximum likelihood estimate of $\theta$ ?

U(\theta)=\frac{3}{\theta}-10,\qquad \widehat\theta=\frac{3}{10}

U(\theta)=\frac{3}{\theta}-7,\qquad \widehat\theta=\frac{3}{7}

U(\theta)=\frac{10}{\theta}-3,\qquad \widehat\theta=\frac{10}{3}

U(\theta)=3\log\theta-10\theta,\qquad \widehat\theta=\frac{3}{10}

← Back to quizzes

Statistics Graduate Level Quiz

Statistics Graduate Level Quiz: Likelihood And Score Function

Practice Likelihood And Score Function in Statistics Graduate Level with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Likelihood And Score Function, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics Graduate Level.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

Which pair gives the score function and the maximum likelihood estimate of $\theta$ ?

$U(\theta)=\frac{3}{\theta}-10,\qquad \widehat\theta=\frac{3}{10}$ (correct answer)
$U(\theta)=\frac{3}{\theta}-7,\qquad \widehat\theta=\frac{3}{7}$
$U(\theta)=\frac{10}{\theta}-3,\qquad \widehat\theta=\frac{10}{3}$
$U(\theta)=3\log\theta-10\theta,\qquad \widehat\theta=\frac{3}{10}$

Explanation: When you see survival data with exponential lifetimes and right censoring, your first move is to write the likelihood contribution for each subject:

\theta e^{-\theta t_i}

if the event occurred (

\delta_i=1

), and

e^{-\theta t_i}

if censored (

\delta_i=0

). Combining these, the log-likelihood is:

\ell(\theta) = \left(\sum_i \delta_i\right)\log\theta - \theta\sum_i t_i = 3\log\theta - 10\theta

The score function is the derivative with respect to

\theta

U(\theta) = \frac{d\ell}{d\theta} = \frac{3}{\theta} - 10

Setting

U(\theta)=0

gives

\widehat\theta = \frac{3}{10}

, confirming answer A is correct. Answer B makes the error of substituting

\sum_i t_i - \sum_i \delta_i = 10 - 3 = 7

as the total exposure, perhaps confusing total observed time with some adjusted quantity — but all observed times, censored or not, contribute to the exposure term. Answer C swaps the roles of

\sum \delta_i

and

\sum t_i

, placing 10 in the numerator and 3 in the denominator. This would arise from differentiating a backwards log-likelihood and produces an MLE greater than 1, which should immediately signal something is wrong for a rate parameter in this context. Answer D reports the log-likelihood itself (

3\log\theta - 10\theta

) as if it were the score function, then correctly solves for

\widehat\theta

. The MLE is right, but the score function is wrong — don't confuse

\ell(\theta)

with

U(\theta) = \ell'(\theta)

. Study tip: Always distinguish the log-likelihood from its derivative. On survival analysis questions, remember that every subject's observed time

t_i

enters the exponent of the likelihood, regardless of censoring status.

Question 2

In one experiment, a fixed sample of $10$ Bernoulli trials produces exactly $4$ successes. In another experiment, sampling stops when the fourth success occurs, and the fourth success is observed on trial $10$ . In both experiments the success probability is $p$ .

Which comparison of the likelihoods for the two observed outcomes is correct?

They have proportional likelihoods, but the second experiment has score $3/p-6/(1-p)$ because the last success is fixed.
They are numerically identical likelihoods, including their combinatorial factors, so both have score $4/p-6/(1-p)$ .
The stopping rule changes the score in the second experiment, giving its MLE as $4/9$ rather than $0.4$ .
They differ only by a factor independent of $p$ , so both have score $4/p-6/(1-p)$ and MLE $0.4$ . (correct answer)

Explanation: Whenever you see a question involving stopping rules and likelihood, your first instinct should be to ask: does the stopping rule affect the likelihood function as a function of $p$ ? This is the heart of the likelihood principle. In the fixed-sample experiment, the likelihood is

L_1(p) = \binom{10}{4}p^4(1-p)^6

. In the negative binomial experiment (stop at the 4th success), the last trial must be a success, so you're arranging the first 3 successes among trials 1–9, giving

L_2(p) = \binom{9}{3}p^4(1-p)^6

. Both likelihoods share the same kernel

p^4(1-p)^6

; they differ only by the constants

\binom{10}{4} = 210

and

\binom{9}{3} = 84

, which do not depend on

p

. Since inference is based on the likelihood up to a multiplicative constant, these two outcomes yield identical inferential content. The score (log-likelihood derivative) is

\frac{d}{dp}\ln[p^4(1-p)^6] = \frac{4}{p} - \frac{6}{1-p}

, and setting this to zero gives MLE

\hat{p} = 0.4

in both cases. This confirms D. Choice A is wrong because it claims the score loses a success term due to the fixed last trial — but the last success still contributes to the likelihood kernel, so the full

4/p

term remains. Choice B is wrong because it claims the combinatorial factors are identical, when in fact they differ (

\binom{10}{4} \neq \binom{9}{3}

); the point is they're both irrelevant constants. Choice C is wrong because stopping rules never change the MLE when the likelihood kernels match — the MLE is still

0.4

. Remember: the likelihood principle states that two likelihoods proportional in

p

carry identical statistical evidence. Stopping rules only affect the normalizing constant, not inference.

Question 3

For three independent binary responses, a logistic regression model specifies $\operatorname{logit}(p_i)=\beta_0+\beta_1x_i$ . The observed pairs $(x_i,y_i)$ are $(-1,1)$ , $(0,0)$ , and $(2,1)$ .

What is the score vector for $(\beta_0,\beta_1)$ at $(\beta_0,\beta_1)=(0,0)$ ?

$U(0,0)=\left(2,1\right)^{\mathsf T}$ , using observed successes without fitted probabilities.
$U(0,0)=\left(-\frac12,-\frac12\right)^{\mathsf T}$ , using residuals $p_i-y_i$ .
$U(0,0)=\left(\frac12,\frac32\right)^{\mathsf T}$ , omitting the negative covariate contribution.
$U(0,0)=\left(\frac12,\frac12\right)^{\mathsf T}$ , using residuals $y_i-p_i$ . (correct answer)

Explanation: When you see a question about the score vector in logistic regression, think: the score is the gradient of the log-likelihood, and for logistic regression it takes the beautifully clean form

U(\boldsymbol{\beta}) = \mathbf{X}^\mathsf{T}(\mathbf{y} - \mathbf{p})

, where

\mathbf{p}

contains the fitted probabilities and

\mathbf{y}

the observed responses. At

(\beta_0, \beta_1) = (0,0)

, every linear predictor equals zero, so every fitted probability is

\text{logit}^{-1}(0) = \frac{1}{2}

. The residuals

y_i - p_i

for the three observations are:

1 - \tfrac{1}{2} = \tfrac{1}{2}

0 - \tfrac{1}{2} = -\tfrac{1}{2}

, and

1 - \tfrac{1}{2} = \tfrac{1}{2}

. The design matrix is

\mathbf{X} = \begin{pmatrix}1 & -1 \\ 1 & 0 \\ 1 & 2\end{pmatrix}

. Multiplying

\mathbf{X}^\mathsf{T}(\mathbf{y}-\mathbf{p})

gives

U_{\beta_0} = \tfrac{1}{2} - \tfrac{1}{2} + \tfrac{1}{2} = \tfrac{1}{2}

and

U_{\beta_1} = (-1)(\tfrac{1}{2}) + (0)(-\tfrac{1}{2}) + (2)(\tfrac{1}{2}) = \tfrac{1}{2}

. This confirms D is correct. Choice A ignores the fitted probabilities entirely, summing raw

y_i

values — this confuses the score with a simple count statistic. Choice B flips the sign, computing

p_i - y_i

instead of

y_i - p_i

; the score uses observed minus fitted, not the reverse. Choice C somehow drops the negative covariate contribution from observation one, an arithmetic error that produces the wrong

U_{\beta_1}

. Your study tip: memorize

U(\boldsymbol{\beta}) = \mathbf{X}^\mathsf{T}(\mathbf{y} - \mathbf{p})

as a single formula. Sign errors and forgetting to compute

\mathbf{p}

are the two most common traps on score-vector questions.

Question 4

A model has scalar parameter of interest $\theta$ . An observation $X$ has score $U_\theta(x)=\partial\log f(x;\theta)/\partial\theta$ . The data are transformed by a one-to-one, differentiable mapping $Y=g(X)$ that does not depend on $\theta$ . Separately, the parameter is reparameterized as $\phi=h(\theta)$ , where $h$ is differentiable with differentiable inverse.

Which statement correctly gives the effects of these two transformations on the score?

$U_\theta^Y(y)=U_\theta(g^{-1}(y))$ and $U_\phi=U_\theta\,d\phi/d\theta$ .
$U_\theta^Y(y)=U_\theta(g^{-1}(y))+d\log|dg^{-1}/dy|/d\theta$ and $U_\phi=U_\theta\,d\phi/d\theta$ .
$U_\theta^Y(y)=U_\theta(g^{-1}(y))$ and $U_\phi=U_\theta\,d\theta/d\phi$ . (correct answer)
$U_\theta^Y(y)=U_\theta(g^{-1}(y))-d\log|dg/dx|/d\theta$ and $U_\phi=U_\theta$ .

Explanation: When you see a question combining data transformations and reparameterizations, tackle each piece independently using the chain rule on log-likelihoods. Data transformation (Y = g(X)): The log-likelihood under Y uses the change-of-variables formula:

\log f_Y(y;\theta) = \log f(g^{-1}(y);\theta) + \log|dg^{-1}/dy|

. Taking

\partial/\partial\theta

, the Jacobian term

\log|dg^{-1}/dy|

drops out because it contains no

\theta

. So

U_\theta^Y(y) = U_\theta(g^{-1}(y))

— the score simply evaluates at the back-transformed point. Reparameterization ( $\phi = h(\theta)$ ): The log-likelihood is unchanged as a function of the data; only the differentiation variable changes. By the chain rule:

U_\phi = \frac{\partial \log f}{\partial \phi} = \frac{\partial \log f}{\partial \theta}\cdot\frac{d\theta}{d\phi} = U_\theta \cdot \frac{d\theta}{d\phi}

. This confirms C as correct. Why the distractors fail: Choice A gets the data transformation right but inverts the chain rule factor for reparameterization, writing

d\phi/d\theta

instead of

d\theta/d\phi

— a classic inversion error. Choice B correctly handles the reparameterization side but wrongly retains the Jacobian term in the score, forgetting that

\log|dg^{-1}/dy|

\theta

-free and vanishes upon differentiation. Choice D compounds errors on both sides: it mishandles the Jacobian sign/form and claims the score is invariant to reparameterization, which contradicts the chain rule. A reliable memory anchor: the score under reparameterization scales by

d\theta/d\phi

(derivative of the old parameter with respect to the new one), while data transformations leave the score structurally intact because Jacobians carry no parameter information.

Question 5

A binomial sample has $n=10$ and $y=6$ , but scientific constraints restrict the success probability to $0<p\le 0.4$ .

Which statement correctly describes the constrained maximum likelihood estimate and the score at that estimate?

$\widehat p=0.4$ and $U(\widehat p)=0$ , because every constrained MLE solves the score equation.
$\widehat p=0.4$ and $U(\widehat p)=25/3>0$ , consistent with an upper-bound maximum. (correct answer)
$\widehat p=0.4$ and $U(\widehat p)=-25/3<0$ , consistent with an upper-bound maximum.
$\widehat p=0.6$ and $U(\widehat p)=0$ , because the unconstrained score root remains valid.

Explanation: When the parameter space is constrained, the MLE lives at whichever point in the feasible region maximizes the likelihood — and that point need not satisfy the score equation

U(p) = 0

. Here, the unconstrained MLE for a binomial is

\widehat{p} = y/n = 6/10 = 0.6

, but

0.6

violates the constraint

p \le 0.4

. Because the log-likelihood is strictly increasing toward

p = 0.6

, it is still increasing at the boundary

p = 0.4

within the feasible region. So the constrained MLE is

\widehat{p} = 0.4

— the closest feasible point to the unconstrained maximizer. Now compute the score at that boundary. The binomial log-likelihood gives:

U(p) = \frac{y}{p} - \frac{n - y}{1 - p} = \frac{6}{0.4} - \frac{4}{0.6} = 15 - \frac{20}{3} = \frac{25}{3} > 0.

A positive score at the upper bound is geometrically correct: the likelihood is still "pushing upward" at

p = 0.4

, confirming we've been stopped by the constraint, not by a true interior maximum. This confirms B. A is wrong because it assumes all MLEs solve the score equation — that only holds for unconstrained interior maxima. C has the right estimate but the wrong sign; a negative score would indicate the likelihood is decreasing at the boundary, which would mean

p = 0.4

isn't even the constrained maximum. D simply ignores the constraint entirely. Study tip: At a binding upper-bound constraint, expect a positive score (likelihood still wants to increase); at a binding lower-bound constraint, expect a negative score. The sign tells you which boundary is active.

Question 6

A proposed family on $0<x<1$ has density proportional to $x^\theta$ , where $\theta>-1$ . For three independent observations, the data satisfy $\sum_{i=1}^3\log x_i=-4$ .

After correctly normalizing the density, which score and maximum likelihood estimate result?

$U(\theta)=-4,\qquad \widehat\theta\text{ has no interior value}$
$U(\theta)=\frac{3}{\theta+1}-4,\qquad \widehat\theta=-\frac14$ (correct answer)
$U(\theta)=\frac{3}{\theta}-4,\qquad \widehat\theta=\frac34$
$U(\theta)=\frac{3}{\theta+1}+4,\qquad \widehat\theta\text{ has no finite value}$

Explanation: When you encounter a density "proportional to" some function, your first move must always be to find the normalizing constant — this is what separates options B and C from each other and from the rest. The density is

f(x|\theta) = c \cdot x^\theta

(0,1)

. Integrating:

\int_0^1 x^\theta\,dx = \frac{1}{\theta+1}

, so

c = \theta+1

, giving the proper density

f(x|\theta) = (\theta+1)x^\theta

. With three independent observations, the log-likelihood is:

\ell(\theta) = 3\log(\theta+1) + \theta\sum_{i=1}^3 \log x_i = 3\log(\theta+1) + \theta(-4).

Differentiating with respect to

\theta

yields the score function:

U(\theta) = \frac{3}{\theta+1} - 4.

Setting

U(\hat\theta)=0

\frac{3}{\theta+1}=4 \Rightarrow \theta+1=\frac{3}{4} \Rightarrow \hat\theta = -\frac{1}{4}.

Since

-\frac{1}{4} > -1

, this is a valid interior MLE. This confirms answer B. Answer A ignores the normalizing constant entirely, keeping only the

\theta \sum \log x_i

term and finding no score equation with a solution — a critical oversight. Answer C writes

\frac{3}{\theta}

in the score, which would arise if the density were

x^{\theta-1}

(a power-law with exponent

\theta

, not

\theta+1

in the denominator) — a normalization error. Answer D gets the score partially right but with the wrong sign on the

-4

term, producing an equation with no finite solution. Study tip: On likelihood problems, always normalize the density first before writing the log-likelihood — forgetting the normalizing constant is one of the most common errors on graduate-level MLE questions, and it cascades into every subsequent calculation.

Question 7

A model has scalar parameter of interest $\psi$ and scalar nuisance parameter $\lambda$ . At the null parameter value, the score vector is $(U_\psi,U_\lambda)^{\mathsf T}=(3,2)^{\mathsf T}$ and the expected information matrix is $$I=\begin{pmatrix}4&1\1&2\end{pmatrix}

What are the efficient score for $\psi$ after adjustment for $\lambda$ and its efficient information, respectively?

$U_{\psi\cdot\lambda}=2,\qquad I_{\psi\cdot\lambda}=\frac92$
$U_{\psi\cdot\lambda}=4,\qquad I_{\psi\cdot\lambda}=\frac72$
$U_{\psi\cdot\lambda}=2,\qquad I_{\psi\cdot\lambda}=\frac72$ (correct answer)
$U_{\psi\cdot\lambda}=3,\qquad I_{\psi\cdot\lambda}=4$

Explanation: When a model has a nuisance parameter, you need the efficient score and efficient information for the parameter of interest — these are the building blocks of the score test after profiling out nuisance parameters. Partition the information matrix as

I_{\psi\psi}=4

I_{\psi\lambda}=I_{\lambda\psi}=1

, and

I_{\lambda\lambda}=2

. The efficient (adjusted) score is obtained by removing from

U_\psi

the component explained by

U_\lambda

U_{\psi\cdot\lambda} = U_\psi - \frac{I_{\psi\lambda}}{I_{\lambda\lambda}}U_\lambda = 3 - \frac{1}{2}(2) = 3 - 1 = 2.

The efficient information is the Schur complement of

I_{\lambda\lambda}

I

I_{\psi\cdot\lambda} = I_{\psi\psi} - \frac{I_{\psi\lambda}^2}{I_{\lambda\lambda}} = 4 - \frac{1}{2} = \frac{7}{2}.

This confirms answer C. Answer A gets the efficient score right (

U_{\psi\cdot\lambda}=2

) but miscalculates the efficient information as

9/2

— a common error from adding rather than subtracting the adjustment term (

4 + 1/2

instead of

4 - 1/2

). Answer B makes the opposite score error: computing

3 + 1 = 4

instead of

3 - 1 = 2

, while coincidentally arriving at the correct efficient information formula but with a wrong numerator. Answer D simply takes the raw score and raw diagonal information with no adjustment at all, ignoring the nuisance parameter entirely. Study tip: Memorize the two-step pattern — the efficient score subtracts a regression-type correction from

U_\psi

, and the efficient information is the Schur complement. Both formulas share the same ratio

I_{\psi\lambda}/I_{\lambda\lambda}

, so compute it once and reuse it.

Question 8

Suppose $X_1,\ldots,X_8$ are independent normal observations with unknown mean $\mu$ and unknown variance $\sigma^2$ . At $\mu=1.5$ , the sample mean is $\overline X=2$ and $\sum_{i=1}^8(X_i-1.5)^2=10$ . Let $\ell_p(\mu)$ denote the log-likelihood after maximizing over $\sigma^2$ for each fixed $\mu$ .

What is the profile score $d\ell_p(\mu)/d\mu$ at $\mu=1.5$ ?

$\frac{16}{5}$ , using the likelihood variance estimate with divisor $8$ . (correct answer)
$\frac{14}{5}$ , using the unbiased variance estimate with divisor $7$ .
$\frac{2}{5}$ , using only the mean residual rather than its sum.
$-\frac{16}{5}$ , using the residual sign in the reverse direction.

Explanation: When you encounter a profile likelihood question, your first move is to derive the profile log-likelihood by substituting the MLE of the nuisance parameter—here

\sigma^2

—as a function of the parameter of interest

\mu

. For normal data, the MLE of

\sigma^2

at fixed

\mu

\hat{\sigma}^2(\mu) = \frac{1}{n}\sum_{i=1}^8(X_i - \mu)^2

. Substituting this into the log-likelihood and simplifying gives the profile log-likelihood:

\ell_p(\mu) = -\frac{n}{2}\log\!\left(\frac{1}{n}\sum_{i=1}^8(X_i-\mu)^2\right) + \text{const}

Differentiating with respect to

\mu

\frac{d\ell_p}{d\mu} = \frac{n \cdot \sum_{i=1}^8(X_i-\mu)}{\sum_{i=1}^8(X_i-\mu)^2}

\mu = 1.5

: the numerator is

8 \cdot \sum(X_i - 1.5) = 8 \cdot n(\bar{X}-\mu) = 8 \cdot 8(2-1.5) = 8 \cdot 4 = 32

, and the denominator is

\sum(X_i-1.5)^2 = 10

, giving

32/10 = \mathbf{16/5}

. This confirms answer A. Answer B is wrong because dividing by 7 (the unbiased estimator) is never used in maximum likelihood—the MLE of

\sigma^2

always uses divisor

n

, and this propagates into the profile score. Answer C is wrong because it uses only

(\bar{X}-\mu) = 0.5

divided by something, ignoring that the score involves the full sum

n(\bar{X}-\mu)

, not just the mean residual alone. Answer D is wrong because the sign is positive: the score

\propto \sum(X_i - \mu)

, which is positive when

\bar{X} > \mu

, as it is here (

2 > 1.5

). Study tip: Always derive the profile score by differentiating after substituting the MLE of the nuisance parameter—never mix in unbiased estimators or forget the factor of

n

in the numerator.

Question 9

A binomial observation has $n=20$ trials and $y=7$ successes. The success probability is parameterized as $p=e^\eta/(1+e^\eta)$ . Consider the value $\eta_0=\log(2/3)$ , for which $p_0=0.4$ .

What is the score with respect to $\eta$ , evaluated at $\eta_0$ ?

$U_\eta(\eta_0)=-1$ , after applying the derivative of the logistic transformation. (correct answer)
$U_\eta(\eta_0)=-\frac{25}{6}$ , which is the score on the probability scale.
$U_\eta(\eta_0)=1$ , after reversing the observed-minus-expected sign.
$U_\eta(\eta_0)=-\frac{6}{25}$ , after multiplying by the Bernoulli variance twice.

Explanation: Whenever you encounter a score function question involving a reparameterization, your first instinct should be the chain rule: the score with respect to a transformed parameter equals the score on the original scale multiplied by the derivative of the transformation. The log-likelihood for a binomial in terms of

p

\ell(p) = y\log p + (n-y)\log(1-p)

, giving score

U_p = \frac{y}{p} - \frac{n-y}{1-p} = \frac{y - np}{p(1-p)}

. At

p_0 = 0.4

, this equals

\frac{7 - 20(0.4)}{0.4 \cdot 0.6} = \frac{-1}{0.24} = -\frac{25}{6}

. Now apply the chain rule:

U_\eta = U_p \cdot \frac{dp}{d\eta}

. For the logistic link,

\frac{dp}{d\eta} = p(1-p)

, which at

p_0

equals

0.4 \times 0.6 = 0.24 = \frac{6}{25}

. Therefore,

U_\eta = -\frac{25}{6} \cdot \frac{6}{25} = -1

. Answer A is correct. Answer B gives

-25/6

, which is actually

U_p

, the score on the probability scale — it skips the crucial chain rule multiplication step. Answer C claims

+1

, simply reversing the sign of the correct answer, reflecting a common error in setting up observed-minus-expected (it should be

y - np

, not

np - y

). Answer D gives

-6/25

, which is the derivative

dp/d\eta

alone — it confuses the Jacobian factor with the score itself. Your key takeaway: reparameterized scores always require the chain rule. A useful check is that

U_\eta = (y - np)

, a beautifully clean result for logistic models that's worth memorizing.