Likelihood & Score Function — Likelihood, log-likelihood, and score function - Statistics Graduate Level

SECTION 1

Historical Context & Motivation

Statistical inference confronts a deceptively simple question: given a sample of observed data, what can we say about the mechanism that generated it? Before the twentieth century, methods for answering this question were largely ad hoc—least squares for regression, the method of moments for distribution fitting—but no unified principle connected them. The introduction of the likelihood function provided exactly that unifying framework, reorienting the entire enterprise of parametric inference around a single, elegant object that measures the plausibility of a parameter value in light of observed data.

The need for such a framework grew acute as scientists in fields ranging from genetics to astronomy demanded principled methods for combining observations. Early contributions by Carl Friedrich Gauss and Pierre-Simon Laplace laid important groundwork through their work on inverse probability, but these approaches relied on prior distributions in ways that many found unsatisfying. It was R. A. Fisher who, starting in the 1920s, distilled the concept of likelihood into a formal principle that avoided the need for priors altogether, creating a cornerstone of frequentist inference and opening the door to maximum likelihood estimation, likelihood ratio tests, and the modern theory of efficient estimation.

1809 • Gauss & Least Squares

Gauss publishes Theoria Motus, using what amounts to maximizing the joint density of errors to justify least squares. Though he did not use the term 'likelihood,' his reasoning contained its essential logic.

1912 • Fisher's Early Insight

As an undergraduate at Cambridge, Ronald Aylmer Fisher circulates a manuscript distinguishing clearly between probability (forward, from model to data) and likelihood (backward, from data to model), a conceptual watershed.

1922 • Foundations of Mathematical Statistics

Fisher's landmark paper formally defines the likelihood function and proposes maximum likelihood estimation (MLE) as a general method, establishing consistency, efficiency, and sufficiency as desirable properties of estimators.

1948 • Information Theory Meets Likelihood

Claude Shannon's information theory provides new language for the score function and Fisher information, connecting the curvature of the log-likelihood to the precision of estimation.

1970s–present • Computational Revolution

The EM algorithm, MCMC methods, and modern optimization make likelihood-based inference feasible for complex, high-dimensional models—from mixture models to deep generative networks—validating Fisher's framework far beyond its original scope.

The central question this lesson addresses is: how do we formalize the relationship between observed data and unknown parameters, and how do the likelihood, log-likelihood, and score function work together to produce optimal estimators and powerful tests?

SECTION 2

Core Principles & Definitions

Understanding likelihood-based inference requires internalizing a subtle but crucial reversal of perspective. In probability, we fix a model (parameter values) and ask about the distribution of data. In likelihood, we fix the data and ask which parameter values are most compatible with what we actually observed. This reversal is not merely philosophical—it reshapes the entire mathematical apparatus we deploy. The following core principles constitute the foundation upon which maximum likelihood estimation, likelihood ratio tests, and score-based inference are built.

Likelihood Function L(θ)

Given observed data x, the likelihood function L(θ) = f(x | θ) treats the joint density or mass function as a function of the parameter θ, with x held fixed. It measures how plausible each θ is, though it is not itself a probability distribution over θ.

Log-Likelihood ℓ(θ)

The natural logarithm of the likelihood, ℓ(θ) = ln L(θ), converts products into sums and is analytically and computationally far more tractable. Because the logarithm is a strictly monotone transformation, maximizing ℓ(θ) yields the same estimator as maximizing L(θ).

Score Function S(θ)

The score function S(θ) = ∂ℓ(θ)/∂θ is the gradient of the log-likelihood. At the true parameter value θ₀, the score has zero mean and its variance equals the Fisher information. Setting S(θ) = 0 yields the maximum likelihood estimator.

Fisher Information I(θ)

The Fisher information I(θ) = E[S(θ)²] = −E[∂²ℓ/∂θ²] quantifies the curvature of the log-likelihood surface. Greater curvature means sharper peaks, implying the data carry more information about θ and the MLE will be more precise.

Likelihood Principle

All evidence about θ contained in the data is captured by the likelihood function. Two experiments yielding the same likelihood function (up to a constant) should lead to identical inferences—a principle that deeply shapes Bayesian and evidential reasoning.

✦

KEY TAKEAWAY

Think of the likelihood function as a landscape over parameter space. Each point on the landscape represents how well that parameter value 'explains' your data. The log-likelihood is the same landscape on a log scale—easier to work with analytically. The score function is the slope of that landscape: it tells you which direction to 'walk uphill' to find the peak. At the summit (the MLE), the slope is zero and the curvature tells you how confident you can be—sharp peaks mean high confidence.

SECTION 3

Visual Explanation — The Likelihood Landscape

The following diagram illustrates the fundamental objects of likelihood inference for a simple one-parameter problem. We observe a sample and then view the likelihood, log-likelihood, and score function as functions of the parameter θ. Notice how the peak of the likelihood corresponds to the peak of the log-likelihood, and the score function crosses zero precisely at that peak. The width of the peak encodes how much the data constrain the parameter, which is formalized by the Fisher information.

Top left: The likelihood function L(θ) with its peak at the MLE θ̂. Top right: The log-likelihood ℓ(θ), a monotone transformation that preserves the location of the maximum; the curvature near the peak is proportional to Fisher information. Bottom: The score function S(θ) = ∂ℓ/∂θ crosses zero at θ̂. Positive values indicate θ should increase; negative values indicate θ should decrease.

Several features in the diagram merit careful attention. First, note that L(θ) and ℓ(θ) achieve their maxima at exactly the same θ̂—this is guaranteed because the natural logarithm is a strictly increasing function, so it preserves the ordering of function values. Second, the curvature of the log-likelihood near the peak determines how 'informative' the data are: a sharply peaked log-likelihood means the data strongly favor a narrow range of parameter values, and the Fisher information quantifies this curvature. Third, the score function S(θ) provides a natural mechanism for finding the MLE—simply solve S(θ) = 0—and its variance at the true parameter value equals the Fisher information, linking estimation precision directly to the shape of the likelihood surface.

SECTION 4

Mathematical Framework

We now formalize the three central objects—likelihood, log-likelihood, and score—for a parametric model. Suppose we observe an independent and identically distributed (i.i.d.) sample X₁, X₂, …, Xₙ from a distribution with probability density (or mass) function f(x | θ), where θ ∈ Θ ⊆ ℝ is an unknown scalar parameter. The formalism extends naturally to vector-valued parameters, but the scalar case illuminates the core ideas most clearly.

LIKELIHOOD FUNCTION

L(θ) = L(θ ; x₁, …, xₙ) = ∏ᵢ₌₁ⁿ f(xᵢ | θ)

where L(θ) is the joint density of the observed sample, viewed as a function of θ with the data x₁, …, xₙ held fixed. It is not a probability distribution over θ; it need not integrate to 1.

LOG-LIKELIHOOD FUNCTION

ℓ(θ) = ln L(θ) = ∑ᵢ₌₁ⁿ ln f(xᵢ | θ)

The logarithm converts the product into a sum, which is crucial for both analytical tractability (derivatives of sums are simpler than derivatives of products) and numerical stability (products of many small probabilities can underflow). Because ln is strictly monotone, arg max_θ ℓ(θ) = arg max_θ L(θ).

SCORE FUNCTION

S(θ) = ∂ℓ(θ)/∂θ = ∑ᵢ₌₁ⁿ ∂ ln f(xᵢ | θ) / ∂θ

The score function is the first derivative of the log-likelihood with respect to θ. Under standard regularity conditions (interchange of differentiation and integration), E[S(θ₀)] = 0 at the true parameter value θ₀. This zero-mean property is fundamental to both MLE theory and score tests.

FISHER INFORMATION

I(θ) = Var[S(θ)] = E[S(θ)²] = −E[∂²ℓ(θ)/∂θ²]

Fisher information equals both the variance of the score and the expected negative curvature of the log-likelihood. The equivalence of these two expressions (the information identity) holds under regularity conditions that permit differentiating under the integral sign. It connects estimation precision to the geometry of the likelihood surface.

⚠

Regularity Conditions

The key results E[S(θ)] = 0 and the information identity require: (1) the support of f(x | θ) does not depend on θ; (2) differentiation under the integral sign is valid (dominated convergence); (3) ℓ(θ) is twice continuously differentiable. These conditions exclude, for example, the Uniform(0, θ) distribution, where the support boundary depends on θ and the MLE is the sample maximum rather than a solution to S(θ) = 0.

The power of this framework becomes apparent when we connect these objects. The MLE θ̂ is obtained by solving S(θ̂) = 0. Under regularity conditions and as n → ∞, the MLE is consistent (θ̂ → θ₀ in probability), asymptotically normal (√n(θ̂ − θ₀) → N(0, 1/I(θ₀))), and asymptotically efficient (it achieves the Cramér–Rao lower bound). The score function thus serves as both a computational tool for finding estimators and a theoretical vehicle for establishing their optimality properties.

SECTION 5

Score Function Properties & Fisher Information

The score function is far more than a mere derivative—it is an object with deep probabilistic structure that underpins the entire asymptotic theory of likelihood-based inference. In this section we examine its key properties in detail and connect them to the geometry of the Fisher information. The diagram below provides a visual roadmap of how the score, Fisher information, and the Cramér–Rao bound relate to one another in the estimation pipeline.

This flowchart traces the logical path from the log-likelihood to the score function, Fisher information, MLE, and the three major asymptotic results: the MLE's normal distribution, the Cramér–Rao lower bound on estimator variance, and the score test statistic's chi-squared distribution under the null.

Proof Sketch: E[S(θ₀)] = 0

Under regularity conditions, we can differentiate under the integral sign. Start from the identity ∫ f(x | θ) dx = 1, which holds for all θ. Differentiating both sides with respect to θ yields ∫ (∂f/∂θ) dx = 0. Now observe that ∂ ln f / ∂θ = (1/f)(∂f/∂θ), so ∂f/∂θ = f · (∂ ln f / ∂θ). Substituting, we obtain ∫ (∂ ln f / ∂θ) · f dx = E[∂ ln f / ∂θ] = E[S(θ)] = 0. This elegant result states that the score function, evaluated at the true parameter, is an unbiased estimating equation—its expectation is zero. This is why solving S(θ̂) = 0 produces consistent estimators.

Information Identity

A second differentiation of ∫ f dx = 1 reveals the information identity. Differentiating E[S(θ)] = 0 with respect to θ gives E[∂S/∂θ] + E[S(θ) · S(θ)] = 0 (using the product rule inside the expectation). Since E[S(θ)] = 0, we have Var[S(θ)] = E[S(θ)²] = −E[∂²ℓ/∂θ²]. This identity is computationally valuable because it allows us to compute Fisher information via the expected negative second derivative of the log-likelihood, which is often easier to evaluate than the variance of the score directly.

Property	Statement	Significance
Zero mean	E[S(θ₀)] = 0	Score is an unbiased estimating equation; root gives consistent estimator
Variance = Information	Var[S(θ₀)] = I(θ₀)	Connects score variability to curvature of ℓ(θ)
Additivity	$I_n$ (θ) = n · I₁(θ) for i.i.d. data	Total information grows linearly with sample size
Asymptotic normality	S(θ₀)/√(nI₁(θ₀)) → N(0,1)	Basis for score tests (Rao test) and confidence intervals

Key properties of the score function under regularity conditions

SECTION 6

Worked Example — Exponential Distribution

Let us work through a complete example using the exponential distribution, which models waiting times and is one of the most common distributions in reliability engineering and survival analysis. Suppose we observe X₁, X₂, …, Xₙ i.i.d. from Exp(λ), where the density is f(x | λ) = λe⁻ˡˣ for x ≥ 0, λ > 0. We will derive the likelihood, log-likelihood, score, Fisher information, and the MLE.

MLE for the Exponential Rate Parameter

1
Step 1 — Write the Likelihood

For i.i.d. observations x₁, …, xₙ, the likelihood function is the product of individual densities: L(λ) = ∏ᵢ₌₁ⁿ λe⁻ˡˣⁱ = λⁿ · exp(−λ ∑ᵢ₌₁ⁿ xᵢ). Note how the likelihood is a function of λ, with all the xᵢ treated as fixed constants.

L(λ) = λⁿ exp(−λ n x̄) where x̄ = (1/n) ∑ xᵢ
2
Step 2 — Compute the Log-Likelihood

Taking the natural logarithm: ℓ(λ) = ln L(λ) = n ln λ − λ ∑ᵢ₌₁ⁿ xᵢ = n ln λ − n λ x̄. The product has become a sum, making differentiation straightforward.

ℓ(λ) = n ln λ − nλx̄
3
Step 3 — Derive the Score Function

Differentiating ℓ(λ) with respect to λ: S(λ) = ∂ℓ/∂λ = n/λ − nx̄ = n(1/λ − x̄). The score is positive when 1/λ > x̄ (i.e., λ < 1/x̄) and negative when λ > 1/x̄, confirming that the log-likelihood is concave and has a unique maximum.

S(λ) = n/λ − nx̄
4
Step 4 — Find the MLE

Setting S(λ̂) = 0: n/λ̂ − nx̄ = 0, which gives 1/λ̂ = x̄, hence λ̂ = 1/x̄. We verify this is a maximum by checking the second derivative: ∂²ℓ/∂λ² = −n/λ² < 0 for all λ > 0, confirming global concavity.

MLE: λ̂ = 1/x̄ = n / ∑xᵢ
5
Step 5 — Compute Fisher Information

The second derivative is ∂²ℓ/∂λ² = −n/λ². Taking the negative expectation: I(λ) = −E[∂²ℓ/∂λ²] = n/λ². Alternatively, since E[X] = 1/λ for Exp(λ), Var[S(λ)] = n · Var(Xᵢ) · ... which also yields n/λ². The single-observation information is I₁(λ) = 1/λ², confirming additivity $I_n$ = nI₁.

I(λ) = n/λ²
6
Step 6 — Asymptotic Distribution of the MLE

By the asymptotic normality of MLEs: √n(λ̂ − λ₀) →ᵈ N(0, 1/I₁(λ₀)) = N(0, λ₀²). Equivalently, for large n, λ̂ ≈ N(λ₀, λ₀²/n). An approximate 95% confidence interval is λ̂ ± 1.96 · λ̂/√n, where we substitute the MLE for the unknown true value.

λ̂ ~ N(λ₀, λ₀²/n) asymptotically

💡

Numerical Check

Suppose n = 20 and x̄ = 4.5 (e.g., average waiting time of 4.5 minutes). Then λ̂ = 1/4.5 ≈ 0.222 events per minute. Fisher information: I(λ̂) = 20/(0.222)² ≈ 405.4. Standard error: SE ≈ 1/√I ≈ 1/√405.4 ≈ 0.050. A 95% CI is approximately 0.222 ± 1.96 × 0.050 = (0.125, 0.320).

SECTION 7

Strengths, Limitations & Comparisons

Likelihood-based inference is one of the most powerful frameworks in statistics, but it is not without limitations. Understanding when and why it excels—and where alternatives may be preferred—is essential for the practicing statistician. The table below contrasts the likelihood approach with two competing paradigms: the method of moments and Bayesian inference. Each brings distinct philosophical commitments and practical trade-offs.

Feature	Maximum Likelihood	Method of Moments	Bayesian Inference
Efficiency	Asymptotically efficient; achieves Cramér–Rao bound	Generally less efficient; not optimal in general	Optimal under correct prior; can outperform MLE in small samples
Computation	Requires numerical optimization; can be complex for multimodal likelihoods	Solving system of equations; usually closed-form	Requires MCMC or variational methods for complex models
Regularity requirements	Needs regularity conditions for standard theory to apply	Minimal; existence of moments suffices	Needs proper posterior; prior specification required
Small sample behavior	Can be biased; asymptotic theory may be inaccurate	Often unbiased but high variance	Regularized by prior; can handle small samples well
Model misspecification	Converges to KL-closest model; pseudo-true parameter	Converges to moment-matched parameter	Posterior concentrates on KL-closest model; sensitive to prior

Comparison of three major estimation paradigms

✦

WHEN TO USE LIKELIHOOD METHODS

Likelihood-based inference is the go-to framework when you have a well-specified parametric model, moderate to large sample sizes, and regularity conditions are satisfied. Think of it as a precision instrument: under the right conditions, no other frequentist method can extract more information from the data per observation. However, just as a precision instrument requires calibration, MLE requires regularity conditions to deliver its advertised optimality. When these conditions fail—non-regular models, very small samples, or heavy misspecification—Bayesian or robust methods may be more appropriate.

SECTION 8

Connection to Advanced Inference Theory

The likelihood, log-likelihood, and score function are not endpoints but launching pads for the most powerful results in statistical inference. They connect directly to the three pillars of asymptotic testing—the Wald test, likelihood ratio test, and score (Rao) test—all of which are asymptotically equivalent under the null hypothesis but differ in finite-sample properties and computational convenience. Moreover, the score function's structure underpins modern developments in semiparametric efficiency theory, estimating equations, and computational statistics.

Concept	Relationship to Score/Likelihood	Key Result
Wald Test	Uses MLE (root of score) and its estimated variance from Fisher information	W = (θ̂ − θ₀)² · I(θ̂) →ᵈ χ²(1) under H₀
Likelihood Ratio Test	Compares log-likelihood at θ̂ versus θ₀ directly	Λ = 2[ℓ(θ̂) − ℓ(θ₀)] →ᵈ χ²(1) under H₀ (Wilks' theorem)
Score (Rao) Test	Evaluates score at θ₀ only; no need to compute MLE	R = S(θ₀)² / I(θ₀) →ᵈ χ²(1) under H₀
Cramér–Rao Bound	Fisher information sets the floor for unbiased estimator variance	Var(T) ≥ [g′(θ)]² / I(θ) for any unbiased estimator T of g(θ)
EM Algorithm	Maximizes expected complete-data log-likelihood; each M-step solves augmented score equations	Monotonically increases observed-data likelihood at each iteration
Quasi-Likelihood	Uses score-like estimating equations without full distributional assumptions	Consistent estimation of mean parameters under correct mean-variance specification

Connections to advanced inference methods

A particularly important connection is to the multiparameter setting. When θ = (θ₁, …, θₖ)ᵀ is a vector, the score becomes a gradient vector S(θ) = ∇ℓ(θ) and the Fisher information becomes a k × k matrix I(θ) with entries Iⱼₖ = −E[∂²ℓ / ∂θⱼ∂θₖ]. The MLE is asymptotically multivariate normal: √n(θ̂ − θ₀) →ᵈ Nₖ(0, I₁(θ₀)⁻¹). This extension is the backbone of generalized linear models, mixed models, and the entire modern statistical modeling enterprise.

ℹ

Looking Ahead

In subsequent lessons, you will see how these foundational ideas extend to nuisance parameters and profile likelihood, semiparametric models where the score lies in a Hilbert space, and computational techniques like stochastic gradient ascent on the log-likelihood for fitting complex machine learning models. The unifying thread through all of this is the score equation and the information geometry it induces on parameter space.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

Explain why the likelihood function L(θ) is not a probability density over θ, even though it is numerically equal to f(x | θ) for fixed x. What implication does this distinction have for interpreting the value L(θ₁)/L(θ₂)?

Show answer

The likelihood L(θ) = f(x | θ) uses the same formula as the density, but the roles of x and θ are reversed. As a density in x, f integrates to 1 over x for fixed θ. But as a function of θ with x fixed, there is no requirement that ∫ L(θ) dθ = 1, and in general it does not. The ratio L(θ₁)/L(θ₂) is a likelihood ratio measuring the relative plausibility of θ₁ versus θ₂ given the data, but it cannot be interpreted as a ratio of probabilities (that θ₁ is more 'probable' than θ₂ would require a prior distribution, as in Bayesian inference). This conceptual point is at the heart of Fisher's distinction between probability and likelihood.

Question 2: BASIC CALCULATION

Let X₁, …, Xₙ be i.i.d. Bernoulli(p). Derive the log-likelihood ℓ(p), the score function S(p), and find the MLE p̂.

Show answer

The Bernoulli PMF is f(x | p) = pˣ(1 − p)¹⁻ˣ for x ∈ {0, 1}. The likelihood is L(p) = ∏ pˣⁱ(1 − p)¹⁻ˣⁱ = p^(∑xᵢ) (1 − p)^(n − ∑xᵢ). Taking logarithms: ℓ(p) = (∑xᵢ) ln p + (n − ∑xᵢ) ln(1 − p). The score is S(p) = ∂ℓ/∂p = (∑xᵢ)/p − (n − ∑xᵢ)/(1 − p). Setting S(p̂) = 0 and solving: (∑xᵢ)/p̂ = (n − ∑xᵢ)/(1 − p̂), which yields p̂ = (∑xᵢ)/n = x̄, the sample proportion. This is the familiar result that the MLE of a Bernoulli probability is the sample mean.

Question 3: INTERMEDIATE

For X₁, …, Xₙ i.i.d. N(μ, σ²) with both μ and σ² unknown, derive the Fisher information matrix I(μ, σ²) and verify that the off-diagonal elements are zero. What does this imply about the MLEs μ̂ and σ̂²?

Show answer

The log-likelihood is ℓ(μ, σ²) = −(n/2) ln(2π) − (n/2) ln σ² − (1/(2σ²)) ∑(xᵢ − μ)². The score vector has components: S_μ = (1/σ²) ∑(xᵢ − μ) and S_σ² = −n/(2σ²) + (1/(2σ⁴)) ∑(xᵢ − μ)². The second derivatives are: ∂²ℓ/∂μ² = −n/σ², ∂²ℓ/∂(σ²)² = n/(2σ⁴) − (1/σ⁶)∑(xᵢ − μ)², and ∂²ℓ/∂μ∂σ² = −(1/σ⁴)∑(xᵢ − μ). Taking negative expectations: I_μμ = n/σ², I_σ²σ² = n/(2σ⁴), and I_μσ² = 0 (since E[Xᵢ − μ] = 0). The zero off-diagonal means the Fisher information matrix is diagonal, implying that μ̂ and σ̂² are asymptotically independent. The MLEs can thus be obtained and analyzed separately: μ̂ = x̄ and σ̂² = (1/n)∑(xᵢ − x̄)².

Question 4: APPLIED

A reliability engineer models the time-to-failure of a component using an exponential distribution. From n = 50 components, she observes a total test time of ∑xᵢ = 12,500 hours. Construct a 95% confidence interval for the failure rate λ using the asymptotic normality of the MLE and the observed Fisher information.

Show answer

The MLE is λ̂ = n/∑xᵢ = 50/12,500 = 0.004 failures per hour. The observed Fisher information evaluated at the MLE is Î(λ̂) = n/λ̂² = 50/(0.004)² = 3,125,000. The estimated standard error is SE = 1/√Î = 1/√3,125,000 ≈ 0.000566. A 95% Wald confidence interval is λ̂ ± 1.96 × SE = 0.004 ± 1.96 × 0.000566 = (0.00289, 0.00511). This corresponds to a mean time between failures (MTBF) of 1/λ̂ = 250 hours, with the CI translating to an approximate MTBF range of (196, 346) hours. Note that for more precise inference with n = 50, one might prefer the profile likelihood interval or an exact interval based on the gamma distribution of ∑Xᵢ.

Question 5: CRITICAL THINKING

Consider X₁, …, Xₙ i.i.d. Uniform(0, θ). Show that the score equation S(θ) = 0 does not yield the MLE. What regularity condition is violated, and what is the actual MLE? Compare its rate of convergence to the standard √n rate.

Show answer

The density is f(x | θ) = 1/θ for 0 ≤ x ≤ θ, so L(θ) = (1/θ)ⁿ · I(x₍ₙ₎ ≤ θ), where x₍ₙ₎ is the sample maximum. On the region θ ≥ x₍ₙ₎, the log-likelihood ℓ(θ) = −n ln θ is strictly decreasing, so there is no interior critical point where S(θ) = ∂ℓ/∂θ = −n/θ = 0. The MLE is θ̂ = x₍ₙ₎, the sample maximum, obtained at the boundary. The violated regularity condition is that the support of f(x | θ) depends on θ—the condition that ∂/∂θ ∫ f(x | θ) dx = ∫ (∂f/∂θ) dx fails because the limits of integration change with θ. Remarkably, the MLE converges at rate n (not √n): P(θ − θ̂ > ε) = (1 − ε/θ)ⁿ ≈

e^{−nε/θ}

, giving n(θ − θ̂) → Exp(1/θ). This super-efficient rate is possible precisely because the problem is non-regular, and the Cramér–Rao bound does not apply.

SUMMARY

Summary & Review

Summary

This lesson developed the three foundational objects of parametric inference. The likelihood function L(θ) = ∏ f(xᵢ | θ) reinterprets the joint density as a function of the parameter, quantifying the plausibility of each parameter value given the data. The log-likelihood ℓ(θ) = ∑ ln f(xᵢ | θ) converts products to sums, making analytical and computational manipulation tractable while preserving the location of the maximum. The score function S(θ) = ∂ℓ/∂θ is the gradient of the log-likelihood; it has zero mean at the true parameter, and setting it to zero yields the maximum likelihood estimator.

The variance of the score equals the Fisher information I(θ) = −E[∂²ℓ/∂θ²], which measures the curvature of the log-likelihood and sets the Cramér–Rao lower bound on the variance of unbiased estimators. Under regularity conditions, the MLE is consistent, asymptotically normal with variance 1/nI(θ), and asymptotically efficient. These results connect directly to the three large-sample tests—Wald, likelihood ratio, and score tests—which are the workhorses of modern statistical hypothesis testing.