Method of Moments & MLE - Statistics Graduate Level

1. Let $X_1,\ldots,X_n$ be independent uniform random variables on $[\theta,\theta+1]$. An observed sample has minimum $2.2$, maximum $2.9$, and mean $2.55$. Which statement correctly describes the MLE and the method-of-moments estimate of $\theta$?
   Answer: Every $\theta\in[1.9,2.2]$ is an MLE, and the moment estimate is $2.05$ When you see a uniform distribution on $[\theta, \theta+1]$, your first instinct should be to identify the constraints that $\theta$ must satisfy given the observed data, then maximize the likelihood over the feasible region. MLE: The likelihood function is $L(\theta) = 1$ if all observations fall in $[\theta, \theta+1]$, and $0$ otherwise. For every $X_i$ to lie in $[\theta, \theta+1]$, you need $\theta \leq X_{(1)}$ (the minimum) and $\theta \geq X_{(n)} - 1$ (the maximum minus 1). With $X_{(1)} = 2.2$ and $X_{(n)} = 2.9$, the feasible set is $\theta \geq 2.9 - 1 = 1.9$ and $\theta \leq 2.2$. So every $\theta \in [1.9, 2.2]$ yields $L(\theta) = 1$, and the MLE is non-unique — a whole interval. Method of Moments: The population mean of $\text{Uniform}[\theta, \theta+1]$ is $\theta + \frac{1}{2}$. Setting this equal to the sample mean: $\theta + 0.5 = 2.55$, giving $\hat{\theta}_{MOM} = 2.05$. This is answer D. Why the others fail: A uses $1.9$ as a unique MLE — the MLE isn't unique here. B claims $2.2$ is the unique MLE — also wrong, and it only identifies the upper boundary of the feasible region. C misidentifies the feasible interval entirely: $[2.2, 2.9]$ violates the constraint derivation (those values of $\theta$ would place the maximum outside the support). Study tip: For uniform distribution MLEs, always write out the support constraints explicitly — the likelihood is flat over its feasible region, so the MLE is often an interval, not a unique point.
2. Suppose $X_i=a+Y_i$, where the $Y_i$ are independent exponential random variables with rate $\lambda$. For an observed sample, the sample mean is $7.4$, the sample minimum is $2.0$, and the sample variance computed with divisor $n$ is $9.0$. Which pair gives the method-of-moments estimates followed by the maximum likelihood estimates?
   Answer: MOM: $(\widehat a,\widehat\lambda)=(4.4,1/3)$; MLE: $(2.0,1/5.4)$ When a question mixes two estimation methods for a shifted exponential model, your first move should be to recall what each method uses as its raw material: method of moments (MOM) uses population moment equations, while maximum likelihood (MLE) exploits the full likelihood structure. MOM derivation. For $X_i = a + Y_i$ with $Y_i \sim \text{Exp}(\lambda)$, the population moments are $E[X] = a + 1/\lambda$ and $\text{Var}(X) = 1/\lambda^2$. Setting $1/\lambda^2 = 9$ gives $\widehat{\lambda}_{MOM} = 1/3$, and then $\widehat{a}_{MOM} = \bar{X} - 1/\widehat{\lambda} = 7.4 - 3 = 4.4$. So MOM yields $(4.4,\ 1/3)$. MLE derivation. The shifted exponential likelihood is maximized by setting $\widehat{a}_{MLE} = X_{(1)} = 2.0$ (the sample minimum, since $a$ must not exceed any observation, and the likelihood increases as $a$ rises). With $a$ fixed, $\widehat{\lambda}_{MLE} = 1/(\bar{X} - \widehat{a}) = 1/(7.4 - 2.0) = 1/5.4$. This gives MLE $(2.0,\ 1/5.4)$, confirming answer A. Answer B swaps the two estimators — assigning the MLE values to MOM and vice versa, a direct reversal of the derivations. Answer C uses the sample variance directly as $1/\lambda$ rather than taking its square root, misapplying the variance equation. Answer D incorrectly uses $\bar{X}$ as $\widehat{a}$ and ignores the MLE's use of the sample minimum entirely. Study tip: Always pair the MLE of $a$ in a shifted distribution with the sample minimum — it's a recurring pattern on graduate exams. MOM, by contrast, always works through moment equations, never directly using order statistics.
3. Consider a zero-inflated Poisson model in which an observation equals a structural zero with probability $\pi$ and otherwise follows a Poisson distribution with mean $\lambda$. In a sample, the first empirical factorial moment is $n^{-1}\sum_iX_i=1.2$ and the second empirical factorial moment is $n^{-1}\sum_iX_i(X_i-1)=2.4$. What are the method-of-moments estimates?
   Answer: $\widehat\lambda=2.0$ and $\widehat\pi=0.4$ When you encounter a zero-inflated Poisson (ZIP) model, your first move should be deriving the theoretical moments in terms of the model parameters, then equating them to their empirical counterparts — that's the method-of-moments recipe. For a ZIP model, an observation is zero with probability $\pi$ (structural zero) and Poisson$(\lambda)$ with probability $(1-\pi)$. The key theoretical factorial moments are: $E[X] = (1-\pi)\lambda$ $E[X(X-1)] = (1-\pi)\lambda^2$ Setting these equal to the sample values gives you: $(1-\pi)\lambda = 1.2 \quad \text{and} \quad (1-\pi)\lambda^2 = 2.4$ Dividing the second equation by the first eliminates $(1-\pi)$ immediately: $\lambda = \frac{2.4}{1.2} = 2.0$ Then substituting back: $(1-\pi)(2.0) = 1.2 \Rightarrow 1-\pi = 0.6 \Rightarrow \widehat{\pi} = 0.4$ This confirms answer C: $\widehat{\lambda}=2.0$ and $\widehat{\pi}=0.4$. Answer A gets $\widehat{\lambda}$ right but flips the roles of $\pi$ and $1-\pi$, reporting $0.6$ instead of $0.4$ — a classic sign-confusion error. Answer B plugs the first moment in directly as $\widehat{\lambda}$, ignoring the inflation adjustment entirely. Answer D appears to split the difference between the two moments arithmetically without any principled derivation. A reliable strategy: always divide consecutive factorial moments to isolate $\lambda$ first — this sidesteps the need to solve a system simultaneously and reduces the chance of algebraic error. Remember, factorial moments are your friends in any count-data mixture model.
4. A sample is drawn from a gamma distribution with shape $\alpha$ and rate $\beta$. The observed summaries are $\bar X=2$, $n^{-1}\sum_{i=1}^n(X_i-\bar X)^2=2$, and $n^{-1}\sum_{i=1}^n\log X_i=0.3$. Which statement correctly gives the method-of-moments estimates and the equations characterizing the MLE, where $\psi$ is the digamma function?
   Answer: MOM: $(2,1)$; MLE satisfies $\beta=\alpha/2$ and $\log\alpha-\psi(\alpha)=\log2-0.3$ When working with the gamma distribution, you need two separate derivations: method of moments (MOM) uses the mean and variance, while MLE requires differentiating the log-likelihood with respect to both parameters. MOM: For a Gamma$(\alpha, \beta)$ with rate parameterization, the mean is $\mu = \alpha/\beta$ and variance is $\sigma^2 = \alpha/\beta^2$. Setting $\bar{X} = \alpha/\beta = 2$ and $s^2 = \alpha/\beta^2 = 2$, dividing gives $1/\beta = 1$, so $\hat{\beta} = 1$, and then $\hat{\alpha} = 2$. The MOM estimates are $(2, 1)$, reported as $(\hat{\alpha}, \hat{\beta})$. MLE: The log-likelihood yields two score equations. Differentiating with respect to $\beta$ gives $\hat{\beta} = \alpha/\bar{X} = \alpha/2$. Differentiating with respect to $\alpha$ gives $\log\alpha - \psi(\alpha) = \bar{X}^{-1}\overline{\log X} \cdot \bar{X} - \overline{\log X}$... more carefully: the equation becomes $\log\alpha - \psi(\alpha) = \log\bar{X} - \overline{\log X} = \log 2 - 0.3$. This matches answer C exactly. Answer A reports the correct MOM pair $(2,1)$ but flips the MLE relationship to $\beta = 2\alpha$ and gets the sign wrong in the digamma equation. Answer B swaps the MOM estimates to $(1, 2)$, confusing $\alpha$ and $\beta$, and also inverts the rate equation. Answer D keeps the right MOM estimates but writes $\beta = 2/\alpha$ (inconsistent with $\bar{X} = \alpha/\beta$) and incorrectly adds rather than subtracts in the digamma equation. A reliable strategy: always derive the MLE $\beta$ equation first (it's algebraically clean), then substitute into the $\alpha$ equation to isolate the digamma condition. Keeping $\log\alpha - \psi(\alpha) = \log\bar{X} - \overline{\log X}$ as a template will prevent sign errors under exam pressure.
5. Let $X_1,\ldots,X_n$ be independent uniform random variables on $[0,\theta]$. Define $\widehat\theta_M=X_{(n)}$ and $\widehat\theta_Q=2\bar X$. Which joint asymptotic statement is correct?
   Answer: $n(\theta-\widehat\theta_M)/\theta\Rightarrow\operatorname{Exp}(1)$ and $\sqrt n(\widehat\theta_Q-\theta)\Rightarrow N(0,\theta^2/3)$ When comparing two estimators with different convergence rates, your first instinct should be to ask: how fast does each estimator concentrate around the truth? Order statistics and sample means behave very differently asymptotically, and this question tests whether you know both. For $\widehat{\theta}_M = X_{(n)}$, the maximum of a uniform sample, the key fact is that the bias shrinks at rate $1/n$, not $1/\sqrt{n}$. Specifically, $\mathbb{E}[X_{(n)}] = \frac{n}{n+1}\theta$, so $X_{(n)}$ underestimates $\theta$ by order $1/n$. The exact CDF of $X_{(n)}$ is $(x/\theta)^n$, which gives $P(n(\theta - X_{(n)})/\theta > t) = (1 - t/n)^n \to e^{-t}$. So $n(\theta - \widehat{\theta}_M)/\theta \Rightarrow \text{Exp}(1)$. Note the centering is $\theta - X_{(n)}$ (positive), since the maximum always undershoots. For $\widehat{\theta}_Q = 2\bar{X}$, since $\text{Var}(X_i) = \theta^2/12$, the variance of $2\bar{X}$ is $4 \cdot \theta^2/(12n) = \theta^2/(3n)$. By the CLT, $\sqrt{n}(\widehat{\theta}_Q - \theta) \Rightarrow N(0, \theta^2/3)$. This confirms answer A. B is wrong because it swaps the two convergence rates — the maximum converges at rate $n$, not $\sqrt{n}$. C uses the correct rate $n$ for the maximum and the correct variance $\theta^2/3$ for $\widehat{\theta}_Q$, but writes $\theta^2/12$ instead — that's the variance of a single observation, not of $2\bar{X}$. D incorrectly applies CLT-rate convergence to the maximum, which is the most common trap here. Remember: whenever you see an order statistic estimating a boundary parameter, the convergence rate is $n$, not $\sqrt{n}$ — this is a signature of non-regular estimation problems.
6. Suppose $X_1,\ldots,X_n$ are independent $N(\theta,1)$ observations. The parameter of interest is $\psi=\theta^2$. For one sample, $\bar X=-1.5$ and $n^{-1}\sum_iX_i^2=3.6$. What are, respectively, the MLE of $\psi$ and the direct method-of-moments estimate obtained by matching the second raw moment?
   Answer: $\widehat\psi_{\mathrm{MLE}}=2.25$ and $\widehat\psi_{\mathrm{MOM}}=2.60$ When you encounter a question like this, recognize it's testing two distinct estimation philosophies — the invariance principle of MLEs and the moment-matching logic of MOM — applied to the same parameter. For the MLE, recall that if $\hat{\theta}$ is the MLE of $\theta$, then for any function $g$, the MLE of $g(\theta)$ is $g(\hat{\theta})$ by the invariance property. The MLE of $\theta$ under $N(\theta,1)$ is simply $\hat{\theta} = \bar{X} = -1.5$. Therefore, $\widehat{\psi}_{\text{MLE}} = \hat{\theta}^2 = (-1.5)^2 = 2.25$. For the MOM via the second raw moment, you match the population moment to its sample counterpart. Since $X \sim N(\theta, 1)$, the second raw moment is $E[X^2] = \text{Var}(X) + (E[X])^2 = 1 + \theta^2 = 1 + \psi$. Setting this equal to the sample version: $1 + \hat{\psi} = n^{-1}\sum_i X_i^2 = 3.6$, so $\widehat{\psi}_{\text{MOM}} = 3.6 - 1 = 2.60$. This confirms answer D. A is wrong because $-2.25$ treats the MLE as $\bar{X}^2$ but keeps the negative sign — you cannot square a number and get a negative result. B confuses the MLE of $\theta$ (which is $|\bar{X}|$) with the MLE of $\psi$, and uses the raw second-moment sample value without subtracting 1. C swaps the two estimates, getting the MOM right but assigning it to the MLE label and vice versa. A reliable study tip: always distinguish which parameter a moment equation solves for. Here, $E[X^2]$ identifies $\psi = \theta^2$, not $\theta$ directly — subtracting the variance is essential.
7. Twenty independent subjects have exponentially distributed event times with rate $\lambda$. Eight events are observed, and their observed event times sum to $19$. The other twelve subjects are administratively right-censored at time $3$. What is the MLE of $\lambda$?
   Answer: $\widehat\lambda=8/55$, using the number of events divided by total observed exposure When survival data includes censoring, the key is to write down the full likelihood correctly. For exponentially distributed event times with rate $\lambda$, each observed event contributes a density $\lambda e^{-\lambda t_i}$, while each censored subject contributes a survival probability $e^{-\lambda c_j}$, where $c_j$ is their censoring time. The log-likelihood is therefore: $\ell(\lambda) = d\log\lambda - \lambda\sum_{i=1}^{d} t_i - \lambda\sum_{j=1}^{m} c_j$ where $d$ is the number of observed events. Setting $d\ell/d\lambda = 0$ gives $\widehat{\lambda} = d / \left(\sum t_i + \sum c_j\right)$, which is the number of events divided by total exposure time — the sum of all observed times, whether they ended in an event or censoring. Here, $d = 8$, the eight observed failure times sum to $19$, and the twelve censored subjects each contribute $3$, giving $12 \times 3 = 36$ additional person-time. Total exposure is $19 + 36 = 55$, so $\widehat{\lambda} = 8/55$. This confirms answer A is correct. Answer B, $8/19$, ignores the censored subjects entirely, throwing away the information that twelve people survived past time $3$ — this severely overestimates $\lambda$. Answer C uses the total sample size $20$ in the numerator instead of the event count $8$, which has no basis in the likelihood derivation. Answer D uses the number censored $12$ in the numerator, which is similarly unjustified. The core study tip: in survival analysis, the MLE denominator is always total person-time at risk (events plus censored exposure), not just failure times or sample size. Memorize the formula $\widehat{\lambda} = \text{events}/\text{total exposure}$.
8. Let $X_1,\ldots,X_5$ be independent observations from the uniform distribution on $[0,\theta]$. The observed sample mean is $2.1$, and the sample maximum is $4.0$. Using the invariance principle separately with the maximum likelihood estimator and the method-of-moments estimator, what are the resulting estimates of $P_\theta(X>3)$?
   Answer: MLE: $1/4$; method of moments: $2/7$ When you see a problem invoking the invariance principle, your job is to first find the estimator for the parameter $\theta$, then plug it into the target function — here, $P_\theta(X > 3) = \frac{\theta - 3}{\theta}$ for $\theta > 3$. For the MLE: on a $\text{Uniform}[0,\theta]$ distribution, the MLE of $\theta$ is the sample maximum, $\hat{\theta}_{MLE} = X_{(5)} = 4.0$. By invariance, the MLE of $P(X>3)$ is $\frac{4.0 - 3}{4.0} = \frac{1}{4}$. For the method of moments: since $E[X] = \theta/2$, setting $\theta/2 = \bar{x} = 2.1$ gives $\hat{\theta}_{MOM} = 4.2$. By invariance, the MOM estimate is $\frac{4.2 - 3}{4.2} = \frac{1.2}{4.2} = \frac{2}{7}$. So the results are MLE: $1/4$ and MOM: $2/7$, confirming answer A. Answer B flips the two estimates — a classic trap if you mix up which estimator uses the sample maximum versus the sample mean. Answer C incorrectly computes the MLE estimate as 0, which would only occur if $\hat{\theta} = 3$ exactly — there's no justification for that here. Answer D correctly identifies the MLE estimate but uses $\hat{\theta}_{MOM} = 4.2$ incorrectly, arriving at $1/2$ through a arithmetic error. Study tip: Always identify the estimator for $\theta$ first, then apply the invariance principle. For Uniform$[0,\theta]$, remember MLE $\to$ sample max; MOM $\to$ twice the sample mean.
9. Let $X_1,\ldots,X_n$ have density $f(x;\alpha)=\alpha x^{\alpha-1}$ for $0<x<1$ and $\alpha>0$. In an observed sample, $\bar X=0.6$ and $n^{-1}\sum_i\log X_i=-0.8$. Using the MLE and the method-of-moments estimator separately, what are the corresponding plug-in estimates of $P_\alpha(X>1/2)$?
   Answer: MLE: $1-2^{-1.25}$; method of moments: $1-2^{-1.5}$ When you see a question combining MLE and method of moments with a parametric family, your first job is to derive each estimator separately, then plug into the survival function. For this power distribution, $P_\alpha(X > 1/2) = 1 - (1/2)^\alpha = 1 - 2^{-\alpha}$, so everything hinges on estimating $\alpha$. MLE: The log-likelihood gives score equation $n/\alpha + \sum_i \log X_i = 0$, so $\hat\alpha_{\text{MLE}} = -1/\bar{\log X} = -1/(-0.8) = 1.25$. Plugging in: $1 - 2^{-1.25}$. Method of Moments: The population mean is $E[X] = \alpha/(\alpha+1)$. Setting this equal to $\bar{X} = 0.6$: $\alpha/(\alpha+1) = 0.6 \Rightarrow \alpha = 0.6\alpha + 0.6 \Rightarrow 0.4\alpha = 0.6 \Rightarrow \hat\alpha_{\text{MOM}} = 1.5$. Plugging in: $1 - 2^{-1.5}$. So the MLE gives $1-2^{-1.25}$ and MOM gives $1-2^{-1.5}$, confirming answer B. Answer A simply swaps the two estimators — a classic trap when you solve both correctly but mislabel which is which. Answer C forgets to subtract from 1; it computes $P(X < 1/2) = (1/2)^\alpha$ rather than the survival probability $P(X > 1/2)$. Answer D skips the estimation entirely and plugs the raw sample statistics $\bar{X}=0.6$ and $\overline{\log X}=-0.8$ directly as if they were $\alpha$ estimates without solving for $\alpha$. Always derive the estimator formula explicitly before substituting numbers — confusing $\bar{X}$ with $\hat\alpha$ is a frequent careless error under exam pressure.
10. Let $X_1,\ldots,X_n$ be independent random variables from a Laplace distribution with density $f(x;\mu,b)=(2b)^{-1}\exp(-|x-\mu|/b)$, where $b>0$. The observed sample is $1,2,2,5,10$. Which choice gives the MLEs of $(\mu,b)$ followed by the method-of-moments estimates based on matching the population mean and variance?
    Answer: MLE: $(\widehat\mu,\widehat b)=(2,2.4)$; MOM: $(\widehat\mu,\widehat b)=(4,\sqrt{5.4})$ When you encounter a question combining MLE and method-of-moments for the Laplace distribution, tackle each estimation method separately and systematically. For MLE, the log-likelihood is $\ell(\mu,b) = -n\ln(2b) - \frac{1}{b}\sum|x_i - \mu|$. Maximizing over $\mu$ means minimizing $\sum|x_i - \mu|$, which is achieved by the sample median. For the data $1,2,2,5,10$, the median is $2$. With $\hat{\mu}=2$, you then maximize over $b$: setting the derivative to zero gives $\hat{b} = \frac{1}{n}\sum|x_i - \hat{\mu}| = \frac{|1-2|+|2-2|+|2-2|+|5-2|+|10-2|}{5} = \frac{1+0+0+3+8}{5} = \frac{12}{5} = 2.4$. So the MLE is $(\hat{\mu}, \hat{b}) = (2, 2.4)$. For MOM, the Laplace distribution has mean $\mu$ and variance $2b^2$. Matching to sample moments: the sample mean is $\bar{x} = \frac{1+2+2+5+10}{5} = 4$, giving $\hat{\mu}=4$. The sample variance is $s^2 = \frac{\sum(x_i-\bar{x})^2}{n} = \frac{9+4+4+1+36}{5} = \frac{54}{5} = 10.8$, so $2\hat{b}^2 = 10.8 \Rightarrow \hat{b} = \sqrt{5.4}$. MOM gives $(4, \sqrt{5.4})$. This matches choice B. Choice A reverses the two methods — a classic trap. Choice C uses incorrect formulas, producing $\hat{b}=3.0$ for MLE (wrong divisor) and $\sqrt{6.75}$ for MOM (likely using $n-1$ variance). Choice D confuses the variance formula by not taking the square root properly for MLE. Key tip: For Laplace MLE, always remember median → mean absolute deviation. For MOM, remember that the Laplace variance is $2b^2$, not $b^2$ — mixing these up is the most common error on this type of question.

Start this quiz