Taylor Expansions for Approximations - Statistics Graduate Level

1. Suppose $\sqrt n\,\hat\theta_n\xrightarrow{d}Z$, where $Z\sim N(0,\sigma^2)$, and define $g(x)=\sin(x)-x$. Which limit follows from the first nonvanishing term in the Taylor expansion of $g$ at zero?
   Answer: $n^{3/2}g(\hat\theta_n)\xrightarrow{d}-\frac{Z^3}{6}$ When you see a composition like $g(\hat\theta_n)$ where $\hat\theta_n \to 0$ in probability, your instinct should be to Taylor-expand $g$ around zero and identify the first nonvanishing term — that term governs the limiting distribution. For $g(x) = \sin(x) - x$, recall the Taylor series: $\sin(x) = x - \frac{x^3}{6} + \cdots$, so $g(x) = -\frac{x^3}{6} + O(x^5)$. The leading term is cubic, not linear — this is the key structural fact. Since $\sqrt{n}\,\hat\theta_n \xrightarrow{d} Z$, we have $\hat\theta_n \approx Z/\sqrt{n}$, so: $g(\hat\theta_n) \approx -\frac{\hat\theta_n^3}{6} \approx -\frac{1}{6}\cdot\frac{Z^3}{n^{3/2}}$ Multiplying both sides by $n^{3/2}$ gives $n^{3/2}g(\hat\theta_n) \xrightarrow{d} -\frac{Z^3}{6}$, confirming answer C. A is wrong because it drops the negative sign — $\sin(x) - x$ has a negative cubic coefficient, so the sign matters critically here. B is wrong because it treats the cubic term as linear in $Z$, which would only be valid if the leading term were $g'(0)x$. But $g'(0) = \cos(0) - 1 = 0$, so the linear term vanishes entirely, and you can't apply the standard delta method. D is tempting if you mistakenly apply the delta method (which requires a nonzero first derivative) and then compute the variance — but since $g'(0) = 0$, the ordinary delta method breaks down completely here. Study tip: Whenever $g'(\hat\theta) = 0$ at the limit point, go to higher-order terms and adjust the normalization accordingly — the exponent in $n^{k/2}$ matches the order of the first nonvanishing derivative.
2. What is the limiting distribution of $n(\hat\theta_n^2-\theta_n^2)$?
   Answer: $2cZ+Z^2$ When you see a question involving a sequence of parameters that shrink with $n$, your first instinct should be the Delta Method combined with careful algebraic expansion — don't try to apply the Delta Method blindly to the transformed quantity. Instead, work directly from what you're given. You know $\sqrt{n}(\hat\theta_n - \theta_n) \xrightarrow{d} Z$, so $\hat\theta_n = \theta_n + Z/\sqrt{n} + o_p(1/\sqrt{n})$. Now expand $n(\hat\theta_n^2 - \theta_n^2)$: $n(\hat\theta_n^2 - \theta_n^2) = n(\hat\theta_n - \theta_n)(\hat\theta_n + \theta_n)$ Write $\hat\theta_n + \theta_n = 2\theta_n + (\hat\theta_n - \theta_n) = \frac{2c}{\sqrt{n}} + \frac{Z}{\sqrt{n}} + o_p(1/\sqrt{n})$. Multiplying: $n \cdot \frac{Z}{\sqrt{n}} \cdot \left(\frac{2c + Z}{\sqrt{n}}\right) = Z(2c + Z) = 2cZ + Z^2$ This confirms D is correct. A ($Z^2 - c^2$) has no algebraic justification — there's no mechanism to produce a $-c^2$ term from this expansion. B ($2cZ$) is a common partial-answer trap: it captures only the first-order term from the product, ignoring the $Z^2$ contribution from $(\hat\theta_n - \theta_n)^2$. Since $\theta_n \to 0$, the quadratic term survives at rate $n$ and cannot be dropped. C ($(c+Z)^2 = c^2 + 2cZ + Z^2$) over-counts by adding an extra $c^2$, which would require $n\theta_n^2 \to c^2$ to appear — but that term cancels out because you're looking at the difference $\hat\theta_n^2 - \theta_n^2$, not $\hat\theta_n^2$ alone. Study tip: When parameters shrink at rate $1/\sqrt{n}$, don't discard quadratic fluctuation terms — they survive scaling and contribute to the limit. Always expand fully before taking limits.
3. Which is the limiting distribution of $T_n$?
   Answer: $N\left(0,\frac{\tau^2}{s^2}\right)$ Whenever you see a studentized statistic, your instinct should be to apply the Delta Method — specifically, treat $T_n = \sqrt{n}(\hat\theta_n - \theta)/\hat s_n$ as a function of the joint estimator $(\hat\theta_n, \hat s_n)$. Define $g(a, b) = a/b$, so $T_n = g(\sqrt{n}(\hat\theta_n - \theta),\, \hat s_n)$... but more carefully, write $T_n = \sqrt{n}(\hat\theta_n - \theta) \cdot \frac{1}{\hat s_n}$. Since $\hat s_n \xrightarrow{p} s > 0$, Slutsky's theorem applies directly: $\frac{1}{\hat s_n} \xrightarrow{p} \frac{1}{s}$. Therefore, $T_n = \frac{\sqrt{n}(\hat\theta_n - \theta)}{\hat s_n} \xrightarrow{d} \frac{1}{s} \cdot Z, \quad Z \sim N(0, \tau^2),$ which gives $T_n \xrightarrow{d} N\!\left(0, \frac{\tau^2}{s^2}\right)$, confirming D. The key insight is that only the numerator $\sqrt{n}(\hat\theta_n - \theta)$ is $O_p(1)$ (i.e., stochastically bounded at rate $\sqrt{n}$); the denominator $\hat s_n$ is already $O_p(1)$ and converges to a constant. Slutsky lets you "pull the limit through" the division without any covariance correction. A is wrong because it applies a Delta Method gradient for $g(a,b) = a - b$, mixing the two components incorrectly. B mistakenly applies $g(\theta, s) = \theta/s$ as if you were estimating a ratio of parameters, dragging $\theta$ into the variance. C correctly uses $g(a,b) = a/b$ with gradient $(1/s,\, -\theta/s^2)$... wait — actually C applies the Delta Method to the joint scaled quantity $\sqrt{n}(\hat\theta_n, \hat s_n)$, treating both components as fluctuating at rate $\sqrt{n}$, which is the critical error. Only $\hat\theta_n - \theta$ is scaled by $\sqrt{n}$; $\hat s_n$ is not centered, so its fluctuation is asymptotically negligible relative to the numerator. Study tip: When a statistic is a ratio where the denominator converges to a nonzero constant (not zero), Slutsky's theorem is your most efficient tool — no covariance terms needed, and the variance simplifies to just the numerator's variance divided by the squared limiting constant.
4. What is the asymptotic covariance matrix of $\sqrt{n}\{(U_n,V_n)^T-(2,2)^T\}$?
   Answer: $\begin{pmatrix}11&-7\\-7&7\end{pmatrix}$ When you see a transformation of asymptotically normal estimators, reach immediately for the Delta Method: if $\sqrt{n}(\hat{\boldsymbol{\theta}} - \boldsymbol{\theta}_0) \xrightarrow{d} N(0, \Sigma)$ and $g$ is differentiable, then $\sqrt{n}(g(\hat{\boldsymbol{\theta}}) - g(\boldsymbol{\theta}_0)) \xrightarrow{d} N(0, J\Sigma J^T)$, where $J$ is the Jacobian of $g$ evaluated at $\boldsymbol{\theta}_0 = (2,1)^T$. Here $g(\theta_1, \theta_2) = (\theta_1\theta_2,\ \theta_1/\theta_2)^T$, so the Jacobian is: $J = \begin{pmatrix} \theta_2 & \theta_1 \\ 1/\theta_2 & -\theta_1/\theta_2^2 \end{pmatrix}\Bigg|_{(2,1)} = \begin{pmatrix} 1 & 2 \\ 1 & -2 \end{pmatrix}.$ Now compute $J\Sigma J^T$. First, $J\Sigma = \begin{pmatrix}1&2\\1&-2\end{pmatrix}\begin{pmatrix}1&1/2\\1/2&2\end{pmatrix} = \begin{pmatrix}2&9/2\\0&-7/2\end{pmatrix}$. Then: $J\Sigma J^T = \begin{pmatrix}2&9/2\\0&-7/2\end{pmatrix}\begin{pmatrix}1&1\\2&-2\end{pmatrix} = \begin{pmatrix}11&-7\\-7&7\end{pmatrix}.$ This is answer B. Choice A has the off-diagonal as $+7$ instead of $-7$ — a sign error that would arise from mishandling the negative partial derivative $\partial(\theta_1/\theta_2)/\partial\theta_2 = -\theta_1/\theta_2^2$. Choice C swaps the diagonal entries, reflecting confusion about which transformation corresponds to which row. Choice D appears to result from using an incorrect Jacobian, perhaps evaluating partial derivatives without applying the quotient rule correctly. Your key study habit: always write the Jacobian carefully and double-check signs on partial derivatives involving quotients. Sign errors in $J$ propagate throughout the entire matrix multiplication and are the most common source of mistakes on Delta Method problems.
5. Writing $m=\exp(\beta_0+2\beta_1)$, which expression is the second-order approximation to $E(\hat m)$?
   Answer: $m\left(1+\frac{2}{n}\right)+o(n^{-1})$ When a nonlinear function of an estimator is itself being estimated, you need the delta method — specifically its second-order (bias-correcting) extension — to approximate the expectation. The key formula: if $\hat\theta \approx \theta$ and $g$ is smooth, then $E[g(\hat\theta)] \approx g(\theta) + \frac{1}{2}\text{tr}\left(H_g(\theta)\,\text{Cov}(\hat\theta)\right)$, where $H_g$ is the Hessian of $g$. Here, $g(\beta_0, \beta_1) = \exp(\beta_0 + 2\beta_1)$. The gradient is $\nabla g = m\,(1,\,2)^T$ and the Hessian is $H_g = m\begin{pmatrix}1&2\\2&4\end{pmatrix}$. The covariance matrix of $(\hat\beta_0, \hat\beta_1)$ is $\Sigma = \frac{1}{n}\begin{pmatrix}4&-1\\-1&1\end{pmatrix}$. The bias correction term is $\frac{1}{2}\text{tr}(H_g\,\Sigma) = \frac{m}{2}\,\text{tr}\!\left(\begin{pmatrix}1&2\\2&4\end{pmatrix}\frac{1}{n}\begin{pmatrix}4&-1\\-1&1\end{pmatrix}\right)$. Computing the product's trace: diagonal entries are $(4-2)/n = 2/n$ and $(-2+4)/n = 2/n$, giving $\text{tr} = 4/n$. So the correction is $\frac{m}{2}\cdot\frac{4}{n} = \frac{2m}{n}$, and $E(\hat m) \approx m\!\left(1+\frac{2}{n}\right)$, confirming answer A. Answer B uses $4/n$ — forgetting the $\frac{1}{2}$ prefactor in the bias formula. Answer C uses $6/n$, possibly from summing raw matrix entries incorrectly. Answer D claims zero bias, ignoring the curvature of the exponential entirely. Study tip: Whenever you apply the second-order delta method for bias, remember the $\frac{1}{2}\,\text{tr}(H\Sigma)$ structure — the factor of $\frac{1}{2}$ is the most common place to lose points on this type of calculation.
6. Which expression gives the second-order approximation to the bias $E(\hat\theta_n)-\theta$?
   Answer: $-\frac{h''(\theta)\sigma^2}{2n\{h'(\theta)\}^3}+o(n^{-1})$ When you see an estimator defined implicitly by $h(\hat\theta_n) = \bar X_n$, your instinct should be to apply the delta method in reverse — specifically, a second-order Taylor expansion of $h^{-1}$ around $h(\theta)$. This is the classic bias calculation for a nonlinear transformation of an unbiased (or nearly unbiased) estimator. Since $h(\hat\theta_n) = \bar X_n$, write $\hat\theta_n = h^{-1}(\bar X_n)$. Expand $h^{-1}$ around $\mu = h(\theta)$ to second order. Setting $g = h^{-1}$, we have $g'(\mu) = 1/h'(\theta)$ and, by differentiating the identity $h(g(y))=y$ twice, $g''(\mu) = -h''(\theta)/[h'(\theta)]^3$. The second-order bias formula for a smooth function of a sample mean is: $E[\hat\theta_n] - \theta \approx \frac{1}{2}g''(\mu)\cdot\operatorname{Var}(\bar X_n) = \frac{1}{2}\cdot\left(-\frac{h''(\theta)}{[h'(\theta)]^3}\right)\cdot\frac{\sigma^2}{n}$ which simplifies directly to $-\dfrac{h''(\theta)\sigma^2}{2n[h'(\theta)]^3}$, confirming C. A is wrong because it carries $[h'(\theta)]^2$ in the denominator instead of $[h'(\theta)]^3$ — this would arise from a first-derivative-only calculation, forgetting that differentiating $g'$ introduces an extra factor of $h'(\theta)$. B has the correct denominator $[h'(\theta)]^3$ but the wrong sign; the negative sign is essential and comes directly from the chain-rule calculation of $g''$. D inverts the roles of $h'$ and $h''$ entirely — a clear indicator of confusing the numerator and denominator structure. Study tip: Always derive $g'' = -h''/[h']^3$ explicitly by differentiating the chain-rule identity twice. The sign and the cube in the denominator are the two most common errors on these bias approximation problems.
7. A second-order Taylor expansion is used to bias-correct the log-odds estimator $\log\{\hat p/(1-\hat p)\}$ through order $n^{-1}$. Which corrected estimator is obtained by replacing unknown quantities with $\hat p$?
   Answer: $\log\frac{\hat p}{1-\hat p}-\frac{2\hat p-1}{2n\hat p(1-\hat p)}$ When you need to bias-correct an estimator using a Taylor expansion, the key tool is the delta method extended to second order. For a function $g(\hat p)$ of $\hat p = X/n$, the expected value expands as: $E[g(\hat p)] \approx g(p) + \frac{1}{2}g''(p)\cdot\frac{p(1-p)}{n}$ So the bias is approximately $\frac{1}{2}g''(p)\cdot\frac{p(1-p)}{n}$, and a bias-corrected estimator subtracts this bias from $g(\hat p)$. Here $g(p) = \log\frac{p}{1-p}$. Computing derivatives: $g'(p) = \frac{1}{p(1-p)}$, and differentiating again gives $g''(p) = \frac{2p-1}{p^2(1-p)^2}$. The bias is therefore: $\text{Bias} \approx \frac{1}{2}\cdot\frac{2p-1}{p^2(1-p)^2}\cdot\frac{p(1-p)}{n} = \frac{2p-1}{2n\,p(1-p)}$ The bias-corrected estimator subtracts this bias and replaces $p$ with $\hat p$: $\log\frac{\hat p}{1-\hat p} - \frac{2\hat p - 1}{2n\hat p(1-\hat p)}$ This is exactly answer A. Answer B adds the bias instead of subtracting it — a sign error that would inflate rather than remove the bias. Answer C uses $1$ in the numerator instead of $2\hat p - 1$, which would only be correct if $g''$ were $1/[p^2(1-p)^2]$ — a miscalculation of the second derivative. Answer D divides by $\sqrt{n}$ rather than $n$, corresponding to a first-order (standard error) correction rather than a second-order bias correction. Study tip: Always track the sign when bias-correcting — you subtract the bias. And remember that second-order corrections scale as $n^{-1}$, not $n^{-1/2}$; if you see $\sqrt{n}$ in a bias-correction context, that's a red flag.
8. Suppose $\sqrt{n}\,\hat\theta_n \xrightarrow{d} N(0,\sigma^2)$, where $\sigma^2>0$. Which limiting distribution follows from an appropriate Taylor expansion of $1-\cos(\hat\theta_n)$?
   Answer: $n\{1-\cos(\hat\theta_n)\}\xrightarrow{d}\frac{\sigma^2}{2}\chi_1^2$ When you see a question like this, think delta method — specifically, what happens when you apply a smooth function to a sequence that's already been shown to converge in distribution. The key tool is a second-order Taylor expansion, used when the first derivative of the function vanishes at zero. Since $\hat\theta_n \to 0$ in probability (because $\sqrt{n}\hat\theta_n = O_p(1)$), expand $1 - \cos(\hat\theta_n)$ around zero: $1 - \cos(\theta) = \frac{\theta^2}{2} - \frac{\theta^4}{24} + \cdots$ So $n\{1-\cos(\hat\theta_n)\} \approx n \cdot \frac{\hat\theta_n^2}{2} = \frac{1}{2}({\sqrt{n}\,\hat\theta_n})^2.$ Since $\sqrt{n}\,\hat\theta_n \xrightarrow{d} N(0,\sigma^2)$, we can write $\sqrt{n}\,\hat\theta_n \xrightarrow{d} \sigma Z$ where $Z \sim N(0,1)$. Therefore: $n\{1-\cos(\hat\theta_n)\} \xrightarrow{d} \frac{(\sigma Z)^2}{2} = \frac{\sigma^2}{2} Z^2 = \frac{\sigma^2}{2}\chi_1^2.$ This confirms answer A. B is wrong because it drops the factor of $1/2$ from the Taylor expansion — a classic error from forgetting the denominator in $\theta^2/2$. C is wrong on two levels: the result cannot be normal (it's a squared quantity, hence non-negative), and the distribution family is entirely incorrect. D is wrong because $\chi_1^2$ is always non-negative, so a negative scaling is impossible; the negative sign has no mathematical justification. Your study tip: whenever the first derivative of your function is zero at the point of expansion, you must use a second-order Taylor expansion — this is the signature of the "second-order delta method," and the $1/2$ factor is non-negotiable.
9. What is the second-order approximation to $E(\bar X/\bar Y)$ through terms of order $n^{-1}$?
   Answer: $2+\frac{15}{4n}+o(n^{-1})$ When you see a ratio of sample means like $\bar X/\bar Y$, the tool to reach for is the delta method — specifically its second-order extension, which captures bias terms of order $n^{-1}$. Let $g(x,y) = x/y$. You need the partial derivatives at $(\mu_x, \mu_y) = (4, 2)$: $g_x = 1/y$, $g_y = -x/y^2$, $g_{xx} = 0$, $g_{yy} = 2x/y^3$, $g_{xy} = -1/y^2$. Evaluating at the means: $g_x = 1/2$, $g_y = -1$, $g_{yy} = 1$, $g_{xy} = -1/4$. The second-order approximation to $E[g(\bar X, \bar Y)]$ is: $g(\mu_x,\mu_y) + \tfrac{1}{2}g_{xx}\operatorname{Var}(\bar X) + \tfrac{1}{2}g_{yy}\operatorname{Var}(\bar Y) + g_{xy}\operatorname{Cov}(\bar X,\bar Y)$ Plugging in: the base value is $4/2 = 2$. The $g_{xx}$ term vanishes. The $g_{yy}$ term gives $\tfrac{1}{2}(1)(9/n) = 9/(2n)$. The cross term gives $(-1/4)(3/n) = -3/(4n)$. Summing: $2 + 9/(2n) - 3/(4n) = 2 + 18/(4n) - 3/(4n) = 2 + 15/(4n)$, confirming answer A. Answer B omits the cross-derivative term entirely, keeping only $9/(2n)$. Answer C keeps only the cross term $-3/(4n)$, dropping the $g_{yy}$ contribution. Answer D adds rather than subtracts the cross term, computing $18/(4n) + 3/(4n) = 21/(4n)$ — a sign error on $g_{xy}$. Study tip: Always compute all three second-order correction terms — variance in numerator, variance in denominator, and the covariance cross term. Sign errors on the cross term ($g_{xy} = -1/y^2 < 0$) are the most common trap here.
10. Let $X\sim\operatorname{Poisson}(\lambda)$, where $\lambda$ is large, and consider $T_c=2\sqrt{X+c}$. Using a second-order Taylor expansion in $X$ about $\lambda$, followed by an expansion in powers of $\lambda^{-1/2}$, which value of $c$ eliminates the leading bias term of order $\lambda^{-1/2}$ in estimating $2\sqrt\lambda$?
    Answer: $c=\frac{1}{4}$ This question tests your ability to apply variance-stabilizing transformations via Taylor expansion — a key technique for making estimators approximately unbiased and normal. When you see $T_c = 2\sqrt{X+c}$, think: "expand around the mean, collect bias terms, then choose $c$ to cancel them." Start with a second-order Taylor expansion of $f(X) = 2\sqrt{X+c}$ around $X = \lambda$: $E[T_c] \approx 2\sqrt{\lambda+c} + \frac{1}{2}f''(\lambda+c)\cdot\operatorname{Var}(X)$ Since $f''(x) = -\tfrac{1}{2}(x+c)^{-3/2}$ and $\operatorname{Var}(X) = \lambda$, the expectation becomes: $E[T_c] \approx 2\sqrt{\lambda+c} - \frac{\lambda}{4(\lambda+c)^{3/2}}$ Now expand $2\sqrt{\lambda+c} = 2\sqrt{\lambda}\left(1 + \tfrac{c}{\lambda}\right)^{1/2} \approx 2\sqrt{\lambda} + \frac{c}{\sqrt{\lambda}} - \frac{c^2}{4\lambda^{3/2}} + \cdots$ and the second term $\approx \frac{1}{4\sqrt{\lambda}}$. The bias relative to $2\sqrt{\lambda}$ at order $\lambda^{-1/2}$ is: $\text{Bias} \approx \frac{c}{\sqrt{\lambda}} - \frac{1}{4\sqrt{\lambda}} = \frac{c - \frac{1}{4}}{\sqrt{\lambda}}$ Setting this to zero gives $c = \tfrac{1}{4}$, confirming answer B is correct. Choice A ($c=0$) leaves a bias of $-\tfrac{1}{4\sqrt{\lambda}}$ — this is the naive square-root transformation with no bias correction. Choice C ($c=\tfrac{3}{8}$) is the famous Anscombe transformation that stabilizes variance and corrects skewness to second order, but it does not minimize the leading bias term alone. Choice D ($c=\tfrac{1}{2}$) overcorrects, leaving a positive residual bias. Study tip: On problems involving stabilizing transformations, always track which order of bias you're eliminating — different values of $c$ optimize different criteria (bias, skewness, variance stability), and exams love to mix these as distractors.

Start this quiz