Influence Functions - Statistics Graduate Level

1. Using the first-order contamination derivative, what is $\operatorname{IF}(4;T,F)$?
   Answer: $3.0$, because both the numerator and denominator change under contamination. When you see a functional of the form $T(F) = E_F(X^2)/E_F(X)$, recognize it as a ratio functional. The influence function requires applying the quotient rule to the contaminated functional $T(F_\epsilon) = E_{F_\epsilon}(X^2)/E_{F_\epsilon}(X)$, where $F_\epsilon = (1-\epsilon)F + \epsilon\delta_z$. Under contamination by a point mass at $z$, the numerator becomes $(1-\epsilon)E_F(X^2) + \epsilon z^2$ and the denominator becomes $(1-\epsilon)E_F(X) + \epsilon z$. Differentiating $T(F_\epsilon)$ with respect to $\epsilon$ at $\epsilon=0$ via the quotient rule gives: $\operatorname{IF}(z;T,F) = \frac{(z^2 - E_F(X^2)) \cdot E_F(X) - E_F(X^2)\cdot(z - E_F(X))}{[E_F(X)]^2}$ Plugging in $z=4$, $E_F(X)=2$, $E_F(X^2)=5$: $= \frac{(16-5)\cdot 2 - 5\cdot(4-2)}{4} = \frac{22 - 10}{4} = \frac{12}{4} = 3.0$ This confirms D is correct. Choice A incorrectly ignores the denominator's contribution entirely. Choice B simply reports $z^2/E_F(X) = 8$, confusing the raw contaminated value with the influence function derivative. Choice C computes only $(z^2 - E_F(X^2))/E_F(X) = 11/2 = 5.5$, applying only the numerator's centered derivative without accounting for the denominator's shift. As a study rule: whenever $T(F)$ is a ratio, always apply the full quotient rule for the IF — forgetting the denominator's derivative is the single most common error on these problems.
2. Which conclusion is justified solely from this information?
   Answer: The first estimator has bounded first-order sensitivity to infinitesimal point-mass contamination at the model. When studying robustness theory, the key is distinguishing precisely what each concept measures. The influence function (IF) quantifies the asymptotic effect of an infinitesimal point-mass contamination on an estimator's value. Formally, if $T$ is a functional and $F$ is the model distribution, the IF at point $x$ is: $\text{IF}(x; T, F) = \lim_{\epsilon \to 0} \frac{T((1-\epsilon)F + \epsilon\delta_x) - T(F)}{\epsilon}$ A bounded influence function means this derivative is uniformly bounded over all contamination points $x$, which is precisely a statement about first-order sensitivity to infinitesimal contamination. This is exactly what answer B states — it's a direct, definitional consequence of having a bounded IF, making B correct. Answer A is tempting but unjustified. Bounded IF and positive breakdown point are related robustness ideas, but neither implies the other. An estimator can have a bounded IF yet still have breakdown point zero (e.g., certain M-estimators with bounded scores but non-robust scale choices). The IF is a local, infinitesimal concept; breakdown point is a global, finite-sample concept. You cannot infer one from the other without additional information. Answer C is false — boundedness of the IF says nothing about asymptotic variance. In fact, high-breakdown estimators often trade efficiency for robustness, meaning the first estimator could have larger variance. Answer D overreaches significantly. A bounded IF only guarantees infinitesimal-contamination stability; it provides no uniform guarantee over a fixed positive contamination fraction $\epsilon > 0$. Study tip: Always match robustness concepts to their precise scope — IF is infinitesimal and asymptotic, breakdown point is finite and global. Exam distractors frequently blur this boundary.
3. Approximately what is the influence function at a contamination point $z=4$?
   Answer: $1.731$, obtained by scaling the clipped score by the inverse expected derivative. When you encounter influence function (IF) questions, the key formula to internalize is: $\text{IF}(z; T, F) = \frac{\psi(z - T(F))}{E_F[\psi'(X - T(F))]}$ This comes from differentiating the estimating equation $E_F[\psi(X - T(F))] = 0$ with respect to contamination at point $z$. The numerator captures how the score function evaluates the outlier; the denominator measures the estimator's average sensitivity. At $T(F) = 0$ and $z = 4$, the numerator is $\psi(4) = \min(4, 1.5) = 1.5$ — the score is clipped. For the denominator, since $\psi$ is the Huber-type clip function, $\psi'(u) = 1$ when $|u| \le 1.5$ and $0$ otherwise. So $E_F[\psi'(X)] = P_F(|X| \le 1.5) = 0.8664$. The influence function is therefore: $\text{IF}(4) = \frac{1.5}{0.8664} \approx 1.731$ This confirms C is correct. A is wrong because it inverts the formula — dividing the expected derivative by the score, rather than the score by the expected derivative. B simply reads off the clipped score (1.5) and ignores the denominator entirely, failing to account for how steeply the estimator responds to perturbations. D uses the raw, unclipped value $z = 4$ in the numerator, as if $\psi$ were the identity function — a fundamental misreading of what the redescending/bounded $\psi$ does to outliers. Study tip: For M-estimator influence functions, always write down the estimating equation first, then differentiate. The denominator $E[\psi']$ is never 1 unless $\psi$ is the identity — don't skip it.
4. What is the influence function for $\theta$ at the contamination point $z=1$?
   Answer: $4.00$, because the probability influence is scaled by the derivative of the log-odds. Whenever you encounter influence function questions, your instinct should be to apply the chain rule through the statistical functional. The influence function of a transformed functional $\theta(F) = g(\mu(F))$ is: $\text{IF}(z; \theta, F) = g'(\mu) \cdot \text{IF}(z; \mu, F)$ Here, $\mu(F) = p = E_F[X]$ and $g(p) = \log\{p/(1-p)\}$. The influence function for the mean functional at point $z$ is simply $\text{IF}(z; \mu, F) = z - p$. At $z = 1$ and $p = 0.25$, this equals $1 - 0.25 = 0.75$. The derivative of the log-odds is $g'(p) = 1/[p(1-p)]$. At $p = 0.25$: $g'(0.25) = 1/(0.25 \times 0.75) = 1/0.1875 \approx 5.33$. Multiplying gives $5.33 \times 0.75 = 4.00$, confirming C is correct. Choice A stops too early — $0.75$ is the influence function for the mean functional, not the log-odds. It ignores the chain rule transformation entirely. Choice B computes $0.75 / 0.25 = 3.00$, dividing only by $p$ rather than by the full Bernoulli variance $p(1-p)$, which misapplies the derivative of the log-odds. Choice D reports $5.33$, which is $g'(p)$ alone — the derivative of the log-odds evaluated at $p$ — but forgets to multiply by the mean's influence function $z - p = 0.75$. As a study tip: always decompose functional influence functions using the chain rule — identify the inner functional's IF first, then scale by the outer function's derivative. This two-step process prevents every trap present in this question.
5. What is the influence function of $V$ at a contamination point $z=4$?
   Answer: $5$, obtained by centering the squared deviation by the population variance. When you encounter influence function questions, your goal is to quantify how a point mass contamination at $z$ shifts a functional — essentially, the derivative of the functional along the direction of that contamination. For the variance functional $V(F) = \int (x - \mu_F)^2 \, dF(x)$, the influence function is derived by considering $V(F_\epsilon) = V((1-\epsilon)F + \epsilon\delta_z)$ and differentiating with respect to $\epsilon$ at $\epsilon = 0$. Working through this carefully (accounting for the fact that $\mu_F$ also shifts with contamination), the influence function evaluates to $(z - \mu_F)^2 - V(F)$. Plugging in $z = 4$, $\mu_F = 1$, and $V(F) = 4$: $(4-1)^2 - 4 = 9 - 4 = 5$. This is choice B — and the description "centering the squared deviation by the population variance" captures precisely this structure. Choice A gives $3$, which is just $z - \mu_F = 4 - 1$, the raw deviation — not squared, not centered. This ignores almost all of the influence function's structure. Choice C gives $9 = (z - \mu_F)^2$, which is the squared deviation alone; this student forgot to subtract $V(F)$, missing the centering step entirely. Choice D gives $-5$, which reverses the subtraction — subtracting the squared deviation from the variance rather than the variance from the squared deviation. A useful memory device: the influence function of variance looks like a "centered" squared deviation — $(z - \mu)^2 - \sigma^2$ — analogous to how variance itself centers squared deviations around their mean. Always remember to subtract $V(F)$.
6. Which expression gives the appropriate first-order approximation to $T(F_n^{\mathrm{rep}})-T(F_n)$?
   Answer: $n^{-1}\{\operatorname{IF}(z;T,F_n)-\operatorname{IF}(x_i;T,F_n)\}$, because replacement both adds and removes mass. When working with influence functions and empirical distributions, the key is tracking exactly how the distribution changes under each operation — adding mass, removing mass, or both simultaneously. When you replace observation $x_i$ with a new value $z$, you are doing two things at once: removing a point mass of size $1/n$ at $x_i$ and adding a point mass of size $1/n$ at $z$. The influence function $\operatorname{IF}(x; T, F)$ captures the first-order effect of adding an infinitesimal point mass at $x$. By linearity of the first-order expansion, removing mass at $x_i$ contributes $-n^{-1}\operatorname{IF}(x_i; T, F_n)$ and adding mass at $z$ contributes $+n^{-1}\operatorname{IF}(z; T, F_n)$, giving the net approximation $n^{-1}\{\operatorname{IF}(z;T,F_n)-\operatorname{IF}(x_i;T,F_n)\}$. This confirms A is correct. Choice B is wrong because it only accounts for the addition of $z$ and completely ignores that $x_i$ is simultaneously removed — a half-treatment of the perturbation. Choice C is wrong on two counts: it uses $(n-1)^{-1}$ instead of $n^{-1}$, and it ignores the contribution from $z$ entirely, focusing only on the deleted point. Choice D is a sign error — it adds both influence function values instead of subtracting, which would correspond to adding mass at both locations simultaneously rather than a replacement. A reliable memory anchor: replacement = addition minus deletion. Whenever a question involves swapping one observation for another, write out the two separate perturbations explicitly and track their signs before combining — this prevents both the sign error in D and the incomplete accounting in B and C.
7. What does the influence-function calculation imply about ordinary least squares in this sequence of contamination points?
   Answer: Its influence is generally unbounded because $M^{-1}xr$ grows with leverage when the residual is fixed. When analyzing robustness of an estimator, your first instinct should be to examine its influence function — the functional derivative measuring how sensitive the estimator is to infinitesimal contamination at a single point. For OLS in the random-design setting, the regression functional $\beta(F) = M^{-1}E(XY)$ yields an influence function proportional to $M^{-1}x r$, where $r = y - x^\mathsf{T}\beta$ is the residual at the contamination point $(x, y)$. This is exactly why C is correct. When $r$ stays fixed but $\|x\| \to \infty$, the term $M^{-1}xr$ grows without bound — its norm scales with $\|x\|$. This is precisely the leverage effect: a point far from the center of the covariate space exerts unbounded influence on the fitted coefficients, even with a modest residual. OLS has no mechanism to down-weight high-leverage points, so its influence function is unbounded in the covariate direction. A reverses the logic. The fixed residual doesn't "cancel out" the growing covariate norm — both enter multiplicatively in $M^{-1}xr$, and the growing norm dominates. B is factually wrong: the estimating equation $E\{X(Y - X^\mathsf{T}\beta)\} = 0$ explicitly involves $X$, so the covariate absolutely enters the influence calculation. D is false — influence functions are perfectly well-defined for smooth statistical functionals, including random-design regression, as long as the functional is Gâteaux-differentiable at the model distribution. As a study strategy: whenever a question pairs "fixed residual" with "growing covariate," immediately think leverage unboundedness. This is a classic argument for why robust alternatives like M-estimators with bounded $\psi$-functions are preferred over OLS when high-leverage contamination is possible.
8. What is the asymptotic variance of $\sqrt{n}\{T_n-T(F)\}$?
   Answer: $6$, because the limiting variance is the second moment of the influence function. Whenever you encounter an asymptotic linear representation, your first instinct should be: the asymptotic variance of $\sqrt{n}(T_n - T(F))$ equals the variance of the influence function. By the CLT applied to the i.i.d. sum, $\sqrt{n}(T_n - T(F)) \xrightarrow{d} N(0, \text{Var}_F(\operatorname{IF}(X;T,F)))$. So the task reduces to computing $\text{Var}_F(X^2 - 2)$. Since $E_F(X^2) = 2$, the influence function $\operatorname{IF}(X;T,F) = X^2 - 2$ is already mean-zero (a requirement for valid influence functions). Its variance is: $\text{Var}_F(X^2 - 2) = E_F[(X^2-2)^2] = E_F[X^4 - 4X^2 + 4] = E_F(X^4) - 4E_F(X^2) + 4 = 10 - 8 + 4 = 6.$ This confirms D is correct: the asymptotic variance is $6$, the second moment of the influence function (i.e., its variance, since it's centered). A is wrong because it claims the variance equals $2 = E_F(X^2)$, confusing the mean of $X^2$ with the variance of the influence function — these are entirely different quantities. B incorrectly equates the asymptotic variance with $E_F(X^4) = 10$, ignoring that you must subtract the squared mean of $X^2$ (and cross terms) when computing the variance. C arrives at $8$ by subtracting only $4E_F(X^2) = 8$ from $10$, forgetting to add back the constant term $+4$ in the expansion. Your study tip: always expand $E_F[(\operatorname{IF})^2]$ fully using $\text{Var}(Y) = E[Y^2] - (E[Y])^2$, and verify the influence function is mean-zero before proceeding.
9. For $F_{\varepsilon}=(1-\varepsilon)F+\varepsilon\Delta_z$, what is the influence function $\operatorname{IF}(z;T,F)$?
   Answer: $2.5$, because the median must move upward to restore cumulative probability one-half. Whenever you encounter influence function questions, your anchor should be the formal definition: $\operatorname{IF}(z; T, F) = \lim_{\varepsilon \to 0} \frac{T(F_\varepsilon) - T(F)}{\varepsilon}$, which measures the first-order sensitivity of a functional to infinitesimal contamination at point $z$. For the sample median, the derivation proceeds by solving for where the contaminated distribution $F_\varepsilon = (1-\varepsilon)F + \varepsilon\Delta_z$ achieves cumulative probability one-half. Setting $F_\varepsilon(m_\varepsilon) = 1/2$ and differentiating implicitly with respect to $\varepsilon$ at $\varepsilon = 0$ yields the closed-form result: $\operatorname{IF}(z; T, F) = \frac{\operatorname{sgn}(z - m)}{2f(m)}$ Since $z > m$, $\operatorname{sgn}(z - m) = +1$, so $\operatorname{IF}(z; T, F) = \frac{1}{2 \times 0.20} = \frac{1}{0.40} = 2.5$. The median must shift upward to compensate for the extra mass above it — confirming B is correct. Choice A gets the sign wrong. Contamination above the median pushes the median upward, not downward; the distribution function at the original median increases, forcing the quantile to move right, not left. Choice C arrives at $5.0$ by using $1/f(m)$ alone, forgetting the factor of $2$ in the denominator — a classic algebra slip. Choice D produces $0.10$ by multiplying $\varepsilon/2$ thinking proportionally, which confuses a finite-sample heuristic with the correct asymptotic derivative formula. Your study tip: always memorize the median's influence function as $\frac{\operatorname{sgn}(z-m)}{2f(m)}$. The factor of 2 reflects the two-sided nature of the median condition, and $f(m)$ in the denominator means the influence is larger when the density is flat — intuition worth retaining.
10. Which statement correctly describes the influence function and the first nondegenerate asymptotic behavior of this estimator?
    Answer: The influence function is zero, and $n\bar X_n^2$ converges in distribution to $\sigma^2\chi_1^2$. When a functional $T(F)$ is smooth, its plug-in estimator inherits a $\sqrt{n}$-rate via the delta method, driven by a nonzero influence function. But when the influence function vanishes, the estimator is second-order, and you need to look one level deeper. The influence function of $T(F) = \{E_F(X)\}^2$ is the Gateaux derivative: $\text{IF}(x; T, F) = \frac{d}{d\epsilon}\Big|_{\epsilon=0} T((1-\epsilon)F + \epsilon\delta_x) = 2E_F(X)\cdot(x - E_F(X)).$ Since $E_F(X) = 0$ by assumption, the influence function equals zero. This means the standard $\sqrt{n}$ central limit theorem gives a degenerate (zero) limit for $\sqrt{n}\,\bar{X}_n^2$, so you must rescale by $n$ instead. By the CLT, $\sqrt{n}\,\bar{X}_n \xrightarrow{d} N(0, \sigma^2)$, and therefore $n\bar{X}_n^2 = \left(\sqrt{n}\,\bar{X}_n\right)^2 \xrightarrow{d} \sigma^2 \chi_1^2,$ confirming answer A. Answer B is wrong on both counts: the influence function is not $2X$ (that would require $E_F(X)\neq 0$), and $\sqrt{n}\,\bar{X}_n^2 \to 0$ in probability, not a normal. Answer C confuses $T(F)$ with the variance functional $\text{Var}_F(X) = E[X^2] - (E[X])^2$; the influence function $X^2 - \sigma^2$ belongs to a different functional entirely. Answer D gets the rescaling right but claims convergence in probability to a constant — that would require a degenerate limit, but $\sigma^2\chi_1^2$ is a genuine random variable, not a constant. Study tip: Whenever $E_F(X) = 0$ makes the first-order influence function vanish, immediately switch to a second-order analysis — rescale by $n$, apply the continuous mapping theorem to the squared CLT limit, and expect a chi-squared (not normal) distribution.

Start this quiz