Kernel Density Estimation - Statistics Graduate Level

1. If the original bandwidth is $h_X$, which bandwidth and density relationship make the kernel density estimator exactly equivariant under this transformation?
   Answer: Use $h_Y=3h_X$, giving $\widehat f_Y(y)=\frac13\widehat f_X\!\left(\frac{y-2}{3}\right).$ Whenever you see a question about kernel density estimation (KDE) under linear transformations, anchor your thinking to two requirements simultaneously: how the bandwidth must scale, and how the density value must transform to preserve proper probability mass. For a linear transformation $Y = aX + b$ with $a > 0$, the KDE achieves exact equivariance — meaning it behaves like the true density under the transformation — when the bandwidth scales as $h_Y = a \cdot h_X$. Here, $a = 3$, so $h_Y = 3h_X$. This makes intuitive sense: stretching the data by a factor of 3 spreads observations apart, so the smoothing window must widen proportionally to maintain the same relative smoothing. For the density relationship, recall that if $Y = 3X + 2$, then the change-of-variables formula requires $f_Y(y) = \frac{1}{|a|} f_X\!\left(\frac{y-b}{a}\right) = \frac{1}{3} f_X\!\left(\frac{y-2}{3}\right)$. The factor $\frac{1}{3}$ ensures the density integrates to 1 after stretching. Combining both requirements confirms C is correct. A uses the wrong bandwidth ($h_Y = h_X$ instead of $3h_X$) but accidentally gets the density formula right — it's a half-correct answer that fails equivariance because the bandwidth doesn't account for the scale change. B has the correct bandwidth $h_Y = 3h_X$ but inverts the density scaling factor, writing $3\widehat{f}_X$ instead of $\frac{1}{3}\widehat{f}_X$. This would produce a density that doesn't integrate to 1 — a fundamental violation. D shrinks the bandwidth ($h_Y = h_X/3$) when it should expand, and omits the necessary $\frac{1}{3}$ normalizing factor entirely. A reliable memory aid: for $Y = aX + b$, bandwidth multiplies by $|a|$ and the density divides by $|a|$. These two adjustments always move in opposite directions.
2. For observations $0,1,2$, consider the kernel density estimator $\widehat f_h(x)=\frac{1}{nh}\sum_{i=1}^n K\!\left(\frac{x-X_i}{h}\right),$ where $h=\frac12$ and $K(u)=\frac{15}{16}(1-u^2)^2\mathbf 1\{|u|\leq 1\}.$ What is $\widehat f_h\!\left(\frac34\right)$?
   Answer: $\frac{45}{128}$ Kernel density estimation questions test your ability to mechanically apply the KDE formula, which requires careful bookkeeping of which kernels actually "activate" at your target point. At $x = \frac{3}{4}$ with $h = \frac{1}{2}$ and $n = 3$, the estimator is: $\widehat{f}_h\!\left(\tfrac{3}{4}\right) = \frac{1}{3 \cdot \frac{1}{2}}\sum_{i=1}^{3} K\!\left(\frac{\frac{3}{4} - X_i}{\frac{1}{2}}\right) = \frac{2}{3}\sum_{i=1}^{3} K\!\left(\frac{\frac{3}{4} - X_i}{\frac{1}{2}}\right)$ The Epanechnikov kernel $K(u) = \frac{15}{16}(1-u^2)^2$ is zero when $|u| > 1$. Check each observation: - $X_i = 0$: $u = \frac{3/4}{1/2} = \frac{3}{2}$, so $|u| > 1$ — kernel is zero. - $X_i = 1$: $u = \frac{-1/4}{1/2} = -\frac{1}{2}$, so $K(-\tfrac{1}{2}) = \frac{15}{16}(1 - \tfrac{1}{4})^2 = \frac{15}{16} \cdot \frac{9}{16} = \frac{135}{256}$. - $X_i = 2$: $u = \frac{-5/4}{1/2} = -\frac{5}{2}$, so $|u| > 1$ — kernel is zero. Only $X_i = 1$ contributes, giving: $\widehat{f}_h\!\left(\tfrac{3}{4}\right) = \frac{2}{3} \cdot \frac{135}{256} = \frac{270}{768} = \frac{45}{128}$ That confirms A. Choice B, $\frac{135}{256}$, is the trap of forgetting to multiply by the prefactor $\frac{2}{3}$ — you evaluated the kernel but skipped the normalization. Choice D, $\frac{135}{128}$, doubles instead of multiplying by $\frac{2}{3}$. Choice C, $\frac{45}{256}$, uses $\frac{1}{3}$ as the prefactor instead of $\frac{2}{3}$, confusing $\frac{1}{n}$ with $\frac{1}{nh}$. Always compute the standardized argument $u = (x - X_i)/h$ first to identify which observations fall within the kernel's support before evaluating anything else.
3. Which approximation to the expectation of the kernel density estimator at $x_0$ is correct?
   Answer: $\operatorname{E}[\widehat f_h(x_0)]\approx f(x_0)-0.03,$ reflecting downward smoothing bias at a local maximum. Whenever you see a question about kernel density estimation bias, your first instinct should be to recall the standard bias expansion. For a symmetric kernel at an interior point, the bias formula is: $\operatorname{E}[\widehat{f}_h(x_0)] \approx f(x_0) + \frac{1}{2}h^2\mu_2(K)f''(x_0)$ This result comes from a Taylor expansion of $f$ inside the convolution integral; the symmetry of the kernel kills the odd-order terms, leaving the leading bias proportional to $h^2 f''(x_0)$. Plugging in the given values — $h = 0.1$, $\mu_2(K) = 1$, and $f''(x_0) = -6$ — you get: $\text{Bias} = \frac{1}{2}(0.1)^2(1)(-6) = \frac{1}{2}(0.01)(-6) = -0.03$ So $\operatorname{E}[\widehat{f}_h(x_0)] \approx f(x_0) - 0.03$, which is answer B. The negative sign makes intuitive sense: a point where $f''(x_0) < 0$ is a local maximum, and smoothing pulls estimates downward toward neighboring lower values — a classic downward bias. A gets the sign wrong. Upward bias would require $f'' > 0$ (a local minimum), not a local maximum. C omits the crucial $\frac{1}{2}$ factor and uses $h^2 f''(x_0) = -0.06$ directly — a common algebraic error that forgets the coefficient in the Taylor expansion. D claims the bias is linear in $h$, which contradicts the theory; the leading bias is always of order $h^2$ for symmetric kernels. The key memory anchor: bias $= \frac{1}{2}h^2\mu_2(K)f''(x_0)$. Never drop the $\frac{1}{2}$, and always check the sign of $f''$ to determine the direction of smoothing bias.
4. Which statement about the resulting estimator is necessarily correct?
   Answer: It integrates to one but can be negative at some points, so it need not be a genuine density. When you encounter a question about higher-order or signed kernels, the key is carefully separating two properties that a genuine probability density must satisfy: nonnegativity everywhere, and integrating to one. These two properties are logically independent, and signed kernels break them asymmetrically. The condition $\int K(u)\,du = 1$ is preserved under the estimator's construction. Since $\widehat{f}_h(x) = \frac{1}{nh}\sum_{i=1}^n K\!\left(\frac{x-X_i}{h}\right)$, integrating over all $x$ gives $\int \widehat{f}_h(x)\,dx = \frac{1}{n}\sum_{i=1}^n \int K\!\left(\frac{x-X_i}{h}\right)\frac{dx}{h} = \frac{1}{n}\sum_{i=1}^n 1 = 1,$ regardless of the sample. So the estimator always integrates to one. However, where $K(u) < 0$, the estimator can take negative values for certain $x$, violating nonnegativity. This makes C correct: the estimator integrates to one but is not guaranteed to be a valid density. Answer A is wrong because the estimator is not necessarily nonnegative — negative kernel lobes directly produce negative estimates. It also incorrectly claims the integral deviates from one, which contradicts the linearity argument above. Answer B is wrong for the same integral reason: the integral is always exactly one, not "less than one," because each kernel shift integrates to one regardless of where observations fall. Answer D is wrong because no sample size, however large, eliminates the possibility of negative values at specific points — that is a structural feature of the signed kernel, not a finite-sample artifact. Your takeaway: for signed kernels, the normalization condition travels through linearity cleanly, but nonnegativity does not. Always ask "which density property does linearity preserve?" — the answer is integration to one, never pointwise positivity.
5. What is the probability limit of the estimator evaluated exactly at the left endpoint, $\widehat f_{h_n}(0)$?
   Answer: It converges to $\frac12$ because only one half of the symmetric kernel mass overlaps the support. Kernel density estimation at boundary points is a classic source of asymptotic bias, and recognizing why requires you to think carefully about how mass integrates over a truncated region. The kernel density estimator is $\widehat{f}_{h_n}(x) = \frac{1}{nh_n}\sum_{i=1}^n K\!\left(\frac{x - X_i}{h_n}\right).$ Its pointwise probability limit equals its limiting expected value (by the law of large numbers under standard bandwidth conditions). At any interior point, $E[\widehat{f}_{h_n}(x)] \to f(x)$ because the full kernel integrates to 1 over the support. But at $x = 0$, the situation changes: the kernel is centered at the boundary, and since the data live only on $[0,1]$, only the right half of the symmetric kernel actually overlaps the support. Formally, $E[\widehat{f}_{h_n}(0)] = \int_0^\infty \frac{1}{h_n} K\!\left(\frac{-t}{h_n}\right) f(t)\,dt \;\to\; f(0)\int_0^\infty K(u)\,du = 1 \cdot \tfrac{1}{2} = \tfrac{1}{2}.$ Because $f \equiv 1$ on $[0,1]$, the limit is $\frac{1}{2}$, confirming answer B. Choice A is wrong because contributions from nearby positive observations do not vanish — they simply get cut in half. Choice C applies interior consistency results where boundary effects are absent; consistency fails at the boundary precisely because the estimator carries systematic bias toward zero. Choice D invents a renormalization that a standard kernel estimator never performs — that would describe a boundary-corrected estimator. As a study tip: whenever a boundary is present, immediately check which fraction of kernel mass lies inside the support. Symmetric kernels lose exactly half their mass at a hard boundary, so the limiting bias factor is always $\frac{1}{2}$.
6. How does the kernel density estimate at $x$ change after adding the observation?
   Answer: It becomes $\frac{n}{n+1}\widehat f_h(x)$ because only the normalization changes at $x$. Whenever you see a question about kernel density estimation, anchor yourself to the formula: $\widehat{f}_h(x) = \frac{1}{nh} \sum_{i=1}^{n} K\!\left(\frac{x - X_i}{h}\right)$ Every observation contributes a scaled kernel bump, but only observations within bandwidth $h$ of $x$ contribute nonzero values when the kernel has compact support $[-1,1]$. Here's the key insight: the new observation is farther than $h$ from $x$, so $K\!\left(\frac{x - X_{n+1}}{h}\right) = 0$. Its numerator contribution is exactly zero. However, the denominator still grows — the sample size increases from $n$ to $n+1$. The updated estimate becomes: $\widehat{f}_{h,\text{new}}(x) = \frac{1}{(n+1)h} \sum_{i=1}^{n} K\!\left(\frac{x - X_i}{h}\right) = \frac{n}{n+1} \cdot \widehat{f}_h(x)$ So the estimate at $x$ is diluted by the factor $\frac{n}{n+1}$, making D correct. A is tempting but incomplete — yes, the new observation contributes zero kernel mass, but you cannot ignore how the normalization constant changes. B is wrong because it assumes the new observation contributes a full $\frac{1}{(n+1)h}$ bump, which requires $K(\cdot) = 1$, impossible when the observation falls outside the kernel's support. C inverts the logic entirely — a larger sample size dilutes the existing contributions rather than inflating them. The study tip: always track both the numerator (kernel evaluations) and the denominator ($nh$) separately. In KDE, adding a remote observation affects only the denominator, quietly shrinking every existing estimate.
7. Which bandwidth order and resulting optimized error order follow from balancing these terms?
   Answer: $h\asymp n^{-1/6}$ and $\operatorname{AMISE}\asymp n^{-2/3},$ reflecting the two-dimensional variance term. Whenever you see a kernel density estimation question involving AMISE balancing, your job is to treat bias and variance as two terms in an equation and find the bandwidth that minimizes their sum — then substitute back to find the optimal error rate. Here, the AMISE has two terms: a bias-squared term $h^4$ and a variance term $(nh^2)^{-1}$. To find the optimal $h$, set them equal in order of magnitude: $h^4 \asymp \frac{1}{nh^2}$ Multiply both sides by $nh^2$: $nh^6 \asymp 1 \implies h \asymp n^{-1/6}$ Substituting back, the bias term gives $(n^{-1/6})^4 = n^{-2/3}$, and the variance term gives $(n \cdot n^{-2/6})^{-1} = n^{-2/3}$. Both terms are $n^{-2/3}$, confirming $\operatorname{AMISE} \asymp n^{-2/3}$. This is answer D. A is the classic univariate result, where the variance term is $(nh)^{-1}$. In 2D with a product kernel, the variance inflates to $(nh^2)^{-1}$ because the effective local bin volume scales as $h^2$, not $h$. Plugging in univariate logic here gives the wrong exponent. B arrives at the right AMISE rate but the wrong bandwidth. It appears to account for the squared bandwidth in the variance but mis-solves the balance equation — setting $h^4 \asymp (nh^2)^{-1}$ does not yield $h \asymp n^{-1/3}$. C incorrectly supposes that adding a second coordinate changes the bias order, but the bias remains $O(h^2)$ per dimension with a squared-kernel product, keeping the squared bias at $h^4$. Study tip: In $d$ dimensions, the product-kernel variance term is $(nh^d)^{-1}$, so the optimal bandwidth scales as $n^{-1/(4+d)}$ and AMISE as $n^{-4/(4+d)}$. For $d=2$: $n^{-1/6}$ and $n^{-2/3}$. Memorizing this general formula saves time on any dimensionality question.
8. Why is the ordinary in-sample criterion unsuitable for bandwidth selection in this setting?
   Answer: It tends to favor extremely small bandwidths because each observation contributes a large kernel value to its own fitted density. When choosing a bandwidth for kernel density estimation, you need a criterion that genuinely measures how well your estimated density generalizes — not just how well it memorizes the data. The in-sample log-likelihood $\sum_{i=1}^n \log \widehat{f}_h(X_i)$ is fatally flawed because each observation $X_i$ participates in its own estimated density value $\widehat{f}_h(X_i)$. With a Gaussian kernel, the contribution of observation $X_i$ to its own density estimate is $\frac{1}{nh}\phi(0) = \frac{1}{nh\sqrt{2\pi}},$ which grows without bound as $h \to 0$. As the bandwidth shrinks, each point's self-contribution dominates and inflates the log-likelihood artificially. The optimizer therefore drives $h$ toward zero, producing degenerate "spike" estimates — one sharp Gaussian centered at each data point — rather than a smooth, meaningful density. This confirms that A is correct. B is wrong because it describes the leave-one-out criterion, not the in-sample one. In leave-one-out cross-validation, $X_i$ is deliberately excluded from $\widehat{f}_{h,-i}(X_i)$, which is exactly why that approach avoids the collapse-to-zero problem. C is wrong because the criterion absolutely depends on $h$. While each Gaussian kernel integrates to one (a property ensuring $\widehat{f}_h$ is a valid density), the pointwise evaluation at each $X_i$ changes with $h$ — the two concepts should not be conflated. D is wrong because the in-sample criterion is not an unbiased estimator of predictive log-likelihood; it is systematically upward-biased due to the self-evaluation problem described above. As a study tip: whenever a selection criterion evaluates a model on the same data used to fit it, ask whether any observation is "cheating" by inflating its own fit — this self-evaluation trap appears frequently in density estimation and model selection questions.
9. For a twice-differentiable density and a symmetric second-order kernel, suppose the leading asymptotic mean integrated squared error has the form $\operatorname{AMISE}(h)=C_1h^4+\frac{C_2}{nh},$ where $C_1$ and $C_2$ are positive constants. Restrict attention to bandwidths of the form $h_n=n^{-\alpha}.$ Which choice yields the fastest decay of the displayed AMISE while also satisfying the usual consistency conditions?
   Answer: $\alpha=\frac15,$ because it balances the integrated squared-bias and variance terms. Kernel density estimation questions hinge on the bias-variance tradeoff baked into the AMISE. When you see a form like $C_1 h^4 + \frac{C_2}{nh}$, your instinct should be: minimize over $h$, then check what power of $n$ that optimal bandwidth takes. Taking the derivative with respect to $h$ and setting it to zero gives $4C_1 h^3 - \frac{C_2}{nh^2} = 0,$ which yields $h^* \propto n^{-1/5}.$ So the optimal bandwidth has the form $h_n = n^{-\alpha}$ with $\alpha = \frac{1}{5}.$ Plugging back in, both terms contribute equally and AMISE decays at rate $n^{-4/5}$ — the fastest possible for this setup. Consistency also requires $h \to 0$ and $nh \to \infty$, both satisfied since $\alpha = 1/5$ means $h \to 0$ and $nh = n^{4/5} \to \infty$. That confirms B is correct. Choice A ($\alpha = 0$) fixes $h$ as a constant, so $h \not\to 0$ and the bias never vanishes — the estimator is inconsistent. Choice C ($\alpha = 1/3$) makes $h$ shrink faster than optimal, so the variance term $C_2/(nh) \propto n^{-2/3}$ dominates and AMISE decays only at rate $n^{-2/3}$, which is slower than $n^{-4/5}$. It's consistent but suboptimal. Choice D ($\alpha = 1$) shrinks $h$ too aggressively: $nh = n^0 = 1$, so $nh \not\to \infty$ and the variance explodes — another inconsistent choice. Study tip: Always verify two things for KDE bandwidth sequences: the AMISE-minimizing rate (set derivative to zero) and the consistency conditions ($h \to 0$, $nh \to \infty$). Forgetting the second step is the most common trap on these questions.
10. Which comparison is correct to leading asymptotic order?
    Answer: $K_1$ has the same leading bias and a smaller leading variance than $K_2$. Kernel density estimation questions like this one test whether you understand which kernel properties govern bias versus variance in the asymptotic MSE decomposition. For a kernel estimator with bandwidth $h \to 0$, the leading bias at an interior point depends on $\mu_2(K) = \int u^2 K(u)\,du$ — the second moment of the kernel — through the expression $\frac{h^2}{2}\mu_2(K) f''(x)$. The leading variance depends on $R(K) = \int K(u)^2\,du$ through $\frac{1}{nh}R(K)f(x)$. These two quantities are entirely separate: bias is controlled by the second moment, variance by the roughness functional $R(K)$. The passage tells you that $K_1$ and $K_2$ share the same second moment but differ in roughness, with $R(K_1) < R(K_2)$. Since both kernels are symmetric, integrate to one, and have identical second moments, their leading bias terms are identical. However, because $R(K_1) < R(K_2)$, $K_1$ produces a strictly smaller leading variance. This confirms A as correct. Choice B is wrong because it claims $K_1$ has a smaller bias — but bias depends only on the second moment, which is equal for both kernels. Choice C compounds this error by also claiming a larger bias for $K_1$, which contradicts the identical second moments. Choice D gets the variance comparison exactly backwards: smaller $R(K)$ means smaller variance, so $K_1$'s advantage is lower variance, not higher. A useful rule to memorize: bias tracks the second moment; variance tracks $R(K)$. These are independent kernel properties, so a question changing one while fixing the other affects only one component of MSE.

Start this quiz