p-Values & Test Inversion - Statistics Graduate Level

1. Without assuming independence among the p-values, which coverage claim follows from test inversion?
   Answer: $P_{\boldsymbol\theta}\{\boldsymbol\theta\in C\}\geq1-\alpha$ for every parameter vector, by the union bound. Whenever you see confidence regions built by test inversion, your first instinct should be to connect coverage of the joint set to the probability that at least one marginal interval fails — that's exactly where the union bound lives. Here's the core logic. The joint set $C = C_1 \times \cdots \times C_m$ contains the true $\boldsymbol{\theta}$ if and only if every coordinate is covered: $\boldsymbol{\theta} \in C \iff \theta_j \in C_j \text{ for all } j$. Its complement is the event that at least one coordinate fails. By the union bound (Bonferroni inequality), $P_{\boldsymbol\theta}\{\boldsymbol\theta \notin C\} \leq \sum_{j=1}^m P_{\boldsymbol\theta}\{\theta_j \notin C_j\} \leq \sum_{j=1}^m \frac{\alpha}{m} = \alpha.$ Each marginal failure probability is at most $\alpha/m$ because $p_j(\theta_j^0) > \alpha/m$ with probability at least $1 - \alpha/m$ (valid p-value guarantee). No independence assumption is needed — the union bound holds for any dependence structure. This confirms A is correct. B is wrong because multiplying marginal coverages — giving $(1-\alpha/m)^m \approx e^{-\alpha} < 1-\alpha$ — requires independence and actually underestimates the joint coverage, yielding a weaker and incorrect guarantee. C is wrong because $1 - \alpha/m$ is the marginal coverage of a single $C_j$, not the joint coverage of the rectangular set $C$. D is wrong because it adds an independence condition that the union bound never requires; the Bonferroni argument is assumption-free about dependence. Study tip: On exam questions mixing multiple testing and confidence sets, always ask whether the guarantee needs independence. If it's Bonferroni/union-bound based, the answer is no — that's its chief advantage over product-based arguments.
2. Which statement correctly describes inversion of $p_{\mathrm{sup}}(\theta_0)$ at level $\alpha$?
   Answer: It equals the projection of the joint nonrejection set onto the $\theta$ coordinate, and this construction has coverage at least $1-\alpha$. Whenever you see a question involving nuisance parameters and confidence set construction, think carefully about the relationship between p-value inversion and set operations — the geometry matters enormously. Start with what p-value inversion means: the set $\{\theta_0 : p_{\text{sup}}(\theta_0) > \alpha\}$ is what we call the inverted confidence set. Because $p_{\text{sup}}(\theta_0) = \sup_{\eta_0} p(\theta_0, \eta_0)$, this condition holds if and only if there exists some $\eta_0$ such that $p(\theta_0, \eta_0) > \alpha$ — meaning $(\theta_0, \eta_0)$ falls in the joint nonrejection set for at least one $\eta_0$. Geometrically, taking the union over $\eta_0$ of all joint nonrejection sets and then reading off which $\theta_0$ values appear is exactly the projection of the joint nonrejection set onto the $\theta$-axis. Since the true $(\theta, \eta)$ is covered by the joint set with probability at least $1-\alpha$, and projection only enlarges sets, the marginal coverage for $\theta$ alone is at least $1-\alpha$. This confirms B is correct. A is wrong on both counts: the intersection over $\eta_0$ of nonrejection sets corresponds to requiring all $\eta_0$ to fail rejection — that's a much smaller, more conservative set, not what the sup construction produces. Calling it undercovering compounds the error. C confuses the nonrejection set with the rejection set. Projecting the rejection set gives you values of $\theta$ that are excluded, not included, and coverage is at least $1-\alpha$, not exactly. D repeats A's intersection error and adds a false ancillarity restriction — the sup construction requires no such condition. Study tip: Remember that $\sup$ corresponds to existential quantification ("there exists"), which maps to union (and hence projection), while $\inf$ corresponds to universal quantification, mapping to intersection. This union/intersection duality is a recurring theme in nuisance-parameter problems.
3. What confidence set results from inverting this family of one-sided tests?
   Answer: $(0.378,\infty)$, because nonrejection requires $\mu_0 > 1.2 - 1.645(0.5) \approx 0.378$, with the boundary excluded. Confidence set inversion is a foundational duality in statistics: a level-$\alpha$ confidence set consists of exactly those parameter values $\mu_0$ that the corresponding level-$\alpha$ test would fail to reject. Your job is to find all $\mu_0$ where $p(\mu_0) > 0.05$. Starting from the nonrejection condition: $p(\mu_0) = 1 - \Phi\!\left(\frac{1.2 - \mu_0}{0.5}\right) > 0.05$, which means $\Phi\!\left(\frac{1.2 - \mu_0}{0.5}\right) < 0.95$. Since $\Phi^{-1}(0.95) = 1.645$, this requires $\frac{1.2 - \mu_0}{0.5} < 1.645$, or equivalently $\mu_0 > 1.2 - 1.645(0.5) = 1.2 - 0.8225 \approx 0.378$. The resulting confidence set is the open interval $(0.378, \infty)$, confirming answer C. Notice the direction carefully. Because the original test is right-tailed (rejecting when $\widehat{\mu}$ is too large relative to $\mu_0$), the test rejects small values of $\mu_0$, so the confidence set excludes values below the boundary — producing a lower-bounded, one-sided interval. A is wrong because $(-\infty, 2.022)$ corresponds to inverting a left-tailed test, not the right-tailed one given here. B makes the opposite directional error: it keeps values below 0.378 when those are precisely the ones that get rejected. D imposes a two-sided interval, which would require inverting two separate families of tests simultaneously — that's a different construction entirely and doesn't apply here. The key study tip: always match the direction of the confidence set to the direction of the test. A right-tailed test rejects low $\mu_0$ values, so inversion yields a lower confidence bound, not an upper one.
4. Which p-value and confidence-interval criterion correctly implement the level-$\alpha$ equivalence test?
   Answer: Use $\max(p_L,p_U)$ and reject equivalence when the two-sided $1-2\alpha$ confidence interval lies entirely inside $(-\Delta,\Delta)$. Equivalence testing requires you to simultaneously reject two one-sided null hypotheses — that $\theta \leq -\Delta$ AND that $\theta \geq \Delta$ — before you can claim equivalence. This "TOST" (Two One-Sided Tests) structure has direct consequences for how you combine p-values and construct confidence intervals. Because both one-sided nulls must be rejected simultaneously, equivalence is declared only when the harder test is cleared — meaning you use $\max(p_L, p_U)$. Reject $H_0$ (and conclude equivalence) when $\max(p_L, p_U) < \alpha$. Intuitively, the weakest link governs: if either one-sided test fails, equivalence fails. The confidence interval criterion follows directly. Each one-sided test at level $\alpha$ corresponds to a one-sided $1-\alpha$ bound, and combining two such bounds yields a two-sided $1-2\alpha$ interval. Concluding equivalence at level $\alpha$ is equivalent to requiring this $1-2\alpha$ interval to lie entirely inside $(-\Delta, \Delta)$. This is precisely D. A is wrong on both counts: $\min(p_L, p_U)$ is too lenient (only one test needs to pass), and checking whether a $1-\alpha$ interval overlaps the equivalence region is the opposite of what you want. B correctly uses $\max$, but "the interval contains both $-\Delta$ and $\Delta$" describes failing equivalence, not achieving it — and the interval width is wrong. C uses the right $1-2\alpha$ interval width but again uses $\min$, which doesn't enforce both rejections simultaneously. Remember the mnemonic: TOST = MAX test + narrow interval inside the zone. When you see equivalence testing, immediately ask yourself: "Do both one-sided tests pass?"
5. For an observed dataset, the p-value function for testing each point null hypothesis $H_0:\theta=\theta_0$ is $p(\theta_0)=\max\left\{\exp[-(\theta_0-2)^2],\exp[-(\theta_0+2)^2]\right\}.$ The testing rule rejects when $p(\theta_0)\leq 0.05$. Which set is obtained by inverting these tests?
   Answer: $(-3.731,-0.269)\cup(0.269,3.731)$, the two open intervals of nonrejected values centered at $\pm2$. When inverting a family of hypothesis tests to form a confidence set, you collect all parameter values $\theta_0$ that are not rejected — that is, all $\theta_0$ for which $p(\theta_0) > 0.05$. The boundary question (open vs. closed intervals) hinges on whether the rejection rule is strict ($\leq$) or non-strict. Here, $p(\theta_0) > 0.05$ means $\max\{\exp[-(\theta_0-2)^2],\exp[-(\theta_0+2)^2]\} > 0.05.$ Taking logs, this requires $\max\{-(\theta_0-2)^2, -(\theta_0+2)^2\} > \ln(0.05) \approx -2.996.$ So you need either $(\theta_0-2)^2 < 2.996 \implies \theta_0 \in (2-\sqrt{2.996},\, 2+\sqrt{2.996}) \approx (0.269, 3.731),$ or $(\theta_0+2)^2 < 2.996 \implies \theta_0 \in (-3.731, -0.269).$ Because the rejection rule uses a strict inequality ($p \leq 0.05$), the boundary points where $p = 0.05$ are rejected, so the nonrejection region is the open union $(-3.731, -0.269) \cup (0.269, 3.731)$. This confirms A. B is wrong because it includes the boundary points $\pm 0.269$ and $\pm 3.731$ as nonrejected, but at those points $p(\theta_0) = 0.05$, which satisfies $p \leq 0.05$ and thus triggers rejection — closed intervals are incorrect. C is wrong because it merges the two disjoint regions into one convex hull. Inversion preserves the exact nonrejection structure; you cannot fill in the middle gap $(-0.269, 0.269)$ where tests are actually rejected. D describes the rejection region itself, not its complement — a classic inversion flip error. Study tip: Always ask two questions when inverting tests: (1) Which values satisfy $p(\theta_0) > \alpha$? (2) Is the inequality strict or non-strict? The strictness directly determines open vs. closed boundaries.
6. What coverage property follows from the stated conditional validity?
   Answer: Conditional coverage is at least $1-\alpha$ for every $a$, and unconditional coverage follows by averaging. When you see a question linking ancillary statistics to confidence sets, the key move is tracing how a conditional probability statement about p-values translates directly into a coverage statement about the inverted set $C(X)$. Start from the given condition: for every $\theta_0$ and every ancillary value $a$, $P_{\theta_0}\{p_{\theta_0}(X) \leq \alpha \mid A = a\} \leq \alpha.$ Taking complements, this says $P_{\theta_0}\{p_{\theta_0}(X) > \alpha \mid A = a\} \geq 1 - \alpha$. But $\theta_0 \in C(X)$ if and only if $p_{\theta_0}(X) > \alpha$, so this is exactly $P_{\theta_0}\{\theta_0 \in C(X) \mid A = a\} \geq 1 - \alpha$ for every $a$. That is conditional coverage holding for each ancillary value. Unconditional coverage then follows by the law of total probability — averaging over $a$ preserves the inequality. This confirms D is correct. A is wrong because it inverts the logic — conditioning on an ancillary strengthens, not weakens, the coverage guarantee here. B is wrong because it imposes an unnecessary independence assumption. The conditional p-value validity already encodes the dependence structure; no additional independence condition is needed. C is wrong on two counts: coverage is at least $1-\alpha$, not exactly equal to it, and ancillary statistics have nothing to do with eliminating discreteness — that is a separate issue altogether. As a study tip: whenever you see a conditional probability inequality, practice immediately taking its complement and recognizing what event that defines. Many exam questions in this area hinge on that one algebraic step.
7. Which conclusion follows, and how would it change if the p-values were valid only at each prespecified fixed time?
   Answer: $C_\tau$ has coverage at least $1-\alpha$; fixed-time validity alone would not generally justify the same claim after optional stopping. When you see sequential testing questions involving stopping times, your first instinct should be to think about the optional stopping problem: classical p-values are calibrated for fixed sample sizes, so using them after data-dependent stopping can invalidate coverage guarantees. The always-valid (or "anytime-valid") property is precisely designed to fix this. The condition $P_{\theta_0}\!\left\{\inf_{t\geq1}p_{\theta_0,t}\leq\alpha\right\}\leq\alpha$ says that no matter when you stop — even adaptively — the probability of the p-value ever dipping below $\alpha$ is controlled. Equivalently, at any stopping time $\tau$, we have $P_{\theta_0}(p_{\theta_0,\tau}\leq\alpha)\leq\alpha$, which means $P_{\theta_0}(\theta_0\notin C_\tau)\leq\alpha$, so $C_\tau$ achieves coverage at least $1-\alpha$ uniformly over all stopping rules. Answer C captures this exactly. A is wrong because it inverts the logic: always-valid p-values are specifically constructed so that optional stopping does not break coverage. Coverage failure would occur only without the always-valid property. B is wrong on two counts: coverage is at least $1-\alpha$, not exactly $1-\alpha$, and conditioning on $\tau$ does not rescue ordinary fixed-time p-values under optional stopping — the conditioning argument fails because $\tau$ is correlated with the test statistic. D introduces a Bonferroni-style $1-2\alpha$ bound with no justification; the always-valid property gives the full $1-\alpha$ guarantee directly, not a degraded one. A useful rule of thumb: whenever a question involves data-dependent stopping, ask whether the inferential tool was designed to be robust to it. Always-valid p-values are; conventional fixed-time p-values are not.
8. Which statement about $D(X)$ is correct?
   Answer: It has coverage at least $0.95$ for $\psi$, but need not equal the inversion of an independently constructed test of $H_0:\theta^2=\psi_0$. When you invert a family of hypothesis tests to form a confidence set, the coverage guarantee travels with the set — and this guarantee is preserved under transformations, even non-injective ones. Here's the key insight: $C(X)$ satisfies $P(\theta \in C(X)) \geq 0.95$ for all $\theta$. Now define $\psi = \theta^2$ and $D(X) = \{\theta^2 : \theta \in C(X)\}$. Whenever the true $\theta$ is in $C(X)$ — which happens with probability at least $0.95$ — we have $\theta^2 \in D(X)$, i.e., the true $\psi$ is captured. So $D(X)$ achieves at least $0.95$ coverage for $\psi$. That's exactly what answer A claims. The second part of A is also correct: there's no reason $D(X)$ must coincide with the inversion of a separately constructed test for $H_0: \theta^2 = \psi_0$, since that test could be built differently (e.g., using a sign-aware statistic). B is wrong because coverage does not drop below $0.95$ from a many-to-one mapping. The event $\{\psi_{\text{true}} \in D(X)\}$ contains the event $\{\theta_{\text{true}} \in C(X)\}$, so coverage can only stay the same or increase — it cannot decrease. C is wrong on both counts: coverage can exceed $0.95$ (making "exactly" false), and coincidence with every valid test inversion is far too strong a claim. D is wrong because symmetry of $C(X)$ is not required. Valid coverage for $\psi$ only requires that $\theta_{\text{true}} \in C(X)$ implies $\psi_{\text{true}} \in D(X)$, which holds regardless of symmetry. Study tip: When a parameter undergoes a transformation, check whether the coverage event is preserved — if $A \subseteq B$ as events, then $P(A) \leq P(B)$. Transformations can only inflate or maintain coverage, never reduce it.
9. Let $C(X)$ be the set of all point-null values not rejected by their corresponding tests. Which statement about the coverage of $C(X)$ is necessarily correct?
   Answer: Its coverage is at least $0.95$ at every parameter value and can be greater where the tests are conservative. Whenever you see a question about inverting hypothesis tests to form confidence sets, anchor your thinking to one key duality: a $1-\alpha$ confidence set $C(X)$ contains $\theta_0$ if and only if the test of $H_0: \theta = \theta_0$ fails to reject. This means the coverage probability of $C(X)$ at a true value $\theta_0$ equals exactly the non-rejection probability of the corresponding test. Because the model is discrete, exact nonrandomized tests cannot always achieve rejection probability precisely equal to $0.05$—the discrete sample space forces the actual size to be at most $0.05$, and often strictly less. Translating through the duality, the non-rejection probability is at least $0.95$ at every $\theta_0$, and strictly greater wherever the test is conservative. Therefore, the coverage of $C(X)$ is guaranteed to be at least $0.95$ everywhere, confirming B as correct. A is wrong because it claims coverage equals exactly $0.95$ at every value—this ignores the conservatism forced by discreteness, which pushes coverage strictly above $0.95$ at some points. C is wrong because it suggests individual coverage can fall below $0.95$, which cannot happen: each test's rejection probability is bounded above by $0.05$ by construction, so non-rejection probability is bounded below by $0.95$. D is wrong and reverses the actual effect. Nonrandomization makes the inverted set larger (more conservative), not narrower, because tests reject less often than the nominal level allows. As a study tip: in discrete settings, always remember that nonrandomization creates conservative tests, which invert into over-covering confidence sets—coverage can only meet or exceed the nominal level, never fall below it.
10. Which statement best characterizes the confidence set obtained by inverting these p-values at level $\alpha$?
    Answer: It is $\{\theta:W(\theta)<\chi^2_{1,1-\alpha}\}$ and generally has only approximate coverage. Whenever you see profile likelihood paired with confidence set construction, your first instinct should be: this is a likelihood-ratio-based inversion, and its validity rests on large-sample chi-squared approximations. The profile log-likelihood $\ell_p(\theta)$ is obtained by maximizing over all nuisance parameters at each fixed $\theta$. The statistic $W(\theta) = 2\{\ell_p(\widehat{\theta}) - \ell_p(\theta)\}$ measures how much the log-likelihood drops when you constrain the parameter to $\theta$. Inverting the test at level $\alpha$ means collecting all $\theta_0$ values you would fail to reject — that is, all $\theta$ where the observed discrepancy is small enough: $\{\theta : W(\theta) < \chi^2_{1,1-\alpha}\}$. The key word in the passage is the dot above the "sim" — it signals an asymptotic approximation, not an exact result. In nonlinear models with nuisance parameters, the chi-squared null distribution holds only as $n \to \infty$, so finite-sample coverage is generally approximate. This makes B correct. A is wrong because it correctly identifies the set but falsely claims exact finite-sample coverage. The $\chi^2_1$ approximation is asymptotic; exact coverage requires either special structure (exponential families, pivots) not assumed here. C has the inequality backwards. Rejecting $\theta_0$ corresponds to large $W(\theta_0)$, so the confidence set retains values where $W$ is small, not large. D is wrong on two counts: profile likelihood intervals are generally not symmetric Wald-type intervals (they respect the curvature of $\ell_p$), and exact conditional coverage is not guaranteed. A useful rule: whenever nuisance parameters are profiled out in a nonlinear model, treat any resulting inference as asymptotic unless a theorem explicitly grants exactness.

Start this quiz