Statistics Graduate Level Quiz: False Discovery Rate Fdr

10 questions · exam conditions

0:00

False Discovery Rate FdrQuestion 1 of 10

A genomics study applies BH at level $q=0.10$ and reports $20$ rejected null hypotheses. Assume the conditions required for the usual BH guarantee hold.

Which interpretation of the reported analysis is most accurate?

Exactly

2

of the

20

reported discoveries are expected to be false, conditional on this observed rejection set

Across repetitions, the expected false-discovery proportion is at most

0.10

, with the proportion defined as zero when there are no rejections

Each of the

20

rejected hypotheses has conditional probability at most

0.10

of having a true null hypothesis

The probability that at least one of the

20

reported discoveries is false is at most

0.10

← Back to quizzes

Statistics Graduate Level Quiz

Statistics Graduate Level Quiz: False Discovery Rate Fdr

Practice False Discovery Rate Fdr in Statistics Graduate Level with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on False Discovery Rate Fdr, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics Graduate Level.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

A genomics study applies BH at level $q=0.10$ and reports $20$ rejected null hypotheses. Assume the conditions required for the usual BH guarantee hold.

Which interpretation of the reported analysis is most accurate?

Exactly $2$ of the $20$ reported discoveries are expected to be false, conditional on this observed rejection set
Across repetitions, the expected false-discovery proportion is at most $0.10$ , with the proportion defined as zero when there are no rejections (correct answer)
Each of the $20$ rejected hypotheses has conditional probability at most $0.10$ of having a true null hypothesis
The probability that at least one of the $20$ reported discoveries is false is at most $0.10$

Explanation: When interpreting multiple testing procedures like Benjamini-Hochberg (BH), the critical skill is distinguishing between what the procedure actually controls versus what students commonly wish it controlled. BH controls the False Discovery Rate (FDR), defined as the expected value of the false discovery proportion (FDP), where FDP = (number of false rejections)/(total rejections) when rejections > 0, and 0 otherwise. Choice B is correct because it states precisely this guarantee: across repeated applications of the procedure under the same conditions, the expected FDP is at most

q = 0.10

. The phrase "with the proportion defined as zero when there are no rejections" correctly captures the standard definition, which avoids a 0/0 issue. This is a statement about a long-run average over hypothetical repetitions, not about this particular experiment alone. Choice A confuses FDR with a conditional expectation given the observed rejection set. The BH guarantee is marginal (unconditional), not conditional on the specific 20 discoveries observed. The posterior expected number of false discoveries given exactly these 20 rejections is a different, harder quantity that BH does not directly control. Choice C describes something resembling a Bayesian local false discovery rate (lfdr) — the posterior probability that an individual hypothesis is null. BH controls nothing at the individual-hypothesis level in this sense; it operates on the procedure's aggregate behavior. Choice D describes control of the familywise error rate (FWER) — the probability of any false rejection — which is a stricter criterion controlled by procedures like Bonferroni, not BH. Your study tip: always anchor your interpretation to the definition of what's being controlled. FDR = expected proportion (long-run average), not a per-experiment conditional probability or an individual-test probability.

Question 2

Suppose all $m$ tested null hypotheses are true, their p-values are independent and continuously uniform under the null, and BH is applied at level $q$ .

Which statement best describes the FDR in this special case?

It equals $P(R > 0) = q$ , because every rejection is a false rejection when all nulls are true (correct answer)
It equals the expected number of rejections divided by $m$ , so it is generally smaller than $q$
It equals the probability that every hypothesis is rejected, since each rejection is necessarily false in this setting
It is undefined when no hypotheses are rejected, so no unconditional FDR statement can be made

Explanation: When all

m

null hypotheses are true, every rejection is necessarily a false discovery. This is the "global null" scenario, and it's a classic setting for understanding what the FDR actually measures. Recall that FDR is defined as

\mathbb{E}\left[\frac{V}{\max(R,1)}\right]

, where

V

is the number of false rejections and

R

is the total rejections. When all nulls are true,

V = R

always, so the ratio

V/\max(R,1)

equals 1 whenever

R > 0

and 0 otherwise. Therefore,

\text{FDR} = 1 \cdot P(R > 0) + 0 \cdot P(R = 0) = P(R > 0)

. Under BH at level

q

with independent uniform p-values, this probability equals exactly

q

, confirming A as correct. The BH procedure is designed so that even in this worst-case scenario, FDR

= q

, not something larger. Answer B is wrong because dividing expected rejections by

m

gives you the expected proportion of all hypotheses rejected — this is not the FDR formula, which conditions on at least one rejection occurring. Answer C confuses the event "each rejection is false" (which is always true here, but doesn't mean all hypotheses get rejected) with the probability that every hypothesis is rejected — a much stronger and unrelated event. Answer D is wrong because the FDR definition elegantly handles the

R = 0

case by defining the ratio as 0, so the unconditional expectation is perfectly well-defined. A key study tip: always return to the formal definition

\mathbb{E}[V/\max(R,1)]

when reasoning about FDR — many traps arise from confusing it with related but distinct quantities like FWER or the expected rejection count.

Question 3

Four ordered p-values are $0.01, 0.04, 0.08, 0.50$ . BH at level $q=0.10$ initially rejects two hypotheses. The analyst then adds four unrelated hypotheses, each having p-value $1$ , and reapplies BH to all eight hypotheses at the same target level.

How many hypotheses does the second BH analysis reject?

None, because doubling the family size halves every BH rank and invalidates the original discoveries
One, because only $0.01$ remains below its new rank-specific BH critical value (correct answer)
Two, because adding p-values equal to $1$ cannot alter any earlier BH rejection
Four, because the larger family provides more ranks at which the BH inequality can hold

Explanation: Whenever you see a BH question involving changes to the family of hypotheses, your instinct should be to recompute the BH critical values from scratch rather than assume prior rejections are preserved. Recall the BH procedure: sort all

m

p-values as

p_{(1)} \leq \cdots \leq p_{(m)}

, then reject all hypotheses up to rank

k^* = \max\{k : p_{(k)} \leq kq/m\}

. The critical value for rank

k

kq/m

, so adding hypotheses changes

m

and rescales every threshold. In the first analysis,

m = 4

and

q = 0.10

. The thresholds are

0.025, 0.050, 0.075, 0.100

. Comparing:

p_{(1)} = 0.01 \leq 0.025

✓,

p_{(2)} = 0.04 \leq 0.050

✓,

p_{(3)} = 0.08 > 0.075

✗. BH rejects two hypotheses. In the second analysis,

m = 8

and

q = 0.10

. The four new p-values are all

1

, so the ordered set is

0.01, 0.04, 0.08, 0.50, 1, 1, 1, 1

. Now the threshold at rank

k

k(0.10)/8 = k/80

. Checking:

p_{(1)} = 0.01 \leq 1/80 = 0.0125

✓,

p_{(2)} = 0.04 \leq 2/80 = 0.025

✗. Only one rejection survives, confirming B is correct. A is wrong because BH rank ratios don't simply halve; each threshold must be computed individually. C is the most tempting trap — it assumes adding null p-values is harmless, but those extra hypotheses inflate

m

and shrink every threshold, costing you a rejection. D is wrong because more ranks help only if corresponding p-values are small enough to clear their new, lower thresholds. Your key takeaway: adding hypotheses always changes

m

, which uniformly lowers BH thresholds — never assume prior rejections automatically persist after the family grows.

Question 4

Before testing differential expression, a researcher removes low-count genes and then applies BH to the retained genes. In one proposed filter, retention depends only on total read count, which is independent of the test p-value under each null. In another proposed filter, the researcher retains the genes having the smallest unadjusted p-values.

Which conclusion about FDR control is most appropriate?

The count-based filter can preserve BH validity under the stated null independence, whereas filtering by small p-values generally invalidates the usual guarantee (correct answer)
Both filters preserve BH validity because BH uses only the number of hypotheses remaining after the filtering step
Only the small-p-value filter preserves BH validity because it increases the proportion of nonnull hypotheses among retained tests
Neither filter can preserve BH validity because any reduction in the tested family necessarily changes the null hypotheses

Explanation: When evaluating FDR control after pre-filtering, the critical question is whether the filter is independent of the test statistics under the null. The Benjamini-Hochberg (BH) procedure's FDR guarantee relies on certain properties of the null p-value distribution — specifically, that null p-values are stochastically super-uniform. Any filter that distorts this distribution among the retained nulls can break the guarantee. A count-based filter retains genes based on total read depth, a quantity determined before observing group-specific signals. Under the null hypothesis for a given gene, its read count is independent of its p-value, so the conditional null p-value distribution among retained genes remains uniform. BH applied to these retained genes therefore preserves its FDR control. This is the reasoning behind answer A, which is correct. Now consider filtering by smallest unadjusted p-values — say, keeping only genes with

p < 0.5

. This truncates the null p-value distribution, removing the large p-values that would otherwise anchor the BH procedure's step-up threshold. The remaining null p-values are no longer uniform; they are concentrated near zero, making the procedure anti-conservative and invalidating the FDR guarantee. B is wrong because BH's validity does not follow automatically from simply plugging in the post-filter count

m

. The shape of the null p-value distribution matters, not just the number of tests. C is wrong because enriching for nonnull genes is a power benefit, not a validity argument — the problem is what happens to the null p-values, not the nonnulls. D is wrong because filtering itself is not disqualifying; only filters that depend on the test statistics under the null are problematic. Key tip: Always ask whether a filter is independent of the test statistic under the null. If yes, BH validity survives. If the filter selects on p-values directly, the null distribution is distorted and the guarantee fails.

Question 5

The same $m$ p-values are analyzed in two ways: Bonferroni testing at family-wise level $q$ and BH testing at FDR level $q$ . Both procedures use the same numerical value of $q$ .

Which relationship between their rejection sets must hold, regardless of the observed p-values?

Every BH rejection is also a Bonferroni rejection, because FDR control is stronger than family-wise error control
The two rejection sets are identical whenever at least one p-value is at most $q/m$
Neither rejection set must contain the other, because their error criteria are mathematically unrelated
Every Bonferroni rejection is also a BH rejection, although BH may reject additional hypotheses (correct answer)

Explanation: When comparing multiple testing procedures, the key is understanding what each procedure's rejection threshold actually is — not just the error concept it controls. Bonferroni rejects hypothesis

i

when

p_{(i)} \leq q/m

. The BH procedure sorts p-values as

p_{(1)} \leq p_{(2)} \leq \cdots \leq p_{(m)}

and rejects all hypotheses up to index

k^* = \max\{i : p_{(i)} \leq iq/m\}

. Notice that the BH threshold for the smallest p-value is

q/m

— identical to Bonferroni's single cutoff. For any larger p-value at rank

i > 1

, BH uses threshold

iq/m > q/m

, which is more lenient. This means any hypothesis Bonferroni rejects (with

p_{(i)} \leq q/m

) automatically satisfies

p_{(i)} \leq iq/m

, so BH rejects it too — but BH can additionally reject hypotheses that Bonferroni misses. That confirms D is correct. A has the logical relationship exactly backwards. FDR control is weaker than FWER control — BH allows some false discoveries, while Bonferroni permits none in expectation. A weaker criterion rejects more, not fewer, hypotheses. B is tempting but confuses a special case for a general rule. Even when no p-value falls at or below

q/m

, the containment relationship in D still holds vacuously (Bonferroni rejects nothing, and the empty set is a subset of BH's rejections). C is wrong because the procedures are mathematically related through the shared threshold

q/m

at rank 1, creating the guaranteed subset relationship. Study tip: Remember that BH's rank-1 threshold equals Bonferroni's cutoff exactly — this single fact implies Bonferroni

\subseteq

BH always.

Question 6

Two disjoint families each contain exactly one hypothesis, and in both cases the null hypothesis is true. The test p-values are independent across families. Each family is tested separately at level $q$ , which is equivalent to applying BH to each family individually.

What is the FDR when the two families' rejection results are pooled and treated as one reported collection?

$q$ , because controlling FDR at level $q$ within every family automatically controls the combined FDR at the same level
$q^2$ , because a pooled false discovery requires both null hypotheses to be rejected simultaneously
$2q$ , because the two family-level FDR values add exactly under independence, with no correction for overlap
$2q - q^2$ , because every pooled rejection is a false rejection, and the probability of at least one rejection equals $1-(1-q)^2$ (correct answer)

Explanation: When every hypothesis in a study is null, every rejection is a false discovery — so the FDR equals the probability of making at least one rejection. This is the key insight that unlocks this problem. Here, you have two independent families, each containing one null hypothesis, each tested at level

q

. Because all hypotheses are null, every possible rejection is a false discovery. The pooled FDR is therefore the probability that at least one of the two tests rejects. Using the complement rule for independent events:

\text{FDR} = 1 - (1-q)(1-q) = 2q - q^2

This confirms D is correct. A is wrong because controlling FDR within each family does not automatically control the combined FDR at the same level when you pool results across families. Within-family control is a local guarantee; pooling creates a new, larger collection where joint error behavior must be re-examined. B is wrong because a pooled false discovery does not require both null hypotheses to be rejected — it requires at least one rejection. The value

q^2

is the probability of rejecting both, which is an intersection event, not the union. C is wrong in a subtle but important way:

2q

overcounts by ignoring the overlap (the event where both are rejected simultaneously). By inclusion-exclusion, the correct union probability is

2q - q^2

, not

2q

. Simple addition only works when the events are mutually exclusive. Study tip: Whenever you see "pooled FDR" with all-null families, immediately think "probability of at least one false rejection" and apply the complement rule — this reframes an FDR question into a clean probability calculation.

Question 7

Eight hypotheses yield ordered p-values $0.004, 0.009, 0.018, 0.021, 0.031, 0.041, 0.20, 0.60$ . The Benjamini–Hochberg procedure is applied at target level $q=0.05$ .

Which set of hypotheses is rejected?

Only the hypotheses corresponding to the first two ordered p-values
The hypotheses corresponding to the first three ordered p-values
The hypotheses corresponding to the first five ordered p-values (correct answer)
The hypotheses corresponding to the first six ordered p-values

Explanation: Whenever you see a Benjamini–Hochberg (BH) procedure question, your job is to find the largest ordered p-value that still falls below its BH threshold, then reject all hypotheses up to and including that one. With

m = 8

hypotheses and target FDR level

q = 0.05

, the threshold for the

i

-th ordered p-value is

\frac{i}{m} \cdot q = \frac{i}{8}(0.05)

. Compare each ordered p-value

p_{(i)}

to its threshold:

$i$	$p_{(i)}$	Threshold $\frac{i}{8}(0.05)$	$p_{(i)} \leq$ threshold?
1	0.004	0.00625	✓
2	0.009	0.01250	✓
3	0.018	0.01875	✗
4	0.021	0.02500	✓
5	0.031	0.03125	✓
6	0.041	0.03750	✗

The largest

i

for which

p_{(i)} \leq \frac{i}{8}(0.05)

i = 5

, so you reject hypotheses 1 through 5, confirming answer C. Answer A is wrong because it stops at

i=2

, ignoring that

p_{(4)}

and

p_{(5)}

also satisfy their thresholds. Answer B rejects only the first three, but

p_{(3)} = 0.018

actually exceeds its threshold of 0.01875 — a common trap. Answer D rejects six hypotheses, but

p_{(6)} = 0.041 > 0.03750

, so that boundary is crossed in the wrong direction. The key study tip: BH is a step-up procedure — scan upward from the largest p-value and find the last crossing, then reject everything below it, even if some intermediate p-values individually exceeded their thresholds.

Question 8

An analyst tests $m$ hypotheses whose null p-values may have an arbitrary and unknown dependence structure. No positive-dependence condition can be justified.

Which procedure has the most defensible general FDR guarantee at nominal level $q$ ?

Apply ordinary BH with critical values $iq/m$ , because valid marginal p-values render dependence irrelevant to FDR control
Apply ordinary BH with critical values $iq/m$ , because dependence affects power but cannot inflate FDR above $q$
Apply the Benjamini–Yekutieli procedure with critical values $iq/(mH_m)$ , where $H_m = \sum_{j=1}^{m} 1/j$ , because it controls FDR under arbitrary dependence (correct answer)
Apply Bonferroni at level $q/m$ and interpret it as an FDR procedure, because its rejection rule implies the expected false-discovery proportion is at most $q$

Explanation: When you encounter FDR control questions, your first instinct should be to ask: what dependence structure am I assuming? The answer determines which procedure is valid. The Benjamini–Hochberg (BH) procedure uses critical values

iq/m

and controls FDR at level

q

— but only under independence or a specific positive-dependence condition called PRDS (positive regression dependence on a subset). When dependence is arbitrary and unknown, BH's FDR guarantee evaporates. Benjamini and Yekutieli (2001) proved that under arbitrary dependence, FDR control is restored by applying the more conservative critical values

iq/(m H_m)

, where

H_m = \sum_{j=1}^{m} 1/j \approx \ln(m) + 0.577

. This BY procedure sacrifices power for universality — it controls FDR at level

q

with no restrictions whatsoever on the joint distribution of p-values. That makes C the correct and most defensible choice. A is wrong because valid marginal p-values do not make dependence irrelevant to FDR control. Dependence among test statistics can cause BH to exceed the nominal FDR level under general dependence structures. B commits the same error more subtly — claiming dependence affects only power but not FDR inflation. This is false; counterexamples exist where BH's FDR exceeds

q

under negative or complex dependence. D is wrong because Bonferroni controls FWER, not FDR. While FWER control implies FDR control (since FWER

\geq

FDR), this inequality means Bonferroni is unnecessarily conservative as an FDR procedure and doesn't directly bound the expected false-discovery proportion at

q

. Your study tip: memorize the trio — BH works under independence/PRDS, BY works under arbitrary dependence, and always distinguish FWER from FDR procedures.

Question 9

A multiple-testing procedure makes no rejections with probability $0.90$ . With probability $0.10$ , it makes exactly $10$ rejections, exactly one of which is a false rejection. These are the only possible outcomes.

What are this procedure's FDR and positive FDR, respectively?

FDR is $0.01$ , and positive FDR is $0.10$ (correct answer)
FDR is $0.10$ , and positive FDR is $0.01$
FDR is $0.10$ , and positive FDR is $1.00$
FDR is $0.01$ , and positive FDR is $0.01$

Explanation: When you see a question about FDR and positive FDR, the key distinction is whether you condition on any rejections occurring. Both metrics measure the expected proportion of false rejections among all rejections, but they differ in how they handle the case of zero rejections. FDR is defined as

E\left[\frac{V}{R}\right]

, where

V

is the number of false rejections and

R

is the total rejections, with the ratio defined as 0 when

R = 0

. Positive FDR (pFDR) conditions on at least one rejection occurring:

E\left[\frac{V}{R} \,\middle|\, R > 0\right]

. In this problem, with probability 0.90 there are no rejections (ratio = 0), and with probability 0.10 there are 10 rejections, exactly 1 false. So the false discovery proportion when rejections occur is

\frac{1}{10} = 0.10

. For FDR:

E\left[\frac{V}{R}\right] = (0.90)(0) + (0.10)(0.10) = 0.01

. For pFDR: you condition on the event that rejections occur (probability 0.10), so

E\left[\frac{V}{R} \,\middle|\, R > 0\right] = 0.10

. The 90% no-rejection scenarios are excluded entirely. This confirms answer A: FDR = 0.01, pFDR = 0.10. Answer B swaps the two values — a direct result of confusing which metric conditions on

R > 0

. Answer C incorrectly sets pFDR = 1.00, perhaps confusing "one false rejection" with "all rejections are false." Answer D sets both equal to 0.01, ignoring the conditioning in pFDR entirely. Your study tip: remember that pFDR is always at least as large as FDR, because conditioning on

R > 0

removes the zero-ratio outcomes that pull the expectation down.

Question 10

Six ordered p-values are $0.006, 0.014, 0.016, 0.041, 0.13, 0.40$ . For the p-value at rank $i$ , define its BH-adjusted p-value as $\min_{j\ge i}\{6p_{(j)}/j\}$ , truncated at $1$ .

What is the BH-adjusted p-value for the hypothesis at rank $2$ ?

$0.014$ , because adjusted p-values preserve the original rank- $2$ value
$0.042$ , because multiplying the rank- $2$ p-value by $6/2$ is sufficient
$0.032$ , because the minimum over ranks $2$ through $6$ occurs at rank $3$ (correct answer)
$0.048$ , because the rank- $3$ p-value should be multiplied by the total number of tests

Explanation: When working with Benjamini-Hochberg (BH) adjusted p-values, the key formula to remember is that the adjusted p-value for rank

i

is not simply

p_{(i)} \cdot m/i

. Instead, it requires a backward minimum sweep:

\tilde{p}_{(i)} = \min_{j \geq i}\left\{\frac{m \cdot p_{(j)}}{j}\right\}

, truncated at 1. This enforces monotonicity — adjusted p-values must be non-decreasing as rank increases. For rank 2, you compute

m \cdot p_{(j)}/j

for all

j \geq 2

and take the minimum. With

m = 6

$j=2$ : $6(0.014)/2 = 0.042$
$j=3$ : $6(0.016)/3 = 0.032$
$j=4$ : $6(0.041)/4 = 0.0615$
$j=5$ : $6(0.13)/5 = 0.156$
$j=6$ : $6(0.40)/6 = 0.40$

The minimum over these values is

0.032

, occurring at rank 3. So the BH-adjusted p-value for rank 2 is

\mathbf{0.032}

, confirming C. A is wrong because BH adjustment always alters the p-value — simply preserving the original value ignores the multiplicity correction entirely. B makes the common mistake of stopping at rank 2 without sweeping forward; it computes

0.042

but misses that rank 3 yields a smaller adjusted value. D multiplies

p_{(3)}

m = 6

rather than

m/j = 6/3

, confusing the total number of tests with the ratio used in the BH formula. A useful strategy: always compute the raw BH values

m \cdot p_{(j)}/j

for all ranks from

i

m

, then take the minimum. Stopping early at rank

i

is the most common trap on questions like this.