Statistics Flashcards: Comparing Treatments Using Randomized Experiments

What this deck covers

This deck focuses on Comparing Treatments Using Randomized Experiments, giving you a quick way to review the definitions, rules, and examples that matter most for Statistics.

How to use these flashcards

Work through these flashcards in short sessions. Try to answer each prompt before flipping the card, then revisit any cards you miss until the explanation feels automatic.

Flashcard 1: What is a Type I error in the context of comparing two treatments with $H_0$ true?

Answer: Rejecting $H_0$ and claiming a treatment effect when none exists. False positive: concluding effect when there's none.

Flashcard 2: What is the purpose of random assignment in a randomized experiment comparing two treatments?

Answer: To reduce confounding and support a cause-and-effect conclusion. Random assignment ensures groups are comparable except for treatment.

Flashcard 3: What does it mean to simulate the randomization distribution under $H_0$ in a randomized experiment?

Answer: Reassign outcomes to groups as if treatments had no effect. Randomly shuffle treatment labels to mimic no effect.

Flashcard 4: What is the typical center of the randomization distribution for $\bar{x}_1-\bar{x}_2$ under $H_0$?

Answer: Approximately $0$. Under $H_0$, no treatment effect means difference centers at 0.

Flashcard 5: Identify the correct scope: In a randomized experiment, what supports a cause-and-effect conclusion?

Answer: Random assignment of experimental units to treatments. Random assignment eliminates confounding variables.

Flashcard 6: What parameter is compared when the response is categorical (success/failure) for two treatments?

Answer: The difference in population proportions, $p_1 - p_2$. Compares success rates between two treatment groups.

Flashcard 7: Decide significance: If a randomization test gives $p=0.42$ at $\alpha=0.05$, what is the conclusion?

Answer: Fail to reject $H_0$; the difference is not statistically significant. $p > \alpha$ means insufficient evidence to reject null.

Flashcard 8: What is the standard randomization-test assumption about the treatment effect under $H_0$?

Answer: Under $H_0$, treatment labels are exchangeable (no real effect). If no effect exists, any assignment gives similar results.

Flashcard 9: Which tail(s) should be used for a $p$-value when the alternative is $H_a:\ \mu_1-\mu_2>0$?

Answer: Right-tail probability (simulated differences $\ge$ observed). One-sided test looks only at differences in predicted direction.

Flashcard 10: Which labels are shuffled in a randomization test for a randomized experiment under $H_0$?

Answer: The treatment labels, keeping the observed responses fixed. Responses stay with units; only treatment assignments change.

Flashcard 11: If the observed difference is in the extreme tail of the randomization distribution, what is the conclusion?

Answer: The difference is statistically significant; evidence of a treatment effect. Extreme results suggest treatment causes the difference.

Flashcard 12: Decide significance: If a randomization test gives $p=0.03$ at $\alpha=0.05$, what is the conclusion?

Answer: Reject $H_0$; the difference is statistically significant. $p < \alpha$ means reject null hypothesis.

Flashcard 13: What is a Type II error in the context of comparing two treatments with a real effect present?

Answer: Failing to reject $H_0$ when a treatment effect actually exists. False negative: missing a real treatment effect.

Flashcard 14: What statistic estimates bc_1 - bc_2 from sample data in a two-treatment experiment?

Answer: The difference in sample means, $\bar{x}_1 - \bar{x}_2$. Sample means estimate population means.

Flashcard 15: Which decision rule uses significance level $\alpha$ with a randomization-test $p$-value?

Answer: Reject $H_0$ if $p\text{-value} \le \alpha$. Standard hypothesis testing decision rule.

Flashcard 16: What is the correct interpretation of a small $p$-value in a randomized experiment?

Answer: The observed result is unlikely if $H_0$ is true. Low probability under null suggests treatment effect.

Flashcard 17: What is the definition of a $p$-value in a randomization test for two treatments?

Answer: Proportion of simulated stats c^6 as or more extreme than observed. Measures how extreme the observed difference is.

Flashcard 18: What parameter is compared when the response is quantitative for two treatments?

Answer: The difference in population means, bc_1 - bc_2. Compares average outcomes between two treatment groups.

Flashcard 19: If the observed difference is near the center of the randomization distribution, what is the conclusion?

Answer: Not statistically significant; insufficient evidence of a treatment effect. Common results suggest random variation, not treatment.

Flashcard 20: What statistic estimates $p_1 - p_2$ from sample data in a two-treatment experiment?

Answer: The difference in sample proportions, c^6p_1 - c^6p_2. Sample proportions estimate population proportions.

Flashcard 21: What does it mean if the randomization-test $p$-value is small (for example, $<0.05$)?

Answer: The observed difference is unlikely by chance; evidence of a treatment effect. Small p-values suggest the difference isn't due to chance.

Flashcard 22: What is the typical alternative hypothesis for comparing two treatments in a randomized experiment?

Answer: $H_a$: a treatment effect exists; parameter difference is not $0$. Claims treatments produce different outcomes.

Flashcard 23: Find the observed statistic: $\bar{x}_1=18.2$, $\bar{x}_2=16.9$; what is $\bar{x}_1-\bar{x}_2$?

Answer: $1.3$. Direct subtraction: $18.2 - 16.9 = 1.3$

Flashcard 24: What is the $p$-value in a randomization test comparing two treatments?

Answer: Proportion of simulated statistics at least as extreme as observed. Counts how often random chance produces results this extreme.

Flashcard 25: What is the null hypothesis for a randomization test comparing two treatments on a mean?

Answer: $H_0:\ \mu_1 - \mu_2 = 0$ (no treatment effect). Tests if treatment means are equal (no effect).

Flashcard 26: What is the typical null hypothesis for comparing two treatments in a randomized experiment?

Answer: $H_0$: no treatment effect; parameter difference equals $0$. Assumes treatments have identical effects.

Flashcard 27: What is the response variable in a randomized experiment comparing two treatments?

Answer: The outcome measured on each experimental unit. The variable we measure to compare treatment effects.

Flashcard 28: What parameter is compared for two treatments when the response variable is categorical (success/failure)?

Answer: The difference in population proportions, $p_1 - p_2$. Compares success rates between two groups.

Flashcard 29: What is the key purpose of random assignment in a randomized experiment comparing two treatments?

Answer: To reduce confounding and support a cause-and-effect conclusion. Random assignment ensures groups are comparable except for treatment.

Flashcard 30: Find the observed statistic: Treatment A has $12/40$ successes and B has $6/40$; what is c^6p_A-c^6p_B?

Answer: $0.15$. $\frac{12}{40} - \frac{6}{40} = 0.30 - 0.15 = 0.15$

Flashcard 31: Which tail(s) should be used for a $p$-value when the alternative is $H_a:\ \mu_1-\mu_2\ne^0$?

Answer: Two tails: simulated differences with $|\text{stat}| \ge |\text{observed}|$. Two-sided test considers extreme differences in either direction.

Flashcard 32: Identify the correct simulation step: What must be kept fixed when shuffling labels in a randomization test?

Answer: The observed responses and group sizes; only treatment labels are shuffled. Simulates null hypothesis by reassigning treatments randomly.

Flashcard 33: Identify the correct scope: In a randomized experiment, to what population can results be generalized?

Answer: Only to the population represented by the random sample (if one exists). Without random sampling, can't generalize beyond participants.

Flashcard 34: What is the null hypothesis for a randomization test comparing two treatments on a proportion?

Answer: $H_0:\ p_1 - p_2 = 0$ (no treatment effect). Tests if treatment proportions are equal (no effect).