Statistics Flashcards: Evaluating Reports Based On Data

What this deck covers

This deck focuses on Evaluating Reports Based On Data, giving you a quick way to review the definitions, rules, and examples that matter most for Statistics.

How to use these flashcards

Work through these flashcards in short sessions. Try to answer each prompt before flipping the card, then revisit any cards you miss until the explanation feels automatic.

Flashcard 1: What is the correct meaning of a $95\%$ confidence level in a report?

Answer: The method captures the true parameter in $95\%$ of samples. It's about the long-run performance of the interval construction method.

Flashcard 2: Choose the better summary for skewed income data: mean or median?

Answer: Median. Median resists the pull of high-income outliers.

Flashcard 3: What is the correct interpretation of a $95\%$ confidence interval for a population parameter?

Answer: $95\%$ of such intervals capture the true parameter. The method produces intervals that contain the parameter 95% of the time.

Flashcard 4: What is the correct interpretation of a $95\%$ confidence statement in a report?

Answer: The method captures the true value in $95\%$ of repeated samples. Not that we're $95\%$ sure about this specific interval.

Flashcard 5: Which conclusion is correct when a report states $p=0.03$ and $\alpha=0.05$?

Answer: Reject $H_0$. Since $p < \alpha$, the result is statistically significant.

Flashcard 6: Which study type most supports a causal claim: randomized experiment or observational study?

Answer: Randomized experiment. Random assignment controls for confounding variables.

Flashcard 7: Choose the correct conclusion: a $95\%$ CI for a mean difference is $(-1.2,0.4)$.

Answer: Not statistically significant at $\alpha=0.05$ (interval includes $0$). CI contains null value $0$, so fail to reject $H_0$.

Flashcard 8: Which statement is correct about statistical significance versus practical importance?

Answer: Significance may occur even when the effect is small. Large samples can detect tiny, meaningless differences.

Flashcard 9: Which design feature reduces confounding in an experiment: random assignment or voluntary response?

Answer: Random assignment. Randomly assigning treatments balances confounders across groups.

Flashcard 10: What is the main risk when a report generalizes from a convenience sample?

Answer: Selection bias; sample likely differs from the target population. Convenience samples are not randomly selected from the population.

Flashcard 11: Choose the correct conclusion: a $95\%$ CI for a proportion difference is $(0.03,0.11)$.

Answer: Statistically significant at $\alpha=0.05$ (interval excludes $0$). All values in CI are positive, showing a real difference.

Flashcard 12: What is the correct meaning of a $p$-value reported for a test of $H_0$?

Answer: Probability of data at least as extreme, assuming $H_0$ is true. Not the probability that $H_0$ is true.

Flashcard 13: Identify the misleading feature: a bar chart axis starts at $50$ instead of $0$.

Answer: Truncated axis exaggerates visual differences. Makes small differences appear larger than they are.

Flashcard 14: Identify the issue: a poll reports results from an online click-in survey about a policy.

Answer: Voluntary response bias. Self-selected respondents don't represent the population.

Flashcard 15: What is the margin of error for a confidence interval written as estimate $\pm$ margin?

Answer: Half the interval width. The $\pm$ value represents the margin of error.

Flashcard 16: What does it mean if a report says its sample is an SRS of the population?

Answer: Every same-size sample has an equal chance of selection. SRS ensures unbiased representation of the population.

Flashcard 17: What does a $p$-value represent in a significance test?

Answer: Probability of data at least as extreme, assuming $H_0$. Measures how likely the observed data would occur if the null hypothesis were true.

Flashcard 18: What is the main risk when a report generalizes from a voluntary response sample?

Answer: Strong selection bias; results may not represent the population. Volunteers differ systematically from non-volunteers.

Flashcard 19: What is the definition of a statistically significant result at level $\alpha$?

Answer: A result with $p \le \alpha$. The p-value must be less than or equal to the significance level.

Flashcard 20: What is the main difference between association and causation in evaluating a report?

Answer: Association does not imply a cause-and-effect link. Correlation doesn't prove one variable causes changes in another.

Flashcard 21: Identify the misleading summary: a report uses the mean for data with extreme outliers.

Answer: Mean is not resistant; median is more appropriate. Outliers inflate the mean but don't affect median.

Flashcard 22: Identify the correct decision rule using a confidence interval to test $H_0: \theta=\theta_0$ at $\alpha=0.05$.

Answer: Reject if $\theta_0$ is not in the $95\%$ CI. If the null value falls outside the CI, it's unlikely given the data.

Flashcard 23: Which value must a report provide to judge statistical significance at level $\alpha$?

Answer: A $p$-value (or enough information to compute it). Compare $p$ to $\alpha$ to determine significance.

Flashcard 24: Find the margin of error for a reported $90\%$ CI of $[48,52]$.

Answer: $2$. Margin = $(52-48)/2 = 2$.

Flashcard 25: Identify the flaw: a report claims causation from an observational study with no random assignment.

Answer: Causal claim is not justified without random assignment. Observational studies can't establish causation due to potential confounders.

Flashcard 26: Which conclusion is correct when a report states $p=0.12$ and $\alpha=0.05$?

Answer: Fail to reject $H_0$. Since $p > \alpha$, the result is not statistically significant.

Flashcard 27: Find the problem: a report compares two treatments but subjects chose their own treatment.

Answer: Self-selection confounding; groups may differ before treatment. Without randomization, treatment groups aren't comparable.

Flashcard 28: What conclusion is justified when a report gives a $p$-value with $p<\alpha$?

Answer: Reject $H_0$; results are statistically significant at level $\alpha$. Small $p$-values indicate data unlikely under null hypothesis.

Flashcard 29: What is the main risk of a voluntary response sample in a report?

Answer: Strong selection bias from self-selection. Only those with strong opinions tend to respond.

Flashcard 30: What is the key question to ask when a report claims a study shows cause and effect?

Answer: Was the study a randomized experiment (not just observational). Only randomized experiments can establish causation, not observational studies.

Flashcard 31: What does a report's margin of error (MOE) typically describe for a survey estimate?

Answer: Likely sampling error range around the estimate (at a stated level). MOE quantifies uncertainty due to random sampling variability.

Flashcard 32: What is the difference between statistical significance and practical importance?

Answer: Significance is about chance; importance is about effect size/impact. Small effects can be significant with large samples.

Flashcard 33: Which conclusion is correct if a reported $95\%$ CI for $\mu$ is $[-1,4]$ and $H_0: \mu=0$?

Answer: Fail to reject $H_0$ at $\alpha=0.05$. Since $0$ is inside $[-1,4]$, the null hypothesis is not rejected.

Flashcard 34: Identify the correct claim: an observational study found a strong association between $X$ and $Y$.

Answer: Association only; causation is not justified without random assignment. Observational studies can't control for all confounders.

Flashcard 35: What is a confounding variable in evaluating a data-based report?

Answer: A variable related to both explanatory and response variables. It affects both variables, creating a spurious relationship.

Flashcard 36: Identify the correct critique: a report highlights $p<0.001$ but the effect size is near $0$.

Answer: Statistically significant but possibly not practically important. Very small p-value doesn't guarantee the effect matters in practice.

Flashcard 37: What is the main warning sign if a report emphasizes relative change but omits baseline rates?

Answer: Relative change can exaggerate impact without the absolute difference. "$200\%$ increase" from $1$ to $3$ is only $2$ units.

Flashcard 38: Which conclusion is correct if a reported $95\%$ CI for $\mu$ is $[2,5]$ and $H_0: \mu=0$?

Answer: Reject $H_0$ at $\alpha=0.05$. Since $0$ is outside $[2,5]$, the null hypothesis is rejected.

Flashcard 39: Identify the confounder risk: a report links ice cream sales to drowning deaths.

Answer: Season/temperature is a confounding variable. Both increase in summer; correlation doesn't imply causation.

Flashcard 40: What feature of a study design most supports a causal conclusion in a report?

Answer: Random assignment to treatment and control groups. Randomization eliminates confounding variables by balancing groups.