Statistics Flashcards: Evaluating Models With Data And Simulation

What this deck covers

This deck focuses on Evaluating Models With Data And Simulation, giving you a quick way to review the definitions, rules, and examples that matter most for Statistics.

How to use these flashcards

Work through these flashcards in short sessions. Try to answer each prompt before flipping the card, then revisit any cards you miss until the explanation feels automatic.

Flashcard 1: What is the main reason to run many simulation repetitions (for example, $10{,}000$) instead of few?

Answer: More repetitions reduce random simulation error in the estimated probability. Law of large numbers ensures convergence to true probability.

Flashcard 2: Which option is correct: Under $P(T)=0.5$, is $5$ tails in a row unusual at the $0.05$ level?

Answer: Yes; 0.03125  0.05. The probability $0.03125$ is less than the threshold $0.05$.

Flashcard 3: What does it mean for observed data to be consistent with a probability model?

Answer: The result is not unusually rare under the model. Events occur with expected frequency, not significantly deviating from model predictions.

Flashcard 4: What model assumption is violated if outcomes in repeated trials influence each other during the process?

Answer: The independence assumption of the probability model is violated. Dependent trials contradict binomial model's independence requirement.

Flashcard 5: What decision rule uses a significance level of $0.05$ to judge model consistency?

Answer: Question the model if $p \le 0.05$; otherwise do not question it. Standard threshold: reject model when probability of observed data is very low.

Flashcard 6: What is $P(\text{at least }5\text{ tails in }5\text{ flips})$ when $P(T)=0.5$?

Answer: $\frac{1}{32}$. Only one way to get all tails: TTTTT.

Flashcard 7: What is the probability of $6$ heads in a row if $P(H)=0.5$?

Answer: $\left(\frac{1}{2}\right)^6=\frac{1}{64}$. Each flip is independent with probability $\frac{1}{2}$.

Flashcard 8: What is the expected number of heads in $n=20$ flips if the model has $p=0.5$?

Answer: $np=20\cdot 0.5=10$. Expected value formula: $E[X] = np$ for binomial distribution.

Flashcard 9: Which conclusion matches $p=\frac{1}{32}$ for $5$ tails in $5$ flips when using a $0.05$ rule?

Answer: Question the $p=0.5$ model because $\frac{1}{32}<0.05$. The probability is below the significance threshold, suggesting model inadequacy.

Flashcard 10: Identify the probability of exactly $8$ heads in $10$ flips if $P(H)=0.5$.

Answer: inom{10}{8}(0.5)^{10}=0.0439453125. Binomial probability with $n=10$, $k=8$, $p=0.5$.

Flashcard 11: Identify the probability of at least one head in $5$ flips if $P(H)=0.5$.

Answer: $1-(0.5)^5=0.96875$. Complement of getting all tails in 5 flips.

Flashcard 12: Which option is correct: If $P(H)=0.5$, is exactly $8$ heads in $10$ flips unusual at $0.05$?

Answer: Yes; 0.0439453125 0.05. The probability $0.044$ is less than the threshold $0.05$.

Flashcard 13: Identify the probability of $0$ successes in $10$ trials if $p=0.2$.

Answer: $(0.8)^{10}=0.1073741824$. All 10 trials fail when each has probability $0.8$ of failing.

Flashcard 14: What is a p-value in a simulation-based test of a model?

Answer: The proportion of simulations at least as extreme as observed. Measures how surprising the observed data are under the model.

Flashcard 15: What does it mean for observed data to be consistent with a probability model?

Answer: The outcome is not unusually unlikely under the model. The observed result has a reasonable probability of occurring by chance.

Flashcard 16: What is a simulated sampling distribution used for in model checking?

Answer: To estimate how often a statistic occurs if the model is true. Shows the likelihood of various outcomes under the assumed model.

Flashcard 17: What is the definition of a simulation in statistical inference?

Answer: Using random trials to imitate a chance process under a model. Generates artificial data matching the model's probability structure.

Flashcard 18: What is the empirical $p$-value if $37$ of $1000$ simulations are at least as extreme as observed?

Answer: $\frac{37}{1000}=0.037$. Divide count of extreme outcomes by total simulations.

Flashcard 19: What is the probability of $5$ tails in a row if a coin has $P(T)=0.5$?

Answer: $\left(\frac{1}{2}\right)^5=\frac{1}{32}$. Each flip has probability $\frac{1}{2}$; multiply for independent events.

Flashcard 20: Identify the correct simulation step to model $n=30$ flips with $p=0.5$ using random digits.

Answer: Treat $0$-$4$ as heads and $5$-$9$ as tails for each trial. Splits digits evenly to simulate fair coin with $p=0.5$.

Flashcard 21: Which option best describes a two-sided (two-tail) extremeness definition?

Answer: Count simulations at least as far from expected in either direction. Considers deviations in both positive and negative directions.

Flashcard 22: What conclusion is appropriate when the p-value is very small, such as at most $0.05$?

Answer: Question the model; the data are inconsistent with it. Small p-value indicates the observed data are unlikely under the model.

Flashcard 23: What conclusion is appropriate when the p-value is not small, such as greater than $0.05$?

Answer: Do not question the model; the data are plausible under it. Large p-value means the data could reasonably occur under the model.

Flashcard 24: Choose the correct interpretation of a small $p$-value in this context of checking a model.

Answer: The observed result would be rare if the model were true. Low probability suggests observed data unlikely under assumed model.

Flashcard 25: What probability threshold is commonly used to label an outcome as unusual in this standard?

Answer: A probability at most $0.05$. Standard significance level for hypothesis testing.

Flashcard 26: What does the phrase "at least as extreme as the observed result" mean in a simulation test?

Answer: Outcomes with deviation from the model at least as large as observed. Includes all results that differ from expected by the observed amount or more.

Flashcard 27: Choose the correct interpretation of a large $p$-value when checking a model with simulation.

Answer: The observed result is plausible under the model; do not question it. High probability indicates observed data aligns with model expectations.

Flashcard 28: What is the null model when checking whether data match a claimed probability process?

Answer: The claimed probability distribution for the data-generating process. The hypothesis being tested about how the data are generated.

Flashcard 29: What probability model is assumed for $n$ independent trials with success probability $p$?

Answer: A binomial model: $X \sim \text{Binomial}(n,p)$. Models repeated independent trials with constant success probability.

Flashcard 30: What is $P(\text{all }10\text{ outcomes match})$ for $10$ independent trials with $P(\text{match})=0.5$?

Answer: $\left(\frac{1}{2}\right)^{10}=\frac{1}{1024}$. Multiply individual probabilities for independent events.

Flashcard 31: Which probability is estimated by the simulation proportion of outcomes at least as extreme as observed?

Answer: An empirical $p$-value for the observed result under the model. Simulation frequency approximates theoretical probability of extreme events.

Flashcard 32: Which conclusion matches an empirical $p$-value of $0.12$ when using a $0.05$ cutoff?

Answer: Do not question the model because $0.12>0.05$. Exceeds significance level, so data consistent with model.

Flashcard 33: Identify the probability of $5$ tails in a row if a coin has $P(T)=0.5$.

Answer: (0.5)^5=0.03125. Each flip has probability $0.5$, multiply for independent events.

Flashcard 34: Which option is correct: If $p=0.2$, is $0$ successes in $10$ trials unusual at $0.05$?

Answer: No; 0.1073741824>0.05. The probability $0.107$ exceeds the threshold $0.05$.

Flashcard 35: Identify the expected number of heads in $n=20$ flips if $P(H)=0.5$.

Answer: $np=20(0.5)=10$. Mean of binomial distribution is $np$.

Flashcard 36: What is a simulation in the context of evaluating a probability model with data?

Answer: Many random repetitions generated according to the model. Mimics the random process to see what typically happens.

Flashcard 37: Which simulation output best estimates the probability of the observed event under the model?

Answer: The proportion of simulated trials at least as extreme as observed. Counts how often simulated data matches or exceeds observed extremity.