Evaluating Models with Data and Simulation - Statistics and Probability

1. What is the main reason to run many simulation repetitions (for example, $10{,}000$) instead of few?

   More repetitions reduce random simulation error in the estimated probability. Law of large numbers ensures convergence to true probability.

2. Which option is correct: Under $P(T)=0.5$, is $5$ tails in a row unusual at the $0.05$ level?

   Yes; $0.03125 0.05$. The probability $0.03125$ is less than the threshold $0.05$.

3. What does it mean for observed data to be consistent with a probability model?

   The result is not unusually rare under the model. Events occur with expected frequency, not significantly deviating from model predictions.

4. What model assumption is violated if outcomes in repeated trials influence each other during the process?

   The independence assumption of the probability model is violated. Dependent trials contradict binomial model's independence requirement.

5. What decision rule uses a significance level of $0.05$ to judge model consistency?

   Question the model if $p \le 0.05$; otherwise do not question it. Standard threshold: reject model when probability of observed data is very low.

6. What is $P(\text{at least }5\text{ tails in }5\text{ flips})$ when $P(T)=0.5$?

   $\frac{1}{32}$. Only one way to get all tails: TTTTT.

7. What is the probability of $6$ heads in a row if $P(H)=0.5$?

   $\left(\frac{1}{2}\right)^6=\frac{1}{64}$. Each flip is independent with probability $\frac{1}{2}$.

8. What is the expected number of heads in $n=20$ flips if the model has $p=0.5$?

   $np=20\cdot 0.5=10$. Expected value formula: $E[X] = np$ for binomial distribution.

9. Which conclusion matches $p=\frac{1}{32}$ for $5$ tails in $5$ flips when using a $0.05$ rule?

   Question the $p=0.5$ model because $\frac{1}{32}<0.05$. The probability is below the significance threshold, suggesting model inadequacy.

10. Identify the probability of exactly $8$ heads in $10$ flips if $P(H)=0.5$.

    $\binom{10}{8}(0.5)^{10}=0.0439453125$. Binomial probability with $n=10$, $k=8$, $p=0.5$.

11. Identify the probability of at least one head in $5$ flips if $P(H)=0.5$.

    $1-(0.5)^5=0.96875$. Complement of getting all tails in 5 flips.

12. Which option is correct: If $P(H)=0.5$, is exactly $8$ heads in $10$ flips unusual at $0.05$?

    Yes; $0.0439453125 0.05$. The probability $0.044$ is less than the threshold $0.05$.

13. Identify the probability of $0$ successes in $10$ trials if $p=0.2$.

    $(0.8)^{10}=0.1073741824$. All 10 trials fail when each has probability $0.8$ of failing.

14. What is a p-value in a simulation-based test of a model?

    The proportion of simulations at least as extreme as observed. Measures how surprising the observed data are under the model.

15. What does it mean for observed data to be consistent with a probability model?

    The outcome is not unusually unlikely under the model. The observed result has a reasonable probability of occurring by chance.

16. What is a simulated sampling distribution used for in model checking?

    To estimate how often a statistic occurs if the model is true. Shows the likelihood of various outcomes under the assumed model.

17. What is the definition of a simulation in statistical inference?

    Using random trials to imitate a chance process under a model. Generates artificial data matching the model's probability structure.

18. What is the empirical $p$-value if $37$ of $1000$ simulations are at least as extreme as observed?

    $\frac{37}{1000}=0.037$. Divide count of extreme outcomes by total simulations.

19. What is the probability of $5$ tails in a row if a coin has $P(T)=0.5$?

    $\left(\frac{1}{2}\right)^5=\frac{1}{32}$. Each flip has probability $\frac{1}{2}$; multiply for independent events.

20. Identify the correct simulation step to model $n=30$ flips with $p=0.5$ using random digits.

    Treat $0$-$4$ as heads and $5$-$9$ as tails for each trial. Splits digits evenly to simulate fair coin with $p=0.5$.

21. Which option best describes a two-sided (two-tail) extremeness definition?

    Count simulations at least as far from expected in either direction. Considers deviations in both positive and negative directions.

22. What conclusion is appropriate when the p-value is very small, such as at most $0.05$?

    Question the model; the data are inconsistent with it. Small p-value indicates the observed data are unlikely under the model.

23. What conclusion is appropriate when the p-value is not small, such as greater than $0.05$?

    Do not question the model; the data are plausible under it. Large p-value means the data could reasonably occur under the model.

24. Choose the correct interpretation of a small $p$-value in this context of checking a model.

    The observed result would be rare if the model were true. Low probability suggests observed data unlikely under assumed model.

25. What probability threshold is commonly used to label an outcome as unusual in this standard?

    A probability at most $0.05$. Standard significance level for hypothesis testing.

26. What does the phrase "at least as extreme as the observed result" mean in a simulation test?

    Outcomes with deviation from the model at least as large as observed. Includes all results that differ from expected by the observed amount or more.

27. Choose the correct interpretation of a large $p$-value when checking a model with simulation.

    The observed result is plausible under the model; do not question it. High probability indicates observed data aligns with model expectations.

28. What is the null model when checking whether data match a claimed probability process?

    The claimed probability distribution for the data-generating process. The hypothesis being tested about how the data are generated.

29. What probability model is assumed for $n$ independent trials with success probability $p$?

    A binomial model: $X \sim \text{Binomial}(n,p)$. Models repeated independent trials with constant success probability.

30. What is $P(\text{all }10\text{ outcomes match})$ for $10$ independent trials with $P(\text{match})=0.5$?

    $\left(\frac{1}{2}\right)^{10}=\frac{1}{1024}$. Multiply individual probabilities for independent events.

31. Which probability is estimated by the simulation proportion of outcomes at least as extreme as observed?

    An empirical $p$-value for the observed result under the model. Simulation frequency approximates theoretical probability of extreme events.

32. Which conclusion matches an empirical $p$-value of $0.12$ when using a $0.05$ cutoff?

    Do not question the model because $0.12>0.05$. Exceeds significance level, so data consistent with model.

33. Identify the probability of $5$ tails in a row if a coin has $P(T)=0.5$.

    $(0.5)^5=0.03125$. Each flip has probability $0.5$, multiply for independent events.

34. Which option is correct: If $p=0.2$, is $0$ successes in $10$ trials unusual at $0.05$?

    No; $0.1073741824>0.05$. The probability $0.107$ exceeds the threshold $0.05$.

35. Identify the expected number of heads in $n=20$ flips if $P(H)=0.5$.

    $np=20(0.5)=10$. Mean of binomial distribution is $np$.

36. What is a simulation in the context of evaluating a probability model with data?

    Many random repetitions generated according to the model. Mimics the random process to see what typically happens.

37. Which simulation output best estimates the probability of the observed event under the model?

    The proportion of simulated trials at least as extreme as observed. Counts how often simulated data matches or exceeds observed extremity.

Study these flashcards