Statistics Quiz: Evaluating Reports Based On Data

What this quiz covers

This quiz focuses on Evaluating Reports Based On Data, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

Question 1

1. The conclusion is not justified because a sample size of 40 is always too small to learn anything useful about grades.
2. The conclusion is not justified because the newsletter does not say what time the library closes.
3. The conclusion is justified because most of the surveyed students in the library reported A or B averages.
4. The conclusion is not justified because students were not randomly assigned to use the library, so higher grades could be due to other factors (like motivation), not the library itself. (correct answer)

Explanation: Evaluating reports based on data involves assessing whether the evidence supports the claims, particularly regarding causation. The newsletter claims that using the library after school causes better grades, based on a survey of 40 students already in the library, where 30 reported A or B averages and 10 reported C or lower. The key limitation is the lack of random assignment to library use, introducing self-selection bias and potential confounding factors like student motivation. This weakens the conclusion because higher grades might stem from traits of students who choose the library, not the library itself. The correct critique is most important as it highlights the inability to establish causation without controlling for other variables. A common misconception is that a clear majority in the sample proves causation, but correlation does not equal causation without experimental design. To evaluate similar reports, check (1) how data were collected (e.g., voluntary response), (2) what comparison is made (e.g., no non-library group), and (3) whether the display is honest (here, no graph issues).

Question 2

1. The conclusion is not justified because random assignment only allows generalizing to the whole school, not making causal claims.
2. The conclusion is not justified because any study with only 30 students can never support a causal claim.
3. The conclusion is not justified because the difference in averages might be due to the teacher’s opinion about music.
4. The conclusion is justified because random assignment to music vs. silence supports a causal conclusion for this class on that quiz. (correct answer)

Explanation: This question tests understanding of when causal conclusions are justified. The teacher claims instrumental music causes faster quiz completion, based on a randomized experiment where students were randomly assigned to music or silence conditions. Random assignment is the key feature that supports causal conclusions because it ensures the two groups are comparable except for the treatment (music). This eliminates confounding variables like ability or test anxiety being systematically different between groups. The 4-minute difference (18 vs 22 minutes) can be attributed to the music rather than other factors. The correct answer recognizes that random assignment within this specific context allows causal inference for this class and quiz. A misconception is thinking sample size alone determines whether causal claims are valid, but it's the random assignment that matters most. When evaluating experiments, check for: random assignment, controlled conditions, and appropriate scope of conclusions.

Question 3

1. The conclusion is not justified because the company did not report the students’ names.
2. The conclusion is not justified because without a comparison group, the score increase could be due to other factors (practice, regular class instruction, or an easier post-test). (correct answer)
3. The conclusion is not justified because 12 students is too small for any average to be computed.
4. The conclusion is justified because the same students improved by 20 points, so tutoring must have caused the increase.

Explanation: When evaluating data-based reports, it's essential to determine if the design allows for causal inferences. The company claims their tutoring raises math scores by 20 points, citing pre-test averages of 60 and post-test of 80 for 12 students, without a comparison group. The key limitation is the absence of a control group, allowing other factors like regular instruction or test familiarity to explain the increase. This limits the conclusion because we can't isolate tutoring as the cause. The correct critique is most important as it emphasizes the need for comparisons in pre-post designs. A misconception is that improvement in the same group proves causation, but without controls, correlation doesn't imply causation. To assess reports, check (1) data collection (pre-post only), (2) comparisons (none to non-tutored), and (3) honest display (no issues noted).

Question 4

1. The claim is well supported because 150 out of 200 is more than half.
2. The claim is not well supported because some students might have been hungry while answering.
3. The claim is not well supported because the sample is a convenience sample taken near the salad bar and may not represent all students’ preferences. (correct answer)
4. The claim is not well supported because 200 students is too small to estimate what most students prefer.

Explanation: In evaluating data-based reports, sample representativeness is critical for generalizing claims. The manager claims most students prefer the new menu, based on surveying 200 students near the salad bar, with 150 approving. The key limitation is the convenience sample, which may overrepresent health-conscious students and not reflect the whole population. This undermines the claim by introducing sampling bias, potentially skewing results. The correct critique is essential as it points out how non-random sampling limits broader inferences. A misconception is that a large sample like 200 ensures validity, but size doesn't correct bias; representativeness matters more. To evaluate, check (1) data collection (convenience method), (2) comparisons (none, just proportion), and (3) honest display (no issues).

Question 5

1. The claim is not credible because confiscations are counts, and only percentages can be used to show change.
2. The claim may be misleading because the number of confiscations could drop due to the new warning policy, even if phone use did not actually drop. (correct answer)
3. The claim is not credible because the announcement did not include data from last year as well.
4. The claim is definitely true because 20 is half of 40, so phone use must have dropped by 50%.

Explanation: Evaluating data-based reports involves assessing if metrics truly measure the claimed phenomenon. The announcement claims phone use dropped 50%, citing confiscations falling from 40 to 20, amid a new warning policy. The key limitation is that the policy change could reduce confiscations without reducing actual use, confounding the measure. This makes the claim misleading by attributing the drop to usage rather than reporting changes. The correct critique is crucial as it reveals how external factors can distort interpretations. A misconception is that halved counts prove halved behavior, but proxies like confiscations may not directly reflect the variable. To evaluate, check (1) data collection (teacher reports), (2) comparisons (monthly), and (3) honest display (no issues).

Question 6

1. The conclusion is not justified because the data are observational and other variables (like studying time) could explain the association. (correct answer)
2. The conclusion is not justified because a sample of 20 students means averages cannot be compared.
3. The conclusion is justified because the average quiz score is higher for students who reported 8 or more hours of sleep.
4. The conclusion is not justified because quiz scores should be reported as percentages, not out of 10.

Explanation: Evaluating reports based on data requires distinguishing between association and causation in observational studies. The blog claims that getting 8 hours of sleep makes students score higher, showing averages of 9/10 for 10 students with 8+ hours versus 7/10 for 10 with less. The key limitation is the observational design, where confounders like study time could explain the difference, not sleep alone. This weakens the causal conclusion by failing to rule out alternative explanations. The correct critique is most important because it prevents overreaching from correlation to causation. A misconception is that clear group differences prove cause, but observational data only show associations. For similar reports, check (1) data collection (self-reported), (2) comparisons (sleep groups), and (3) display honesty (no graph).

Question 7

1. The conclusion is not justified because random assignment was used, which only shows association, not causation.
2. The conclusion is justified for these students because random assignment makes a causal interpretation reasonable, though it may not generalize beyond this class. (correct answer)
3. The conclusion is not justified because the two groups had the same number of students, which makes it impossible to compare averages.
4. The conclusion is justified for all students everywhere because experiments always generalize to any population.

Explanation: Evaluating reports involves assessing experimental designs for causal validity and generalizability. The slide claims listening to music improves test scores, based on a randomized assignment of 60 students to music (average 84) or silence (78). The key strength is random assignment, supporting causation for these students, but the limitation is the single-class sample, limiting broader generalization. This justifies the conclusion narrowly but not universally. The correct critique is most important as it highlights when experiments allow causal claims. A misconception is that random assignment only shows association, but it enables causation; however, small scopes don't generalize widely. For similar reports, check (1) data collection (randomized), (2) comparisons (treatment groups), and (3) display honesty (no graph).

Question 8

1. The claim is supported because 12/30 = 40% and 14/70 = 20%, but the data show an association and do not prove energy drinks cause lateness. (correct answer)
2. The claim is supported because 12 is less than 14, so energy drink users are not more likely to be late.
3. The claim is not supported because the survey should have included at least 1,000 students to compare proportions.
4. The claim is not supported because the article uses the phrase “twice as likely,” which is too informal for statistics.

Explanation: Evaluating data-based reports means verifying numerical claims and their implications. The article claims energy drink users are twice as likely to be late, with data showing 12/30 (40%) users late versus 14/70 (20%) non-users. The key limitation is that the observational data show association but not causation, so while the 'twice as likely' holds, it doesn't prove drinks cause lateness. This tempers the claim by cautioning against causal overreach. The correct critique is essential as it balances support for the statistic with interpretive limits. A misconception is that small samples like 100 prevent proportion comparisons, but valid ratios can emerge; correlation isn't causation. To evaluate, check (1) data collection (survey), (2) comparisons (user groups), and (3) honest display (no issues).

Question 9

1. The conclusion is justified because club members have higher attendance than non-members.
2. The conclusion is not justified because the non-member group is much larger than the member group.
3. The conclusion is not justified because the report does not describe what activities the club does.
4. The conclusion is not justified because the membership requirement creates a confounding factor: club members already tended to have high attendance. (correct answer)

Explanation: When evaluating reports, identifying confounding variables is key to valid causal claims. The report claims the after-school club increased attendance, comparing members (96%, n=35) to non-members (92%, n=300), but membership required prior 95% attendance. The key limitation is confounding from the eligibility rule, as members already had high attendance, not necessarily due to the club. This limits the conclusion by suggesting selection bias rather than a club effect. The correct critique is most important because it exposes how prerequisites create non-comparable groups. A misconception is that larger non-member groups invalidate results, but confounding is the core issue; big samples don't fix it. For assessment, check (1) data collection (group comparisons), (2) comparisons (biased by requirement), and (3) display honesty (no graph).

Question 10

1. The claim is misleading because the y-axis is truncated, making a small increase look much larger, and the data do not show support “doubled.” (correct answer)
2. The claim is questionable because the bars should be a different color to be easier to read.
3. The claim is unreliable because percent should never be used; only counts are valid.
4. The conclusion is justified because the percent increased from 46% to 52%, which is an increase.

Explanation: Evaluating data-based reports requires examining how visuals and language represent the data accurately. The poster claims support for longer lunch has doubled, showing a bar chart with percentages rising from 46% to 52% on a y-axis starting at 45%. The key limitation is the truncated y-axis, which exaggerates the small 6% increase, and the word 'doubled' misrepresents the change. This weakens the claim by making the increase appear more dramatic than it is, potentially misleading viewers. The correct critique is crucial because it addresses how graphical distortions can bias interpretations. A misconception is that any increase justifies strong language like 'doubled,' but precise calculations show it's only a 13% relative increase. For similar reports, check (1) data collection method (not specified here), (2) comparisons (year-to-year), and (3) if the display is honest (truncated axis).

Question 11

1. The conclusion is not justified because without a control group and random assignment, other factors could explain the change, so causation is not supported. (correct answer)
2. The conclusion is justified because the percent increase is large, so it cannot be due to chance.
3. The conclusion is not justified because the report used percentages instead of counts, which makes the comparison impossible.
4. The conclusion is justified because volunteers are more reliable than randomly selected students.

Explanation: When evaluating data-based reports, we examine if study design supports causal claims, especially without controls or randomization. The headline claims a new planner app boosts homework completion by 40%, based on 25 volunteers self-reporting from 50% to 70% completion after two weeks, without a control group or random assignment. The key limitation is the lack of a control group and randomization, allowing confounding factors like novelty effects or external motivations to explain the change. This weakens the causal conclusion because we can't isolate the app's effect from other influences. The critique in choice B is most important as it addresses the inability to establish causation without proper experimental design. A misconception is that large percentage changes prove causation, but correlation does not equal causation without controls. Always check data collection methods for controls, the comparisons made, and if displays accurately represent changes without exaggeration.

Question 12

1. The conclusion is justified because 120 students is a large enough sample for any school.
2. The conclusion is not justified because the sample was taken only from students in the cafeteria line at one time, so it may not be representative of all students. (correct answer)
3. The conclusion is not justified because 78 is not a majority of 120.
4. The conclusion is not justified because the word “most” is too informal for a poll result.

Explanation: Evaluating data reports requires checking if the sample represents the population to support broad claims. The student council claims most students want longer lunch, based on polling 120 students in the cafeteria line on Tuesday, where 78 agreed. The key limitation is the non-random, convenience sample from one location and time, potentially biasing toward students who eat in the cafeteria and excluding others. This limits the conclusion because the sample may not reflect the entire school's views, weakening generalizability. Choice A is the strongest critique as it identifies sampling bias as the core issue preventing valid extrapolation. A misconception is that a large sample like 120 guarantees representativeness, but bias persists without random selection. Evaluate by examining data collection for representativeness, the comparisons to the population, and if results are presented honestly without overreach.

Question 13

1. The conclusion is not justified because the counselor did not report the median number of absences.
2. The conclusion is justified because the sample sizes are large, so joining a club must reduce absences.
3. The conclusion is not justified as a causal claim because this is observational; factors like motivation or schedule flexibility could confound the association. (correct answer)
4. The conclusion is not justified because averages cannot be compared across groups.

Explanation: Report evaluation requires caution with causal claims from observational data. The counselor claims joining a club reduces absences, finding club members (150) averaged 3 absences versus 5 for non-members (200). The key limitation is the observational nature, allowing confounders like inherent motivation in club joiners to explain lower absences. This overreaches causation as differences might not result from clubs but from self-selection. Choice B is most important as it emphasizes the confounding risk in non-randomized studies. A misconception is that large samples prove causation, but correlation doesn't equal causation without experiments. Check data collection for design type, group comparisons, and if displays accurately reflect associations without causal leaps.

Question 14

1. The conclusion is weak because the y-axis is truncated (3.8–4.2), which exaggerates small changes and the data do not increase each month. (correct answer)
2. The conclusion is justified because any upward trend on a line graph means satisfaction is soaring.
3. The conclusion is weak because line graphs should only be used when there are at least 12 months of data.
4. The conclusion is weak because the title uses an exclamation point, which is unprofessional.

Explanation: Evaluating data reports includes analyzing graphs for misleading scales and accurate trend descriptions. The website claims cafeteria satisfaction is soaring each month, with ratings from 3.9 to 4.1 on a truncated y-axis (3.8–4.2), but data plateau at 4.1. The key limitation is the truncated axis exaggerating minor fluctuations and the claim overstating non-increasing trends. This weakens the 'soaring' assertion as visuals distort small changes and data don't support continual rise. Choice A is the strongest critique for highlighting graphical deception and factual inaccuracy. A misconception is that any upward graph trend justifies hype, but truncated scales mislead and data must match claims. Evaluate by reviewing data collection, trend comparisons, and if displays like axes are honest without exaggeration.

Question 15

1. The conclusion is justified because people who bought Brand X are the best judges of which brand is preferred.
2. The conclusion is not justified because the survey sampled only Brand X buyers, creating selection bias and preventing generalization to all students. (correct answer)
3. The conclusion is not justified because the word “excellent” is subjective and should be removed from the survey.
4. The conclusion is not justified because 24 out of 30 is not enough to be called “preferred.”

Explanation: Evaluating reports involves spotting sampling biases that skew generalizations. The site claims students prefer Brand X over Y, surveying 30 Brand X buyers where 24 rated it excellent, then generalizing to all students. The key limitation is selection bias from sampling only Brand X purchasers, who are predisposed to favor it. This weakens the broad preference claim as the sample doesn't represent unbiased student opinions. Choice A is the best critique because it addresses how biased sampling invalidates generalization. A misconception is that buyers' opinions best judge preferences, but this ignores non-representative sampling. Assess by examining data collection for bias, comparisons to the population, and if conclusions are honestly limited to the sample.

Question 16

1. Whether the rule was written in a short or long paragraph, because shorter rules are usually more effective.
2. Whether the principal personally likes the new rule.
3. Whether 48 and 30 are even numbers, since even counts are more reliable than odd counts.
4. Whether the observations were collected on many comparable days (not just two) and whether anything else changed at the same time (e.g., special events or extra staff reminders). (correct answer)

Explanation: This question focuses on identifying what additional information is needed to evaluate a causal claim. The principal claims a new hallway rule reduced phone use based on comparing counts from one day before (48) to one day after (30) the rule. To properly evaluate this claim, we need to know if observations were collected across multiple comparable days and whether other factors changed simultaneously. Single-day comparisons can't account for natural variation—perhaps the "before" day had unusual circumstances increasing phone use, or the "after" day had a test period reducing hallway traffic. Other changes like special events, different staff supervision, or even weather could affect phone use. The correct answer identifies these temporal and confounding concerns as most critical. A misconception is focusing on irrelevant details rather than fundamental validity issues. When evaluating before-after comparisons, always ask: were multiple time points measured, and were other variables controlled or documented?

Question 17

1. The conclusion may be misleading because it uses store sales without context; changes in stock, promotions, or number of shopping days could explain the increase, not necessarily increased student interest. (correct answer)
2. The conclusion is invalid because the report did not include a photo of the bottles.
3. The conclusion is definitely correct because 45 is greater than 30, so interest must have increased.
4. The conclusion is invalid because percent change cannot be computed from counts like 30 and 45.

Explanation: This question tests understanding of contextual factors in interpreting data. The school store claims student interest in reusable bottles increased 50% based on sales rising from 30 to 45 bottles between months. The key issue is that sales figures alone don't necessarily reflect interest changes—many factors affect sales. October might have had more shopping days, special promotions, increased stock availability, or seasonal factors (like sports seasons starting). The store might have displayed bottles more prominently or run an environmental awareness campaign. The correct critique recognizes that sales changes could result from supply-side factors rather than demand changes. A misconception is assuming sales directly measure consumer interest without considering other variables. When evaluating business metrics as proxies for attitudes or behaviors, consider: what else affects the measured outcome, whether supply matched demand, and what contextual changes occurred.

Question 18

1. The conclusion is not well supported because doubling is impossible for test scores.
2. The conclusion is not well supported because the ad did not list each student’s individual score.
3. The conclusion is not well supported because without a comparison group, the score increase could be partly due to normal learning, repeated testing, or other factors besides tutoring. (correct answer)
4. The conclusion is well supported because the average increased, which proves tutoring caused the change.

Explanation: This question addresses the need for comparison groups when making causal claims. The tutoring center claims their program doubles test scores, showing students improved from 40 to 80 after 6 weeks. The critical weakness is the absence of a control group—we don't know what would happen without tutoring. Students might improve through normal classroom learning, familiarity with the test format, or natural development over 6 weeks. Without comparing to similar students who didn't receive tutoring, we can't isolate the tutoring effect from other factors. The correct critique identifies that improvement alone doesn't prove the intervention caused it. A common misconception is thinking any improvement proves program effectiveness, ignoring alternative explanations. When evaluating intervention claims, look for: appropriate comparison groups, consideration of other factors affecting outcomes, and recognition that change over time has multiple potential causes.

Question 19

1. The conclusion is justified because selecting every 5th student guarantees a perfectly random sample of the entire school.
2. The conclusion is not justified because 65% is a percentage and percentages are less accurate than counts.
3. The conclusion is not justified because 60 students is too small to estimate a percentage for any school.
4. The conclusion is not justified because the sample only includes students who bought lunch that day, so it may not represent all students at the school. (correct answer)

Explanation: This question tests your ability to evaluate sampling methods in data collection. The cafeteria manager claims 65% of all students prefer the new sandwich, based on surveying every 5th student in the lunch line. The critical limitation is that the sample only includes students who bought lunch that day, creating sampling bias. Students who bring lunch from home or skip lunch entirely aren't represented, and their preferences might differ from lunch-buyers. This makes the sample non-representative of the entire school population. The systematic selection (every 5th student) doesn't fix this bias because it still only samples from the biased pool of lunch-line students. A misconception is thinking systematic sampling automatically creates a representative sample, but the sampling frame (lunch line) must first represent the target population (all students). When evaluating surveys, check whether the sampling method captures all relevant subgroups of the population being studied.

Question 20

1. The conclusion is weak because the blog did not define “tardy,” even though the numbers are clearly different.
2. The conclusion is weak because the blog used counts instead of percentages, and only percentages can show change.
3. The conclusion is strong because a 50% decrease always proves the new format caused the change.
4. The conclusion is weak because it compares only two specific days, so the result could be due to normal day-to-day variation or other changes that week. (correct answer)

Explanation: This question examines evaluating claims based on limited data points. The blog claims the new announcement format reduced tardiness by 50%, comparing just two Mondays (20 tardies before, 10 after). The fundamental weakness is using only two data points, which makes it impossible to distinguish the format's effect from normal day-to-day variation. Tardiness naturally fluctuates due to weather, tests, events, or random chance. Without multiple observations before and after the change, we can't know if this was a typical reduction or just coincidence. The correct critique identifies that comparing single days provides insufficient evidence for causal claims. A common error is assuming any observed change must be due to the intervention, ignoring natural variability. To evaluate such reports properly, look for: adequate sample size across time, control for other variables, and recognition that single comparisons can't establish patterns or causation.