Evaluating Reports Based on Data - Statistics and Probability

1. A school blog reports: "Switching to reusable water bottles reduced plastic waste by 50%." The blog compares the number of plastic bottles found in hallway trash during one week in September (200 bottles) to one week in December (100 bottles). The blog does not mention any other changes at school. Which statement best describes whether the conclusion is justified?
   Answer: The conclusion is not fully justified because comparing two different weeks months apart allows other factors (events, attendance, weather) to confound the result. Data report evaluation includes checking for confounding when comparing non-equivalent time periods. The blog claims switching to reusable bottles reduced plastic waste by 50%, comparing 200 bottles in September to 100 in December without noting other changes. The key limitation is the time gap allowing confounders like seasonal events or attendance variations to influence results. This weakens the causal attribution as the reduction might not stem solely from the switch. Choice B is the strongest critique because it identifies confounding as undermining the isolated effect claim. A misconception is that matching percentages like 50% prove causation, but temporal comparisons need controls for other factors. Evaluate by scrutinizing data collection timing, comparisons made, and if presentations honestly account for potential confounders.
2. A school store advertises: "Students prefer Brand X water bottles 3 to 1 over Brand Y." The store's data come from sales last month: 300 Brand X bottles sold and 100 Brand Y bottles sold. The ad does not mention that Brand X was priced at $8 while Brand Y was priced at $14. Which critique best evaluates the claim?
   Answer: The conclusion about preference is questionable because price is a likely confounding variable; higher sales of the cheaper bottle may not reflect true preference. In evaluating claims, we identify confounding variables that could explain observed differences. The store claims students prefer Brand X water bottles 3 to 1 over Brand Y, based on sales of 300 versus 100, without mentioning Brand X's lower price ($8 vs. $14). The limitation is the confounding factor of price, which could drive sales rather than true preference. This questions the conclusion by suggesting affordability, not preference, might account for the ratio. Choice A's critique is crucial because it highlights this confounder, key to interpreting sales data accurately. A misconception is that sales directly equal preference, but external factors like price must be controlled. For such reports, examine (1) data collection for all variables, (2) comparisons isolating the claimed factor, and (3) displays that don't omit influential details.
3. A student council poster claims: "Support for longer lunch has doubled this year!" The poster includes a bar chart of the percent of students who support longer lunch. Chart description: The y-axis is labeled "Percent supporting longer lunch" and starts at 45% (not 0%). The bars show 46% last year and 52% this year. Which critique best evaluates the claim?
   Answer: The claim is misleading because the y-axis is truncated, making a small increase look much larger, and the data do not show support "doubled." Evaluating data-based reports requires examining how visuals and language represent the data accurately. The poster claims support for longer lunch has doubled, showing a bar chart with percentages rising from 46% to 52% on a y-axis starting at 45%. The key limitation is the truncated y-axis, which exaggerates the small 6% increase, and the word 'doubled' misrepresents the change. This weakens the claim by making the increase appear more dramatic than it is, potentially misleading viewers. The correct critique is crucial because it addresses how graphical distortions can bias interpretations. A misconception is that any increase justifies strong language like 'doubled,' but precise calculations show it's only a 13% relative increase. For similar reports, check (1) data collection method (not specified here), (2) comparisons (year-to-year), and (3) if the display is honest (truncated axis).
4. A sports drink ad says: "Drinking HydrateNow makes runners faster." In a study, 30 runners were randomly assigned to drink HydrateNow or plain water before a 1-mile run. The HydrateNow group averaged 7:50, and the water group averaged 8:05. Which critique best evaluates the claim?
   Answer: The claim is reasonably supported for these runners because random assignment allows a causal conclusion, though the result may not generalize beyond similar runners and conditions. This question evaluates a causal claim based on a randomized experiment. The sports drink company randomly assigned 30 runners to drink either HydrateNow or water, finding a 15-second average difference in mile times. Because of random assignment, this study can support a causal conclusion—for these specific runners under these conditions, HydrateNow appeared to improve performance. However, the claim's scope matters: the effect might not generalize to all runners, distances, or conditions. The correct answer recognizes both the validity of the causal conclusion (due to randomization) and its limited generalizability. A misconception is thinking small differences are always meaningless—even 15 seconds can matter in running. When evaluating experimental claims, consider both internal validity (does X cause Y here?) and external validity (does this apply broadly?).
5. A cafeteria manager says: "Students prefer the new sandwich." For one lunch period, the manager stood by the register and asked every 5th student in line to answer a question. Of the 60 students asked, 39 chose the new sandwich and 21 chose the old sandwich. The manager concludes: "About 65% of all students at our school prefer the new sandwich." Which statement best describes whether the conclusion is justified?
   Answer: The conclusion is not justified because the sample only includes students who bought lunch that day, so it may not represent all students at the school. This question tests your ability to evaluate sampling methods in data collection. The cafeteria manager claims 65% of all students prefer the new sandwich, based on surveying every 5th student in the lunch line. The critical limitation is that the sample only includes students who bought lunch that day, creating sampling bias. Students who bring lunch from home or skip lunch entirely aren't represented, and their preferences might differ from lunch-buyers. This makes the sample non-representative of the entire school population. The systematic selection (every 5th student) doesn't fix this bias because it still only samples from the biased pool of lunch-line students. A misconception is thinking systematic sampling automatically creates a representative sample, but the sampling frame (lunch line) must first represent the target population (all students). When evaluating surveys, check whether the sampling method captures all relevant subgroups of the population being studied.
6. A student newspaper claims: "The new tutoring center helps students pass math." It reports that among 40 students who visited the tutoring center at least twice, 34 passed math this semester. The article does not report pass rates for students who did not use the tutoring center. What information is most needed to judge this report?
   Answer: The pass rate for a comparable group of students who did not use the tutoring center (or pass rates before the center existed). To evaluate data reports, identify missing information needed for valid comparisons, like baseline or control data. The newspaper claims the tutoring center helps students pass math, reporting 34 of 40 users passed, without non-user pass rates. The key limitation is the absence of a comparison group, preventing assessment of whether tutoring improved outcomes relative to others. This limits the claim as high pass rates might reflect self-selection of motivated students. Choice A is most needed because comparable group data would enable judging the center's impact. A misconception is that success in a single group proves effectiveness, but correlation without comparison doesn't show causation. Check data collection for completeness, what comparisons are possible, and if reports transparently include necessary benchmarks.
7. A school store report states: "Reusable water bottles are becoming more popular." It notes that in September the store sold 30 reusable bottles, and in October it sold 45 reusable bottles. Based on this, the report concludes: "Student interest in reusable bottles increased by 50%." Which critique best evaluates the claim?
   Answer: The conclusion may be misleading because it uses store sales without context; changes in stock, promotions, or number of shopping days could explain the increase, not necessarily increased student interest. This question tests understanding of contextual factors in interpreting data. The school store claims student interest in reusable bottles increased 50% based on sales rising from 30 to 45 bottles between months. The key issue is that sales figures alone don't necessarily reflect interest changes—many factors affect sales. October might have had more shopping days, special promotions, increased stock availability, or seasonal factors (like sports seasons starting). The store might have displayed bottles more prominently or run an environmental awareness campaign. The correct critique recognizes that sales changes could result from supply-side factors rather than demand changes. A misconception is assuming sales directly measure consumer interest without considering other variables. When evaluating business metrics as proxies for attitudes or behaviors, consider: what else affects the measured outcome, whether supply matched demand, and what contextual changes occurred.
8. A student council post says: "Joining a club reduces stress." The post cites a survey of 120 students at one high school: 70 club members reported an average stress rating of 4.8 (out of 10), and 50 non-members reported an average stress rating of 6.1. Which critique best evaluates the claim?
   Answer: The claim is too strong because the survey is observational; club membership may be associated with stress, but it does not show that joining a club causes lower stress. This question examines whether joining clubs causes lower stress based on survey data. The student council found club members reported average stress of 4.8 while non-members reported 6.1, then claimed joining clubs reduces stress. The fundamental limitation is that this is observational data without random assignment—students chose whether to join clubs. This creates potential confounding: perhaps less-stressed students are more likely to join clubs, or students with better time management join clubs AND experience less stress. The correct critique recognizes that observational studies cannot prove causation, only association. A common misconception is that any difference between groups proves one thing causes another, but correlation doesn't equal causation. To evaluate causal claims, check whether participants were randomly assigned to groups or self-selected.
9. A student council post claims: "Most students support extending lunch by 10 minutes." The post is based on an online poll link shared on the student council's social media page; 312 students responded out of about 1,800 students at the school, and 68% of respondents clicked "support." Which statement best describes whether the conclusion is justified?
   Answer: The conclusion is limited because the poll used a voluntary response sample, so respondents may not represent the whole student body. When evaluating data-based reports, we examine if the sampling method allows for generalization to the broader population. The student council claims most students support extending lunch by 10 minutes, based on a voluntary online poll where 312 out of 1,800 students responded, with 68% in favor. The primary limitation is voluntary response bias, as those who chose to respond may have stronger opinions, not representing the entire student body. This limits the conclusion by potentially overrepresenting supporters, making it unreliable for the whole school. Choice B's critique is crucial because it addresses this bias, which undermines the claim's validity more than sample size alone. A misconception is that a large number of responses guarantees representativeness, but bias persists if the sample isn't random. For future reports, verify (1) data collection for randomness, (2) comparisons to the population, and (3) honest presentation without overstating generalizability.
10. A cafeteria manager claims: "Most students prefer the new pizza recipe." To support this, the manager asked the first 40 students who entered the cafeteria on Monday and found 28 said they preferred the new recipe. Which statement best describes whether the conclusion is justified?
    Answer: The conclusion is limited because the sample is a convenience sample (first 40 students on Monday) and may not represent all students' preferences. This question tests your ability to evaluate whether a sample represents a larger population. The cafeteria manager surveyed only the first 40 students entering on Monday morning, which is a convenience sample. The key limitation is sampling bias: students who arrive early on Monday might not represent all students—perhaps they're athletes with morning practice, or students who prefer breakfast items and have different lunch preferences. Even though 28 out of 40 (70%) preferred the new recipe, this percentage might not reflect the whole school's preferences. The correct critique identifies the convenience sampling as the main weakness. A misconception is thinking any majority result represents everyone, but how you collect data matters more than the results themselves. When evaluating surveys, always check if the sample was randomly selected or just convenient.
11. A student newspaper writes: "Energy drink users are twice as likely to be late to first period." The article reports a survey of 100 students: 30 students said they drink an energy drink on school mornings; among them, 12 reported being late at least once in the past month. Of the 70 who said they do not drink energy drinks, 14 reported being late at least once. Which critique best evaluates the claim?
    Answer: The claim is supported because 12/30 = 40% and 14/70 = 20%, but the data show an association and do not prove energy drinks cause lateness. Evaluating data-based reports means verifying numerical claims and their implications. The article claims energy drink users are twice as likely to be late, with data showing 12/30 (40%) users late versus 14/70 (20%) non-users. The key limitation is that the observational data show association but not causation, so while the 'twice as likely' holds, it doesn't prove drinks cause lateness. This tempers the claim by cautioning against causal overreach. The correct critique is essential as it balances support for the statistic with interpretive limits. A misconception is that small samples like 100 prevent proportion comparisons, but valid ratios can emerge; correlation isn't causation. To evaluate, check (1) data collection (survey), (2) comparisons (user groups), and (3) honest display (no issues).
12. A school social media post says: "Joining a club leads to higher GPA." The post summarizes a survey of 120 students: club members had an average GPA of 3.4, and non-members had an average GPA of 3.0. The post concludes: "Clubs raise GPA by 0.4." Which critique best evaluates the claim?
    Answer: The conclusion is not justified because this is an observational comparison; students who join clubs may already differ (e.g., time management), so the difference may be due to confounding rather than clubs causing higher GPA. This question addresses the classic correlation versus causation issue in observational studies. The post claims joining clubs leads to higher GPA based on observing that club members average 3.4 while non-members average 3.0. The critical flaw is that this is observational data without random assignment—students self-select into clubs. Students who join clubs likely differ in motivation, time management skills, or family support, which could also explain their higher GPAs. These confounding variables make it impossible to conclude clubs cause the GPA difference. The correct critique identifies that observational comparisons can't establish causation due to potential confounding. A common misconception is that large sample sizes (120 students) can overcome the fundamental limitation of observational data. To evaluate such reports, distinguish between observational studies (which show association) and experiments (which can show causation through random assignment).
13. A school website headline reads: "New online homework system improves grades." The school piloted the system by letting students choose whether to use it. At the end of the quarter, the average grade in math was 88 for students who chose the system (n=70) and 82 for students who did not (n=90). Which critique best evaluates the claim?
    Answer: The conclusion is limited because students self-selected into groups, so the difference could be due to preexisting differences rather than the system. Evaluating reports on data involves checking for biases in group formation that affect conclusions. The website claims the online homework system improves grades, comparing self-selected users (average 88, n=70) to non-users (82, n=90). The key limitation is self-selection, where preexisting differences like study habits could explain the grade gap, not the system. This weakens the claim by confounding the effect of the system with participant characteristics. The correct critique is vital because it identifies selection bias as a barrier to causal claims. A misconception is that unequal sample sizes invalidate comparisons, but the real issue is non-random groups; larger samples don't fix bias. For evaluation, check (1) data collection (opt-in), (2) comparisons (self-selected groups), and (3) display honesty (no graph mentioned).
14. A teacher says: "Students who sit in the front row participate more." She observed one class period and tallied participation. Front row (6 students): 18 total comments. Other seats (24 students): 24 total comments. She concludes: "Sitting in the front causes more participation." Which critique best evaluates the claim?
    Answer: The conclusion is not justified because this was observational and students were not randomly assigned seats, so self-selection or other factors could explain the difference. Evaluating reports means distinguishing observational data from experimental to avoid causation overreach. The teacher claims sitting in the front causes more participation, observing 18 comments from 6 front-row students versus 24 from 24 others, without random seat assignment. The key limitation is the observational design, allowing self-selection where engaged students choose front seats, confounding the results. This limits the causal claim because we can't rule out other factors explaining the difference. Choice B is most critical as it highlights the need for randomization to establish causation. A misconception is that observed differences prove cause, but correlation isn't causation without controls. Assess by reviewing data collection for randomization, comparisons across groups, and if conclusions are presented without overstating evidence.
15. A student blog post says: "Students who sleep at least 8 hours score higher on quizzes." The blogger collected data from one class: 10 students who reported sleeping at least 8 hours averaged 9/10 on a quiz, while 10 students who reported less than 8 hours averaged 7/10. The blogger concludes that getting 8 hours of sleep makes students score higher. Which critique best evaluates the claim?
    Answer: The conclusion is not justified because the data are observational and other variables (like studying time) could explain the association. Evaluating reports based on data requires distinguishing between association and causation in observational studies. The blog claims that getting 8 hours of sleep makes students score higher, showing averages of 9/10 for 10 students with 8+ hours versus 7/10 for 10 with less. The key limitation is the observational design, where confounders like study time could explain the difference, not sleep alone. This weakens the causal conclusion by failing to rule out alternative explanations. The correct critique is most important because it prevents overreaching from correlation to causation. A misconception is that clear group differences prove cause, but observational data only show associations. For similar reports, check (1) data collection (self-reported), (2) comparisons (sleep groups), and (3) display honesty (no graph).
16. A school board report claims: "The new after-school club increased attendance." The report compares attendance for club members vs non-members during the same semester. Data summary: Club members (n=35) averaged 96% attendance; non-members (n=300) averaged 92% attendance. Membership required a minimum 95% attendance in the previous semester to join. Which critique best evaluates the claim?
    Answer: The conclusion is not justified because the membership requirement creates a confounding factor: club members already tended to have high attendance. When evaluating reports, identifying confounding variables is key to valid causal claims. The report claims the after-school club increased attendance, comparing members (96%, n=35) to non-members (92%, n=300), but membership required prior 95% attendance. The key limitation is confounding from the eligibility rule, as members already had high attendance, not necessarily due to the club. This limits the conclusion by suggesting selection bias rather than a club effect. The correct critique is most important because it exposes how prerequisites create non-comparable groups. A misconception is that larger non-member groups invalidate results, but confounding is the core issue; big samples don't fix it. For assessment, check (1) data collection (group comparisons), (2) comparisons (biased by requirement), and (3) display honesty (no graph).
17. A tutoring center advertisement says: "Our tutoring program doubles students' test scores." The ad reports that 12 students who signed up had an average pre-test score of 40 and an average post-test score of 80 after 6 weeks. There is no comparison group of students who did not receive tutoring. Which critique best evaluates the claim?
    Answer: The conclusion is not well supported because without a comparison group, the score increase could be partly due to normal learning, repeated testing, or other factors besides tutoring. This question addresses the need for comparison groups when making causal claims. The tutoring center claims their program doubles test scores, showing students improved from 40 to 80 after 6 weeks. The critical weakness is the absence of a control group—we don't know what would happen without tutoring. Students might improve through normal classroom learning, familiarity with the test format, or natural development over 6 weeks. Without comparing to similar students who didn't receive tutoring, we can't isolate the tutoring effect from other factors. The correct critique identifies that improvement alone doesn't prove the intervention caused it. A common misconception is thinking any improvement proves program effectiveness, ignoring alternative explanations. When evaluating intervention claims, look for: appropriate comparison groups, consideration of other factors affecting outcomes, and recognition that change over time has multiple potential causes.
18. A student council flyer claims: "Most students want longer lunch." The council posted a poll link on its own social media account and received 310 responses. Of those, 240 said "Yes, longer lunch" and 70 said "No." The flyer concludes: "About 77% of students want longer lunch." Which critique best evaluates the claim?
    Answer: The conclusion may not be reliable because the poll is a voluntary response sample from the council's followers, so it may be biased and not representative of all students. This question examines voluntary response bias in survey sampling. The student council claims 77% of students want longer lunch based on a poll posted on their social media, receiving 310 responses. The fundamental problem is voluntary response bias—only students who follow the council's account and feel strongly enough to respond participated. This self-selected sample likely overrepresents students already engaged with student government and those with strong opinions about lunch length. The 77% figure may not represent the broader student body's preferences. The correct critique identifies this sampling bias makes the result non-representative. A misconception is believing large response numbers (310) guarantee representativeness, but sample size doesn't fix biased sampling methods. To evaluate polls, check: how participants were recruited, whether all students had equal opportunity to respond, and whether the sample characteristics match the population.
19. A student blog posts: "The new morning announcement format reduced tardiness by 50%!" The blog reports that on one Monday there were 20 tardies, and on the next Monday (after the new format started) there were 10 tardies. No other days are mentioned. Which critique best evaluates the claim?
    Answer: The conclusion is weak because it compares only two specific days, so the result could be due to normal day-to-day variation or other changes that week. This question examines evaluating claims based on limited data points. The blog claims the new announcement format reduced tardiness by 50%, comparing just two Mondays (20 tardies before, 10 after). The fundamental weakness is using only two data points, which makes it impossible to distinguish the format's effect from normal day-to-day variation. Tardiness naturally fluctuates due to weather, tests, events, or random chance. Without multiple observations before and after the change, we can't know if this was a typical reduction or just coincidence. The correct critique identifies that comparing single days provides insufficient evidence for causal claims. A common error is assuming any observed change must be due to the intervention, ignoring natural variability. To evaluate such reports properly, look for: adequate sample size across time, control for other variables, and recognition that single comparisons can't establish patterns or causation.
20. A poster shows a bar chart titled "Average Quiz Score Before vs After New Review Sessions." The chart shows "Before: 78" and "After: 82." The y-axis is labeled from 75 to 85 (not from 0). The poster claims: "Review sessions dramatically increased quiz scores." Which critique best evaluates the claim?
    Answer: The conclusion is weak because the truncated y-axis (75–85) visually exaggerates a small 4-point increase. In evaluating data reports, we assess if graphical displays accurately represent changes without misleading visuals. The poster claims review sessions dramatically increased quiz scores, showing bars from 78 to 82 on a y-axis truncated from 75 to 85. The key limitation is the truncated y-axis, which exaggerates a small 4-point increase by compressing the scale. This weakens the 'dramatic' claim as the visual distortion misleads viewers about the change's magnitude. Choice B is the best critique because it points out how the graph's design dishonestly amplifies the effect. A misconception is that any increase on a graph justifies strong language, but truncated axes can mislead even with real data. Check data collection details, comparisons shown, and especially if displays like axes are honest and not manipulated.

Start this quiz