Analyzing Decisions and Strategies with Probability
Help Questions
Statistics › Analyzing Decisions and Strategies with Probability
A tech company is deciding how to handle login attempts that look suspicious. Internal data show:
- On a typical day, $0.5%$ of login attempts are truly malicious.
- The detection system flags $92%$ of malicious attempts.
- It also falsely flags $4%$ of legitimate attempts.
They are choosing a policy for flagged attempts. The goal is to minimize the chance that a malicious attempt is allowed through, while avoiding treating every user as malicious.
Which strategy is most reasonable given the probabilities?
- Strategy 1: Allow flagged attempts but send a warning email.
- Strategy 2: Require a second factor (2FA) only for flagged attempts.
- Strategy 3: Block all flagged attempts immediately.
Strategy 2, because flagged attempts include many false alarms, so adding 2FA reduces risk without assuming every flag is truly malicious.
Strategy 3, because malicious logins are scary, so it’s best to block anything that looks suspicious regardless of the false-flag rate.
Strategy 1, because only $0.5%$ of attempts are malicious overall, so flags are probably harmless.
Strategy 3, because $92%$ detection means a flagged attempt is malicious $92%$ of the time.
Explanation
This question involves the skill of analyzing decisions using probability. The company's goal is to minimize the chance that a malicious login attempt is allowed through, while avoiding treating every user as malicious. The relevant probabilities include a 0.5% base rate of malicious attempts, 92% detection of malicious ones, and 4% false flags on legitimate attempts, resulting in flagged attempts being malicious only about 10% of the time. Requiring 2FA for flagged attempts best aligns with this goal because it adds security to suspicious logins, stopping most malicious ones among them while inconveniencing but not blocking legitimate users. One incorrect option is to block all flagged attempts, which commits the flaw of base rate neglect by equating the 92% detection rate with the probability that a flag is malicious, leading to over-blocking good users. Remember that probabilities inform decisions by weighing risks, but they do not guarantee outcomes for any individual attempt. To apply this strategy elsewhere, first identify the decision goal, then compare the probabilities that matter most, such as the posterior probability of malice given a flag.
A city is planning snow response for a weekend. Forecast data for the region:
- $20%$ chance of heavy snow
- $50%$ chance of light snow
- $30%$ chance of no snow
If the city pre-treats roads Friday night, it reduces the chance of major traffic disruption to:
- $15%$ if heavy snow occurs
- $5%$ if light snow occurs
- $1%$ if no snow occurs
If the city does not pre-treat, the chance of major traffic disruption is:
- $40%$ if heavy snow occurs
- $12%$ if light snow occurs
- $1%$ if no snow occurs
The city’s goal is to minimize the overall probability of major traffic disruption. Which strategy is most reasonable given the probabilities?
Do not pre-treat, because heavy snow is unlikely ($20%$), so it’s not worth planning for.
Do not pre-treat, because if there is no snow ($30%$), disruption is only $1%$ either way.
Pre-treat, because it lowers the disruption probability in both heavy and light snow scenarios while not increasing it when there is no snow.
Pre-treat, because the worst-case outcome (heavy snow disruption) is severe, so probability doesn’t matter.
Explanation
This question involves the skill of analyzing decisions using probability. The city's goal is to minimize the overall probability of major traffic disruption across possible snow scenarios. The relevant probabilities include 20% heavy snow, 50% light, and 30% none, with pre-treatment reducing disruption to 15%, 5%, and 1% respectively, versus 40%, 12%, and 1% without, yielding a lower expected disruption (5.8% vs. 14.3%). Pre-treating best aligns with this goal because it substantially lowers disruption risks in heavy and light snow without increasing it in no-snow cases. One incorrect option is not pre-treating because heavy snow is unlikely, which commits the flaw of ignoring weighted probabilities by dismissing the impact of the 20% chance. Remember that probabilities inform decisions by calculating expected risks, but they do not guarantee outcomes for any weather event. To apply this strategy elsewhere, first identify the decision goal, then compare the probabilities that matter most, such as the expected value of disruption under each option.
A warehouse is choosing between two barcode scanners for order picking. Each scanned item is either correct (right product) or incorrect (wrong product). Based on trials:
- Scanner 1: correctly identifies the right product $98%$ of the time, but when it misreads, it fails to alert the worker $70%$ of the time.
- Scanner 2: correctly identifies the right product $96%$ of the time, but when it misreads, it fails to alert the worker only $20%$ of the time.
The warehouse’s goal is to minimize the chance that a wrong product is shipped (a wrong shipment happens when the scanner misreads and fails to alert). Which strategy is most reasonable given the probabilities?
Choose Scanner 2, because even though it misreads slightly more often, its much lower fail-to-alert rate makes a wrong shipment less likely overall.
Choose Scanner 2, because a worker once said it ‘felt’ more reliable during a busy shift.
Choose Scanner 1, because when it misreads, it fails to alert $70%$ of the time, which means it is more cautious.
Choose Scanner 1, because $98%$ accuracy is higher, so it will ship fewer wrong products.
Explanation
This question involves the skill of analyzing decisions using probability. The warehouse's goal is to minimize the chance that a wrong product is shipped, which occurs when the scanner misreads and fails to alert. The relevant probabilities show Scanner 1 misreads 2% of the time and fails to alert 70% of those, versus Scanner 2 misreading 4% but failing to alert only 20%, resulting in a lower wrong shipment rate for Scanner 2 (0.8% vs. 1.4%). Choosing Scanner 2 best aligns with this goal because its better alerting on misreads reduces overall errors, despite the slightly higher misread rate. One incorrect option is choosing Scanner 1 for its higher accuracy, which commits the flaw of incomplete analysis by ignoring the conditional failure-to-alert rate. Remember that probabilities inform decisions by combining error components, but they do not guarantee outcomes for any scan. To apply this strategy elsewhere, first identify the decision goal, then compare the probabilities that matter most, such as the joint probability of misread and no alert.
A clinic uses a screening test for a condition in a low-risk population. In this population, about $3%$ of people actually have the condition. The test has:
- Sensitivity $80%$
- Specificity $90%$
A patient tests positive. The clinic must decide what to do next. Options: A) Start treatment immediately. B) Order a more accurate confirmatory test before treatment. C) Ignore the result because the patient is low-risk.
The clinic’s goal is to minimize unnecessary treatment while still taking positives seriously. Which strategy is most reasonable given the probabilities?
Ignore the result, because $90%$ specificity means positives are usually wrong in any population.
Order a confirmatory test, because with a low base rate ($3%$) and imperfect specificity, many positives can be false positives, so confirmation reduces unnecessary treatment.
Start treatment immediately, because the condition is serious, so any chance is too high to wait for another test.
Start treatment immediately, because a positive result means the patient has the condition with $80%$ probability.
Explanation
This question involves the skill of analyzing decisions using probability. The clinic's goal is to minimize unnecessary treatment while still taking positive results seriously. The relevant probabilities include a 3% base rate of the condition, 80% sensitivity, and 90% specificity, meaning a positive test indicates only about a 20% chance of the condition due to false positives. Ordering a confirmatory test best aligns with this goal because it verifies the initial positive, reducing the risk of treating false positives while addressing true cases. One incorrect option is to start treatment immediately, which commits the flaw of base rate neglect by assuming the 80% sensitivity means an 80% chance of the condition given a positive. Remember that probabilities inform decisions by quantifying diagnostic accuracy, but they do not guarantee outcomes for any patient. To apply this strategy elsewhere, first identify the decision goal, then compare the probabilities that matter most, such as the positive predictive value in low-prevalence settings.
An airline is deciding how to schedule a spare aircraft to reduce cancellations caused by mechanical issues. Historical data:
- On any given day, Route A has a $6%$ chance of a mechanical issue that would cancel the flight unless a spare is available.
- Route B has a $3%$ chance of such an issue.
- If the spare is positioned for a route that has an issue, it prevents cancellation $90%$ of the time (sometimes the spare can’t be swapped fast enough).
The airline can position the spare for Route A or Route B each day. The goal is to maximize the probability of preventing at least one cancellation (not to minimize costs). Which strategy is most reasonable given the probabilities?
Position the spare at Route A, because the higher issue probability makes it more likely the spare will be needed and able to prevent a cancellation.
Position the spare at Route B, because Route B has fewer issues so the spare will be in better condition when needed.
Alternate days between A and B, because fairness is the best way to maximize prevention probability.
Position the spare at Route B, because once Route B had an issue on a day when no spare was available, causing a major disruption.
Explanation
This question involves the skill of analyzing decisions using probability. The airline's goal is to maximize the probability of preventing at least one cancellation using the spare aircraft. The relevant probabilities include a 6% issue rate for Route A and 3% for Route B, with the spare preventing a cancellation 90% of the time if positioned where an issue occurs. Positioning the spare at Route A best aligns with this goal because it yields a higher chance of successful prevention (5.4%) compared to Route B (2.7%), due to A's higher issue likelihood. One incorrect option is positioning at Route B based on an anecdote, which commits the flaw of anecdotal reasoning by prioritizing a single past event over probabilistic data. Remember that probabilities inform decisions by estimating expected benefits, but they do not guarantee outcomes on any given day. To apply this strategy elsewhere, first identify the decision goal, then compare the probabilities that matter most, such as the product of issue rates and prevention success.
A subscription app is deciding whether to show a retention offer to users who look likely to cancel. Their model produces a ‘high-risk’ flag. Based on past users:
- $10%$ of users would cancel within 30 days without any offer.
- Of those who would cancel, the model flags $70%$ as high-risk.
- Of those who would not cancel, the model still flags $15%$ as high-risk.
The company can choose one policy:
- Show the offer to all users.
- Show the offer only to high-risk users.
- Show the offer to no one.
The company’s goal is to target the offer so that most recipients are genuinely at risk of canceling (i.e., reduce wasted offers), while still reaching many would-be cancellers. Which strategy is most reasonable given the probabilities?
Policy 2, because $70%$ of flagged users will cancel, since the model catches $70%$ of cancellers.
Policy 2, because the high-risk flag concentrates would-be cancellers more than the overall $10%$ base rate, even though some flagged users would not cancel.
Policy 1, because offering to everyone guarantees that all would-be cancellers see it.
Policy 3, because $15%$ false flags means the model is unreliable, so targeting cannot work.
Explanation
This question involves the skill of analyzing decisions using probability. The company's goal is to target the retention offer so that most recipients are genuinely at risk of canceling, while still reaching many would-be cancellers. The relevant probabilities include a 10% base cancellation rate, 70% flagging of cancellers, and 15% false flags on non-cancellers, making flagged users about 34% likely to cancel versus 10% overall. Policy 2 best aligns with this goal because it concentrates offers on the higher-risk group (34% cancellation rate), reducing waste while covering 70% of potential cancellers. One incorrect option is Policy 1, which commits the flaw of not targeting by offering to all, resulting in only 10% of recipients being at risk and more wasted offers. Remember that probabilities inform decisions by assessing targeting efficiency, but they do not guarantee outcomes for any user. To apply this strategy elsewhere, first identify the decision goal, then compare the probabilities that matter most, such as the positive predictive value of the flag for cancellation risk.
A factory uses an automated scanner to flag defective parts before shipping. Historically, about $2%$ of parts are truly defective. The scanner has:
- Sensitivity: $90%$ (it flags $90%$ of truly defective parts)
- Specificity: $95%$ (it correctly clears $95%$ of truly good parts)
If a part is flagged, the factory can choose one of these strategies:
- Ship it anyway.
- Do a manual inspection on flagged parts only.
- Scrap all flagged parts with no further checks.
The factory’s goal is to minimize the chance that a defective part gets shipped, even if that means extra inspections or scrapping some good parts. Which strategy is most reasonable given the probabilities?
Ship flagged parts but manually inspect a few at random, because one time a flagged part turned out to be fine.
Do a manual inspection on flagged parts only, because the scanner will miss about $10%$ of defects and also falsely flag some good parts.
Scrap all flagged parts, because a flagged result means the part is defective about $90%$ of the time.
Ship it anyway, because most flagged parts will still be good since only $2%$ of all parts are defective.
Explanation
This question involves the skill of analyzing decisions using probability. The factory's goal is to minimize the chance that a defective part gets shipped, even if it means extra inspections or scrapping some good parts. The relevant probabilities include a 2% base defect rate, 90% sensitivity (flagging 90% of defects), and 95% specificity (clearing 95% of good parts), meaning flagged parts have about a 27% chance of being defective due to false positives. Manually inspecting flagged parts best aligns with this goal because it allows verification of the suspicious parts, catching true defects while potentially shipping good flagged ones, and unflagged parts are very unlikely to be defective (about 0.2% chance). One incorrect option is to scrap all flagged parts, which commits the flaw of base rate neglect by assuming the 90% sensitivity means flagged parts are defective 90% of the time, when it's actually much lower. Remember that probabilities inform decisions by quantifying risks, but they do not guarantee outcomes for any single part. To apply this strategy elsewhere, first identify the decision goal, then compare the probabilities that directly impact that goal, such as the posterior probability of defect given a flag.
A school is deciding whether to use a quick screening quiz to place students into an advanced workshop. Historically, 15% of students would truly benefit from the advanced workshop.
The screening quiz has these properties:
- Among students who would benefit, it recommends “advanced” 80% of the time.
- Among students who would not benefit, it still recommends “advanced” 20% of the time.
The school can either:
- Use the quiz recommendation directly, or
- Require that a student be recommended “advanced” twice by taking two independent versions of the quiz (same accuracy each time) before placing them.
Goal: increase the likelihood that a placed student truly benefits, even if fewer students are placed.
Which strategy is most reasonable given the probabilities?
Require two recommendations, because if a student is recommended twice, that guarantees they will benefit.
Use one quiz, because 80% accuracy among those who benefit means most placed students will benefit.
Require two recommendations, because it reduces false placements by making it less likely a non-benefiting student is recommended twice.
Use one quiz, because requiring two quizzes would unfairly exclude some students who could benefit.
Explanation
This problem involves analyzing decisions using probability to decide on workshop placement criteria. The decision goal is to increase the likelihood that a placed student truly benefits. The key probabilities are the 15% base benefit rate and the quiz's 80% true positive and 20% false positive rates, with two quizzes being independent. Requiring two recommendations best aligns with the goal because it raises the positive predictive value to about 74% versus 41% for one quiz by reducing false positives. One incorrect option is D, which overstates the certainty, assuming two positives guarantee benefit, a flaw in misunderstanding conditional probability. Remember, probabilities inform better placement but do not guarantee every placed student benefits. To transfer this strategy, identify the goal, then compare predictive values for different thresholds.
A warehouse is deciding how to screen incoming shipments for damaged items. Historically, 4% of items arrive damaged.
Two screening strategies are available:
- Strategy 1 (Quick Scan): correctly flags 70% of damaged items, but also incorrectly flags 8% of undamaged items.
- Strategy 2 (Thorough Scan): correctly flags 90% of damaged items, but also incorrectly flags 18% of undamaged items.
Each flagged item is pulled for manual inspection, and the warehouse wants to minimize the chance that an item released without manual inspection is actually damaged, even if that means inspecting more items overall.
Which strategy is most reasonable given the probabilities?
Strategy 1, because it flags fewer undamaged items, so it must also release fewer damaged items.
Strategy 2, because one time a thorough scan found damage that a quick scan missed, so it is safer.
Strategy 1, because damage is rare (4%), so the difference between 70% and 90% detection will not matter in practice.
Strategy 2, because it catches a higher fraction of damaged items, reducing the chance damaged items slip through uninspected.
Explanation
This problem involves analyzing decisions using probability to select the optimal screening strategy for damaged items. The decision goal is to minimize the probability that an item released without inspection is actually damaged. The key probabilities include a 4% base rate of damage, Strategy 1's 70% detection rate and 8% false positive rate, and Strategy 2's 90% detection rate and 18% false positive rate. Strategy 2 best aligns with the goal because its higher detection rate results in a lower conditional probability of damage given release (about 0.5% versus 1.34% for Strategy 1), prioritizing fewer missed damages over more inspections. One incorrect option is C, which commits the base rate fallacy by downplaying the detection difference due to rarity without calculating the actual impact on released items. Remember, these probabilities guide the decision to minimize risk but do not guarantee that no damaged items will ever be released. To transfer this strategy, identify the goal, then compare the conditional probabilities like P(damaged | released) that matter most.
A bike-sharing program is deciding how to place maintenance crews for flat-tire reports. Based on past data:
- On weekdays, 60% of flat-tire reports come from downtown and 40% from suburbs.
- On weekends, 35% come from downtown and 65% from suburbs.
Forecast: there is a 70% chance tomorrow is a weekday-like demand day and a 30% chance it is a weekend-like demand day (due to a major event).
The program can choose one of two crew placements for the whole day:
- Place the crew downtown.
- Place the crew in the suburbs.
Goal: maximize the probability the crew starts the day in the area where most reports will occur.
Which strategy is most reasonable given the probabilities?
Place the crew downtown, because 60% is greater than 65%, so downtown is always the best bet.
Place the crew downtown, because downtown has the higher share of reports on the more likely (weekday-like) day type.
Place the crew in the suburbs, because weekends have more reports in suburbs and the event could make it feel like a weekend.
Place the crew in the suburbs, because a friend once saw far more flats in the suburbs than downtown.
Explanation
This problem involves analyzing decisions using probability to decide crew placement for flat-tire reports. The decision goal is to maximize the probability that the crew is in the area with the most reports tomorrow. The relevant probabilities are a 70% chance of weekday-like demand (60% downtown, 40% suburbs) and 30% chance of weekend-like (35% downtown, 65% suburbs). Placing the crew downtown best aligns with the goal because it matches the higher-probability weekday scenario where downtown has the most reports, yielding a 70% overall success probability versus 30% for suburbs. One incorrect option is C, which incorrectly compares percentages without weighting by scenario likelihood, a flaw in ignoring expected values. Remember, probabilities inform the best placement but do not guarantee the crew will always be in the right area. To transfer this strategy, identify the goal, then compare the weighted probabilities of success for each option.