Business Analytics Quiz: Precision Recall And Roc Auc

10 questions · exam conditions

0:00

Precision Recall And Roc AucQuestion 1 of 10

A lender evaluates a default-prediction model on $1{,}000$ applicants. Of the $80$ applicants who actually defaulted, the model identified $60$ as high risk. It also classified $90$ applicants who did not default as high risk.

What is the model's precision for the high-risk classification?

40.0\%

, because

60

of the

150

high-risk classifications were actual defaults

75.0\%

, because

60

of the

80

actual defaults were classified as high risk

85.0\%

, because

850

of the

1{,}000

applicants were not classified as high risk

92.4\%

, because

850

of the

920

nondefaulting applicants were classified correctly

← Back to quizzes

Business Analytics Quiz

Business Analytics Quiz: Precision Recall And Roc Auc

Practice Precision Recall And Roc Auc in Business Analytics with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Precision Recall And Roc Auc, giving you a quick way to practice the rules, question types, and explanations that matter most for Business Analytics.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

What is the model's precision for the high-risk classification?

$40.0\%$ , because $60$ of the $150$ high-risk classifications were actual defaults (correct answer)
$75.0\%$ , because $60$ of the $80$ actual defaults were classified as high risk
$85.0\%$ , because $850$ of the $1{,}000$ applicants were not classified as high risk
$92.4\%$ , because $850$ of the $920$ nondefaulting applicants were classified correctly

Explanation: Whenever you see a question about model evaluation, your first task is to identify which metric is being asked about — precision, recall, and accuracy are easy to confuse because they all use similar numbers from the same scenario. Precision answers the question: "Of all the applicants the model flagged as high risk, what fraction actually defaulted?" It focuses on the quality of positive predictions. The formula is:

\text{Precision} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Positives}}

From the passage: the model flagged

60 + 90 = 150

applicants as high risk. Of those,

60

actually defaulted (true positives) and

90

did not (false positives). So:

\text{Precision} = \frac{60}{150} = 0.40 = 40.0\%

That confirms A is correct. B describes recall (also called sensitivity), not precision. It measures how many actual defaults the model caught —

60

out of

80

, or

75\%

— but that's a different question entirely. C calculates nothing meaningful here. The

850

applicants not flagged as high risk is irrelevant to precision, and the percentage doesn't correspond to any standard metric in this context. D describes the true negative rate for nondefaulters — correctly identifying

850

920

nondefaulting applicants — which relates to specificity, not precision. A useful memory trick: precision is about the predictions (denominator = all flagged), while recall is about reality (denominator = all actual positives). Keeping that distinction clear will protect you from the B trap, which is the most tempting wrong answer on questions like this.

Question 2

An insurer expects $200$ truly fraudulent claims among $10{,}000$ claims. At a proposed threshold, a model is expected to achieve recall of $70\%$ . Management requires precision of at least $80\%$ before automating investigations.

What is the largest false-positive rate that can satisfy the precision requirement, assuming the expected counts are accurate?

Approximately $0.36\%$ , corresponding to no more than $35$ false positives among $9{,}800$ legitimate claims (correct answer)
Approximately $1.43\%$ , corresponding to no more than $140$ false positives among $9{,}800$ legitimate claims
Approximately $2.00\%$ , corresponding to no more than $196$ false positives among $9{,}800$ legitimate claims
Approximately $20.00\%$ , corresponding to no more than $1{,}960$ false positives among $9{,}800$ legitimate claims

Explanation: When a question ties precision to false positives, your job is to work backwards from the precision formula to find the maximum allowable false positives, then convert that count into a rate. Start with what you know. Recall of

70\%

200

true fraudulent claims means the model catches

0.70 \times 200 = 140

true positives. Precision is defined as

\text{Precision} = \frac{TP}{TP + FP}

. Management requires precision

\geq 80\%

, so:

0.80 = \frac{140}{140 + FP}

. Solving gives

140 + FP = 175

, so

FP \leq 35

. The false-positive rate is then

\frac{35}{9{,}800} \approx 0.357\%

. This confirms answer A — at most

35

false positives, roughly

0.36\%

of legitimate claims flagged incorrectly. Answer B (

140

false positives,

1.43\%

) is a classic trap: it confuses the number of true positives (

140

) with the maximum allowable false positives. These are entirely different quantities. Answer C (

196

false positives) likely comes from using

200 \times (1 - 0.02) = 196

, mixing up recall and precision arithmetic without a clear framework. Answer D (

1{,}960

false positives,

20\%

) inflates the false-positive pool dramatically — possibly from misapplying the

80\%

precision as a

20\%

error rate directly against the full legitimate-claim pool, ignoring the true-positive anchor entirely. A reliable habit: always anchor your precision calculation to the confirmed true-positive count first, then solve for FP. Confusing TP counts with FP limits is the most common error on classifier-performance questions.

Question 3

A collections model is applied to a cohort containing $100$ customers who will default and $900$ who will not. A higher threshold produces a true-positive rate of $60\%$ and a false-positive rate of $5\%$ . A lower threshold produces a true-positive rate of $80\%$ and a false-positive rate of $12\%$ . Each missed default costs $200$ monetary units, while each false-positive review costs $20$ monetary units.

Which threshold has the lower expected cost for this cohort, and by how much?

The higher threshold, by $2{,}730$ monetary units, because it generates fewer false-positive reviews
The higher threshold, by $1{,}840$ monetary units, because its false-positive rate is lower
The lower threshold, by $2{,}740$ monetary units, because avoiding additional missed defaults outweighs added reviews (correct answer)
The lower threshold, by $4{,}000$ monetary units, because it identifies $20$ additional defaults

Explanation: When a model has two possible thresholds, choosing between them is a cost-benefit calculation — you must weigh the savings from catching more true positives against the added expense of generating more false positives. Start by translating rates into counts. Your cohort has 100 defaulters and 900 non-defaulters. Higher threshold:

0.60 \times 100 = 60

true positives caught, so

40

missed defaults. False positives:

0.05 \times 900 = 45

. Cost:

(40 \times 200) + (45 \times 20) = 8{,}000 + 900 = \$8{,}900

. Lower threshold:

0.80 \times 100 = 80

true positives caught, so

20

missed defaults. False positives:

0.12 \times 900 = 108

. Cost:

(20 \times 200) + (108 \times 20) = 4{,}000 + 2{,}160 = \$6{,}160

. The lower threshold saves

8{,}900 - 6{,}160 = \$2{,}740

, confirming C is correct — catching 20 additional high-cost defaults more than compensates for the extra reviews. A is wrong because the higher threshold does not win on total cost — its higher miss rate is too expensive despite fewer false positives. B repeats the same flawed logic and invents a figure (

1{,}840

) that matches no actual calculation. D correctly identifies the 20 additional defaults caught but overstates the savings at

\$4{,}000

, ignoring the offsetting cost of the additional 63 false-positive reviews. A useful habit: never compare thresholds based on a single rate in isolation. Always compute total dollar cost using both error types, their frequencies, and their respective penalties — asymmetric costs can completely reverse what "better" looks like.

Question 4

Two fraud models are evaluated on the same representative holdout sample. Model A has ROC AUC $0.89$ , and Model B has ROC AUC $0.86$ . Because investigation capacity is limited, the business must operate at a false-positive rate no greater than $5\%$ . At a $5\%$ false-positive rate, Model A has recall $48\%$ and Model B has recall $62\%$ .

Which model-selection decision is best supported by these results?

Select Model A because a higher overall AUC guarantees better recall at every feasible threshold
Select Model B because it captures more fraud within the business's required false-positive-rate region (correct answer)
Select Model A because its lower recall indicates that its positive alerts have higher precision
Treat the models as equivalent because their AUC values differ by less than $5$ percentage points

Explanation: When selecting a model for a real-world deployment, overall AUC tells only part of the story. AUC summarizes performance across all possible thresholds, but businesses rarely operate across all thresholds — they operate at one specific operating point dictated by capacity, cost, or policy. When a constraint like "no more than

5\%

false-positive rate" is imposed, the only metric that matters is performance within that constrained region. Here, Model B delivers

62\%

recall at the

5\%

FPR operating point versus Model A's

48\%

— meaning Model B catches 14 percentage points more fraud under the exact conditions the business must use. That makes B the correct choice: it captures more fraud where it counts, making it the superior tool for this deployment context. A is wrong because a higher AUC absolutely does not guarantee better performance at every threshold. AUC reflects average performance; two ROC curves can cross, meaning one model dominates in some FPR ranges while the other dominates elsewhere. This is precisely what's happening here. C contains a logical error. Lower recall doesn't imply higher precision — those two metrics can move independently depending on threshold placement and class distribution. You cannot infer precision from recall alone without additional information. D applies an arbitrary rule of thumb (

5

percentage point AUC difference) and ignores the operational evidence entirely. No such threshold for "practical equivalence" is a recognized standard, and the recall gap at the business's required FPR is substantial and decision-relevant. Study tip: On model-selection questions, always check whether a specific operating constraint is given. If it is, local performance at that constraint trumps global metrics like AUC every time.

Question 5

At a selected threshold, a renewal model is evaluated on $120$ customers who renewed and $880$ customers who did not renew. The model correctly classifies $78$ renewing customers as positive and incorrectly classifies $88$ nonrenewing customers as positive.

Using the conventional ROC coordinates of false-positive rate on the horizontal axis and true-positive rate on the vertical axis, which point represents this threshold?

$(0.10,\ 0.65)$ , using the actual-negative and actual-positive groups as the respective denominators (correct answer)
$(0.65,\ 0.10)$ , using the actual-positive and actual-negative groups as the respective denominators
$(0.47,\ 0.65)$ , using predicted positives and actual positives as the respective denominators
$(0.10,\ 0.47)$ , using actual negatives and predicted positives as the respective denominators

Explanation: When working with ROC curves, you need to keep two rates straight: the true-positive rate (TPR) and the false-positive rate (FPR). Both are defined using actual group sizes as denominators — this is what distinguishes ROC metrics from other classification measures like precision. From the passage: 120 actual renewers, 880 actual non-renewers, 78 correctly classified renewers (true positives), and 88 non-renewers incorrectly flagged (false positives).

\text{TPR} = \frac{\text{True Positives}}{\text{Actual Positives}} = \frac{78}{120} = 0.65

\text{FPR} = \frac{\text{False Positives}}{\text{Actual Negatives}} = \frac{88}{880} = 0.10

On a standard ROC plot, FPR goes on the horizontal axis and TPR on the vertical axis, giving the point

(0.10,\ 0.65)

— confirming A is correct. B is wrong because it swaps the axes, placing TPR on the horizontal and FPR on the vertical. This reverses the conventional ROC orientation entirely. C uses

\frac{88}{78+88} = 0.56

(not 0.47, and regardless) — it substitutes predicted positives as a denominator, which would define precision-style metrics, not ROC metrics. D mixes denominators incorrectly: it uses actual negatives for one rate but predicted positives for the other, producing a hybrid that corresponds to no standard evaluation metric. A useful memory anchor: ROC rates always use actual group sizes in the denominator — actual positives for TPR, actual negatives for FPR. If you see "predicted positives" in the denominator, you've drifted into precision territory, not ROC territory.

Question 6

An analyst replaces a model's probability scores with a strictly increasing transformation of those scores. For example, every original score is transformed in a way that preserves the ordering of all customers. The analyst then continues to classify customers using the same numerical cutoff of $0.50$ .

Which conclusion is necessarily valid?

ROC AUC may decrease, but precision must increase, because the strictly increasing transformation compresses the score distribution and reduces false positives at the $0.50$ cutoff
ROC AUC increases, while precision and recall remain unchanged, because the order-preserving transformation improves the model's discrimination without altering its classifications
ROC AUC remains unchanged, and precision and recall must also remain unchanged, because preserved rankings guarantee that the same customers are classified as positive at any fixed cutoff
ROC AUC remains unchanged, but precision and recall at the fixed numerical cutoff may change because the transformation can shift which customers fall above or below $0.50$ (correct answer)

Explanation: Whenever you see a question involving score transformations and classification metrics, pause and ask yourself two separate questions: Does this transformation affect the ranking of predictions? And separately: Does it affect which specific customers fall above or below a fixed numerical threshold? ROC AUC is built entirely on rankings. It measures how well a model separates positives from negatives across all possible thresholds, so it depends only on the ordering of scores — not their exact values. A strictly increasing transformation preserves every pairwise ranking, meaning the ROC curve is identical before and after. ROC AUC is therefore unchanged. That confirms the first half of answer D. Now consider precision and recall at a fixed cutoff of

0.50

. These metrics depend on which customers are classified as positive — specifically, who lands above

0.50

. A strictly increasing transformation preserves order, but it can dramatically shift the actual score values. A customer originally scoring

0.45

could be pushed to

0.55

after transformation, suddenly crossing the threshold. This reshuffles the predicted-positive group, so precision and recall can change — sometimes substantially. That confirms the second half of D. Answer A is wrong because ROC AUC cannot decrease under an order-preserving transformation, and there's no guarantee precision increases. Answer B is wrong because while ROC AUC is correctly identified as unchanged, claiming precision and recall are also unchanged ignores how the fixed numerical cutoff interacts with shifted scores. Answer C is the most tempting trap — it correctly identifies that rankings are preserved but incorrectly concludes that a fixed numerical cutoff captures the same customers. Order preservation only guarantees consistency across rank-based thresholds, not fixed numerical ones. Your takeaway: always separate rank-based metrics (like ROC AUC) from threshold-based metrics (like precision and recall at a fixed cutoff). They respond very differently to score transformations.

Question 7

A subscription company lowers a churn-model threshold. Before the change, the model produced $48$ true positives, $12$ false positives, and $32$ false negatives. Lowering the threshold adds $40$ customers to the predicted-churn group; $12$ of those customers actually churn.

How do precision and recall change after the threshold is lowered?

Precision rises from $80\%$ to $90\%$ , while recall remains at $60\%$
Precision falls from $80\%$ to $60\%$ , while recall rises from $60\%$ to $75\%$ (correct answer)
Precision falls from $60\%$ to $48\%$ , while recall rises from $80\%$ to $90\%$
Precision rises from $60\%$ to $75\%$ , while recall falls from $80\%$ to $60\%$

Explanation: Whenever you see a threshold-adjustment question, your instinct should be to recalculate precision and recall from scratch using the updated confusion matrix values — don't try to reason about direction alone. Starting values: TP = 48, FP = 12, FN = 32. Precision =

\frac{48}{48+12} = \frac{48}{60} = 80\%

. Recall =

\frac{48}{48+32} = \frac{48}{80} = 60\%

. After lowering the threshold, 40 more customers are flagged as churners; 12 actually churn (true positives) and 28 do not (false positives). Update the matrix: TP = 48 + 12 = 60, FP = 12 + 28 = 40, FN = 32 − 12 = 20. New precision =

\frac{60}{60+40} = \frac{60}{100} = 60\%

. New recall =

\frac{60}{60+20} = \frac{60}{80} = 75\%

. Precision fell from 80% to 60%; recall rose from 60% to 75%. That's answer B. A is wrong because it claims precision rises — the opposite of what happens when you add more false positives. C swaps the direction of both metrics and uses incorrect starting values (60% precision, 80% recall), which would describe a high-recall, low-precision model before the change — not the one described. D reverses the relationship entirely, claiming precision rises and recall falls, which contradicts how lowering a threshold works. The key study tip: lowering a threshold always widens your predicted-positive net, which pulls in more true positives (boosting recall) but also more false positives (hurting precision). This trade-off is the heart of every threshold question.

Question 8

A lead-scoring model has a true-positive rate of $80\%$ and a false-positive rate of $10\%$ at its selected threshold. These rates remain stable when the model is deployed in a market where only $5\%$ of leads ultimately purchase.

Approximately what precision and recall should the sales team expect in the new market?

Precision of $80.0\%$ and recall of $10.0\%$
Precision of $44.4\%$ and recall of $50.0\%$
Precision of $29.6\%$ and recall of $80.0\%$ (correct answer)
Precision of $88.9\%$ and recall of $29.6\%$

Explanation: Whenever you see a model's true-positive rate (recall) and false-positive rate given alongside a new base rate, your job is to recalculate precision using Bayes' theorem — because precision is highly sensitive to how rare the positive class actually is. Start by imagining 1,000 leads in the new market where only 5% purchase, so 50 are true buyers and 950 are non-buyers. The model catches 80% of real buyers:

50 \times 0.80 = 40

true positives. It also flags 10% of non-buyers as positives:

950 \times 0.10 = 95

false positives. Your model therefore flags

40 + 95 = 135

leads total. Precision = true positives ÷ total predicted positives =

\frac{40}{135} \approx 29.6\%

Recall = true positives ÷ all actual positives =

\frac{40}{50} = 80\%

That confirms C as the correct answer. Choice A confuses recall with precision and swaps true-positive rate with false-positive rate — a simple label mixup. Choice B reflects neither metric correctly; it may result from incorrectly averaging rates without grounding them in the actual population counts. Choice D inverts the two metrics, swapping precision and recall while using plausible-looking numbers — a classic distractor for students who set up the problem correctly but assign values to the wrong labels. The key study tip: precision depends on base rate; recall does not. When a model moves to a low-prevalence market, expect precision to drop sharply even if recall stays constant. Always anchor your calculation to a concrete population (e.g., 1,000 leads) to avoid algebraic errors.

Question 9

A payment processor reviews $10{,}000$ transactions, of which $100$ are fraudulent. At its current threshold, the model has recall of $90\%$ and a false-positive rate of $2\%$ .

Which statement best describes the operational performance of the model?

About $90$ alerts are true frauds and $200$ are false alarms, giving recall of approximately $31.0\%$
About $90$ alerts are true frauds and $20$ are false alarms, giving precision of approximately $81.8\%$
About $100$ alerts are true frauds and $198$ are false alarms, giving precision of approximately $33.6\%$
About $90$ alerts are true frauds and $198$ are false alarms, giving precision of approximately $31.3\%$ (correct answer)

Explanation: Whenever you see a question combining recall and false-positive rate, your job is to translate those percentages into actual counts using the confusion matrix before calculating precision. Start by partitioning the 10,000 transactions: 100 are fraudulent and 9,900 are legitimate. Recall of 90% means the model catches 90% of actual frauds, so true positives = 0.90 × 100 = 90. A false-positive rate of 2% means 2% of legitimate transactions are incorrectly flagged, so false positives = 0.02 × 9,900 = 198. Total alerts = 90 + 198 = 288, and precision =

\frac{90}{90 + 198} = \frac{90}{288} \approx 31.3\%

. That matches D, the correct answer. Choice A gets the true-positive count right (90) but uses only 200 false positives instead of 198, then mislabels the resulting metric as "recall" rather than precision — a double error. Choice B dramatically understates false positives at just 20, which would require a false-positive rate of only 0.2%, not 2% — a decimal-place mistake that inflates precision to a misleadingly optimistic 81.8%. Choice C claims 100 true positives, implying perfect recall (100%), but the problem states recall is 90%, so only 90 frauds are caught, not all 100. The key study tip: always convert rates into raw counts before computing precision. Precision depends on both the true-positive pool and the false-positive pool, and a small false-positive rate across a large legitimate population (9,900 here) can produce a surprisingly large number of false alarms — exactly the real-world challenge in fraud detection.

Question 10

A supplier-risk model assigns a higher score to suppliers considered more likely to experience disruption. On an independent test set, the model's ROC AUC is $0.82$ .

Which interpretation of this result is most appropriate?

At every possible threshold, the model correctly classifies approximately $82\%$ of all suppliers
Approximately $82\%$ of suppliers predicted as risky will actually experience a disruption
A randomly selected disrupted supplier has about an $82\%$ chance of receiving a higher score than a randomly selected nondisrupted supplier (correct answer)
The threshold that maximizes accuracy will identify approximately $82\%$ of all disrupted suppliers

Explanation: Whenever you encounter an ROC AUC score, anchor your thinking to its precise probabilistic definition rather than treating it as a simple accuracy percentage — that's the trap this question is built around. ROC AUC measures rank discrimination: the probability that the model assigns a higher risk score to a randomly chosen positive case (disrupted supplier) than to a randomly chosen negative case (nondisrupted supplier). So an AUC of

0.82

means that if you randomly draw one disrupted supplier and one nondisrupted supplier, there's an

82\%

chance the model ranks the disrupted one higher. That's exactly what C states, making it the correct interpretation. A is wrong because ROC AUC says nothing about classification accuracy at every threshold — accuracy varies threshold by threshold, and no single threshold produces uniform

82\%

accuracy across all of them. This conflates AUC with a per-threshold accuracy metric. B describes precision (or positive predictive value) — the share of flagged suppliers that truly experience disruption. AUC is a threshold-free ranking metric and tells you nothing about what fraction of positive predictions are actually correct. D describes recall (sensitivity) — the fraction of truly disrupted suppliers caught at a specific threshold. Again, AUC is aggregated across all thresholds and doesn't identify what any single threshold will capture. Study tip: Memorize this one-sentence definition of AUC: "The probability that the model ranks a random positive higher than a random negative." On exams, any answer that ties AUC to a specific threshold, precision, or simple accuracy is a distractor.