Data Quality Assessment - Business Analytics

1. After adjusting for the identified data-quality problem, what is the best estimate of design B's relative conversion lift over design A?
   Answer: Approximately $16.7\%$, because dividing B's recorded purchases by the capture rate gives about $933$ purchases and a corrected conversion rate of about $9.33\%$. When data is missing systematically — not randomly — you must correct for undercounting before computing any metrics. The key principle: if a capture rate tells you only a fraction of true events were recorded, you recover the true count by dividing the observed count by that fraction, not by multiplying by the complement. For Design B, the audit confirms only 90% of purchases were captured. To recover the true purchase count, divide: $840 \div 0.90 \approx 933$. Design B's corrected conversion rate is then $933 / 10{,}000 \approx 9.33\%$. Design A's rate remains $800 / 10{,}000 = 8.00\%$ (fully captured). The relative lift is $(9.33\% - 8.00\%) / 8.00\% \approx 16.7\%$, confirming D is correct. Choice A makes no correction at all, treating flawed observed data as ground truth — this understates B's true performance and is exactly the error the question warns you about. Choice B adds 10 percentage points directly to the rate difference, which confuses a multiplicative capture-rate correction with an additive adjustment; the correction doesn't work that way. Choice C multiplies $840 \times 1.10 = 924$, which is the classic inversion mistake — multiplying by $(1 + \text{missing rate})$ instead of dividing by the capture rate. Multiplying by 1.10 only recovers 10% of the observed count, not 10% of the true count. Study tip: Whenever you see a partial capture rate, always divide the observed value by the capture rate to recover the true value. Multiplying by $(1 + \text{missing fraction})$ is a tempting but incorrect shortcut that systematically underestimates the true count.
2. Which assessment best determines whether the missing income values could bias the churn model?
   Answer: Compare income missingness across enrollment channels and churn outcomes, then test models with a missingness indicator. Whenever you see a question involving missing data, your first instinct should be to ask why the data is missing — not just how much is missing. This distinction is the heart of Missing Not at Random (MNAR) vs. Missing Completely at Random (MCAR) analysis, and it's critical for avoiding biased models. In this scenario, income missingness is heavily concentrated among store-enrolled customers ($18\%$) versus online customers ($<1\%$), and store-enrolled customers also have higher churn rates. This means missingness is systematically linked to both a predictor (enrollment channel) and the outcome (churn) — a textbook bias risk. Option B is correct because it directly investigates this pattern: comparing missingness across channels and churn outcomes reveals whether the missing data is informative, and adding a missingness indicator variable lets the model capture that signal rather than ignore it. Option A fails because a low overall rate ($3\%$) masks the dangerous $18\%$ rate within a high-churn subgroup. Thresholds on aggregate missingness are blind to subgroup patterns. Option C — replacing missing values with the overall median — imposes a false uniformity that erases the very signal you're trying to detect; it also doesn't address bias, it just preserves sample size. Option D evaluates income's predictive power among observed records only, which tells you nothing about whether the missing records are systematically different from those with data present. The key study tip: always disaggregate missing data by relevant subgroups before deciding how to handle it. A low overall missing rate can hide serious bias when missingness clusters in a high-risk segment.
3. Which conclusion is most appropriate?
   Answer: The records may contain multivariable inconsistencies that require a cross-field reconciliation rule. When validating data quality, checking each field in isolation is only the first layer of scrutiny. A deeper and often more revealing check is cross-field validation — verifying that related fields are mutually consistent with one another. This question tests exactly that concept. Here, the expected relationship is straightforward: $\text{Line-Item Revenue} = (\text{Unit Price} \times \text{Quantity}) \pm \text{Discounts/Taxes}$ Even though unit price and quantity each pass their individual range checks, the computed revenue doesn't match the reported revenue. That mismatch signals a multivariable inconsistency — the fields are internally valid but externally incoherent as a group. Answer B is correct because it correctly identifies this as a cross-field problem requiring a reconciliation rule (i.e., enforcing the formula above). A is the most tempting trap. It's true that each field passes individual screening, but that conclusion is incomplete — passing univariate checks does not guarantee multivariate consistency. Data quality requires both. C is a misclassification: duplicates refer to repeated records, not to multiple fields describing one transaction. Nothing in the passage suggests records are repeated. D invents a sampling bias narrative unsupported by the passage — there's no evidence that high-revenue orders are selectively over-represented or more prone to formula violations. A useful rule of thumb: whenever a dataset contains fields that are mathematically related by definition (price × quantity = revenue, hours × rate = payroll, etc.), always build explicit cross-field validation rules. Individual field checks alone will miss formula-level corruption.
4. Which quality check would most directly reveal the duplication affecting the revenue analysis?
   Answer: Check whether order IDs occur more than once after the join and compare revenue before and after joining. When a join multiplies rows due to a one-to-many relationship, the danger isn't duplicate complete rows — it's duplicate key values inflating your metrics. Here, each order legitimately appears multiple times (once per address), so the rows look unique on the surface, yet revenue gets counted repeatedly. The right diagnostic targets that exact mechanism. Answer A is correct because checking whether order IDs repeat after the join directly exposes the fan-out problem. An order ID should appear exactly once in a clean dataset; if it appears three times, that order's revenue is triple-counted. Comparing total revenue before and after the join then quantifies the distortion, giving you both detection and impact assessment in one step. Answer B fails because it checks for exact row duplicates — but as the passage explicitly states, no rows are exact duplicates (the address fields differ). This check would return zero duplicates and give you false confidence that nothing is wrong. Answer C is tempting but answers a different question. Customers having multiple orders is normal and expected; filtering to only the latest order would actually delete legitimate data and destroy historical revenue analysis rather than fix the join problem. Answer D addresses data completeness — missing addresses — which is a real data quality concern but completely unrelated to the fan-out duplication causing revenue inflation. Imputing missing values wouldn't reduce the row count at all. Strategy tip: Whenever a join involves a one-to-many relationship, always audit your join key (here, order ID) for unexpected repetition — not complete-row duplicates. Inflated aggregates after a join are almost always caused by multiplied keys, not identical rows.
5. Which statement best assesses this data-preparation procedure?
   Answer: It introduces leakage because validation-set values influence the imputation rule; the median should be learned from training data only. Whenever you see a question about model validation, your first instinct should be to ask: did any information from the validation set leak into the training process? This is the core concept of data leakage, and it's one of the most common — and costly — mistakes in predictive modeling. Here, the analyst computes the median account balance using the entire dataset before the train/validation split. That means the median already "knows" about the validation customers' balances. When the model is later evaluated on the validation set, those observations were partially used to construct the very imputation rule the model depends on. This is classic preprocessing leakage: the validation set silently influenced a preparation step, causing the reported validation performance to be overly optimistic. Answer A correctly identifies this — the median should be fit on training data only, then applied to validation data. Answer B is wrong because it confuses preprocessing leakage with target leakage. Target leakage means the outcome variable (default status) is used as a predictor. Median imputation uses account balances, not default outcomes — so B misidentifies both the type and the mechanism of the problem. Answer C is tempting but false. The fact that a median is a "descriptive statistic" doesn't make it harmless. Any statistic computed on pooled data carries information from both splits and can inflate performance estimates. Answer D arbitrarily ties the issue to missingness rate. The leakage problem exists regardless of how many values are missing — even one imputed validation observation is enough. Study tip: Always fit your preprocessing steps (imputation, scaling, encoding) on training data alone, then transform validation and test data using those fitted parameters. If you compute any statistic on the full dataset first, flag it as potential leakage.
6. Which next step provides the strongest assessment of possible nonresponse bias in the reported score?
   Answer: Compare respondents and nonrespondents on known customer attributes and cancellation reasons, then perform weighting sensitivity analyses. Whenever you see a question about survey data and potential bias, your first instinct should be to ask: who didn't respond, and how might they differ from those who did? Nonresponse bias occurs when the people who skip a survey have systematically different characteristics — and thus different satisfaction levels — than those who complete it. A single reported mean like $7.1$ can be deeply misleading if certain groups are dramatically underrepresented. B is correct because it directly attacks the core problem: the dramatically unequal response rates ($40\%$ vs. $12\%$) mean service-problem customers are badly underrepresented in the $7.1$ figure. By comparing respondents to nonrespondents on observable attributes — cancellation reason, tenure, spending — you can detect where the gaps are. Weighting sensitivity analyses then let you model what the score might look like if nonrespondents had responded, giving you a range of plausible true values. This is the gold-standard approach to diagnosing and quantifying nonresponse bias. A is a trap because sending more invitations doesn't fix differential response rates. If service-problem customers systematically ignore surveys, more invitations still produce a skewed sample — you're just scaling up the same bias. C artificially balances group sizes by discarding valid data, which destroys representativeness rather than restoring it. Deliberately removing respondents is not a bias-correction method; it's data manipulation. D calculates precision (how tightly the sample estimates the mean), not accuracy (whether the mean is measuring the right population). A narrow confidence interval around a biased estimate is still a biased estimate. Your study tip: distinguish precision from accuracy. More data or tighter intervals don't fix bias — only understanding and correcting for who is missing from your sample does.
7. What range contains the true average handling time for all $100$ calls?
   Answer: From $8.6$ to $9.6$ minutes, using the minimum and maximum possible totals for the missing calls combined with the observed total. When data is partially missing but bounded, your job is to compute the worst-case and best-case scenarios to define a valid range for the true average — a core technique in sensitivity analysis. Here's the logic for the correct answer, C. You have $95$ observed calls totaling $760$ minutes. The five missing calls each lasted between $20$ and $40$ minutes. The minimum possible total for the missing calls is $5 \times 20 = 100$ minutes; the maximum is $5 \times 40 = 200$ minutes. Combined with the observed total, the overall handling time falls between $760 + 100 = 860$ and $760 + 200 = 960$ minutes. Dividing by all $100$ calls gives an average ranging from $\frac{860}{100} = 8.6$ to $\frac{960}{100} = 9.6$ minutes — exactly what C states. A is wrong because you cannot simply exclude the missing calls. Doing so gives $\frac{760}{95} = 8.0$ minutes, but this ignores real call time that occurred, producing a biased and incomplete average. B makes two compounding errors: assigning the midpoint ($30$ minutes) is an assumption, not a bound, and adding a standard error conflates statistical sampling uncertainty with bounded missing-data analysis — these are different problems. D is an unsupported assumption. Nothing in the passage says the missing calls were the longest; you may only use what the operational log confirms ($20$–$40$ minutes). Study tip: Whenever a question gives you a known range for missing values, immediately calculate the minimum-total and maximum-total scenarios. That's your analytical boundary — don't average, don't assume, don't exclude.
8. Which metric comparison is most directly biased by the duplicate events?
   Answer: Treatment-versus-control revenue per member, because duplicated amounts inflate treatment revenue more strongly. When analyzing data quality issues in A/B tests, your first instinct should be to ask: which metric is most mechanically affected by the specific defect described? Here, duplicate purchase events inflate the dollar amounts recorded in the event stream — and they do so asymmetrically (3% duplication in treatment vs. 1% in control). Revenue per member is calculated as total recorded revenue divided by member count. Since duplicated events copy entire purchase amounts, treatment revenue is overstated by roughly 3% and control by 1%, creating an artificial gap of approximately 2 percentage points in the revenue ratio. This is a direct, quantifiable inflation of the metric you're comparing — making A the correct answer. B is the most tempting trap. Duplicates copy existing purchase events from members who already converted, so no new member crosses the conversion threshold. Conversion is defined as having at least one purchase — a duplicate of an existing purchase doesn't change a member's converted status from 0 to 1. The conversion rate is essentially unaffected. C is incorrect because the passage explicitly states that member IDs and assignment records are unaffected. Duplicate purchase events do not create duplicate members, so member counts remain clean. D is incorrect for the same reason — eligibility and assignment records are untouched. Duplicate events live in the purchase log, not the enrollment or qualification logic. Study tip: When a data defect is described, trace exactly which tables or fields it contaminates, then ask which metric formula uses those fields directly. Defects only bias metrics that incorporate the corrupted data in their numerator or denominator.
9. What is the most appropriate initial treatment of these observations?
   Answer: Reconcile flagged transactions with source records and merchant context, then analyze valid extremes separately from errors. Whenever you encounter a question about outlier treatment, the core decision hinges on why the values are extreme — not just that they are extreme. The IQR rule flags observations for investigation, not automatic removal. Your goal is to distinguish legitimate extreme values from data errors before choosing any treatment. Here, the flagged transactions have two plausible origins: valid high-value luxury purchases and decimal-placement errors. These require completely different handling — one should be kept, the other corrected or removed. Option C is correct because it prescribes exactly this diagnostic step: reconcile each flagged record against source documentation and merchant context, then treat confirmed valid extremes separately from confirmed errors. This preserves data integrity for both downstream uses — revenue forecasting needs accurate large transactions included, while fraud detection needs clean, error-free inputs. Option A is wrong because the IQR rule is a detection heuristic, not a validity test. Winsorizing all flagged values would distort legitimate luxury sales, systematically biasing revenue forecasts downward. Option B is wrong on two counts: $0.2\%$ of transactions in a high-volume payment processor can represent enormous dollar amounts, and "small fraction" does not mean "small impact." Deleting without investigation also destroys potentially useful fraud signals. Option D is wrong because the passage explicitly acknowledges that some flagged values may be errors — assuming luxury transactions explain everything ignores that risk entirely, which would corrupt the fraud detection model. Your strategy tip: on outlier questions, always ask "Do we know these are errors, or do we only suspect it?" If the answer is "suspect," investigation before treatment is almost always the right first step.
10. Which assessment best addresses the resulting data-quality and bias concern?
    Answer: Recognize selective labels and evaluate with external outcomes, audited cases, or a carefully controlled approval sample. When a model is trained only on individuals who were approved under a previous policy, you're dealing with selective labels — a classic sampling bias in predictive modeling. The key insight is that the historical data doesn't represent the full applicant population; it reflects the outcomes of a filtered subset, shaped by the very variables your new model will use. This creates a feedback loop where bias from past decisions gets baked into future ones. Option D is correct because it acknowledges the root problem honestly. Rather than patching over incomplete data, it calls for evaluation strategies — external outcome data, audited edge cases, or a controlled random-approval sample — that can reveal how the model performs on applicants it was never trained to see. This is the only approach that doesn't assume the biased sample is representative. Option A is dangerously wrong: rejected applicants have no observed outcome, not a confirmed non-default. Labeling them as non-defaulters introduces systematic false negatives and distorts your model's calibration entirely. Option B commits a generalization fallacy. Approved borrowers are a selected group — typically lower-risk — so performance metrics on them won't transfer to the broader population. Deleting the rejected group doesn't make the bias disappear; it just hides it. Option C sounds methodologically sophisticated, but it's misleading. Imputing outcomes from similar approved applicants still inherits the same selection bias — you're extrapolating from the biased group into itself. Treating imputed values as known labels overstates your confidence in fabricated data. Study tip: On bias questions, watch for answers that "solve" missing data by filling it in without addressing why it's missing. The source of missingness matters as much as the missingness itself.

Start this quiz