Business Analytics Quiz: Data Documentation

10 questions · exam conditions

0:00

Data DocumentationQuestion 1 of 10

A support organization defines first-contact resolution rate as cases resolved without a follow-up interaction divided by resolved cases. For records created before a system migration, the follow-up indicator is null because the legacy system did not capture follow-up interactions. One analyst treats null as no follow-up; another excludes null records. Both approaches produce internally consistent but materially different rates.

Which metric-documentation revision best addresses this issue without overstating measurement certainty?

Treat every null indicator as no follow-up because no follow-up interaction appears in the current system.

Exclude all pre-migration cases and describe the remaining observations as the complete historical population.

Classify null as unknown, define its inclusion policy, and publish metric coverage alongside the resolution rate.

Replace null indicators with the monthly modal value and document the imputation as a source-system correction.

← Back to quizzes

Business Analytics Quiz

Business Analytics Quiz: Data Documentation

Practice Data Documentation in Business Analytics with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Data Documentation, giving you a quick way to practice the rules, question types, and explanations that matter most for Business Analytics.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

Which metric-documentation revision best addresses this issue without overstating measurement certainty?

Treat every null indicator as no follow-up because no follow-up interaction appears in the current system.
Exclude all pre-migration cases and describe the remaining observations as the complete historical population.
Classify null as unknown, define its inclusion policy, and publish metric coverage alongside the resolution rate. (correct answer)
Replace null indicators with the monthly modal value and document the imputation as a source-system correction.

Explanation: When you encounter metric-definition questions involving missing data, ask yourself: does the proposed solution acknowledge uncertainty honestly, or does it hide uncertainty behind a convenient assumption? That distinction is at the heart of this question. The fundamental problem here is that null doesn't mean "no follow-up occurred" — it means the legacy system never recorded whether a follow-up occurred. Those are very different things. The best documentation practice treats them differently, too. Option C does exactly this: it labels nulls as unknown rather than forcing them into a category, explicitly states how those records will be handled, and publishes a coverage statistic so stakeholders understand what share of history the metric actually reflects. This approach is transparent, reproducible, and doesn't overstate what the data can confirm. Option A commits the most common trap — assuming absence of evidence is evidence of absence. Because no follow-up appears in the system, the analyst concludes none happened, silently inflating the resolution rate for pre-migration cases. Option B avoids the null problem by discarding pre-migration records entirely, but then mislabels the truncated dataset as the "complete historical population" — that's a factually false claim that could mislead decision-makers reviewing long-term trends. Option D imputes missing values using the modal value and calls it a "source-system correction," but imputation is an analytical choice, not a correction of a known error; presenting it otherwise overstates certainty and obscures that the values were estimated, not observed. Study tip: On data-governance and analytics questions, watch for answers that resolve ambiguity by relabeling it as certainty. The stronger answer almost always names the uncertainty explicitly and documents it — that's the professional standard.

Question 2

A subscription business calculates average revenue per customer by joining invoice records to a customer master. Billing accounts can map to multiple customer profiles after mergers, and historical mappings are corrected retroactively. The metric catalog lists the source systems and formula but does not explain the join. A recent mapping correction causes revenue to be duplicated for some customers.

Which addition to the documentation would most directly prevent or detect this failure?

Document join keys, expected cardinality, effective-dated mapping logic, and reconciliation checks against invoice totals. (correct answer)
Document the database names, invoice refresh frequency, customer profile owner, and dashboard publication schedule along with the data-retention policy.
Document the average-revenue formula, currency display format, rounding precision, preferred dashboard filter order, and column display names.
Document the profile attributes used for segmentation, the date each attribute was last verified, and exclude customers whose profiles changed after invoicing.

Explanation: When a metric fails due to a data-join issue, the root cause almost always traces back to missing data lineage documentation — specifically, how records are linked, at what granularity, and how edge cases (like retroactive corrections) are handled. That's the concept being tested here: what documentation actually prevents or detects calculation errors caused by join logic and mapping changes? Answer A directly addresses every layer of the failure. Documenting join keys catches ambiguous or incorrect linkages. Specifying expected cardinality (e.g., one invoice maps to exactly one customer) makes duplication detectable the moment a many-to-one mapping breaks that rule. Effective-dated mapping logic ensures historical corrections are applied correctly rather than retroactively inflating counts. Reconciliation checks against invoice totals create an automatic alarm when revenue figures drift from the source. Each element maps precisely to the described failure mode. Answer B documents operational metadata — database names, refresh schedules, retention policies — which is useful for infrastructure management but does nothing to catch duplication caused by a flawed join or mapping correction. Answer C focuses entirely on presentation and formatting details (rounding, display names, filter order), which are cosmetic concerns with no bearing on calculation integrity. Answer D addresses segmentation and profile verification, which might improve data quality in some contexts, but excluding changed profiles doesn't fix the join logic or detect duplication already embedded in the metric. The strategic takeaway: when a question describes a calculation failure tied to joins or mappings, look for documentation that addresses join structure, cardinality constraints, and reconciliation — not display settings, ownership details, or segmentation rules. Those distractors sound technical but solve different problems entirely.

Question 3

A global marketplace reports daily completed orders. The metric is documented as orders completed on the order date. Order records contain a customer-local creation timestamp, a warehouse completion timestamp in UTC, and an ingestion timestamp. Late-arriving records may be loaded up to two days after completion. Regional teams disagree about which day receives an order completed near midnight.

Which documentation addition is sufficient to make the daily metric reproducible across regions and reruns?

Name the warehouse completion field and require analysts to use the date displayed by their local reporting tools.
Specify the completion timestamp, business time zone, day-boundary rule, and policy for late-arriving records. (correct answer)
Use the ingestion timestamp as the order date and freeze each daily result immediately after its first publication.
Use the customer-local creation date for regional reports and the UTC completion date for company-wide reports.

Explanation: Whenever you see a question about metric reproducibility, think about all the decisions that must be identical across every analyst and every rerun — not just the data source, but how that data is interpreted. A metric is truly reproducible only when there is zero ambiguity at each decision point. Option B is correct because it closes every gap that causes disagreement. Specifying the completion timestamp field tells everyone which column to use. Specifying the business time zone means a midnight order lands on the same calendar day whether the analyst sits in Tokyo or Toronto. A day-boundary rule (e.g., "a day runs from 00:00 to 23:59:59 in the business time zone") resolves exactly the midnight dispute the passage describes. Finally, a late-arrival policy (e.g., "records arriving up to two days late are backdated to their completion date; the window closes after 48 hours") ensures reruns produce consistent historical numbers. Together, these four elements eliminate every documented source of disagreement. Option A fails because it defers to each analyst's local reporting tool, which is the exact source of the regional conflict — different tools may render the same UTC timestamp as different local dates. Option C is wrong because ingestion timestamps reflect when data was loaded, not when orders were completed, so the metric no longer measures what it claims. Freezing results also prevents correcting legitimate late-arriving records. Option D worsens the situation by intentionally creating two incompatible versions of the same metric, making cross-regional and company-wide comparisons unreliable by design. Your study tip: when evaluating metric definitions, run through the checklist — which field, which time zone, which boundary, which correction policy — and flag any answer that leaves even one of those undefined.

Question 4

A software company defines monthly active accounts as distinct customer accounts with at least one qualifying product event during the calendar month. A dashboard also breaks the metric down by industry. An account's industry classification can change during the month, and the event log stores the industry value that was current when each event occurred. As a result, an account can appear in more than one industry while appearing only once in the company-wide total.

Which documentation change would best make the industry-level results additive and reproducible?

Require each qualifying event to use its event-time industry and sum the distinct-account results across industries.
Count all qualifying events within each industry and divide each result by the account's number of monthly events.
Assign each account to one documented industry snapshot, such as its classification at month end, before aggregation. (correct answer)
Use the most frequent industry found among an account's events and retain every tied industry classification.

Explanation: Whenever you see a metric decomposition question — where a company-wide total is broken into sub-groups — ask yourself: can the sub-group numbers be summed back to the total without double-counting? That property is called additivity, and it requires each unit of analysis (here, each account) to belong to exactly one group at aggregation time. The core problem in this passage is that an account can carry different industry labels across its events within a single month. This means one account might be counted in both "Healthcare" and "Finance," inflating industry totals beyond the company-wide figure. Option C solves this cleanly: by assigning every account to exactly one industry snapshot — say, its month-end classification — before aggregation, each account appears in precisely one industry bucket. The industry counts then sum to the company-wide total, making the metric both additive and reproducible (anyone running the same query gets the same result). Option A is tempting but wrong: summing distinct-account counts across industries still allows one account to be counted in multiple industries, so the totals won't add up correctly. Option B introduces an averaging mechanism (dividing by event count) that has no logical connection to a distinct-account metric — it conflates event volume with account presence and produces uninterpretable numbers. Option D compounds the problem: retaining tied industries means a single account can still appear in multiple industry rows, preserving the very double-counting issue you're trying to eliminate. The study tip here: when a metric requires distinct counts broken out by a dimension that can change, always pin each entity to one value of that dimension before aggregating — this is the standard fix for non-additive dimensional breakdowns.

Question 5

A delivery company uses a prescriptive model to recommend driver schedules. Management wants to report projected weekly savings from the recommendations. The draft metric definition states only: projected savings equals current cost minus optimized cost. The optimizer may reduce overtime by leaving some low-priority deliveries unassigned, even though operations requires at least $98\%$ of eligible deliveries to be completed within the promised window.

Which revision would make projected savings a decision-relevant and auditable metric?

Compare optimized cost with the prior week's actual cost, regardless of changes in delivery volume or service requirements.
Define a volume-matched baseline, optimization horizon, cost components, and required service constraints for both scenarios. (correct answer)
Compare optimized cost with the unconstrained minimum cost and describe unassigned deliveries in a separate operational report.
Define savings as overtime reduction alone and hold regular labor cost constant across every scheduling scenario.

Explanation: When evaluating a metric definition in prescriptive analytics, ask yourself three questions: Does the baseline fairly represent "what we're replacing"? Are all cost components explicitly defined? And do the constraints match real operational requirements? A metric that fails any of these tests can mislead decision-makers or be impossible to audit later. B is correct because it addresses all three dimensions at once. A volume-matched baseline ensures you're comparing equivalent workloads — not last week's lighter Tuesday against this week's peak Friday. Specifying the optimization horizon prevents cherry-picking favorable time windows. Defining cost components makes the arithmetic reproducible by any auditor. Crucially, requiring that service constraints appear in both scenarios means the 98% delivery-completion rule applies to the optimized schedule too, so savings can't be manufactured simply by abandoning low-priority deliveries. A introduces a confound by ignoring volume and service-requirement changes. If delivery volume drops 20%, costs fall naturally — that's not optimizer savings. Comparing across unequal conditions produces a meaningless number. C is tempting but problematic: comparing against an unconstrained minimum treats an operationally illegal solution as the benchmark. Real savings must be measured against a feasible baseline, not a theoretical floor that violates the 98% service requirement. Relegating unassigned deliveries to a separate report conceals the true trade-off. D narrows savings to overtime alone and freezes other labor costs artificially. This ignores routing costs, fuel, and variable labor — omitting components that the optimizer directly affects. As a study tip: whenever a metric question involves optimization, always check whether constraints are consistently applied to both the baseline and the optimized scenario. Missing constraints are the most common source of inflated savings figures.

Question 6

A retailer defines the $90$ -day repeat-purchase rate as customers making another purchase within $90$ days divided by newly acquired customers. A June 30 report includes customers acquired during May and June. Analysts disagree whether customers who have not yet had a full $90$ -day opportunity should remain in the denominator as non-repeaters. They also disagree whether a purchase exactly $90$ days after acquisition qualifies.

Which metric definition would provide the most comparable cohort measure?

Include every customer acquired by the report date, classify immature customers as non-repeaters, and include purchases on day $90$ .
Include every customer with any observed follow-up, weight customers by observed days, and exclude purchases on day $90$ .
Include only customers with a fully elapsed follow-up window and explicitly define whether the day- $90$ boundary is inclusive. (correct answer)
Include only customers who made a repeat purchase or reached day $90$ , and omit the precise boundary convention.

Explanation: When measuring repeat-purchase rates across cohorts, comparability depends on two things: every customer in the denominator having had an equal opportunity to repeat, and the metric boundary being unambiguous. If some customers haven't yet reached day

90

, their presence as "non-repeaters" artificially deflates the rate — not because they won't buy again, but because they simply haven't had the chance yet. C is correct because it solves both problems simultaneously. By restricting the denominator to customers whose full

90

-day window has elapsed, every person in the calculation faced the same observation period. Equally important, it requires an explicit statement about whether day

90

itself counts — a small but meaningful detail that, if left undefined, makes comparisons across reports inconsistent. A is wrong because classifying immature customers as non-repeaters contaminates the denominator. A customer acquired June 1st has had

29

days by June 30th — calling them a non-repeater isn't a measurement, it's a premature label that biases the rate downward. B is wrong on two fronts: weighting by observed days introduces complexity that reduces interpretability and comparability, and arbitrarily excluding day-

90

purchases without a principled reason creates a definition gap rather than resolving one. D is wrong because omitting the boundary convention — exactly the second disagreement the analysts raised — leaves the metric partially undefined. A cohort measure that ignores edge-case rules will produce subtly different numbers across reporting periods. Your study tip: whenever a question involves a rate metric with a time window, always ask "Did every unit in the denominator have the same exposure time?" That single question catches most cohort comparability errors.

Question 7

A retailer defines monthly net merchandise revenue as merchandise revenue recognized when an order ships, after discounts and refunds, but excluding tax and shipping fees. In January, shipped merchandise after discounts totaled $\$500{,}000$ . This amount included $\$20{,}000$ associated with orders refunded in February. January also contained $\$15{,}000$ of refunds for orders shipped in December. One analyst reports January revenue of $\$480{,}000$ , while another reports $\$485{,}000$ .

Which addition to the metric documentation would most directly resolve the discrepancy?

Specify whether merchandise discounts are recorded at checkout or allocated proportionally when each order ships.
Specify whether refunds are attributed to the original shipment month or to the month in which each refund is processed. (correct answer)
Specify whether tax and shipping fees are excluded before or after revenue is aggregated across individual orders.
Specify whether shipped revenue is rounded at the order level or only after the monthly total is calculated.

Explanation: When a metric definition contains a timing ambiguity, small differences in interpretation can produce different but equally "defensible" numbers — and that's exactly what's happening here. Your job is to identify which ambiguity actually generates the observed gap. The two analysts agree on shipped merchandise after discounts:

\$500{,}000

. The disagreement is about refunds. January contains two refund-related figures:

\$20{,}000

in future refunds (orders shipped in January, refunded in February) and

\$15{,}000

in past refunds (orders shipped in December, refunded in January). If one analyst deducts only the December refunds processed in January —

\$500{,}000 - \$15{,}000 = \$485{,}000

— and the other deducts only the January-shipped orders that will be refunded —

\$500{,}000 - \$20{,}000 = \$480{,}000

— you get exactly the two reported figures. The definition never says whether refunds follow the shipment month or the processing month, which is precisely what B would resolve. A is wrong because the passage already states revenue is recognized when an order ships after discounts — there's no ambiguity about discount timing that could produce a

\$5{,}000

discrepancy. C is wrong because tax and shipping exclusions are not in dispute; both analysts presumably exclude them the same way. D is wrong because rounding at the order versus monthly level would create tiny, fractional differences — not a clean

\$5{,}000

gap. When you see a discrepancy question, work backward from the dollar difference to identify which ambiguity in the definition mathematically produces that exact gap. That's your answer.

Question 8

A lender publishes a monthly default rate by model risk band. Customer risk scores are recalculated daily and overwritten in the operational database. A default is defined as becoming at least $90$ days past due within the next $90$ days. When analysts rerun the report several months later, customers move between historical risk bands and the newest month shows an unusually low default rate.

Which documentation requirement is most critical for producing valid and reproducible model-monitoring metrics?

Preserve the score as of a specified prediction timestamp and report outcomes only after the full forward window has matured. (correct answer)
Use each customer's latest available score and annualize defaults observed during any partial forward outcome window.
Preserve the score from the date of default and classify nondefaulting customers using their latest available score.
Use the monthly average score for each customer and count defaults recorded before or after the scoring month.

Explanation: When you see a question about model monitoring and reproducibility, the core concept being tested is point-in-time data integrity — the idea that a valid prediction-outcome pair must lock in both the input (the score) and the outcome window at fixed, well-defined moments in time. Answer A is correct because it captures exactly this discipline. To produce a valid default rate for a given month, you must freeze the risk score at a specific prediction timestamp (say, the first of that month) and then wait until the full

90

-day outcome window has elapsed before reporting results. This prevents two forms of contamination: score drift (customers' scores change daily and overwrite the record) and immature outcome windows (defaults that haven't yet had time to materialize are miscounted as non-defaults). Without this discipline, neither the numerator nor the denominator of your default rate is stable, making the metric unreproducible. Answer B is wrong because using the latest available score retroactively reassigns customers to risk bands they didn't occupy at prediction time — this is the exact data-leakage problem causing customers to "move between historical risk bands" in the passage. Annualizing from a partial window compounds the error by extrapolating from incomplete outcome data. Answer C introduces a particularly dangerous asymmetry: defaulters are scored at default, while non-defaulters are scored at the latest date. The two groups are measured at completely different points in time, making any comparison between them meaningless. Answer D's monthly average score blurs the temporal anchor entirely, and counting defaults outside the defined outcome window violates the

90

-day definition. Study tip: On model-monitoring questions, always ask yourself: Are the score and the outcome window both frozen at consistent, defined points in time? If either floats, the metric is invalid.

Question 9

Beginning April 1, a media company changes its engaged-session KPI to exclude sessions generated by known bots. Executives want current dashboards, prior board reports, and analyst queries to remain reconcilable. Some users expect all history to be recomputed under the new rule, while others expect historical reports to preserve the definition used when they were issued.

Which documentation and governance approach best supports both expectations?

Replace the old formula in the shared query and allow each report owner to decide whether historical differences are material.
Keep the metric name unchanged, add a footnote about bots, and recompute only months with unusually high bot traffic.
Maintain the old formula indefinitely for published reports and use the new formula only in undocumented ad hoc analyses.
Version the metric with an effective date and state whether each dataset is restated or preserves its as-published definition. (correct answer)

Explanation: When a KPI definition changes mid-stream, your core governance challenge is reconcilability: different audiences need the same number to mean the same thing across time. The key framework here is metric versioning — the practice of treating a business definition as a versioned artifact with an effective date, just like software code. Answer D is correct because it directly solves the dual requirement. By versioning the metric (e.g., "Engaged Sessions v1" vs. "Engaged Sessions v2, effective April 1") and explicitly labeling each dataset as either restated or as-published, both audiences get what they need. Analysts who want the new rule applied historically can use restated datasets; executives reconciling prior board reports can reference the as-published definition. Nothing is ambiguous or hidden. Answer A fails because silently replacing the formula in a shared query creates a single, unlabeled break in the data. Report owners are left guessing whether discrepancies are definitional or represent real business changes — exactly the reconcilability problem you're trying to avoid. Answer B is a partial fix at best. Selective recomputation based on "unusually high bot traffic" introduces a subjective, inconsistent threshold. A footnote doesn't constitute governance; it just shifts the interpretive burden to the reader. Answer C is the worst outcome: it creates a shadow metric system where undocumented ad hoc analyses use a different definition than published reports. This destroys auditability and violates basic data governance principles. Study tip: On business analytics questions involving KPI changes, always look for the answer that separates versions with explicit labels rather than one that patches or hides the change. Governance questions reward precision and traceability.

Question 10

An online retailer randomizes eligible users into a new-checkout treatment or control group when they first load the checkout page. Its experiment report defines conversion as purchasers divided by visitors. The event repository contains visits that occurred before assignment, repeat visits after assignment, and visits by users who were never eligible for randomization. Analysts agree on the purchaser count but produce different conversion rates.

Which metric specification is most appropriate for the experiment's primary conversion measure?

Use all site visitors during the experiment as the denominator and count any purchase occurring before the report is generated.
Use post-assignment checkout visits as the denominator and count each repeat visit as another conversion opportunity.
Use users who loaded checkout at any time as the denominator and attribute them by their most recent interface exposure.
Use randomized eligible users as the denominator and define a fixed post-assignment purchase window for each user. (correct answer)

Explanation: When designing an A/B test metric, your goal is clean causal attribution: the denominator must contain only units that were actually randomized, and the measurement window must be fixed and identical for everyone. Any contamination — from non-eligible users, pre-assignment behavior, or inconsistent timing — breaks the comparison between treatment and control. D is correct because it satisfies both requirements. Using randomized eligible users as the denominator ensures every person in the calculation was actually exposed to either the new or old checkout. Defining a fixed post-assignment purchase window (e.g., 7 days from first assignment) means each user gets the same observation period, making treatment and control groups directly comparable. This isolates the checkout interface as the only systematic difference driving any observed conversion gap. A fails because "all site visitors" includes people who were never eligible for randomization — browsers, logged-out users, or users on ineligible devices. These dilute and distort the true treatment effect since they were never exposed to either condition. B is flawed because counting repeat visits as independent conversion opportunities inflates the denominator artificially and introduces dependency: a single user who visits ten times is treated as ten separate trials, violating the assumption of independent observations and biasing the rate. C misattributes conversions using "most recent exposure," which is problematic if users encounter the interface multiple times. This destroys the integrity of the original random assignment — a user initially placed in control could be reclassified as treatment, corrupting the experimental groups. As a study rule: whenever a question involves experiment design, immediately ask yourself — who was randomized, when, and what window follows? Those three constraints define a valid primary metric.