- Which conclusion is best supported by these results?
Answer: The results are consistent with a positive app effect, but preexisting customer engagement confounds a causal estimate based on adoption.
Whenever you see a question comparing groups before and after an intervention, you're being tested on causal inference — specifically, whether a difference in outcomes truly reflects the intervention's effect or simply reflects pre-existing differences between groups.
Here's the key insight: future app adopters were already higher spenders before the app launched ($215 vs. $145, a $70 gap). After launch, adopters spent $240 vs. $150 for nonadopters — a $90 gap. The pre-existing $70 gap tells you these groups weren't comparable to begin with. More engaged customers self-selected into using the app, which is called confounding by preexisting engagement. Answer C correctly identifies that the data is consistent with a positive app effect, but you cannot isolate a clean causal estimate because of this selection bias.
Answer A is wrong because it treats the entire $90 post-launch difference as the app's causal effect, ignoring that $70 of that gap existed before the app was even available. Answer B is wrong in a different way — it looks only at adopters' before-and-after change ($215 to $240 = $25) without accounting for what nonadopters' spending did over the same period, so it omits a necessary control comparison. Answer D overcorrects: just because most of the gap was pre-existing doesn't mean the app had zero effect — the remaining $20 widening of the gap may reflect a real, if modest, impact.
Your takeaway: when evaluating any intervention, always ask "were these groups comparable before the treatment?" If not, pre-existing differences confound your causal claim — even if a real effect also exists. - Under the parallel-trends assumption, which estimate and interpretation are most defensible?
Answer: The campaign effect is approximately 6 percentage points because the nonadvertised region estimates the concurrent holiday-related change.
Whenever you see a question combining a treatment group, a control group, and a concurrent external event, you're in difference-in-differences (DiD) territory. The core idea: if both regions were trending similarly before the campaign, any sales movement in the nonadvertised region during the campaign period must reflect forces other than advertising — in this case, the nationwide holiday promotion.
The math is straightforward. The advertised region grew 18%; the nonadvertised region grew 12%. The holiday-driven baseline lift is captured by the control region's 12%. Subtracting gives you the incremental effect attributable to the campaign: 18%−12%=6 percentage points. That's answer B, and it's defensible precisely because the parallel-trends assumption holds — both regions moved together before the campaign, so the control region is a credible counterfactual.
A is wrong because it ignores the counterfactual entirely. Taking the full 18% as the campaign effect assumes sales would have been flat without advertising, which ignores the holiday lift affecting both regions.
C gets the logic backwards. The 12%$figurerepresentsthe∗confounding∗holidayeffect,notanisolatedadvertisingeffect.Attributing12\%$$ to the campaign would actually measure what advertising didn't cause.
D overcorrects. A nationwide event doesn't destroy the comparison — it's exactly what the control region helps you adjust for. DiD is designed for situations where a common shock affects both groups simultaneously.
Study tip: In DiD problems, always ask "what does the control group tell me?" It estimates the counterfactual trend, so the treatment effect is always treated minus control, never treated alone. - Which statement most accurately distinguishes the two analyses?
Answer: The randomized-price relationship can support causation, while the operational relationship may be confounded by expected demand.
Whenever you see a question contrasting observational data with randomized experiments, your core task is to identify confounding — a third variable that distorts the apparent relationship between two others.
In the operational data, managers deliberately lower prices on products they expect to sell poorly. This means low prices and low demand share a common cause: managerial expectations. The observed negative correlation between price and units sold could reflect true price sensitivity, managerial selection behavior, or both — you simply cannot tell. This is a classic confound, and C correctly identifies it: the operational correlation may be driven by expected demand rather than price alone.
The randomized test breaks this problem. By randomly assigning prices across eligible sessions, the team ensures that which price a customer sees is unrelated to any product characteristic or managerial forecast. Any remaining difference in units sold across price conditions can be attributed to price itself. This is why randomization enables causal inference — and why C is the accurate answer.
Choice A is wrong because consistency with the law of demand does not eliminate confounding. A result can "look right" for the wrong reasons. Choice B has the logic backwards — the fact that managers use demand forecasts when setting prices is precisely what creates confounding in the operational data, not what removes it. Choice D overcorrects: correlation from a well-designed randomized experiment absolutely can support causal claims; the limitation applies to observational correlations, not all correlations universally.
Your study tip: on any question involving correlation versus causation, immediately ask, "Was assignment random or chosen by someone?" If chosen, expect confounding.
- Why might the redesign coefficient in this regression fail to estimate the redesign's total causal effect?
Answer: Page-load time is a post-assignment mediator, so controlling for it can remove part of the redesign's causal effect on conversion.
Whenever you see a question involving causal inference from an experiment, ask yourself: is the control variable measured before or after the treatment was assigned? That distinction is everything.
In a properly randomized experiment, the treatment (here, redesign assignment) is independent of all pre-existing characteristics, so you can estimate the total causal effect without controlling for anything. But when you control for a post-treatment variable — one that sits on the causal pathway between treatment and outcome — you risk "blocking" part of the mechanism through which the treatment operates. This is called controlling for a mediator, and it's a classic mistake in causal analysis.
Here, page-load time is measured after assignment, and the redesign itself causes faster page loads. Faster load times then boost conversions. By including page-load time as a covariate, the regression absorbs some of the redesign's effect, leaving the redesign coefficient to capture only the direct path (redesign → conversion, not through load time). The coefficient no longer reflects the total causal effect. That makes A correct.
B is wrong because page-load time is not a pre-existing confounder — it's caused by the redesign. It cannot confound a randomized assignment, since randomization already handles pre-treatment imbalance. C is wrong because binary outcomes don't preclude causal estimation; linear probability models and logistic regression are both used in randomized settings. D is wrong because while randomization protects the total-effect estimate without additional controls, adding a mediator actively distorts it.
The study tip: always trace the causal timing of any control variable. If it's post-treatment, controlling for it can introduce bias rather than remove it.
- Which statement best reconciles the unadjusted and adjusted findings?
Answer: Low prior output confounds the unadjusted relationship because it affects both training assignment and likely future output.
Whenever you see a gap between an unadjusted (raw) comparison and an adjusted one, your instinct should be to ask: is there a third variable that simultaneously influences both who gets treated and what the outcome would have been anyway? That's the definition of a confounder.
Here, managers deliberately assign training to employees with unusually low prior output. Those same employees — because of regression to the mean and their underlying performance level — are also likely to have lower future output regardless of training. Prior output therefore drives both the selection into training and the next month's results. This makes it a classic confounder, and B correctly identifies that structure: low prior output inflates the apparent harm of training in the raw comparison, masking the modest true benefit that surfaces once you control for it.
A is wrong because adjusted associations can reverse or change the direction of unadjusted ones — that's precisely what happens in Simpson's Paradox-style confounding. Claiming a reversal is impossible reflects a fundamental misunderstanding of multivariate analysis.
C misapplies the term mediator. A mediator is on the causal pathway from treatment to outcome (training → mediator → output). Prior output happens before training is assigned, so it cannot mediate training's effect — it's upstream, not in between.
D overclaims. Controlling for one confounder reduces some selection bias but rarely eliminates it entirely. Other unmeasured differences between trained and untrained workers may still exist, so the adjusted estimate is not guaranteed to equal the true causal effect.
Study tip: When an adjusted finding contradicts a raw one, always map out the timeline — a variable measured before treatment is a potential confounder; one measured after treatment could be a mediator. Getting that sequence right unlocks the correct interpretation.
- Which explanation best accounts for the unexpected association?
Answer: Support contact is a selection variable affected by slow delivery or other dissatisfaction, so conditioning on it can induce an association.
Whenever you see a question about surprising correlations within a restricted sample, think about collider bias (also called selection bias or Berkson's paradox). The key question to ask is: "Did something filter which observations made it into this dataset, and could that filter create a spurious association?"
Here, the company only studies customers who contacted support. Support contact is triggered by either slow delivery or other dissatisfaction (like damaged items). This means support contact is a collider — a variable causally downstream of both delivery speed and satisfaction. When you condition on a collider (restrict your sample to support-contactors), you inadvertently link its two causes together. Fast delivery customers who contacted support must have done so because of the other reason — dissatisfaction. This artificially makes fast delivery look associated with low satisfaction within the sample. Answer D captures this exactly: support contact is a selection variable affected by multiple dissatisfiers, and conditioning on it induces a spurious association.
A is wrong because support contact isn't a common cause of delivery speed and satisfaction — it's a downstream consequence of them. That's the opposite causal structure, and restricting on it doesn't remove bias; it creates it.
B is wrong because item damage isn't being described as a mediator on the causal path from delivery speed to satisfaction. The passage presents damage as an independent reason for dissatisfaction, not a mechanism through which speed operates.
C is wrong because it assumes the observed association reflects a real causal effect — precisely the error collider bias produces.
Study tip: When a sample is restricted by contact, complaint, or any event triggered by multiple causes, immediately suspect collider bias as the explanation for any unexpected correlation.
- Which assessment best identifies the flaw in the executive's reasoning?
Answer: Underlying product problems may cause both repeated calls and later cancellation, so suppressing calls may not reduce churn.
Whenever a question describes a correlation being used to justify a policy intervention, your first instinct should be to ask: is this association causal, or could something else explain both variables? This is the classic confounding variable problem in causal reasoning.
Here, the executive sees that frequent support calls predict cancellation and concludes that eliminating calls will eliminate churn. But consider what's actually happening: customers may be calling repeatedly because the product itself is broken or poorly designed. That underlying product failure is the true driver — it frustrates customers into calling and eventually drives them to cancel. If you block the calls without fixing the product, customers still experience the same frustration and churn at the same rate. You've removed the symptom, not the disease. This is exactly what B identifies: a common third cause (product problems) drives both variables simultaneously, so suppressing one doesn't reduce the other.
A is tempting because it invokes the legitimate causal rule that causes must precede effects. However, the calls do precede the cancellations (calls happen this month, cancellation happens next month), so there's no temporal violation here — A invents a problem that doesn't exist in the scenario.
C is irrelevant. Subscription length uniformity is not a prerequisite for a predictive association to be meaningful or actionable.
D misunderstands spuriousness. Measuring two variables in different time periods doesn't make an association spurious — in fact, that lag is often by design in churn analysis.
When you see a proposed intervention based on correlation, always ask: could a hidden third variable explain both sides? If yes, the causal claim is likely flawed.
- What is the principal causal concern with the executive's forecast?
Answer: Forecast customer traffic is a confounder that raises both staffing and revenue, so the regression may overstate the gain from adding staff.
Whenever you see a regression coefficient being used to justify a managerial intervention, your first instinct should be to ask: is this correlation driven by a third variable rather than a true cause-and-effect relationship? This is the classic confounding problem in causal inference.
Here, managers deliberately schedule more staff on days when forecasts predict high customer traffic. That means forecast traffic drives both staffing levels and revenue simultaneously. Stores don't earn more because they added staff — both staffing and revenue are elevated by the same underlying cause: anticipated demand. The regression coefficient absorbs this shared variation and will therefore overstate the true revenue benefit of simply mandating one additional employee on any given day. This is exactly what A describes, making it the correct answer.
B gets the causal arrow backwards. Staffing doesn't cause forecast traffic — managers respond to forecasts, not the other way around. Labeling staffing as the confounder here contradicts the sequence of events described in the passage.
C misapplies the concept of a mediator. A mediator sits on the causal path between the treatment and outcome. Revenue is the outcome variable, not a mechanism linking traffic forecasts to staffing decisions — so calling it a mediator is structurally incorrect.
D is simply false. Causal effects routinely produce imperfect, noisy correlations. A positive coefficient can absolutely represent a causal effect; the issue here is specifically about confounding, not the sign or magnitude of the correlation.
Study tip: When a policy question uses observational regression, always ask whether a third variable could be driving both sides of the relationship — that's the confounding trap exam writers love to test.
- Which conclusion is most justified?
Answer: The respondent difference is descriptive, but differential post-assignment response can bias the causal effect for all assigned customers.
Whenever you see a question combining randomized assignment with a voluntary survey, you need to think carefully about two separate causal questions: what randomization actually guarantees, and whether post-assignment behavior (like choosing to respond) can reintroduce bias even after a clean randomization.
Randomization at the start ensures the two groups are statistically comparable at the moment of assignment. However, if the treatment itself influences who chooses to respond to the survey, then the respondents you actually observe are no longer comparable across groups. Here, response rates differ dramatically — 70% vs. 40% — and the passage explicitly warns that the application experience may affect response likelihood. This means the observed satisfaction scores (8.2 vs. 7.4) reflect only the self-selected respondents, not all assigned customers. The 0.8-point gap is a real descriptive finding among respondents, but it cannot be cleanly attributed as a causal effect for the full assigned population. That is precisely what D captures: the difference is descriptive, and differential post-assignment response is a form of selection bias that threatens the causal interpretation.
A is wrong because randomization protects internal validity only when outcomes are observed for all (or a representative sample of) assigned units — it does not automatically survive selective non-response. B misunderstands confounders entirely; response behavior happens after assignment, making it a post-treatment variable, not a pre-assignment confounder. Re-randomizing among responders would destroy the original experimental logic. C has the reasoning exactly backwards — higher response rates do not necessarily include more dissatisfied customers; dissatisfied customers often disengage rather than respond.
When you see differential response rates in an experiment, treat it as a red flag for post-treatment selection bias — randomization is not a shield against it. - What is the most appropriate interpretation of the resulting aggregate purchase rates?
Answer: Exposure has no demonstrated effect because purchase rates are equal within each intent group, while group composition creates the aggregate gap.
Whenever you see a question comparing aggregate statistics across groups with different compositions, you should immediately think about Simpson's Paradox — the phenomenon where a trend that appears in combined data disappears or reverses when the data is broken into subgroups.
Here, the math tells a clear story. Among exposed visitors, 80 are high-intent (20% purchase rate) and 20 are low-intent (5% purchase rate), giving an aggregate rate of 10080(0.20)+20(0.05)=10017=17%. Among unexposed visitors, the composition flips: 20 high-intent and 80 low-intent, yielding 10020(0.20)+80(0.05)=1008=8%. The gap exists entirely because high-intent visitors — who purchase at higher rates regardless — are overrepresented among the exposed group. Within each intent segment, purchase rates are identical across exposed and unexposed visitors. This confirms B: exposure has no demonstrated effect; the aggregate gap is a compositional artifact.
A is wrong because it treats the 9-percentage-point aggregate difference as causal evidence of exposure's impact, ignoring that the difference is fully explained by group composition — a classic Simpson's Paradox trap. C is wrong because exposure doesn't affect low-intent visitors at all; their 5% rate is identical in both groups. The answer confuses representation with causation. D is wrong because high-intent visitors purchase at 20% in both groups — exposure isn't responsible for their purchases either.
Your study tip: whenever aggregate trends conflict with subgroup trends, always break the data down by a potential confounding variable before drawing causal conclusions.
