IV & Regression Discontinuity — Instrumental variables and regression discontinuity concepts (intro) - Business Analytics

SECTION 1

Historical Context & Motivation

At the heart of modern business decision-making lies a deceptively simple question: does X actually cause Y, or do they merely move together? The gold standard for answering this question is the randomized controlled trial (RCT), where subjects are randomly assigned to treatment and control groups. Yet in business and economics, randomization is often impractical, unethical, or prohibitively expensive. Imagine trying to randomly assign MBA degrees to workers to measure their causal effect on wages, or randomly imposing tariffs on countries to estimate their impact on trade. The impossibility of such experiments motivated econometricians and statisticians to develop quasi-experimental methods — techniques that exploit naturally occurring variation in data to approximate the conditions of a true experiment.

Two of the most influential quasi-experimental designs are instrumental variables (IV) estimation and regression discontinuity (RD) design. Both emerged from a recognition that observational data are plagued by endogeneity — the problem that the variable you want to study is correlated with unobserved factors that also affect your outcome. The intellectual lineage of these methods stretches across decades of innovation in economics, education policy, and public health research.

1928 • Philip Wright & Instrumental Variables

Philip Wright, in a study of supply and demand for butter and flaxseed, introduced the concept of instrumental variables — using an external variable to isolate causal variation in an endogenous regressor. This foundational idea addressed the simultaneous equations problem in economics.

1953 • Theil's Two-Stage Least Squares

Henri Theil formalized the two-stage least squares (2SLS) estimator, providing a practical and generalizable algorithm for IV estimation that became the workhorse of applied econometrics.

1960 • Thistlethwaite & Campbell — RD Origins

Donald Thistlethwaite and Donald Campbell published a landmark study on the effect of merit awards on career aspirations, exploiting a test-score cutoff. This introduced regression discontinuity design — leveraging a known threshold to approximate random assignment.

1991–2001 • The Credibility Revolution

Angrist and Imbens formalized the Local Average Treatment Effect (LATE) interpretation of IV. Hahn, Todd, and van der Klaauw developed formal econometric theory for RD. These advances, part of the broader credibility revolution in economics, emphasized research designs with transparent identification strategies.

2021 • Nobel Prize Recognition

Joshua Angrist and Guido Imbens shared the Nobel Memorial Prize in Economic Sciences for their methodological contributions to the analysis of causal relationships, including foundational work on IV and natural experiments that underpin modern business analytics.

The central question these methods address is: when you cannot randomly assign treatment, how can you still credibly estimate a causal effect? Instrumental variables answer this by finding an external source of variation that affects the outcome only through the treatment. Regression discontinuity exploits situations where treatment is assigned based on whether a running variable exceeds a known threshold, creating a local randomization around the cutoff. Both methods have become essential tools in the modern business analyst's toolkit for data-driven causal inference.

SECTION 2

Core Principles & Definitions

Before diving into the mechanics of IV and RD, it is essential to understand the core concepts that underpin both methods. Each technique addresses the fundamental problem of endogeneity bias — the distortion that arises in ordinary least squares (OLS) regression when the explanatory variable of interest is correlated with the error term. In business settings, this correlation frequently arises from omitted variables (factors you cannot measure), reverse causality (the outcome influencing the treatment), or selection bias (subjects self-selecting into treatment based on expected outcomes).

Endogeneity

Occurs when an explanatory variable is correlated with the regression error term, meaning OLS estimates capture both the causal effect and confounding influences. This is the central problem both IV and RD are designed to solve.

Instrumental Variable

A variable (Z) that is correlated with the endogenous regressor (X) but is uncorrelated with the error term (ε). It must satisfy both the relevance and exclusion restriction conditions.

Running Variable & Cutoff

In RD design, treatment assignment is determined by whether a continuous running variable (also called a forcing variable) crosses a known cutoff. Units just above and just below the cutoff are assumed to be nearly identical on average.

Local Average Treatment Effect (LATE)

IV estimates identify a LATE — the causal effect for compliers, the subpopulation whose treatment status is actually changed by the instrument. This is not necessarily the average treatment effect (ATE) for the entire population.

Identification Strategy

The logical argument, grounded in institutional knowledge or policy rules, that explains why a particular instrument is valid or why treatment near a cutoff is as-if random. A credible identification strategy is the cornerstone of any quasi-experimental analysis.

✦

KEY TAKEAWAY

Think of endogeneity like trying to measure the effect of a fertilizer on crop yield, but the farmers who use the most fertilizer also happen to have the richest soil. You cannot tell whether the higher yield comes from the fertilizer or the soil. An instrumental variable is like a government subsidy randomly distributed across farms — it changes fertilizer use but has no direct effect on soil quality, allowing you to isolate the fertilizer's true effect. Regression discontinuity is like comparing farms that just barely qualified for the subsidy (score = 51) with those that barely missed it (score = 49) — these two groups are virtually identical except for the subsidy, so any difference in yield is attributable to the program.

SECTION 3

Visual Explanation — How IV and RD Work

Instrumental Variables: The Causal Pathway

The diagram below illustrates the fundamental logic of instrumental variables estimation. In a standard regression of outcome Y on treatment X, the presence of an unobserved confounder U creates a spurious correlation (the dashed red path) that biases OLS estimates. The instrument Z provides a source of variation in X that is independent of U, effectively isolating only the exogenous part of X's influence on Y. The key insight is that the instrument affects Y only through its effect on X — this is the exclusion restriction.

The causal diagram shows the instrument Z (cyan box) influencing treatment X (violet box), which in turn affects outcome Y (green box). The unobserved confounder U (red box) creates spurious paths (dashed red arrows) that bias OLS. The instrument must satisfy relevance (Z predicts X) and the exclusion restriction (Z does not directly affect Y).

In the diagram, notice that the dashed red arrows represent the confounding bias introduced by U. If you simply regress Y on X using OLS, your estimate of X's effect picks up influence from both the true causal path (X → Y) and the backdoor path (X ← U → Y). The instrument Z provides a 'clean' source of variation in X — variation that flows only through the solid cyan arrow, untainted by U. By using only this Z-driven variation in X to estimate the effect on Y, IV estimation removes the confounding bias.

SECTION 4

Mathematical Framework

The IV Estimator

Consider the structural equation relating outcome Y to treatment X and an error term ε that captures all unobserved factors. When X is endogenous — meaning Cov(X, ε) ≠ 0 — OLS produces biased and inconsistent estimates. The IV approach proceeds in two conceptual stages, formalized as two-stage least squares (2SLS).

STRUCTURAL EQUATION

Y = β₀ + β₁X + ε

Y = outcome variable, X = endogenous treatment variable, β₁ = causal effect of interest, ε = error term containing all unobserved determinants of Y. The problem: Cov(X, ε) ≠ 0, so OLS(β₁) is biased.

FIRST STAGE

X = π₀ + π₁Z + v

Regress X on the instrument Z. The fitted values X̂ = π̂₀ + π̂₁Z capture only the variation in X that is driven by Z. Relevance requires π₁ ≠ 0 (i.e., Z must predict X).

SECOND STAGE

Y = β₀ + β₁X̂ + η

Regress Y on X̂ (the predicted values from Stage 1). Because X̂ is constructed solely from Z — which is exogenous — this regression yields a consistent estimate of β₁, the causal effect of X on Y.

WALD ESTIMATOR (SIMPLE IV)

β̂ᵢᵥ = Cov(Y, Z) / Cov(X, Z)

When there is a single binary instrument and a single endogenous variable, the IV estimator simplifies to this ratio — the Wald estimator. It divides the reduced-form effect of Z on Y by the first-stage effect of Z on X.

The Regression Discontinuity Estimator

In the sharp RD design, treatment is a deterministic function of a running variable R and a cutoff c: individuals receive treatment if and only if R ≥ c. The causal effect is identified by the discontinuous jump in the conditional expectation of Y at the cutoff. Under the assumption that all other factors affecting Y change continuously across the cutoff, any discrete jump in Y at c must be caused by the treatment.

SHARP RD TREATMENT EFFECT

τ_RD = lim(r→c⁺) E[Y | R = r] − lim(r→c⁻) E[Y | R = r]

τ_RD is the causal effect at the cutoff, estimated as the difference between the right-limit and left-limit of the conditional expectation of Y as the running variable R approaches the cutoff c. This is a local estimate — it applies specifically to units near the threshold.

ℹ

Sharp vs. Fuzzy RD

In a sharp RD, the probability of treatment jumps from 0 to 1 at the cutoff — compliance is perfect. In a fuzzy RD, the cutoff only changes the probability of treatment (e.g., from 30% to 80%). Fuzzy RD is essentially an IV estimator where the instrument is the indicator for being above the cutoff, making it a ratio of the jump in Y to the jump in treatment probability at c.

SECTION 5

Regression Discontinuity in Detail

Regression discontinuity design is often considered the most credible quasi-experimental method because the identification strategy is transparent and visually verifiable. The core idea rests on a simple premise: when individuals or firms are assigned to treatment based on whether a continuous score exceeds a threshold, those scoring just above and just below the threshold are — in expectation — identical in all observed and unobserved characteristics. This local randomization near the cutoff allows the researcher to estimate a causal effect by comparing outcomes on either side of the discontinuity.

In this sharp RD diagram, the running variable (horizontal axis) determines treatment assignment at the cutoff c (dashed yellow line). Violet dots represent untreated observations (R < c), and cyan dots represent treated observations (R ≥ c). The treatment effect τ is the vertical gap between the right-limit and left-limit of the fitted regression lines at the cutoff.

Several practical considerations govern the implementation of RD designs in business analytics. First, the researcher must verify that individuals cannot precisely manipulate their running variable to sort around the cutoff — a violation known as manipulation. This is typically assessed using a McCrary density test, which checks for bunching just above or below the threshold. Second, the choice of bandwidth — how wide a window around the cutoff to include — involves a bias-variance tradeoff: a narrow bandwidth reduces bias but increases variance due to fewer observations, while a wider bandwidth increases sample size but risks including observations too far from the cutoff to be comparable.

Feature	Sharp RD	Fuzzy RD
Treatment Assignment	Deterministic: D = 1 if R ≥ c, D = 0 otherwise	Probabilistic: probability of treatment jumps at c, but compliance is imperfect
Probability Jump at Cutoff	0 → 1 (100%)	Partial (e.g., 20% → 70%)
Estimation Method	Local polynomial regression comparing outcomes at the cutoff	IV / 2SLS using the cutoff indicator as the instrument for actual treatment
Interpretation	Average treatment effect at the cutoff	LATE for compliers at the cutoff
Business Example	Loyalty program tier: customers with spend ≥ $500 receive Gold status	Scholarship eligibility: students above a GPA cutoff are offered (but may decline) financial aid

Comparison of Sharp and Fuzzy RD Designs

SECTION 6

Worked Example — IV Estimation in a Business Context

Consider a marketing analytics team at an e-commerce company trying to determine the causal effect of email marketing on customer spending. Simply regressing spending on the number of marketing emails received would produce biased results because the company targets its most engaged (and highest-spending) customers with more emails. The team identifies a plausible instrument: a server outage that randomly prevented some customers from receiving a promotional email blast — the outage affected email delivery independently of customer characteristics.

Estimating the Effect of Email Marketing on Spending Using IV

1
Step 1 — Define Variables

Let Y = monthly spending ($), X = indicator for receiving the promotional email (1 = received, 0 = not), and Z = indicator for server functioning normally during the email blast (1 = server up, 0 = server down). The concern is that the company's targeting algorithm makes X endogenous — customers who receive emails are already predisposed to higher spending.
2
Step 2 — Check Instrument Validity (Relevance)

The first-stage regression is: X = π₀ + π₁Z + v. Suppose we estimate π̂₁ = 0.65, meaning that customers whose server was up were 65 percentage points more likely to receive the email than those affected by the outage. The first-stage F-statistic is 89.4, well above the rule-of-thumb threshold of 10 for strong instruments.

First-stage coefficient: π̂₁ = 0.65, F = 89.4 ≫ 10 → strong instrument
3
Step 3 — Argue Exclusion Restriction

The exclusion restriction requires that the server outage affected spending only through its effect on email receipt — not through any direct channel. This is plausible if the outage was localized to the email delivery system and did not affect the website, app, or other marketing channels. We verify that the outage was orthogonal to customer demographics, past purchase behavior, and geographic location by running balance tests.
4
Step 4 — Compute the Wald (IV) Estimate

From the data: E[Y | Z = 1] = $142 (customers whose server was up), E[Y | Z = 0] = $119 (customers affected by outage), E[X | Z = 1] = 0.78, E[X | Z = 0] = 0.13. The Wald estimate is:

β̂ᵢᵥ = (142 − 119) / (0.78 − 0.13) = 23 / 0.65 = $35.38
5
Step 5 — Interpret the Result

The IV estimate suggests that receiving the promotional email causes an average increase of $35.38 in monthly spending among compliers — those customers whose email receipt was actually affected by the server outage. For comparison, the naïve OLS estimate was $52.10, which was inflated by selection bias (the company targeted high-spending customers with more emails). The IV estimate provides a more credible causal estimate, though it applies locally to the complier subpopulation.

Causal effect (LATE): $35.38 per email vs. biased OLS: $52.10

SECTION 7

Strengths, Limitations, and When to Use Each Method

Both IV and RD are powerful tools for causal inference, but they come with important tradeoffs. Understanding these tradeoffs is critical for the business analyst deciding which method to apply — or whether quasi-experimental methods are appropriate at all for a given question.

Dimension	Instrumental Variables	Regression Discontinuity
Key Strength	Can address endogeneity even when treatment is not assigned by a threshold; applicable to a wide range of causal questions	Highly credible and visually transparent; identification assumptions are testable (e.g., density tests, covariate balance)
Key Limitation	Exclusion restriction is untestable — must be argued on theoretical or institutional grounds; finding valid instruments is difficult	Estimates are local to the cutoff and may not generalize; requires a known, rule-based assignment mechanism
Weak Instrument Risk	Weak instruments (low first-stage F) lead to severely biased and unreliable estimates, often worse than OLS	Not directly applicable; bandwidth choice is the analogous concern
External Validity	LATE applies only to compliers; may not represent the average treatment effect for the full population	Effect applies at the cutoff; may differ for observations far from the threshold
Data Requirements	Requires a valid instrument — a variable correlated with treatment but not directly with the outcome	Requires a continuous running variable with a known cutoff and sufficient observations near the threshold
Business Use Cases	Supply/demand estimation, returns to education, advertising effectiveness using weather or distance as instruments	Loyalty program tier effects, scholarship impacts, regulatory thresholds (e.g., firms above/below a size threshold)

IV vs. RD — Comparative Strengths and Limitations

✦

KEY TAKEAWAY

Think of IV and RD as complementary tools in a causal inference toolkit, analogous to different types of surgical instruments. IV is like a general-purpose scalpel — it can be applied in many settings but requires the analyst to make a difficult-to-verify assumption (exclusion restriction). RD is like a precision laser — it works only in specific threshold-based settings but produces especially credible results because you can visually inspect the discontinuity and empirically test the identifying assumptions. Neither method gives you the full ATE across all units; both provide local estimates for specific subpopulations.

SECTION 8

Connection to Advanced Causal Inference

Instrumental variables and regression discontinuity sit within a broader ecosystem of causal inference methods. Understanding where they fit relative to more advanced techniques helps you appreciate both their power and their boundaries. The table below maps the introductory concepts covered in this lesson to their advanced extensions, giving you a roadmap for further study.

Introductory Concept	Advanced Extension	What It Adds
Simple IV (one instrument, one treatment)	Generalized Method of Moments (GMM)	Handles multiple instruments efficiently; provides tests of overidentifying restrictions to assess instrument validity
Wald estimator / 2SLS	LIML, JIVE, Weak-IV Robust Methods	Address finite-sample bias when instruments are weak; limited information maximum likelihood (LIML) is less biased than 2SLS with weak instruments
Sharp RD	Geographic / Multi-dimensional RD	Extends RD to settings with spatial boundaries or multiple running variables (e.g., latitude/longitude thresholds for policy zones)
Fuzzy RD	RD with Covariates / RD-DID	Combines RD with difference-in-differences to control for time trends; adds covariates to improve precision near the cutoff
Local bandwidth selection	Optimal Bandwidth (Calonico, Cattaneo, Titiunik)	Data-driven bandwidth selection with bias-corrected confidence intervals; the rdrobust package automates this

From Introductory to Advanced Methods

A particularly important connection is between fuzzy RD and IV. As noted earlier, fuzzy RD is formally equivalent to an IV estimator where the indicator for crossing the cutoff serves as the instrument for actual treatment receipt. This unifying perspective reveals that IV and RD are not entirely separate methods — they are different instantiations of the same underlying logic of using exogenous variation to identify causal effects. In advanced coursework, you will encounter the potential outcomes framework (Rubin causal model) and directed acyclic graphs (DAGs) as formal languages for reasoning about identification strategies — tools that generalize beyond any single estimator.

💡

Looking Ahead

In your business analytics career, you will likely encounter settings where IV and RD can be combined with machine learning. For instance, causal forests can estimate heterogeneous treatment effects in RD designs, and LASSO-based instrument selection can help identify strong instruments from high-dimensional candidate sets. The intersection of causal inference and machine learning is one of the most active areas of methodological innovation in business analytics.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A retail company wants to estimate the causal effect of offering free shipping on customer order value. They notice that customers who receive free shipping tend to place larger orders. Explain why simply comparing average order values between customers who received free shipping and those who did not would likely produce a biased estimate of the causal effect. What type(s) of endogeneity could be at play?

Show answer

A naïve comparison would be biased because free shipping is not randomly assigned — the company likely offers it to customers who are already high-value or highly engaged (selection bias / omitted variable bias). Additionally, customers who are predisposed to make large purchases may self-select into promotions or loyalty tiers that include free shipping. The unobserved variable 'customer purchase intent' is correlated with both receiving free shipping (X) and order value (Y), violating the OLS assumption that Cov(X, ε) = 0. This endogeneity could arise from omitted variables (unobserved purchase intent, income), reverse causality (high spenders qualifying for free shipping), or simultaneous selection (the firm targets free shipping to likely high spenders).

Question 2: BASIC CALCULATION

A researcher uses distance from a distribution warehouse as an instrument for whether a customer receives same-day delivery (X) to estimate the effect on customer satisfaction scores (Y). From the data: E[Y | Z = 'close'] = 82, E[Y | Z = 'far'] = 76, E[X | Z = 'close'] = 0.85, E[X | Z = 'far'] = 0.25. Calculate the Wald (IV) estimate of the effect of same-day delivery on satisfaction.

Show answer

The Wald estimator is β̂ᵢᵥ = (E[Y | Z = close] − E[Y | Z = far]) / (E[X | Z = close] − E[X | Z = far]) = (82 − 76) / (0.85 − 0.25) = 6 / 0.60 = 10.0. This means that same-day delivery is estimated to increase customer satisfaction scores by 10 points for compliers — those customers whose delivery speed was actually changed by proximity to the warehouse. The numerator (6 points) is the reduced-form effect of proximity on satisfaction, and the denominator (0.60) is the first-stage effect of proximity on same-day delivery probability. The ratio scales the reduced-form effect to account for the fact that proximity only partially determines delivery speed.

Question 3: INTERMEDIATE

A state government offers a small business grant to any firm with annual revenue below $500,000. A business analyst wants to estimate the effect of the grant on employment growth using a regression discontinuity design. (a) Identify the running variable, cutoff, treatment, and outcome. (b) Why might a sharp RD not be appropriate here, and what variant should be used instead? (c) What test should the analyst run to check for manipulation of the running variable?

Show answer

(a) Running variable: annual revenue; cutoff: $500,000; treatment: receiving the small business grant; outcome: employment growth. (b) A sharp RD assumes perfect compliance — all firms below $500K receive the grant and none above do. In practice, some eligible firms may not apply, and some ineligible firms near the cutoff might appeal or misreport revenue. If compliance is imperfect, a fuzzy RD is more appropriate, where crossing the threshold increases the probability of receiving the grant but does not guarantee it. The fuzzy RD would use the cutoff indicator as an instrument for actual grant receipt. (c) The analyst should run a McCrary density test (or the updated Cattaneo, Jansson, and Ma test) to check whether the density of firms is continuous across the $500K threshold. A suspicious bunching of firms just below $500K would suggest that firms are manipulating their reported revenue to qualify, which would violate the identifying assumption.

Question 4: APPLIED

A tech company wants to estimate the causal effect of its premium support subscription on customer retention rates. A product manager suggests using the timing of a platform-wide price change as an instrument: customers who signed up before the price increase paid $9.99/month, while those who signed up after paid $14.99/month — the price change was driven by corporate strategy and was unrelated to individual customer characteristics. (a) Explain how this instrument satisfies (or potentially fails) the relevance and exclusion restriction conditions. (b) What is the complier group in this context? (c) How would you interpret the resulting LATE?

Show answer

(a) Relevance: the price increase likely reduced the take-up rate of the premium subscription (customers are price-sensitive), so Z (pre/post price change) should predict X (subscription). This can be verified with a first-stage regression and F-test. Exclusion restriction: the price change must affect retention only through its effect on subscription take-up, not through other channels. A potential violation arises if the price change coincided with other changes (e.g., new features, interface redesign, or a general economic downturn) that independently affected retention — careful institutional knowledge and timing controls are needed. (b) The compliers are customers who would subscribe at $9.99 but not at $14.99 — they are the 'marginal' customers whose behavior is actually changed by the price instrument. Always-takers (would subscribe at any price) and never-takers (would never subscribe) do not contribute to the LATE. (c) The LATE estimates the causal effect of premium support on retention specifically for these price-sensitive marginal customers — not for all customers. If the company wants to extrapolate to always-takers (e.g., enterprise clients), the LATE may not apply, and additional analysis (e.g., examining effect heterogeneity) would be needed.

Question 5: CRITICAL THINKING

Critically evaluate the following claim: 'Regression discontinuity provides causal estimates that are as credible as a randomized experiment, so there is no reason to ever run an RCT when you have a natural threshold.' Discuss at least three specific limitations of RD relative to an RCT, and explain a scenario where an RD estimate might be misleading for business decision-making even though it is internally valid.

Show answer

The claim overstates RD's advantages. Three key limitations: (1) External validity — RD estimates the treatment effect only at the cutoff, not across the full distribution of the running variable. A loyalty program might boost spending by $20 for customers right at the qualifying threshold, but the effect could be $50 for customers well above it or $5 for those well below. An RCT, by contrast, identifies the average effect across the full population of interest. (2) Statistical power — RD uses only observations near the cutoff, discarding most of the data, so precision is often poor relative to an RCT that randomizes across the full sample. This can lead to wide confidence intervals and difficulty detecting economically meaningful effects. (3) Manipulation and sorting — even with density tests, subtle sorting (e.g., customer service representatives helping borderline customers qualify) can bias RD estimates in ways that are hard to detect. RCTs eliminate this concern by design. A misleading scenario: suppose a company uses RD to estimate the effect of a $1,000 bonus on employee productivity using a performance score cutoff. The RD shows a large positive effect at the cutoff. However, if the company then gives the bonus to all employees (not just those near the threshold), the average effect may be much smaller because employees far below the threshold may lack the skills to convert the bonus into productivity gains, while those far above are already performing at capacity. The internally valid RD estimate would be a poor guide for a company-wide policy rollout.

SUMMARY

Lesson Summary

Summary

This lesson introduced two foundational quasi-experimental methods for causal inference in business analytics. Instrumental variables (IV) estimation addresses endogeneity by leveraging an external variable (the instrument Z) that affects the treatment X but is uncorrelated with the error term. A valid instrument must satisfy two conditions: relevance (Z predicts X, verified by a first-stage F-statistic > 10) and the exclusion restriction (Z affects Y only through X, argued on institutional grounds). The two-stage least squares (2SLS) procedure operationalizes IV estimation, and the resulting estimate has a LATE interpretation — it captures the causal effect for compliers whose treatment status is changed by the instrument.

Regression discontinuity (RD) design exploits threshold-based treatment assignment: when a continuous running variable crosses a known cutoff, treatment is assigned, creating local randomization near the threshold. In sharp RD, compliance is perfect; in fuzzy RD, the cutoff indicator serves as an instrument for actual treatment — linking RD directly to IV. Both methods produce local estimates that may not generalize to the full population, a key consideration for business decision-makers. Together, IV and RD equip analysts with rigorous tools for extracting causal insights from observational data when randomized experiments are not feasible.