IV & Regression Discontinuity — Instrumental variables and regression discontinuity concepts (intro)

Historical Context & Motivation

At the heart of modern business decision-making lies a deceptively simple question: does X actually cause Y, or do they merely move together? The gold standard for answering this question is the randomized controlled trial (RCT), where subjects are randomly assigned to treatment and control groups. Yet in business and economics, randomization is often impractical, unethical, or prohibitively expensive. Imagine trying to randomly assign MBA degrees to workers to measure their causal effect on wages, or randomly imposing tariffs on countries to estimate their impact on trade. The impossibility of such experiments motivated econometricians and statisticians to develop quasi-experimental methods — techniques that exploit naturally occurring variation in data to approximate the conditions of a true experiment.

Two of the most influential quasi-experimental designs are instrumental variables (IV) estimation and regression discontinuity (RD) design. Both emerged from a recognition that observational data are plagued by endogeneity — the problem that the variable you want to study is correlated with unobserved factors that also affect your outcome. The intellectual lineage of these methods stretches across decades of innovation in economics, education policy, and public health research.

The central question these methods address is: when you cannot randomly assign treatment, how can you still credibly estimate a causal effect? Instrumental variables answer this by finding an external source of variation that affects the outcome only through the treatment. Regression discontinuity exploits situations where treatment is assigned based on whether a running variable exceeds a known threshold, creating a local randomization around the cutoff. Both methods have become essential tools in the modern business analyst's toolkit for data-driven causal inference.

Core Principles & Definitions

Before diving into the mechanics of IV and RD, it is essential to understand the core concepts that underpin both methods. Each technique addresses the fundamental problem of endogeneity bias — the distortion that arises in ordinary least squares (OLS) regression when the explanatory variable of interest is correlated with the error term. In business settings, this correlation frequently arises from omitted variables (factors you cannot measure), reverse causality (the outcome influencing the treatment), or selection bias (subjects self-selecting into treatment based on expected outcomes).

Visual Explanation — How IV and RD Work

Instrumental Variables: The Causal Pathway

The diagram below illustrates the fundamental logic of instrumental variables estimation. In a standard regression of outcome Y on treatment X, the presence of an unobserved confounder U creates a spurious correlation (the dashed red path) that biases OLS estimates. The instrument Z provides a source of variation in X that is independent of U, effectively isolating only the exogenous part of X's influence on Y. The key insight is that the instrument affects Y only through its effect on X — this is the exclusion restriction.

In the diagram, notice that the dashed red arrows represent the confounding bias introduced by U. If you simply regress Y on X using OLS, your estimate of X's effect picks up influence from both the true causal path (X → Y) and the backdoor path (X ← U → Y). The instrument Z provides a 'clean' source of variation in X — variation that flows only through the solid cyan arrow, untainted by U. By using only this Z-driven variation in X to estimate the effect on Y, IV estimation removes the confounding bias.

Mathematical Framework

The IV Estimator

Consider the structural equation relating outcome Y to treatment X and an error term ε that captures all unobserved factors. When X is endogenous — meaning Cov(X, ε) ≠ 0 — OLS produces biased and inconsistent estimates. The IV approach proceeds in two conceptual stages, formalized as two-stage least squares (2SLS).

The Regression Discontinuity Estimator

In the sharp RD design, treatment is a deterministic function of a running variable R and a cutoff c: individuals receive treatment if and only if R ≥ c. The causal effect is identified by the discontinuous jump in the conditional expectation of Y at the cutoff. Under the assumption that all other factors affecting Y change continuously across the cutoff, any discrete jump in Y at c must be caused by the treatment.

Regression Discontinuity in Detail

Regression discontinuity design is often considered the most credible quasi-experimental method because the identification strategy is transparent and visually verifiable. The core idea rests on a simple premise: when individuals or firms are assigned to treatment based on whether a continuous score exceeds a threshold, those scoring just above and just below the threshold are — in expectation — identical in all observed and unobserved characteristics. This local randomization near the cutoff allows the researcher to estimate a causal effect by comparing outcomes on either side of the discontinuity.

Several practical considerations govern the implementation of RD designs in business analytics. First, the researcher must verify that individuals cannot precisely manipulate their running variable to sort around the cutoff — a violation known as manipulation. This is typically assessed using a McCrary density test, which checks for bunching just above or below the threshold. Second, the choice of bandwidth — how wide a window around the cutoff to include — involves a bias-variance tradeoff: a narrow bandwidth reduces bias but increases variance due to fewer observations, while a wider bandwidth increases sample size but risks including observations too far from the cutoff to be comparable.

Worked Example — IV Estimation in a Business Context

Consider a marketing analytics team at an e-commerce company trying to determine the causal effect of email marketing on customer spending. Simply regressing spending on the number of marketing emails received would produce biased results because the company targets its most engaged (and highest-spending) customers with more emails. The team identifies a plausible instrument: a server outage that randomly prevented some customers from receiving a promotional email blast — the outage affected email delivery independently of customer characteristics.

Strengths, Limitations, and When to Use Each Method

Both IV and RD are powerful tools for causal inference, but they come with important tradeoffs. Understanding these tradeoffs is critical for the business analyst deciding which method to apply — or whether quasi-experimental methods are appropriate at all for a given question.

Connection to Advanced Causal Inference

Instrumental variables and regression discontinuity sit within a broader ecosystem of causal inference methods. Understanding where they fit relative to more advanced techniques helps you appreciate both their power and their boundaries. The table below maps the introductory concepts covered in this lesson to their advanced extensions, giving you a roadmap for further study.

A particularly important connection is between fuzzy RD and IV. As noted earlier, fuzzy RD is formally equivalent to an IV estimator where the indicator for crossing the cutoff serves as the instrument for actual treatment receipt. This unifying perspective reveals that IV and RD are not entirely separate methods — they are different instantiations of the same underlying logic of using exogenous variation to identify causal effects. In advanced coursework, you will encounter the potential outcomes framework (Rubin causal model) and directed acyclic graphs (DAGs) as formal languages for reasoning about identification strategies — tools that generalize beyond any single estimator.

Practice Problems

Lesson Summary