Binary Outcomes & Log-Odds — Binary outcomes and log-odds...

SECTION 1

Historical Context & Motivation

Medical researchers and epidemiologists frequently confront outcomes that fall into exactly two categories: a patient either survives or does not, a diagnostic test returns positive or negative, and a clinical trial participant either responds to therapy or fails to improve. These binary outcomes pose a fundamental modeling challenge because the standard linear regression framework, which assumes a continuous and unbounded dependent variable, cannot naturally confine predictions to the 0–1 probability range. The intellectual journey toward solving this problem spans more than two centuries and draws on contributions from actuarial science, mathematical biology, and the emerging discipline of biostatistics.

1838

Verhulst's Logistic Curve

Belgian mathematician Pierre-François Verhulst introduced the logistic function to model bounded population growth, providing the S-shaped curve that would later become central to binary outcome modeling.

1944

Berkson's Logit Model

Joseph Berkson at the Mayo Clinic coined the term logit and proposed using the log-odds transformation as an alternative to the probit model for bioassay data, arguing it was computationally simpler and biologically interpretable.

1958

Cox's Logistic Regression

David Cox formalized logistic regression as a general-purpose statistical model for binary data, establishing the theoretical framework used across modern biostatistics and epidemiology.

1970s–80s

Widespread Epidemiological Adoption

The Framingham Heart Study and other landmark cohort investigations adopted logistic regression to quantify risk factors for disease, making the odds ratio one of the most reported effect measures in medical literature.

The central question that drove this historical arc remains directly relevant today: how can we build a regression model that maps any real-valued linear combination of predictors onto the bounded probability scale of [0, 1]? The answer lies in the elegant log-odds transformation, which converts probabilities into an unrestricted real-number scale, enabling us to apply the familiar tools of linear modeling to inherently nonlinear, dichotomous data.

SECTION 2

Core Principles & Definitions

Before diving into the mathematical formalism, it is essential to build a conceptual scaffold around four interrelated ideas. Each represents a link in the chain that connects an observed binary event—such as disease presence—to a tractable regression framework. Mastering these core definitions allows you to read the logistic regression literature fluently and to interpret coefficient tables that appear in biomedical journals.

Binary Outcome

A response variable Y that takes only two values, conventionally coded 1 (event) and 0 (non-event). Examples include mortality (dead/alive), infection status (positive/negative), and treatment response (responder/non-responder).

Probability (p)

The probability of the event occurring, constrained to the interval [0, 1]. In a sample, we estimate p as the proportion of observations where Y = 1. This quantity is intuitive but cannot serve directly as a regression outcome because linear predictors may produce values outside [0, 1].

Odds

The ratio p / (1 − p), expressing the event's likelihood relative to the non-event. Odds range from 0 to +∞, eliminating the upper bound of probability but retaining a lower bound of zero.

Log-Odds (Logit)

The natural logarithm of the odds, ln[p / (1 − p)]. This transformation maps the entire probability range (0, 1) onto (−∞, +∞), creating a scale perfectly suited to linear modeling.

✦ KEY TAKEAWAY

Think of probability, odds, and log-odds as three different thermometer scales measuring the same underlying temperature—the likelihood of an event. Probability is like a Celsius thermometer capped at 0 and 100; odds are like a Kelvin scale that eliminates the lower boundary issue but still has a floor at zero; and log-odds are like an imaginary scale with no caps at all, stretching from −∞ to +∞. It is precisely this unboundedness that makes log-odds the natural 'language' for regression coefficients in logistic models.

SECTION 3

Visual Explanation — The Probability-to-Logit Pipeline

The diagram below traces how a single probability value p = 0.75 is converted step-by-step through the odds transformation and then the natural logarithm to yield a log-odds (logit) value. Alongside the numeric path, the diagram shows the permissible range for each scale, reinforcing why the final logit scale is the one we model linearly.

The pipeline shows p = 0.75 converted to odds = 3.00 via p/(1 − p), then to logit = ln(3) ≈ 1.099. The lower bar chart illustrates how each scale progressively expands, with log-odds spanning the entire real line.

Notice that the probability scale occupies only a finite interval, the odds scale frees the upper bound, and the log-odds scale eliminates both bounds entirely. This is the mathematical rationale for why logistic regression models the logit of the probability rather than the probability itself. A linear predictor β₀ + β₁X₁ + β₂X₂ + ⋯ can take any real value, so it maps naturally onto a scale that is itself unrestricted. Back-transforming through the inverse logit (the logistic function) then guarantees predicted probabilities remain in [0, 1].

SECTION 4

Mathematical Framework

The mathematics connecting probability, odds, and log-odds can be encapsulated in a small set of interrelated equations. These equations form the backbone of logistic regression and provide the foundation for interpreting model coefficients as changes in log-odds. We begin with the definition of odds and derive the logit function, then present the inverse logistic function that converts log-odds back to probabilities.

ODDS DEFINITION

Odds = p / (1 − p)

Where p = P(Y = 1), the probability of the event. When p = 0.5, odds = 1 (equal chance). When p > 0.5, odds > 1; when p < 0.5, odds < 1.

LOGIT (LOG-ODDS) FUNCTION

logit(p) = ln(p / (1 − p)) = β₀ + β₁X₁ + β₂X₂ + ⋯ + βₖXₖ

The logit maps p ∈ (0, 1) to the real line (−∞, +∞). In a logistic regression, we set the logit equal to a linear combination of predictors. Each coefficient βⱼ represents the change in log-odds for a one-unit increase in Xⱼ, holding other predictors constant.

INVERSE LOGIT (LOGISTIC FUNCTION)

p = 1 / (1 + e^(−(β₀ + β₁X₁ + ⋯ + βₖXₖ)))

This is the sigmoid function that maps any real-valued linear predictor back to a probability between 0 and 1. The function is symmetric about p = 0.5, which occurs when the linear predictor equals zero.

ODDS RATIO INTERPRETATION

OR = e^(βⱼ)

Exponentiating a logistic regression coefficient yields the odds ratio (OR). An OR > 1 indicates that higher Xⱼ is associated with increased odds of the event; OR < 1 indicates decreased odds; OR = 1 indicates no association.

💡 Why Not Just Model Probability Directly?

A linear model P(Y = 1) = β₀ + β₁X can produce predicted probabilities less than 0 or greater than 1 for extreme values of X. The logit link function is a bijective mapping from (0, 1) to ℝ, ensuring that any real-valued linear predictor maps to a valid probability via the inverse logit. This is the key insight that makes logistic regression mathematically coherent.

SECTION 5

The Logistic (Sigmoid) Curve in Detail

The logistic curve (also called the sigmoid curve) is the graphical embodiment of the inverse logit function. It shows how predicted probability changes as the linear predictor η = β₀ + β₁X varies across the real line. Understanding the curve's anatomy—its asymptotes, inflection point, and the steepness governed by the coefficient magnitude—is essential for interpreting logistic regression output in clinical and epidemiological contexts.

The sigmoid curve maps linear predictor values (x-axis) to probabilities (y-axis). At the inflection point (η = 0), p = 0.5 and the curve is steepest. The curve flattens near the asymptotes at 0 and 1, meaning very large or very small linear predictors produce probabilities that change slowly.

A crucial consequence of the sigmoid shape is that the effect of a one-unit change in X on the predicted probability depends on where you are on the curve. Near the extremes (p close to 0 or 1), a one-unit increase in X barely shifts the probability. Near p = 0.5, the same one-unit increase produces the maximum probability change, approximately β₁/4. This contrasts with a linear probability model, where the effect is constant everywhere, and it explains why logistic regression coefficients are interpreted on the log-odds scale rather than the probability scale: on the logit scale, the effect of a one-unit change in Xⱼ is always βⱼ, regardless of baseline probability.

Correspondence between probability, odds, and log-odds for selected values

Probability (p)	Odds [p/(1−p)]	Log-Odds [ln(odds)]
0.01	0.0101	−4.595
0.10	0.111	−2.197
0.25	0.333	−1.099
0.50	1.000	0.000
0.75	3.000	1.099
0.90	9.000	2.197
0.99	99.000	4.595

SECTION 6

Worked Example — Interpreting a Logistic Regression Coefficient

Suppose a biostatistician fits a logistic regression model predicting 30-day mortality (Y = 1 if dead, 0 if alive) among ICU patients using a single predictor, APACHE II score (X, ranging from 0 to 71). The estimated model is: logit(p) = −3.50 + 0.12 × APACHE. We wish to interpret the coefficient, compute the predicted probability for a patient with APACHE = 25, and calculate the corresponding odds ratio.

Predicting 30-Day Mortality from APACHE II Score

Step 1 — Identify the Model ComponentsThe estimated intercept is β₀ = −3.50 and the slope for APACHE is β₁ = 0.12. The model equation is: logit(p) = −3.50 + 0.12 × X. The coefficient β₁ = 0.12 means that each one-point increase in APACHE II score increases the log-odds of 30-day mortality by 0.12, holding everything else constant.

Step 2 — Compute the Linear Predictor for APACHE = 25Substitute X = 25 into the linear predictor: η = −3.50 + 0.12 × 25 = −3.50 + 3.00 = −0.50. This value of −0.50 represents the estimated log-odds of death for a patient with an APACHE II score of 25.

η = −0.50 (log-odds)

Step 3 — Convert Log-Odds to OddsExponentiate the log-odds to obtain the odds: Odds = e^(−0.50) = 0.6065. This means that for every 1 patient who dies within 30 days, approximately 1.65 survive (or equivalently, the odds against death are about 1.65 to 1).

Odds = 0.607

Step 4 — Convert Odds to ProbabilityApply the inverse logit: p = 1 / (1 + e^(0.50)) = 1 / (1 + 1.6487) = 1 / 2.6487 ≈ 0.3775. Alternatively, p = Odds / (1 + Odds) = 0.6065 / 1.6065 ≈ 0.3775. The predicted probability of 30-day mortality for a patient with APACHE = 25 is approximately 37.8%.

p ≈ 0.378 (37.8% mortality risk)

Step 5 — Compute and Interpret the Odds RatioThe odds ratio for a one-unit increase in APACHE is OR = e^(β₁) = e^(0.12) ≈ 1.1275. This means that each additional point on the APACHE II scale is associated with a 12.75% multiplicative increase in the odds of 30-day mortality. For a 10-point increase, the cumulative odds ratio would be e^(0.12 × 10) = e^(1.2) ≈ 3.32, indicating a more than threefold increase in odds.

OR = 1.128 per APACHE point

SECTION 7

Strengths & Limitations of the Log-Odds Framework

The logit transformation is an elegant solution to the binary outcome modeling problem, but like all modeling choices, it carries trade-offs. Understanding these strengths and limitations is critical for selecting the right approach in a given research context and for communicating findings accurately to clinical audiences.

Strengths	Limitations
Predicted probabilities are always bounded within (0, 1), unlike the linear probability model.	Log-odds are not intuitive to non-statistical audiences; clinicians often struggle to interpret raw coefficients without exponentiation.
The odds ratio provides a single, unitless summary of effect size that is invariant to the baseline probability.	Odds ratios can overestimate relative risk when the event is common (prevalence > 10%), leading to potential misinterpretation.
The logit link is the canonical link for the Bernoulli distribution, maximizing computational efficiency in estimation.	The model assumes a linear relationship on the logit scale; violations of this assumption can produce biased coefficients.
Coefficients have a consistent interpretation as the change in log-odds per unit change in X, regardless of covariate values.	With rare events or small samples, maximum likelihood estimation can produce inflated coefficient estimates (separation/quasi-complete separation).

✦ KEY TAKEAWAY

The log-odds transformation is to binary outcome modeling what the square root transformation is to stabilizing variance in count data—a mathematically motivated re-expression that unlocks the power of linear methods. However, just as you would report counts rather than square roots in a final presentation, you should always convert log-odds back to odds ratios or predicted probabilities when communicating with clinical collaborators. The logit is the engineer's workspace; the probability is the client's deliverable.

SECTION 8

Connections to Advanced Topics

The binary logistic regression model sits at a foundational node in a web of more advanced techniques. Understanding log-odds interpretation prepares you for extensions that handle more complex data structures, including ordered and unordered multi-category outcomes, correlated observations, and time-to-event data.

Binary Logistic Regression	Advanced Extension	Key Difference
Single binary outcome (Y = 0 or 1)	Multinomial logistic regression	Extends to ≥ 3 unordered outcome categories; models log-odds of each category relative to a reference.
Independent observations	Mixed-effects logistic regression	Adds random effects (e.g., random intercepts for hospitals) to handle clustered or longitudinal binary data.
Logit link function	Probit regression	Uses the inverse standard normal CDF (Φ⁻¹) as the link function instead of the logit; coefficients are on the z-score scale.
Probability of event at a fixed time	Cox proportional hazards	Models the log hazard rate rather than log-odds, accommodating varying follow-up times and censoring.

All of these extensions share a common structural principle with binary logistic regression: they apply a link function to transform a bounded or constrained response into an unrestricted scale, then model that transformed quantity as a linear function of predictors. The logit link you have learned here is the prototype. In generalized linear model (GLM) theory, the logistic regression model is a GLM with a Bernoulli distribution and a logit link—a framing that unifies binary, count, and continuous outcome models under a single theoretical umbrella.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

Explain in your own words why we cannot simply use ordinary least squares (OLS) linear regression to model a binary outcome. What specific problems arise, and how does the logit transformation address them?

PROBLEM 2 — BASIC CALCULATION

A disease has a prevalence of 20% in a population. Compute the odds of disease and the log-odds (logit) of disease.

PROBLEM 3 — INTERMEDIATE

A logistic regression model predicting hospital readmission yields the equation logit(p) = −2.00 + 0.50 × (number of comorbidities). (a) What is the odds ratio for each additional comorbidity? (b) What is the predicted probability of readmission for a patient with 4 comorbidities? (c) By how much do the odds of readmission increase when moving from 2 to 4 comorbidities?

PROBLEM 4 — APPLIED

A clinical study reports a logistic regression model for developing Type 2 diabetes: logit(p) = −6.00 + 0.08 × BMI + 0.05 × Age. (a) Interpret both coefficients in terms of odds ratios. (b) A 55-year-old patient with BMI = 32 asks about their estimated diabetes risk. Compute the predicted probability. (c) How would the predicted probability change if the patient lost enough weight to reduce their BMI to 27?

PROBLEM 5 — CRITICAL THINKING

A colleague reports an odds ratio of 3.5 for the association between a biomarker and one-year mortality, where the baseline mortality rate in the study population is 40%. They claim this means 'exposed patients are 3.5 times as likely to die.' Critically evaluate this claim. Under what conditions is the odds ratio a good approximation of the relative risk, and does that condition hold here? Compute the actual relative risk from the available information to support your argument.

SUMMARY

Summary — Binary Outcomes & Log-Odds

Binary outcomes are response variables that take only two values (0 or 1), appearing throughout biomedical research as disease status, survival, treatment response, and test results. Because direct linear modeling of a probability can yield impossible predictions outside [0, 1], we transform the probability first into odds (p / (1 − p), bounded below by 0) and then into log-odds (ln[p / (1 − p)], spanning the entire real line). This logit transformation is the link function in logistic regression, enabling us to model a linear predictor β₀ + β₁X₁ + ⋯ that is unrestricted in value while guaranteeing valid predicted probabilities via the inverse logit (sigmoid) function.

Each coefficient βⱼ in a logistic model represents the change in log-odds per one-unit increase in Xⱼ. Exponentiating yields the odds ratio (OR = e^βⱼ), which is the most commonly reported effect measure in clinical research. Remember that the OR approximates the relative risk only when the outcome is rare. This foundational framework extends naturally to multinomial logistic regression, mixed-effects models, and the broader generalized linear model framework.