Transformations & Jacobians — Transformations of random...

SECTION 1

Historical Context & Motivation

The problem of determining the distribution of a function of a random variable is one of the oldest and most fundamental questions in probability theory. When early probabilists studied games of chance in the seventeenth and eighteenth centuries, they naturally encountered situations where the quantity of interest was not the original random outcome itself but some transformation of it — a sum of dice, a ratio of measurements, or a nonlinear function of an observable. The formal resolution of this problem required a tool from multivariable calculus: the Jacobian determinant, which tracks how volumes (and hence probability densities) stretch and compress under a change of variables. Understanding this bridge between calculus and probability is essential for deriving sampling distributions, constructing estimators, and performing Bayesian inference in modern statistics.

1841

Jacobi's Determinant

Carl Gustav Jacob Jacobi publishes his systematic treatment of functional determinants — now called Jacobians — providing the calculus machinery for multivariable change of variables in integrals.

1886

Helmert's Chi-Squared Derivation

Friedrich Robert Helmert uses the transformation technique to derive the distribution of the sum of squared standard normals, laying groundwork for the chi-squared distribution central to hypothesis testing.

1908

Student's t-Distribution

William Sealy Gosset ("Student") derives the t-distribution as a ratio transformation of normal and chi-squared random variables, demonstrating the power of the Jacobian method in small-sample inference.

1930s

Fisher & Multivariate Transformations

R. A. Fisher systematizes maximum likelihood estimation and uses Jacobian-based transformations to derive sampling distributions of estimators, including the F-distribution and variance-stabilizing transformations.

Modern Era

Normalizing Flows & Deep Generative Models

In machine learning, normalizing flows chain invertible transformations with tractable Jacobians to model complex distributions, showcasing the enduring relevance of this classical technique.

The central question this lesson addresses is deceptively simple: if X is a random variable (or random vector) with a known density, and Y = g(X) for some function g, what is the density of Y? The answer hinges on accounting for how g locally expands or contracts the probability mass — precisely the role of the Jacobian.

SECTION 2

Core Principles & Definitions

Before diving into formulas, it is important to build intuition for why a correction factor is needed at all. A probability density function is not a probability — it is a probability per unit of the variable. When you change variables, the "units" change, and the density must be rescaled so that total probability remains one. The Jacobian is precisely this rescaling factor, encoding the local rate at which the transformation stretches or compresses infinitesimal regions of the sample space.

Monotone Transformations

A monotone (one-to-one) transformation g has a unique inverse g⁻¹. The density of Y = g(X) is obtained by composing the original density with g⁻¹ and multiplying by the absolute value of the derivative of g⁻¹.

The Jacobian Determinant

For multivariate transformations Y = g(X), the Jacobian matrix J = [∂gᵢ/∂xⱼ] captures all partial derivatives. Its determinant |det J| measures the local volume distortion of the mapping.

Absolute Value Requirement

Because probability densities must be non-negative, we always take the absolute value of the Jacobian determinant. A negative determinant indicates an orientation reversal, not negative probability mass.

Support of the Transformed Variable

The support of Y is the image of the support of X under g. Correctly identifying this new support is essential — the transformed density is zero outside it.

Non-Injective Transformations

When g is not one-to-one (e.g., Y = X²), the support of X is partitioned into regions where g is monotone, and contributions from each branch are summed.

✦ KEY TAKEAWAY

Think of the Jacobian as a unit-conversion factor for probability densities. Just as converting speed from miles per hour to meters per second requires multiplying by a conversion ratio, converting a density from one set of variables to another requires multiplying by |det J⁻¹|. If the transformation stretches a region by a factor of 3, the density must shrink by a factor of 3 in that region so the total probability integral stays equal to one.

SECTION 3

Visual Explanation — How Transformations Reshape Densities

The left panel shows the original density f_X(x) in violet. Applying the transformation Y = g(X) reshapes and possibly shifts the density. The right panel (cyan) shows f_Y(y), where the Jacobian factor |J⁻¹| ensures the area under each curve integrates to one. Notice the peak position and shape have changed — reflecting both the nonlinearity of g and the compensating Jacobian.

The diagram above illustrates the core intuition: a transformation g acts on the random variable X to produce Y, and in doing so it warps the density. Where g stretches an interval (its derivative has large magnitude), the density of Y is compressed — the same probability is now spread over a wider interval. Conversely, where g compresses, the density concentrates. The Jacobian determinant quantifies this stretching and compression at every point, ensuring the fundamental axiom P(Ω) = 1 is maintained after the transformation.

SECTION 4

Mathematical Framework

Univariate Case: Monotone Transformation

Let X be a continuous random variable with density f_X(x) and support S_X. Suppose Y = g(X) where g is strictly monotone (either strictly increasing or strictly decreasing) and differentiable on S_X. Then g has a differentiable inverse g⁻¹, and the density of Y is given by the change-of-variable formula.

UNIVARIATE TRANSFORMATION FORMULA

f_Y(y) = f_X(g⁻¹(y)) · |d/dy [g⁻¹(y)]|

Here g⁻¹(y) maps back from the y-space to the x-space, and the absolute derivative |d[g⁻¹]/dy| is the univariate Jacobian. This factor equals 1/|g′(g⁻¹(y))|, the reciprocal of the original function's derivative evaluated at the pre-image.

Non-Injective Univariate Case

When g is not one-to-one, partition the support S_X into disjoint sets A₁, A₂, …, A_k on each of which g is monotone, with local inverses g₁⁻¹, g₂⁻¹, …, gₖ⁻¹. Then:

NON-INJECTIVE FORMULA

f_Y(y) = Σᵢ₌₁ᵏ f_X(gᵢ⁻¹(y)) · |d/dy [gᵢ⁻¹(y)]|

Each branch contributes a term. A classic example: if X ~ N(0,1) and Y = X², then g maps both x and −x to the same y, requiring two branches.

Multivariate Case: The Full Jacobian

Let X = (X₁, X₂, …, Xₙ)ᵀ be an n-dimensional continuous random vector with joint density f_X(x). Let Y = g(X), where g: ℝⁿ → ℝⁿ is a bijection with continuously differentiable inverse. The Jacobian matrix of the inverse transformation is the n × n matrix of partial derivatives.

JACOBIAN MATRIX

J = ∂(x₁, x₂, …, xₙ) / ∂(y₁, y₂, …, yₙ) where Jᵢⱼ = ∂xᵢ/∂yⱼ

The (i, j)-entry of J is the partial derivative of the i-th component of g⁻¹ with respect to the j-th component of y.

MULTIVARIATE CHANGE-OF-VARIABLE FORMULA

f_Y(y) = f_X(g⁻¹(y)) · |det J|

|det J| is the absolute value of the Jacobian determinant. It generalizes the univariate |dx/dy| to n dimensions, measuring how infinitesimal n-dimensional volumes transform under the mapping.

⚠ Convention Note

Some textbooks define J as the Jacobian of the forward transformation g (with entries ∂yᵢ/∂xⱼ) and write |det J|⁻¹ in the density formula. Others define J as the Jacobian of the inverse g⁻¹ (entries ∂xᵢ/∂yⱼ) and write |det J| directly. Both are equivalent. In this lesson, J denotes the Jacobian of the inverse transformation, so the formula uses |det J| without the exponent.

SECTION 5

Detailed Techniques & Common Transformations

In practice, the change-of-variable technique is applied through a systematic sequence of steps. First, express the inverse transformation explicitly. Second, compute the Jacobian matrix (or scalar derivative in one dimension). Third, evaluate the determinant and take its absolute value. Fourth, substitute into the density formula and identify the support of the new variable. The following diagram illustrates this workflow, along with a concrete bivariate example: the transformation from Cartesian to polar coordinates, one of the most frequently encountered transformations in probability.

The workflow at top shows the four-step Jacobian method. Below, the Cartesian-to-polar transformation is visualized: a point (x₁, x₂) is re-expressed as (r, θ). The inverse mapping x₁ = r cos θ, x₂ = r sin θ yields a 2 × 2 Jacobian matrix whose determinant is r. This is why the factor r appears in polar-coordinate integrals — it accounts for the stretching of area elements as one moves away from the origin.

Common Transformations in Probability

A reference table of common univariate and bivariate transformations with their Jacobian factors.

Transformation	Jacobian \|det J\|	Application
Y = aX + b (linear)	1/\|a\|	Location-scale families, standardization of normal RVs
Y = eˣ (exponential)	1/y for y > 0	Log-normal distribution derivation
Y = X² (square)	1/(2√y) per branch	Chi-squared distribution from standard normal
Y = −ln(X), X ~ Unif(0,1)	e⁻ʸ	Inverse CDF method: generates Exp(1) samples
(R, Θ) from (X₁, X₂) polar	r	Deriving Rayleigh, chi distributions; Box-Muller algorithm

SECTION 6

Worked Example — Deriving the Log-Normal Density

Suppose X ~ N(μ, σ²) and we define Y = eˣ. We wish to find the density of Y. This is a classic application of the univariate change-of-variable formula and it produces the log-normal distribution, which models positive-valued quantities such as asset prices, income distributions, and particle sizes.

Deriving f_Y(y) When Y = eˣ and X ~ N(μ, σ²)

Step 1 — Identify the Transformation and Its InverseThe transformation is g(x) = eˣ, which is strictly increasing on all of ℝ, so it is monotone and injective. The inverse is g⁻¹(y) = ln y, defined for y > 0. The support of Y is therefore (0, ∞).

g⁻¹(y) = ln y, support of Y: (0, ∞)

Step 2 — Compute the Jacobian (Derivative of the Inverse)In one dimension, the Jacobian is simply the derivative of the inverse function with respect to y. We have d/dy [ln y] = 1/y. Taking the absolute value: |d/dy [g⁻¹(y)]| = 1/y.

|dx/dy| = 1/y

Step 3 — Write the Original DensityThe density of X ~ N(μ, σ²) is f_X(x) = (1/(σ√(2π))) exp(−(x − μ)² / (2σ²)).

Step 4 — Substitute Into the Transformation FormulaApply f_Y(y) = f_X(g⁻¹(y)) · |dx/dy|. Substituting x = ln y and the Jacobian 1/y: f_Y(y) = (1/(σ√(2π))) × exp(−(ln y − μ)² / (2σ²)) × (1/y).

f_Y(y) = (1/(yσ√(2π))) exp(−(ln y − μ)² / (2σ²)) for y > 0

Step 5 — Verify and InterpretThis is the standard log-normal density LogN(μ, σ²). Notice the 1/y factor — it came directly from the Jacobian and is responsible for the characteristic right-skewed shape of the log-normal distribution. One can verify ∫₀^∞ f_Y(y) dy = 1 by substituting u = ln y and recovering the integral of the standard normal density.

SECTION 7

Strengths, Limitations & Alternative Methods

The Jacobian change-of-variable technique is powerful but not the only tool for finding distributions of transformed random variables. It is worth comparing it with alternative approaches — the CDF method and the moment-generating function (MGF) technique — to understand when each is most appropriate and where the Jacobian method excels or struggles.

Comparison of three methods for finding distributions of transformed random variables.

Criterion	Jacobian Method	CDF Method	MGF Technique
Input requirement	Explicit invertible transformation with differentiable inverse	Ability to express P(Y ≤ y) in terms of X	Closed-form MGF or characteristic function
Strength	Yields density directly in one step; generalizes cleanly to n dimensions	Works for non-invertible g; handles mixed discrete-continuous cases	Elegant for sums of independent RVs; bypasses explicit inversion
Limitation	Requires g to be (piecewise) invertible and differentiable	Requires differentiation of CDF, which may be intractable	Not all distributions have closed-form MGFs; uniqueness issues
Multivariate	Fully general via Jacobian determinant; handles joint densities	Becomes complex; requires careful region tracking	Mainly for sums; limited for general multivariate transforms
Typical use case	Polar/spherical transforms, ratio distributions, Box-Muller	Y = X², Y = max(X₁,…,Xₙ), order statistics	Sum of normals, convolutions, reproductive properties

✦ KEY TAKEAWAY

The Jacobian method is analogous to having a GPS that gives you turn-by-turn directions — it directly outputs the new density. The CDF method is more like having a compass and a map — you can always navigate, but you may need extra computation (differentiation) to get to the same destination. The MGF technique is like knowing the terrain from above — powerful for certain problems like sums of independent variables, but it requires a bird's-eye view (a closed-form MGF) that isn't always available. A well-rounded probabilist uses all three, choosing the tool that best fits the structure of the problem.

SECTION 8

Connections to Advanced Theory

The Jacobian transformation technique forms the conceptual backbone for several major areas of modern statistics and machine learning. Understanding how it generalizes illuminates deep structural connections across the mathematical sciences.

How the Jacobian technique connects to advanced topics in statistics, machine learning, and geometry.

Classical Concept	Advanced Extension	Key Insight
Jacobian for bijective g	Normalizing flows in deep learning	Chain K invertible layers: log f_Y = log f_X(g⁻¹(y)) + Σ log \|det Jₖ\|. Architectures are designed to make Jacobian computation O(n) instead of O(n³).
Finite-dimensional Jacobian	Radon–Nikodym derivative	In measure theory, \|det J\| is a special case of the Radon–Nikodym derivative dν/dμ, which generalizes 'density ratio' to arbitrary σ-finite measures.
Scalar \|det J\|	Area/coarea formula (geometric measure theory)	For maps between spaces of different dimensions (e.g., marginalizing out variables), the coarea formula replaces the Jacobian determinant with the appropriate co-dimensional volume factor.
Change of variables in integration	Reparameterization trick (variational inference)	In VAEs, the encoder outputs parameters of a simple distribution; samples are transformed via a differentiable g. Gradients flow through the transformation, enabled by the same Jacobian calculus.
1-to-1 smooth transformation	Diffeomorphisms on manifolds	Densities on Riemannian manifolds transform via \|det J\| times the ratio of metric determinants, extending the Jacobian to curved spaces.

These connections underscore that mastery of the Jacobian method is not merely a technical exercise — it is an investment in a language that pervades modern quantitative science. Whether you are deriving a posterior distribution in Bayesian inference, implementing a generative model, or studying geometric probability, the logic of tracking volume distortion under smooth mappings remains the same.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

Explain in your own words why the Jacobian determinant appears in the change-of-variable formula for densities. Why can't we simply substitute y = g(x) into f_X(x) and call the result f_Y(y)?

PROBLEM 2 — BASIC CALCULATION

Let X ~ Uniform(0, 1) and Y = −2 ln(X). Find the density of Y and identify the distribution by name.

PROBLEM 3 — INTERMEDIATE

Let X ~ Exp(1), so f_X(x) = e^(−x) for x > 0. Define Y = √X. Find f_Y(y) using the Jacobian method.

PROBLEM 4 — APPLIED

Let (X₁, X₂) be independent standard normal random variables. Define R = √(X₁² + X₂²) and Θ = arctan(X₂/X₁), the polar coordinate transformation. Using the multivariate Jacobian method, find the joint density f_{R,Θ}(r, θ) and show that R and Θ are independent.

PROBLEM 5 — CRITICAL THINKING

Let X have density f_X(x) = 2x for 0 < x < 1 (a Beta(2,1) distribution). Define Y = X². (a) Partition the support of X into regions where g(x) = x² is monotone. (b) Derive f_Y(y) using the Jacobian method. (c) Comment on whether a partition into multiple branches is actually needed in this case and explain why.

SUMMARY

Summary & Review

The transformation of random variables is a foundational technique that answers the question: if X has a known density and Y = g(X), what is the density of Y? The answer rests on the Jacobian determinant, which measures how the transformation locally stretches or compresses probability mass. In the univariate case, the formula f_Y(y) = f_X(g⁻¹(y)) × |d[g⁻¹]/dy| applies whenever g is monotone and differentiable; for non-injective transformations, one partitions the support into monotone branches and sums the contributions.

In the multivariate case, the scalar derivative generalizes to the Jacobian matrix of partial derivatives, and |det J| replaces |dx/dy|. This framework produces classical results like the log-normal, chi-squared, and Rayleigh distributions, and extends to modern applications in normalizing flows and variational inference. Mastery of the four-step workflow — define the transformation, find the inverse, compute the Jacobian, and substitute into the density formula — equips you with a versatile tool that appears across statistics, machine learning, and applied mathematics.