Extrema & Saddle Points — Maxima/Minima and Saddle Points...

SECTION 1

Historical Context & Motivation

The problem of finding maxima and minima has ancient roots: Greek geometers such as Apollonius sought the shortest distances to conic sections, and Euclid recognized that a square maximizes the area of a rectangle with a fixed perimeter. However, extending optimization to functions of more than one variable required entirely new mathematical machinery. The development of partial derivatives and the formal classification of critical points in several variables unfolded over the eighteenth and nineteenth centuries, driven by both pure mathematical curiosity and the practical demands of physics, engineering, and economics.

1760s

Lagrange's Early Work on Extrema

Joseph-Louis Lagrange systematically studied conditions for maxima and minima of functions of several variables, laying the groundwork for the necessary conditions involving first-order partial derivatives vanishing simultaneously.

1810s

Gauss and Curvature of Surfaces

Carl Friedrich Gauss investigated the curvature of surfaces, connecting the behavior of quadratic forms (built from second-order partial derivatives) to the geometric shape of a surface near a point — an insight central to the second derivative test.

1830s

Hessian Matrix Introduced

Ludwig Otto Hesse formalized the matrix of second-order partial derivatives — now called the Hessian matrix — providing a compact algebraic object whose determinant governs the classification of critical points.

1860s–1900s

Weierstrass and Rigorous Foundations

Karl Weierstrass and his school placed the theory of extrema on rigorous ε-δ foundations, distinguishing local from global extrema and clarifying the role of higher-order conditions when the second derivative test is inconclusive.

20th Century

Modern Optimization Theory

The classification of critical points expanded into fields such as Morse theory, nonlinear programming, and machine learning, where the Hessian serves as the basis for algorithms like Newton's method and informs concepts like loss-surface geometry in deep neural networks.

The fundamental question that drives this lesson is deceptively simple: given a function f(x, y) whose first-order partial derivatives both equal zero at a point (a, b), how can we determine whether that point corresponds to a local maximum, a local minimum, or a saddle point? The answer lies in examining second-order information — specifically, the signs and relationships among the second partial derivatives evaluated at that critical point.

SECTION 2

Core Principles & Definitions

Before we can classify critical points, we need to establish a clear vocabulary and a set of foundational ideas. The theory rests on extending single-variable calculus concepts — specifically the first and second derivative tests — into the multivariable setting, where the interplay between partial derivatives in different directions introduces subtleties that have no one-dimensional analogue.

Critical Point

A point (a, b) in the domain of f(x, y) where both f_x(a, b) = 0 and f_y(a, b) = 0, or where one or both partials fail to exist. At such a point, the tangent plane (if it exists) is horizontal.

Local Maximum

A critical point (a, b) is a local maximum if f(a, b) ≥ f(x, y) for all (x, y) in some open disk centered at (a, b). The surface curves downward in every direction from this peak.

Local Minimum

A critical point (a, b) is a local minimum if f(a, b) ≤ f(x, y) for all (x, y) in some open disk centered at (a, b). The surface curves upward in every direction from this valley.

Saddle Point

A critical point where the function increases in some directions and decreases in others — resembling the shape of a horse saddle or a mountain pass. It is neither a local max nor a local min.

The Hessian Determinant (D)

The quantity D = f_xx · f_yy − (f_xy)², evaluated at the critical point, determines whether the second-order behavior is definite (max or min) or indefinite (saddle).

✦ KEY TAKEAWAY

Think of standing at the top of a hill: if you walk in any direction you go downhill, it is a local max. If every direction is uphill, it is a local min. But imagine standing at the center of a Pringles chip — along the chip's length you descend, and across its width you ascend. That 'mixed curvature' scenario is exactly a saddle point. The second derivative test uses the Hessian determinant D to detect whether the curvatures in orthogonal directions agree (D > 0, giving max or min) or conflict (D < 0, giving a saddle).

SECTION 3

Visual Explanation — Surfaces & Critical Points

A powerful way to build intuition is to visualize the surface z = f(x, y) in three dimensions and observe how it curves near a critical point. The following diagram presents three canonical scenarios side by side: a local minimum (a bowl opening upward), a local maximum (a bowl opening downward), and a saddle point (a hyperbolic paraboloid). In each case the tangent plane at the critical point is horizontal, yet the surrounding surface behavior differs dramatically.

Three canonical critical-point types for z = f(x, y). Left: a local minimum where the surface curves upward in all directions (D > 0, f_xx > 0). Center: a local maximum where the surface curves downward in all directions (D > 0, f_xx < 0). Right: a saddle point where one cross-section curves up and another curves down (D < 0).

Notice that in the local minimum panel, every cross-sectional slice through the critical point produces a parabola opening upward — the curvature is consistently positive in all directions. In the local maximum panel, every slice yields a downward-opening parabola. At the saddle point, the x-direction slice and the y-direction slice have opposite concavities — the curvatures 'disagree,' and this disagreement is precisely what the Hessian determinant D detects. When D < 0, the eigenvalues of the Hessian have opposite signs, confirming the indefinite nature of the quadratic form.

SECTION 4

Mathematical Framework — The Second Derivative Test

We now formalize the classification procedure. Suppose f(x, y) has continuous second-order partial derivatives in an open region containing a critical point (a, b) — that is, f_x(a, b) = 0 and f_y(a, b) = 0. The behavior of f near (a, b) is governed by its second-order Taylor approximation, which is determined by the Hessian matrix evaluated at (a, b).

HESSIAN MATRIX

H = [ f_xx f_xy ] [ f_yx f_yy ]

f_xx = ∂²f/∂x², f_yy = ∂²f/∂y², and f_xy = f_yx = ∂²f/(∂x ∂y) (by Clairaut's theorem, when these partials are continuous).

DISCRIMINANT (HESSIAN DETERMINANT)

D = f_xx · f_yy − (f_xy)²

D is the determinant of the 2×2 Hessian matrix. It equals the product of the eigenvalues of H and captures whether the quadratic form is definite or indefinite.

Classification Rules

With D computed at the critical point (a, b), the second derivative test states the following.

If D > 0 and f_xx > 0, then (a, b) is a local minimum.
If D > 0 and f_xx < 0, then (a, b) is a local maximum.
If D < 0, then (a, b) is a saddle point, regardless of the sign of f_xx.
If D = 0, the test is inconclusive — higher-order analysis or other methods are needed.

SECOND-ORDER TAYLOR EXPANSION (MOTIVATION)

f(a+h, b+k) ≈ f(a,b) + ½(f_xx h² + 2f_xy hk + f_yy k²)

Because f_x = f_y = 0 at (a, b), the first-order terms vanish, and the sign of the change f(a+h, b+k) − f(a, b) depends entirely on the quadratic form Q(h, k) = f_xxh² + 2f_xyhk + f_yyk². The discriminant D determines whether Q is positive definite, negative definite, or indefinite.

💡 Why f_xx and not f_yy?

When D > 0, f_xx and f_yy necessarily have the same sign (since D = f_xxf_yy − (f_xy)² > 0 implies f_xxf_yy > 0, so their signs match). Therefore you may equivalently check f_yy instead of f_xx — the result is the same.

SECTION 5

Detailed Classification & Decision Flowchart

In practice, classifying a critical point is a systematic procedure. The following flowchart diagram encodes the entire decision process: compute the partial derivatives, find critical points, evaluate the Hessian quantities, and classify. Keeping this flowchart in mind prevents the common errors of forgetting to check both partial derivatives equal zero or misinterpreting the sign conditions on D and f_xx.

Decision flowchart for the second derivative test. Start at the top by computing partial derivatives, then follow the branches based on the sign of the discriminant D and, if D > 0, the sign of f_xx.

Summary of the Second Derivative Test outcomes

Condition	Classification	Geometric Meaning
D > 0, f_xx > 0	Local minimum	Surface curves upward in all directions; bottom of a bowl.
D > 0, f_xx < 0	Local maximum	Surface curves downward in all directions; top of a dome.
D < 0	Saddle point	Surface curves up in one direction and down in another; mountain pass.
D = 0	Inconclusive	Could be any type; higher-order terms dominate the behavior.

SECTION 6

Worked Example

Let us apply the full procedure to a concrete function. Consider f(x, y) = x³ + y³ − 3xy. We will find all critical points and classify each one using the second derivative test.

Classify the critical points of f(x, y) = x³ + y³ − 3xy

Step 1 — Compute the first partial derivativesDifferentiate f with respect to x and y separately. We obtain f_x = 3x² − 3y and f_y = 3y² − 3x.

f_x = 3x² − 3y, f_y = 3y² − 3x

Step 2 — Solve for critical pointsSet f_x = 0: 3x² − 3y = 0, so y = x². Set f_y = 0: 3y² − 3x = 0, so x = y². Substituting y = x² into x = y²: x = (x²)² = x⁴, giving x⁴ − x = 0, i.e., x(x³ − 1) = 0. Thus x = 0 or x = 1.

Critical points: (0, 0) and (1, 1)

Step 3 — Compute the second partial derivativesDifferentiating again: f_xx = 6x, f_yy = 6y, and f_xy = −3.

f_xx = 6x, f_yy = 6y, f_xy = −3

Step 4 — Evaluate D at (0, 0)At (0, 0): f_xx = 0, f_yy = 0, f_xy = −3. So D = (0)(0) − (−3)² = 0 − 9 = −9.

D = −9 < 0 → Saddle point at (0, 0)

Step 5 — Evaluate D at (1, 1)At (1, 1): f_xx = 6, f_yy = 6, f_xy = −3. So D = (6)(6) − (−3)² = 36 − 9 = 27. Since D > 0 and f_xx = 6 > 0, the critical point is a local minimum.

D = 27 > 0, f_xx = 6 > 0 → Local minimum at (1, 1)

Step 6 — State function valuesf(0, 0) = 0 and f(1, 1) = 1 + 1 − 3 = −1. The local minimum value is −1, occurring at the point (1, 1).

f(1, 1) = −1 (local min value)

SECTION 7

Strengths & Limitations of the Second Derivative Test

The second derivative test is among the most frequently used tools in multivariable optimization, but it is important to understand both its power and its boundaries. Like any classification criterion built from finitely many derivatives, it cannot resolve every situation, and alternative strategies exist for the cases it cannot handle.

Strengths vs. limitations of the multivariable second derivative test

Strengths	Limitations
Provides a definitive classification (local max, local min, or saddle) in the generic case where D ≠ 0.	Fails when D = 0 — the degenerate case — which arises more often than one might expect, e.g., f(x, y) = x⁴ + y⁴ at the origin.
Computationally straightforward: requires only three second partial derivatives and simple arithmetic.	Only classifies local extrema; says nothing about global extrema, which require additional techniques such as comparison over the entire domain or on the boundary.
Connects naturally to the eigenvalue structure of the Hessian, offering deeper geometric insight into the curvature of the surface.	Assumes second-order partials are continuous (C² condition); if the function lacks this smoothness, the test may not apply.
Extends systematically to functions of n variables using the bordered Hessian or Sylvester's criterion for definiteness.	For constrained optimization problems, the unconstrained second derivative test is insufficient — the method of Lagrange multipliers with bordered Hessians is needed instead.

✦ KEY TAKEAWAY

Think of the second derivative test as a powerful but finite-resolution lens: it resolves the character of a critical point whenever the curvature is 'strong enough' to dominate the higher-order terms (D ≠ 0), much like how a seismograph reliably classifies earthquakes above a certain magnitude but becomes inconclusive for micro-tremors at the noise floor. When D = 0, you need a higher-resolution instrument — analogous to examining third- or fourth-order derivatives, or using specialized methods like completing the square on the Taylor expansion.

SECTION 8

Connection to Advanced Theory

The second derivative test for two variables is the gateway to a rich landscape of advanced optimization and geometry. Understanding how it generalizes prepares you for topics you will encounter in advanced calculus, linear algebra, differential geometry, and applied mathematics. The core idea — classifying a quadratic form by the eigenvalues of a symmetric matrix — scales elegantly to higher dimensions and more complex constraint structures.

From the 2D second derivative test to advanced generalizations

Two-Variable Setting (This Lesson)	General / Advanced Setting
2×2 Hessian matrix with determinant D = f_xx f_yy − (f_xy)²	n×n Hessian matrix; classification via eigenvalue signs or Sylvester's criterion (all leading principal minors positive ⇒ positive definite)
Saddle point when D < 0 (eigenvalues of opposite sign)	In n variables, the Hessian can be indefinite with various combinations of positive and negative eigenvalues — giving 'generalized saddle points' of different indices (Morse index)
Inconclusive when D = 0	Morse theory classifies 'non-degenerate' critical points (all eigenvalues nonzero); degenerate cases require catastrophe theory or blowup analysis
Unconstrained optimization only	Constrained optimization uses the bordered Hessian (Lagrange multipliers) or the projected Hessian onto the constraint manifold

In machine learning, the Hessian of a loss function with millions of parameters determines whether an algorithm is near a local minimum, a saddle point, or a plateau. Research has shown that saddle points proliferate exponentially in high-dimensional spaces, making them far more common than local minima. Algorithms such as Newton's method use the Hessian directly, while approximations like Adam and L-BFGS sidestep computing the full Hessian by estimating its action. The conceptual foundation you are building in this lesson — understanding how the second derivative test works for two variables — is the prototype for all of this advanced machinery.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

Explain in your own words why D < 0 implies a saddle point, regardless of the sign of f_xx. What does D < 0 tell you about the eigenvalues of the Hessian matrix?

PROBLEM 2 — BASIC CALCULATION

Find and classify all critical points of f(x, y) = x² + y² − 4x + 6y + 13.

PROBLEM 3 — INTERMEDIATE

Find and classify all critical points of f(x, y) = xy − x² − y² − 2x − 2y + 4.

PROBLEM 4 — APPLIED

A firm's profit is modeled by P(x, y) = 80x + 120y − x² − 2y² − xy, where x and y represent units of two products (in thousands). Find the production levels that maximize profit, and verify the result with the second derivative test.

PROBLEM 5 — CRITICAL THINKING

Consider the family of functions f(x, y) = x² + cxy + y² for a real parameter c. (a) Show that the origin is always a critical point. (b) Determine for which values of c the origin is a local minimum, a local maximum, or a saddle point. (c) What happens at the boundary values of c, and what does this tell you about the geometry of the level curves?

SUMMARY

Summary

To classify a critical point of a function f(x, y), first locate all points where f_x = 0 and f_y = 0 simultaneously. At each critical point, compute the three second partial derivatives f_xx, f_yy, and f_xy, then form the discriminant D = f_xx · f_yy − (f_xy)². If D > 0 with f_xx > 0, the point is a local minimum; if D > 0 with f_xx < 0, it is a local maximum; if D < 0, it is a saddle point; and if D = 0, the test is inconclusive.

The underlying mechanism is the Hessian matrix, whose eigenvalues determine whether the second-order Taylor approximation at the critical point defines a positive definite (min), negative definite (max), or indefinite (saddle) quadratic form. This two-variable test generalizes naturally to n dimensions via eigenvalue analysis or Sylvester's criterion, and it forms the conceptual backbone for constrained optimization via Lagrange multipliers and the bordered Hessian encountered in more advanced coursework.