Intro to Optimization — Introduction to Optimization...

SECTION 1

Historical Context & Motivation

Humans have always wanted to make the best use of limited resources. Ancient farmers wanted the most land from a fixed amount of fencing. Merchants wanted to ship goods in the fewest trips. These everyday concerns drove mathematicians to develop tools for finding optimal solutions — answers that maximize or minimize some quantity. The branch of calculus that tackles these questions is called optimization, and it remains one of the most powerful applications of derivatives today.

~300 BCE

Euclid's Extremal Problems

Greek mathematicians, including Euclid, explored geometric problems such as finding the shortest path between two points and the largest area enclosed by a fixed perimeter.

~50 CE

Heron's Reflection Principle

Heron of Alexandria proved that light takes the shortest path when reflecting off a surface, foreshadowing the idea that nature itself optimizes.

1665–1687

Newton & Leibniz Develop Calculus

Isaac Newton and Gottfried Wilhelm Leibniz independently invented calculus, providing systematic methods to find maxima and minima using derivatives.

1744

Euler's Calculus of Variations

Leonhard Euler extended optimization from single-variable functions to entire curves and surfaces, laying groundwork for modern engineering and physics.

Present

Optimization Everywhere

Today, optimization techniques built on calculus are used in machine learning, logistics, finance, medicine, and virtually every field of science and engineering.

The central question optimization addresses is deceptively simple: given certain constraints, what value of a variable produces the largest or smallest result for a quantity we care about? Calculus, and specifically the derivative, gives us a precise and elegant way to answer this question.

SECTION 2

Core Principles & Definitions

Before diving into calculations, you need a clear vocabulary for optimization. Every optimization problem has the same underlying structure: you have a quantity you want to maximize or minimize, you have one or more variables you can control, and you often have constraints that limit your choices. Understanding these pieces is the key to setting up any problem correctly.

Objective Function

The formula that represents the quantity you want to optimize. For instance, if you want to maximize the area of a garden, the objective function is A = l × w.

Decision Variable

The variable whose value you adjust to achieve the best outcome. You search for the specific value of this variable that makes the objective function hit its maximum or minimum.

Constraint

A condition that restricts the decision variable. Common constraints include a fixed perimeter, a limited budget, or a required volume. These are expressed as equations or inequalities.

Feasible Domain

The set of all values the decision variable can actually take, given the constraints. You only look for your optimum inside this domain — values outside it are physically or mathematically impossible.

Critical Points

Values where the derivative of the objective function equals zero or is undefined. These are the candidate locations where a maximum or minimum can occur.

✦ KEY TAKEAWAY

Think of an optimization problem like planning the perfect road trip. Your objective function is what you want to optimize — say, minimize total driving time. Your decision variable is the route you choose. Your constraints are things like 'I must stop for gas' and 'the highway has a speed limit.' You can't just teleport — you have to work within your constraints to find the best possible route.

SECTION 3

Visualizing Optimization

The graph below shows a typical objective function on a closed interval. Notice how the function rises, reaches a peak, dips, and rises again. The absolute maximum is the highest point on the entire interval, while the absolute minimum is the lowest. At the critical points where the tangent line is horizontal, the derivative equals zero. But notice that not every critical point is the absolute best — you must also check the endpoints of the feasible domain.

The curve shows an objective function f(x) on the closed interval [a, b]. The absolute maximum occurs at critical point c₁, and the absolute minimum occurs at c₃. Notice that both endpoints must also be checked when searching for extrema on a closed interval.

This visual makes an important point: the derivative tells you where to look, but you still need to compare values. At every critical point (where f′(x) = 0) and at every endpoint of the feasible domain, plug the x-value back into f(x). The largest output is your absolute maximum; the smallest is your absolute minimum. This process is sometimes called the Closed Interval Method.

SECTION 4

Mathematical Framework

Setting up an optimization problem mathematically follows a consistent pattern. You translate the real-world scenario into functions, use the constraint to reduce to one variable, and then apply calculus to find the optimum. Here are the key equations and ideas you will use over and over.

GENERAL OPTIMIZATION STRATEGY

Find x such that f(x) is maximized or minimized, subject to g(x) = k

f(x) = objective function (what you optimize), g(x) = k is the constraint equation that limits your choices.

CRITICAL POINT CONDITION

f′(x) = 0 or f′(x) is undefined

Set the first derivative of the objective function (in one variable) equal to zero. Solutions are critical points — candidates for the maximum or minimum.

SECOND DERIVATIVE TEST

If f″(c) > 0, then f has a local minimum at x = c. If f″(c) < 0, then f has a local maximum at x = c.

The sign of the second derivative at a critical point tells you whether the curve is concave up (minimum) or concave down (maximum) at that point.

CLOSED INTERVAL METHOD

Compare f(a), f(c₁), f(c₂), …, f(b) — the largest is the absolute max, the smallest is the absolute min.

On a closed interval [a, b], evaluate f at every critical point inside the interval and at both endpoints. The largest value wins for the maximum; the smallest wins for the minimum.

⚠️ Important Note

In many optimization word problems, you start with two variables (like length and width). The constraint lets you eliminate one variable so that your objective function depends on only one variable. This step — substitution using the constraint — is critical and often where students make errors.

SECTION 5

The Step-by-Step Setup Process

The hardest part of optimization is not the calculus — it is the setup. Translating a word problem into a function you can differentiate requires a systematic approach. The flowchart below shows the five steps you should follow every single time.

The five-step optimization setup process. Each step builds on the last: start with a diagram, identify what to optimize, write the constraint, substitute to get one variable, then differentiate and solve.

Step 1 — Read & Draw: Sketch the physical situation. Label dimensions with variables (x, y, r, h, etc.). This visual will guide every equation you write.
Step 2 — Identify the Objective: Write the quantity to optimize as a formula. Ask yourself: am I looking for maximum area? Minimum cost? Shortest distance?
Step 3 — Write the Constraint: Translate the restriction into an equation. If you have 200 feet of fencing, then 2l + 2w = 200.
Step 4 — Reduce to One Variable: Solve the constraint for one variable and substitute into the objective function. Now you can differentiate.
Step 5 — Differentiate & Solve: Take the derivative, set it equal to zero, solve for the critical point, and verify it gives a max or min.

SECTION 6

Worked Example — Maximizing Area

Let's work through a classic optimization problem step by step. A farmer has 120 meters of fencing and wants to enclose the largest possible rectangular area against a straight river (so only three sides need fencing). What dimensions should the farmer choose?

Maximizing a Rectangular Area with a Constraint

Step 1 — Draw and LabelSketch a rectangle along a river. Let x be the length of the side parallel to the river, and let y be the length of each side perpendicular to the river. The river forms the fourth side, so it needs no fencing.

Step 2 — Write the Objective FunctionWe want to maximize the area: A = x × y. This is our objective function.

Step 3 — Write the ConstraintThe farmer has 120 m of fencing for three sides: one side of length x and two sides of length y. So the constraint is x + 2y = 120.

Step 4 — Reduce to One VariableSolve the constraint for x: x = 120 − 2y. Substitute into the area formula: A(y) = (120 − 2y) × y = 120y − 2y². Now the area depends only on y.

A(y) = 120y − 2y²

Step 5 — Differentiate and Set Equal to ZeroTake the derivative: A′(y) = 120 − 4y. Set it equal to zero: 120 − 4y = 0, so y = 30.

y = 30 meters

Step 6 — Find the Other DimensionSubstitute y = 30 back into x = 120 − 2y: x = 120 − 2(30) = 60.

x = 60 meters

Step 7 — Verify It's a MaximumCheck the second derivative: A″(y) = −4. Since A″ < 0, the function is concave down, confirming this critical point is a maximum. The maximum area is A = 60 × 30 = 1800 m².

Maximum area = 1800 m²

✅ Feasible Domain Check

Since y represents a physical length, y must be positive. Also, x = 120 − 2y must be positive, so y < 60. The feasible domain is 0 < y < 60. Our answer of y = 30 falls comfortably within this interval.

SECTION 7

Common Pitfalls & Tips

Optimization problems can trip you up in several ways. Below is a comparison of common mistakes alongside the correct approach. Keeping these in mind will save you time and frustration on homework and exams.

Common optimization mistakes and how to avoid them

Common Mistake	Why It's Wrong	Correct Approach
Forgetting to use the constraint	You can't differentiate a function of two variables (in Calculus 1). You need one variable only.	Always solve the constraint for one variable and substitute before differentiating.
Not checking the feasible domain	A critical point that gives a negative length or a width greater than the available material is physically meaningless.	Determine valid range for the variable (e.g., 0 < y < 60) and discard solutions outside it.
Assuming the critical point is automatically the answer	A critical point could be a local min when you want a max, or the absolute extremum might occur at an endpoint.	Use the Second Derivative Test or compare f-values at critical points and endpoints.
Confusing max and min problems	Setting up a correct equation but then finding the wrong type of extremum leads to a completely incorrect answer.	Re-read the problem after solving: does it ask for maximum or minimum? Does your second derivative confirm it?
Skipping the diagram	Without a picture, it's easy to misidentify which dimensions are constrained, leading to the wrong constraint equation.	Always draw and label the scenario before writing any equations.

✦ KEY TAKEAWAY

Think of the constraint as the rules of a game. You can't just ignore the rules and hope for the best score. The constraint keeps your answer realistic, and the derivative finds the best possible play within those rules. Skipping either step means you're not actually solving the problem.

SECTION 8

Connection to Advanced Optimization

The single-variable optimization you're learning right now is the foundation for a vast landscape of more advanced techniques. As problems grow more complex — more variables, more constraints, or functions without nice formulas — the methods evolve, but the core idea remains the same: find where the rate of change is zero and verify it's the kind of extremum you want.

Calculus 1 optimization versus advanced optimization

Feature	Calculus 1 Optimization	Advanced Optimization
Number of Variables	One (after substitution)	Two or more (multivariable calculus)
Constraint Handling	Direct substitution into the objective function	Lagrange multipliers (a method from Calculus 3)
Derivative Tool	Single derivative f′(x)	Partial derivatives and gradients
Verification	Second Derivative Test or endpoint comparison	Bordered Hessian matrix or numerical methods
Applications	Fencing, box design, shortest distance	Machine learning, rocket trajectories, economic models

The good news is that mastering the five-step setup process now will serve you in every future optimization course. Whether you're using Lagrange multipliers in Calculus 3 or gradient descent in a machine learning class, you'll always need to clearly identify the objective, the constraints, and the domain before you start computing.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

In your own words, explain why we set the derivative of the objective function equal to zero when solving an optimization problem. What does the derivative being zero tell us geometrically about the graph?

PROBLEM 2 — BASIC CALCULATION

A farmer has 80 meters of fencing to enclose a rectangular garden with all four sides fenced. Find the dimensions that maximize the area of the garden.

PROBLEM 3 — INTERMEDIATE

Find two positive numbers whose sum is 50 and whose product is as large as possible.

PROBLEM 4 — APPLIED

A company manufactures open-top boxes by cutting equal squares from each corner of a 24 cm × 24 cm sheet of cardboard and folding up the sides. What size squares should be cut to maximize the volume of the box?

PROBLEM 5 — CRITICAL THINKING

A lifeguard is on the beach, 60 meters from the nearest point on a straight shoreline. A swimmer is in distress in the water, located along the shore 80 meters from that nearest point. The lifeguard runs at 5 m/s on sand and swims at 3 m/s in water. If the swimmer is right at the water's edge, explain how you would set up (but do not fully solve) an optimization problem to minimize the lifeguard's total travel time. Identify the objective function, constraint, and feasible domain.

SUMMARY

Lesson Summary

Optimization is the process of finding the maximum or minimum value of a quantity, given certain restrictions. Every optimization problem has three key ingredients: an objective function (what you want to optimize), a constraint (a fixed condition that limits your choices), and a feasible domain (the set of valid inputs). You use the constraint to reduce the objective function to a single variable, then take the derivative and set it equal to zero to locate critical points.

To confirm whether a critical point is a maximum or minimum, use the Second Derivative Test or compare function values at critical points and endpoints using the Closed Interval Method. Always follow the five-step setup process: draw a diagram, identify the objective, write the constraint, reduce to one variable, and then differentiate and solve. Mastering this framework prepares you for increasingly complex optimization problems in future math, science, and engineering courses.