Introduction to Optimization

Imagine you’re trying to pack a suitcase for a vacation. You have a limited amount of space, but you want to fit in everything essential — clothes, toiletries, maybe even a few luxury items like a book or a camera. Your goal? To maximize utility while minimizing wasted space. Congratulations, you’ve just encountered optimization in its simplest form!

Optimization Idea: Finding the Best Possible Solution

At its heart, optimization is about making the best possible choice from a set of alternatives. Whether it's minimizing costs, maximizing profits, reducing energy consumption, or increasing efficiency, optimization helps us find the most effective way to reach our goals.

In mathematical terms, we define an optimization problem as:

min f (x), where x \in Ω \subseteq R^{n}

Let’s break this down:

$x$ represents the variables we can control (like how much of each item to pack).
$f (x)$ is the objective function — this is what we’re trying to minimize (or maximize).
$Ω$ (omega) is the feasible region, the set of all possible choices we can make. Basically, it’s something about restrictions on what we can choose.

Remark

$f (x) : R^{n} \to R$ . In simple terms, it’s a function that takes in a set of variables $x$ and gives us a single number as output.
For more deep understanding, you can refer to this page: Optimization Formula

Constraints vs. No Constraints

Optimization problems fall into two broad categories:

Unconstrained optimization

The choice space is the entire set of real numbers $R^{n}$ .
This is like packing a suitcase without worrying about weight limits — you can just throw everything in.

Ω = R^{n}

Constrained optimization

The feasible region is a subset of $R^{n}$ , meaning there are limitations on what we can choose.
This is like trying to pack within an airline’s luggage weight and size restrictions.

Ω \subset R^{n}

The Magic of Maximization and Minimization

Here’s a cool trick: maximization can always be turned into minimization. If we want to maximize $f (x)$ , we can simply minimize $- f (x)$ instead:

max f (x) = - min (- f (x))

It’s like looking at a mountain from the perspective of a valley — the peaщk of the mountain is just the lowest point of the inverted landscape!