Epigraph of a Function. Jensen’s Inequality

Epigraph of a convex function

To understand convexity deeply, we look at the epigraph of a function.

We already know this:
The graph of a function $f (x) : R^{n} \to R$ is the set of all points $(x, f (x))$ in the domain of $f$ . It's defined as:

{(x, f (x)) ∣ x \in D (f)}

where $D (f)$ is the domain of $f$ .

What is the epigraph?
The epigraph of a function $f$ is the set of all points that lie on or above its graph:

epi (f) = {(x, α) ∣ x \in D (f), α \geq f (x)}

Epigraphs.png|600

Key property:
A function is convex if and only if its epigraph is a convex set.

This means that if you look at the "shadow" or "area above" the curve, it has no dents or dips — just a smooth "roof" shape.

Jensen’s inequality: The fingerprint of convexity

Recall

For convex functions, the graph lies below every chord between points:
$f (λ x_{1} + (1 - λ) x_{2}) \leq λ f (x_{1}) + (1 - λ) f (x_{2})$

Jensen’s inequality is a powerful rule that applies to all convex functions. It says:

f (\sum_{i = 1}^{n} λ_{i} x_{i}) \leq \sum_{i = 1}^{n} λ_{i} f (x_{i})

where:

$λ_{i} \geq 0$ for all $i$ (weights are non-negative)
$\sum_{i = 1}^{n} λ_{i} = 1$ (weights sum to 1)
$x_{i}$ are input with index $i$ for the function $f$ . $x_{i}$ can be vectors of any dimension.

What does Jensen’s inequality mean?
It means that if you average inputs before applying the function, the result is always better (or equal) than averaging outputs after applying the function.

So, previously we had $i = 2$ and formula was:

\begin{aligned} f (λ x_{1} + (1 - λ) x_{2}) & \leq λ f (x_{1}) + (1 - λ) f (x_{2}) \\ f (\sum_{i = 1}^{2} λ_{i} x_{i}) & \leq \sum_{i = 1}^{2} λ_{i} f (x_{i}) \end{aligned}

Operations that preserve convexity

Convexity is stable — it is not easy to "break" a convex function. Here are two rules that help build new convex functions from old ones:

Sum of convex functions is convex:
If $f_{1}, f_{2}, \dots, f_{k}$ are convex and $λ_{1}, λ_{2}, \dots, λ_{k} \geq 0$ , then:
$f (x) = \sum_{i = 1}^{k} λ_{i} f_{i} (x) is convex$
Why? The "average" of several bowls is still a bowl.
Composition with affine functions is convex:
If $f$ is convex and $g (x) = A x + b$ is an affine function, then:
$h (x) = f (g (x)) = f (A x + b) is convex$
Why? Affine transformations only "tilt" or "shift" the bowl — they cannot create dents or dips.

Summary of key ideas about convex functions

Concept	Meaning
Convex function	Graph lies below every chord between points.
Strictly convex function	Graph lies strictly below chords — ensures a unique minimum.
Epigraph	The region above the graph — convex if the function is convex.
Jensen’s inequality	Blending inputs is always better than blending outputs.
Gradient condition	Tangent lines stay below the graph.
Hessian condition	Positive curvature (Hessian positive semidefinite).
Convexity-preserving operations	Sums and affine transformations preserve convexity.

Final intuition: Why convex functions are powerful

Think of a bowl-shaped landscape:

Anywhere you drop a ball, it rolls down to the same lowest point — the global minimum.
There are no traps or false valleys, just one solution.
The path to the lowest point is always the fastest and shortest.

That’s why optimization problems love convex functions — they guarantee a clear, simple route to the best solution. 🚀