Projections

When we working with constrained optimization problems, the concept of projection plays a crucial role. It helps us find the closest point in a convex set to a given point outside the set.

The projection of a point onto a convex set is a fundamental concept in constrained optimization and mathematical analysis. It represents the closest point in a convex set to a given point outside the set.

Definition

Given a convex set $C$ (associate with Constrained Convex set), the projection of a point $y$ onto $C$ is denoted by $P_{C} (y)$ and is defined as:

P_{C} (y) = \arg min_{x \in C} ∥ x - y ∥^{2}

This means that $P_{C} (y)$ is the point in $C$ that is closest to $y$ in terms of Euclidean distance.

In simpler terms:

If $y$ is outside the convex set $C$ , the projection $P_{C} (y)$ is the nearest point on the boundary of $C$ .
If $y$ is inside the convex set, then the projection is simply $P_{C} (y) = y$ .

Properties of Projection

Let $C \subseteq R^{n}$ be a closed and convex set. The projection operator $P_{C} (y)$ satisfies the following properties:

1. Non-expansiveness

The projection operator is non-expansive, meaning that it does not increase distances:

∥ P_{C} (y) - P_{C} (z) ∥ \leq ∥ y - z ∥ \forall y, z \in R^{n} .

This ensures that the projection does not introduce additional stretching or distortion.

2. Idempotency

Applying the projection twice does not change the result:

P_{C} (P_{C} (y)) = P_{C} (y) \forall y \in R^{n} .

This means that once a point is projected onto the set, projecting it again does nothing — it remains in place.

3. Optimality Condition

The projection $P_{C} (y)$ is the closest point to $y$ in $C$ , meaning it satisfies the first-order optimality condition:

(y - P_{C} (y)) (x - P_{C} (y)) \leq 0 \forall x \in C, \forall y \in R^{n} .

This condition ensures that the difference vector $y - P_{C} (y)$ is orthogonal to the set $C$ at the projection point.

Visual Understanding

Projection 1.png
Here the distance between $x$ and $P_{C} (x)$ is minimized, making $P_{C} (x)$ the closest point in the convex set $C$ .

Projection 2.png
Here the inner product condition confirms that the projection direction is perpendicular to the tangent at $P_{C} (x)$ . Thus, the inner product is non-positive.

Remember:

This stuff: $(y - P_{C} (y)) (x - P_{C} (y))$ is the same as $⟨ y - P_{C} (y), x - P_{C} (y) ⟩$ .
Just a different way to write it 🤷‍♀️

These visualizations emphasize that projection always follows the shortest path to the set.

Summary

Property	Formula	Interpretation
Projection definition	$P_{C} (y) = \arg min_{x \in C} \| \| x - y \| \|^{2}$	Finds the closest point in $C$ .
Non-expansiveness	$\| \| P_{C} (y) - P_{C} (z) \| \| \leq \| \| y - z \| \|$	Projection does not increase distances.
Idempotency	$P_{C} (P_{C} (y)) = P_{C} (y)$	Repeated projections do not change the point.
Optimality condition	$(y - P_{C} (y)) (x - P_{C} (y)) \leq 0$	Projection direction is orthogonal to the boundary at $P_{C} (y)$ .