Cauchy – Schwarz Inequality

For all real vectors (?) $\mathbf{x}$ and $\mathbf{y}$, the Cauchy – Schwarz inequality states:

This inequality is equivalent to the triangle inequality for a norm in a space with a scalar product...

Before we dive into proving the Cauchy – Schwarz inequality, let’s clear up an important distinction between absolute value and norm — two concepts that often look similar but are different in meaning.

What’s the Difference Between $|x|$ and $\parallel x\parallel$?

• $|x|$ (Absolute value):
  Measures the distance of a number from zero on the real number line.
• If $x$ is a single number (a scalar):$$|x|=\{\begin{array}{ll}x,& \text{if }x\ge 0\\ -x,& \text{if }x<0\end{array}$$

Example: $|5|=5$, $|-5|=5$.

• $\parallel \mathbf{x}\parallel$ (Norm):
  Measures the length (magnitude) of a vector in space. For a vector $\mathbf{x}=(x_1,x_2,\dots ,x_n)$, the most common norm is the Euclidean norm:$$\parallel \mathbf{x}\parallel =\sqrt{x_1^2+x_2^2+\cdots +x_n^2}$$

Example: For $\mathbf{x}=(3,4)$,

Key Difference

• $|x|$ measures the length of a single number on a line. It's just a modulus.
• $\parallel \mathbf{x}\parallel$ measures the length of a vector in multi-dimensional space. Norms could be different, but the Euclidean norm is the most common.

Cauchy – Schwarz Inequality: The Statement

The Cauchy – Schwarz inequality says:

Where:

• $\mathbf{x}\cdot \mathbf{y}$ is the dot product of vectors $\mathbf{x}$ and $\mathbf{y}$, defined as:$$\mathbf{x}\cdot \mathbf{y}=x_1{y}_1+x_2{y}_2+\cdots +x_n{y}_n$$OR$$\mathbf{x}\cdot \mathbf{y}=\parallel \mathbf{x}\parallel \parallel \mathbf{y}\parallel \cos (\theta )$$where $\theta$ is the angle between the vectors.
• $|\mathbf{x}\cdot \mathbf{y}|$ is the absolute value of the dot product (just a scalar).
• $\parallel \mathbf{x}\parallel$ and $\parallel \mathbf{y}\parallel$ are the magnitudes (lengths) of the vectors.

So, basically, looking at the dot product formula with $\cos (\theta )$, the Cauchy – Schwarz inequality tells us that the dot product is always bounded by the lengths of the vectors.
$cos(\theta )$ can never exceed 1, so the dot product can never exceed the product of the lengths.

What Does It Mean?

It’s like comparing the "shadow" of one vector on another to their total lengths. The dot product measures the alignment between vectors:

• If the vectors are in the same direction, the dot product is at its maximum.
• If they are at right angles (90°), the dot product is zero.
• If they point in opposite directions, the dot product is negative but still within the bound.

Proof of the Cauchy – Schwarz Inequality

Let’s prove it step by step using a clever trick with quadratic equations:

Step 1: Start with a General Vector Expression

For any real vectors $\mathbf{x}$ and $\mathbf{y}$, define:

Since norms are always non-negative, we know:

Step 2: Expand the Norm

Using the definition of the dot product and norm:

So, basically:

Similarly, we can say:

Expand the dot product:

Step 3: Rewrite in a Quadratic Form

Let’s set it up as a quadratic expression in terms of $\lambda$:

Step 4: Analyze the Quadratic Inequality

Since $f(\lambda )\ge 0$ for all $\lambda$, the quadratic equation:

must have no real roots or a non-positive discriminant. The discriminant ($\mathrm{\Delta }$) for a quadratic $a{\lambda }^2+b\lambda +c$ is:

Here:

• $a=\parallel \mathbf{y}{\parallel }^2$
• $b=-2(\mathbf{x}\cdot \mathbf{y})$
• $c=\parallel \mathbf{x}{\parallel }^2$

Step 5: Set the Discriminant Condition

Since $f(\lambda )\ge 0$, the discriminant must be less than or equal to zero:

Step 6: Simplify the Discriminant

Divide both sides by 4 and a little rearrangement gives:

Step 7: Take the Square Root

Finally, taking the square root of both sides:

And there you have it — Cauchy – Schwarz proven!

Useful Links

• Dot products and duality | 3Blue1Brown

...