Suppose that $X$ and $Y$ are metric spaces. A metric space is a set equipped with a *metric*, which is a function that defines the intuitive notion of *distance* between the elements of the set. For example, the set of real numbers, $\mathbb{R}$, equipped with the metric induced by the absolute value function (which is a norm). More precisely, the metric $d$ can be defined as $$d(x,y)=|x - y|, \forall x, y \in \mathbb{R} \tag{1}\label{1}.$$

Let $f$ be a function from the metric space $X$ to the metric space $Y$, that is, $f: X \rightarrow Y$. Then, $f$ is a *non-expansive map* (also known as metric map) *if and only if*

$$d_{Y}(f(x),f(y)) \leq d_{X}(x,y) \tag{2} \label{2}$$

where the subscript $_X$ in $d_X$ means that the metric $d_X$ is the metric associated with the metric space $X$. Therefore, any function $f$ between two metric spaces that satisfies \ref{2} is a non-expansive operator.

To show that the max operator is non-expansive, consider the set of real numbers, $\mathbb{R}$, equipped with the absolute value metric defined in \ref{1}. Then, in this case, $f=\operatorname{max}$, $d(x, y) = |x - y|$ and $X = Y = \mathbb{R}$, so \ref{2} becomes

$$|\operatorname{max}(x) - \operatorname{max}(y)| \leq | x - y | \tag{3} \label{3}$$

Given that $\operatorname{max}(x) = x, \forall x$, then \ref{3} trivially holds, that is

\begin{align}
|\operatorname{max}(x) - \operatorname{max}(y)| &\leq | x - y | \iff \\
|x - y| &\leq | x - y | \iff \\
|x - y| &= | x - y | \tag{4} \label{4}
\end{align}

For example, suppose that $x=6$ and $y=9$, then \ref{4} becomes

\begin{align}
|\operatorname{max}(6) - \operatorname{max}(9)| &\leq | 6 - 9 | \iff \\
|6 - 9| &\leq | -3 | \iff \\
|-3| &= 3
\end{align}

There are other examples of non-expansive maps. For example, $f(x) = k x$, for $0 \leq k \leq 1$, where $f : \mathbb{R} \rightarrow \mathbb{R}$.

See also https://en.wikipedia.org/wiki/Contraction_mapping and https://en.wikipedia.org/wiki/Contraction_(operator_theory).

I haven't gone through your whole answer in detail, but you consider only the L2 norm, the Euclidean distance metric. Is the $\max$ operation non-expansive for other notions of distance? – Philip Raeisghasem – 2019-03-15T23:22:09.293

1@PhilipRaeisghasem That's a good question. I tried to keep the answer closer to laymen's terms. In fact, I think the analysis will hold for any metric (it requires the triangle inequality). The more formal definition of a non-expansive operator does state that it holds only for functions between metric spaces, so I think that is correct. – John Doucette – 2019-03-15T23:41:19.037