Let $X\subset \mathbb{R}^n, Z\subset \mathbb{R}^m$ and $\phi:X\times Z\to \mathbb{R}$. We define a saddle point of $\phi$ as follows:
Definition. $(x^*, z^*)\in X\times Z$ is said to be a saddle point if for any $x\in X, z\in Z$, we have $\phi(x^*, z)\leq \phi(x^*, z^*) \leq \phi(x,z^*)$.
According to Convex Optimization by Bertsekas, the following theorem holds for a saddle point.
Theorem. $(x^*,z^*)$ is a saddle point if and only if:
(1)The minimax equality holds, i.e., $\sup_{z}\inf_{x}\phi(x,z)=\inf_{x}\sup_{z}\phi(x,z)$.
(2) $x^*$ is a solution to $\textrm{minimize } \sup_{z}\phi(x,z)$ subject to $x\in X$.
(3) $z^*$ is a solution to $\textrm{maximize } \inf_{x}\phi(x,z)$ subject to $z\in Z$.
I was wondering if we can put (2) and (3) together as a minmax problem, that is, we want to prove the following.
Conjecure. $(x^*,z^*)$ is a saddle point if and only if:
(1)The minimax equality holds, i.e., $\sup_{z}\inf_{x}\phi(x,z)=\inf_{x}\sup_{z}\phi(x,z)$.
(2) $(x^*,z^*)$ is a solution to the minimax problem $\inf_{x}\sup_{z}\phi(x,z)$ i.e. $\phi(x^*,z^*)=\inf_{x}\sup_{z}\phi(x,z)$($=\sup_{z}\inf_{x}\phi(x,z)$ by (1).).
I proved that a saddle point is indeed a solution to the minimax problem, but I could not prove the other way. I feel like this is false, but I could not come up with a counterexample. Can somebody give me any hint?
The proof that a saddle point is a solution to the minimax problem. By definition, it is clear that $(x^*,z^*)$ is a saddle point if and only if \begin{equation} \sup_{z}\phi(x^*,z)=\phi(x^*,z^*)= \inf_{x}\phi(x,z^*) \end{equation} for any $x,z$. Since $x^*$ solves the minimization problem for $\sup_{z}\phi(x,z)$, we have $\inf_{x}\sup_z\phi(x,z)= \sup_{z}\phi(x^*,z)$. By the equality above, we have $\inf_{x}\sup_z\phi(x,z)=\phi(x^*,z^*)$. Therefore, $(x^*,z^*)$ is a solution to the minimax problem.
