3

Consider a minimax problem $$\inf_{x\in X} \sup_{y\in Y} L(x,y)$$ where $L$ is convex in $x$ and concave in $y$. Assume that $$\inf_x \sup_y L(x,y) = \sup_y \inf_x L(x,y)$$ and all the inf's and sup's are attainable(if they are not infinity). I also want to add that $X$, $Y$ are typically closed convex set in $\mathbb{R}^n$(not necessarily compact).

The background of the problem is that such convex-concave saddle point problems often occur in the primal-dual formulation of convex optimization. For example, in a standard linear programming, the primal-dual problem is $\min_{x\ge 0}\max_{y\in \mathbb{R}^m} L(x,y)= c^T x+y^T b-y^T A x$.

Now let $z=(x,y)$ and $Z^*$ denote the solution set of $z$ for the minimax problem $L(z)=L(x,y)$.

Let $X^*$ denote the set of $x$ which minimizes $\sup_y L(x,y)$. Similarly, let $Y^*$ denote the set of $y$ which maximizes $\inf_x L(x,y)$.

I wonder if there is some relation beween $Z^*$ and $X^* \times Y^*$. I want to show that $X^*\times Y^* = Z^*$. I have shown that $X^*\times Y^* \subset Z^*$, but I cannot prove the other way. Can somebody give me a hint on whether the statement is true or not?

Remark 1 This is a "concurrent" question with this. That question is still open and I tried to answer it but finally found that the two questions are equivalent, i.e., if I can show that $Z^*=X^*\times Y^*$, then I can also show that the solution of a minimax problem is a saddle point and vice versa.

Remark 2 To show $X^*\times Y^* \subset Z^*$, we only need to notice that for any $(x^*,y^*)\in (X^*,Y^*)$,

$$\sup_{y} \inf_{x} L(x,y)= \inf_{x} L(x,y^*)\le L(x^*,y^*)\le \sup_y L(x^*,y)= \inf_x \sup_{y} L(x,y) $$ Since we also assume minimax equality holds, $\sup_{y} \inf_{x} L(x,y)= \inf_x \sup_{y} L(x,y)$, the equalities holds throughout above.

PPP
  • 346
  • 1
    So we assume that the $\inf$s and $\sup$s are actually $\min$s and $\max$s right ? Otherwise the sets could be empty ? – P. Quinton Feb 14 '24 at 09:12
  • @P. Quinton Yes, I've edited to add these assumptions. – PPP Feb 14 '24 at 09:23
  • Could you provide the precise definition of $Z^\ast$ ? I'm asking because every attempt I had at defining it is essentially saying that $Z^\ast=X^\ast\times Y^\ast$. – P. Quinton Feb 14 '24 at 09:30
  • @ P. Quinton $Z^={(x,y): L(x,y)=\min_x \max_y L(x,y)}$. What confuses me is that I cannot prove the solution $z=(x^,y^)$ in $Z^$ have the property $y^= \arg \max L(x^,y),x^=\arg\min L(x,y^)$ – PPP Feb 14 '24 at 09:32
  • If $Z^\ast={ z : L(z)=\inf_x\sup_y L(x,y) }$ then this is certainly wrong. – P. Quinton Feb 14 '24 at 09:33
  • @P. Quinton because the convexity and concavity may not be strict? Then when would $Z^=X^\times Y^*$ hold? For example, what conclusion can we get when $L(x,y)$ is the Largangian of a linear programming, i.e., the saddle point problem is $\min_{x\ge 0}\max_{y\in \mathbb{R}^m} c^T x+y^T b-y^T A x$? – PPP Feb 14 '24 at 09:35
  • I am not sure what the condition should be, I believe something like $L(x,\cdot)$ or $L(\cdot, y)$ is constant for all $x$ or $y$ would do, but I do not have any proof that otherwise $Z^\ast$ contains non optimal values. – P. Quinton Feb 16 '24 at 09:37
  • Based on the counterexample given in the Quinton's answer we know that $X^\times Y^= Z^$ cannot be hold generally. Are you looking for conditions under which $X^\times Y^* = Z^*$? – Amir Feb 19 '24 at 22:10
  • @ Amir. Yes. Because a minimax problem is commonly derived from a Lagrangian of a convex optimization problem(strong duality holds), I think it would be interesting to find out what is the relationship of primal-dual optimal pairs($Z^$) and $X^\times Y^*$ – PPP Feb 21 '24 at 18:01

1 Answers1

2

If $L(x,y)=x^2-y^2$, then $Z^\ast=\{ (a,\pm a): a\in\mathbb R \}$ while $X^\ast=Y^\ast=\{ 0\}$.

In case $L(x,y)$ is the lagrangian of some linear programing, the same problem arises, in order to study the saddle point, I would resort to the KKT conditions which gives a full characterization of the solutions.

P. Quinton
  • 6,031
  • Consider LP or any smooth convex optimization and assume strong duality holds. By KKT conditions, an optimal primal-dual pair $(x^,y^)$ satisfy $x^$ is the minimizer of the lagrangian $L(x,y^)$. In this case, I guess it may hold that $Z^=X^\times Y^*$ – PPP Feb 14 '24 at 10:01
  • I think it can hold in particular cases, but it won't in general. – P. Quinton Feb 14 '24 at 12:13