For $n,k \in \mathbb{N}$ with $k \leq n$ let $f_k: \mathbb{R}^n \to \mathbb{R}$ be the following function $$ f_k(x) := \sum_{i=1}^k x_{[i]}, $$ where $x_{[i]}$ is the $i$-th largest component of $x \in \mathbb{R}^n$. I know that $f_k(x)$ equals the optimal value of the following linear program (LP) \begin{align*} \max\limits_{y} \quad & x^T y \quad \\ \text{subject to}\quad & \mathbb{1}^T y = k\\ \quad & 0 \leq y \leq \mathbb{1}. \end{align*} One solution can be found by Boyd et al in the book Convex Optimization. They argued that if the components are sorted in a nonincreasing fashion, than $y_1 = y_2 = ... = y_k = 1$ and $y_{k+1} = ... = y_n = 0$ is an optimal solution and hence $f_k(x)$ equals the optimal value of the above problem. But for me this shows only that $f_k(x)$ is bounded above by the optimal value of (LP). I also have to show that $f_k(x)$ is lower bounded by the optimal value of (LP). How can I show this? Or maybe there is an alternative proof to show it analytically? My approach was to use the KKT conditions. The Langragian is given by \begin{align*} L(y,\mu,t) = x^Ty + \mu^T g(y) + t \cdot h(y), \end{align*} where $g(y) = (g_1(y),\dots,g_n(y),g_{n+1}(y),\dots,g_{2n}(y))$ is the vector of the $2n$ constraints corresponding to the constraint $0 \leq y \leq \mathbb{1}$: \begin{align*} &g_1(y) = -y_1 \\ &\vdots \\ &g_n(y) =-y_n \\ &g_{n+1}(y) = 1 - y_1 \\ &\vdots \\ &g_{2n}(y) = 1 - y_n, \end{align*} and $\mu = (\mu_0,\dots,\mu_{2n+1}) \in \mathbb{R}^{2n+1}$ is the appropiate multiplier. $h(y) = \mathbb{1}^T y - k$. Now the KKT conditions are given by \begin{align*} \nabla_{y} L(y,\mu,t) &= x + \sum_{i=1}^{2n}\mu_i \nabla g_{i}(y) + t\nabla h(y)\qquad \qquad \qquad \qquad \text {(Stationarity)} \\ g_i(y) &\leq 0 \quad \text{and} \quad h(y) = 0 \qquad\qquad \qquad \qquad \qquad \qquad \text{(Primal feasibility)} \\ \mu & \geq 0 \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \text{(Dual feasibility)}\\ \mu^T g(y) &= 0 \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad\text{(Complementary slackness)} \end{align*}
Asked
Active
Viewed 44 times
1
-
Which part confuses you? Showing that the optimal solution of LP assigns $y_i=1$ to the $k$ biggest entries in $x$, or showing that this implies that the objective value is equal to $f_k(x)$? Sorting is just for convenience of notation (without loss of generality we can assume....) – Michal Adamaszek Feb 18 '22 at 11:02
-
To show that the optimal value of the LP equals $f_k(x)$. For this, I have to show that $f_k(x)$ is lower and upper bounded by the optimal value of the LP – wayne Feb 18 '22 at 11:04
-
Forget for a moment about $f_k$. Just look at the LP. The book gives you a complete, explicit, optimal solution to the LP, with proof. Is there a part in that solution that worries you? If no, then you take that optimal point, compute the objective, and it turns out to be equal $f_k(x)$. – Michal Adamaszek Feb 18 '22 at 11:10
-
I understand the proof. I just wanted an alternative solution using the function $f_k(x)$. But whatever. Thanks anyway :) – wayne Feb 18 '22 at 11:16
-
@MichalAdamaszek I have edited my question. Honestly, I just want an alternative approach to show that the optimal solution equals the $k$ largest elements. – wayne Feb 18 '22 at 12:41