I only recently started studying the Lagrange Multipliers, and was given a task to create some challenging problems on them and also provide solutions. Could somebody please suggest how I could get started on this? Some example problems would be welcome!
Thanks very much!
Here is a moderately complicated example, cooked up in such a way that it can be solved explicitly all the way to the end.
Given are the two points $A=(0,-7)$, $B=(-7,0)$, and the circle $\gamma: \>x^2+y^2=4$. Determine two points $P$, $Q\in\gamma$ such that the quantity $$d(P,Q):=|AP|^2+|PQ|^2+|QB|^2$$ becomes maximal, resp., minimal.
You will obtain four conditionally stationary situations $(P_k,Q_k)$: the two extremal situations and two "saddle points". The latter had to be expected, since the surface determined by the conditions is a torus.
A full solution (in German) is given on pp. 212–214 of the following pdf-file. The file contains a full chapter (pages 128–236) of a textbook for engineering students. Scroll down to page 212; there you will see the figure printed above.
Sure, here's 2:
a really good one to start out with is the optimization problem behind Principal Component Analysis.
$$\text{max$_{\bf v}$ } \langle \bf{v},\Sigma_n \bf{v}\rangle$$ $$\text {s.t.}$$ $$||\bf {v}||_2=1$$
where $\bf v$ is a vector and $\Sigma_n= \frac 1n\sum_{i=1}^n(\bf x_i -\mu_n)(\bf x_i - \mu_n)^T$ is the Covariance matrix of the data points $\bf x_i$
The answer for $\bf v$ is the vector that points in the direction of the eigenvector of $\Sigma_n$ corresponding to its largest eigenvalue $\lambda_{max}$
Another really good application of Lagrange multipliers/ difficult problem involving Lagrange multipliers is solving for the Euler equation in Economics for logarithmic utility. This is extremely important in the theory of dynamic programming as well.
$$max \sum_{t=0}^{T-1} lnc_t + lnx_T$$ $$\text{s.t.}$$ $$x_{t+1} =\alpha (x_t-c_t)$$
Be careful, $x_t$ shows up twice. Interpret as maximizing consumption and final wealth where $x_t$ is the wealth at period $t$ with $t=0$ corresponding to initial wealth and $c_t$ is the consumption for period $t$.
The solution is: $x_t^*=\frac {T+1-t}{T+1}(\frac 1{\alpha})^{-t}x_0$ and $c_t^*=\frac {(\frac 1{\alpha})^{-t}x_0}{T+1}$
These two problems include applications of lagrange multipliers to 1. a problem involving matrix calculus. and 2. a problem in which an optimal consumption path is determined (function), which has ties to optimal control and the functional optimization.
Here two other ones :
Exercice 1 :
Let $\alpha\in\mathbb{R}_{+}$. For all $\left(x,y,z\right)\in\mathbb{R}^{3}$, find the optimal constant $C\left(\alpha\right)\in\mathbb{R}$ such that $$x^{3}-\left(\frac{z}{\alpha}\right)^{3}\leq C\left(\alpha\right)\left(x^{2}+y^{2}+z^{2}\right)^{3/2}$$ and precise the case of equality in term of $\alpha$.
Solution :
We notice that the both sizes of the above equation are homogeneous functions of degree $3$ : it suffices to consider the problem on the unit sphere $\mathcal{S}=\left\{ \left(x,y,z\right)\in\mathbb{R}^{3};x^{2}+y^{2}+z^{2}=1\right\}$. Soient $f$ et $g$ the functions defined on $\mathbb{R}^{3}$ by $$\begin{cases} f\left(x,y,z\right)=x^{3}-\left(\frac{z}{\alpha}\right)^{3}\\ g\left(x,y,z\right)=x^{2}+y^{2}+z^{2}-1 \end{cases}.$$
As $f$ is continuous and $\mathcal{S}$ is compact, $f_{\mid\mathcal{S}}$ is bounded and attain its maximum values. For all $\left(x,y,z\right)\in\mathbb{R}^{3}$, we have $$\begin{cases} \mathrm{d}f\left(x,y,z\right)=3x^{2}dx-3\frac{z^{2}}{\alpha^{3}}dz\\ \mathrm{d}g\left(x,y,z\right)=2xdx+2ydy+2zdz \end{cases}$$ or in matricial notations in the basis $\left(\mathrm{d}x,\mathrm{d}y,\mathrm{d}z\right)$ of $\left(\mathbb{R}^{3}\right)^{*}$
$$\begin{pmatrix}3x^{2} & 0 & -3\frac{z^{2}}{\alpha^{3}}\\ 2x & 2y & 2z \end{pmatrix}.$$
Cancelling all the $\begin{pmatrix}3\\2\end{pmatrix}=3$ $2\times2$-determinants so that $\mathrm{d}f\left(x,y,z\right)$ and $\mathrm{d}g\left(x,y,z\right)$ are linearly dependant yields :
$$\begin{vmatrix}3x^{2} & 0\\ 2x & 2y \end{vmatrix}=6x^{2}y=0\Longleftrightarrow\left(x=0\right)\textrm{ ou }\left(y=0\right).$$ $$ \begin{vmatrix}3x^{2} & -3\frac{z^{2}}{\alpha^{3}} 2x & 2z \end{vmatrix}=6xz\left(x+\frac{z}{\alpha^{3}}\right)=0\Longleftrightarrow\left(x=0\right)\textrm{ ou }\left(z=0\right)\textrm{ ou }\left(x=-\frac{z}{\alpha^{3}}\right).$$ $$\begin{vmatrix}0 & 3\frac{z^{2}}{\alpha^{3}}\\ 2y & 2z \end{vmatrix}=6y\frac{z^{2}}{\alpha^{3}}=0\Longleftrightarrow\left(y=0\right)\textrm{ ou }\left(z=0\right).$$
Then, for all $\left(x,y,z\right)\in\mathbb{R}^{3}$ :
$(i)$ if $\alpha<1$, $$f\left(x,y,z\right)\leq\frac{1}{\alpha^{3}}\left(x^{2}+y^{2}+z^{2}\right)^{3/2}$$ with equality iff $\left(x,y,z\right)\in\left\{ \left(0,0,-\lambda\right);\lambda\in\mathbb{R}_{+}^{*}\right\}$ ;
$(ii)$ if $\alpha=1$, $$f\left(x,y,z\right)\leq\left(x^{2}+y^{2}+z^{2}\right)^{3/2}$$ with equality iff $\left(\left(x,y,z\right)\in\left\{ \left(\lambda,0,0\right);\lambda\in\mathbb{R}_{+}^{*}\right\} \right)$ or $\left(\left(x,y,z\right)\in\left\{ \left(0,0,-\lambda\right);\lambda\in\mathbb{R}_{+}^{*}\right\} \right)$ ;
$(iii)$ If $\alpha>1$, $$f\left(x,y,z\right)\leq\left(x^{2}+y^{2}+z^{2}\right)^{3/2}$$ with equality iff $\left(x,y,z\right)\in\left\{ \left(\lambda,0,0\right);\lambda\in\mathbb{R}_{+}^{*}\right\} $.
Exercice 2 :
Let $n$,$\alpha\in\mathbb{N}^{*}$, $1\leq\alpha\leq n$. For all $\left(x_{1},\ldots,x_{n}\right)\in\mathbb{R}^{n}$, find the optimal constant $C\left(n,\alpha\right)\in\mathbb{R}$ such that $$x_{1}\ldots x_{n}\leq C\left(n,\alpha\right)\left(x_{1}^{\alpha}+\ldots+x_{n}^{\alpha}\right)^{n/\alpha}$$ and precise the case of equality in term of $\alpha$.
Solution :
We notice that the both sizes of the above equation are homogeneous functions of degree $n$ : it suffices to consider the problem on the hypersurface $\mathcal{S}=\left\{ \left(x_{1},\ldots,x_{n}\right)\in\mathbb{R}^{n};x_{1}^{\alpha}+\ldots+x_{n}^{\alpha}=1\right\}$. Let $f$ and $g$ the functions defined on $\mathbb{R}^{n}$ by
$$\begin{cases} f\left(x_{1},\ldots,x_{n}\right)=x_{1}\ldots x_{n}\\ g\left(x_{1},\ldots,x_{n}\right)=x_{1}^{\alpha}+\ldots+x_{n}^{\alpha}-1 \end{cases}.$$
We can assume that $x_{j}>0$ for all $1\leq j\leq n$ (the inequality is stronger in the case where there is an even number of $x_j<0$, which is the same that consider them all non-negative). For all $\left(x_{1},\ldots,x_{n}\right)\in\mathbb{R}_{+}^{n}$, we have
$$\begin{cases} df\left(x_{1},\ldots,x_{n}\right)=\sum_{j=1}^{n}\left(\prod_{k\neq j}x_{k}\right)dx_{j}\\ dg\left(x_{1},\ldots,x_{n}\right)=\alpha\sum_{j=1}^{n}\left(x_{j}^{\alpha-1}\right)dx_{j} \end{cases}$$ or in matricial notations
$$\begin{pmatrix}x_{2}x_{3}\ldots x_{n} & x_{1}x_{3}x_{4}\ldots x_{n} & \cdots & x_{1}x_{2}\ldots x_{n-2}x_{n-1}\\ \alpha x_{1}^{\alpha-1} & \alpha x_{2}^{\alpha-1} & \cdots & \alpha x_{n}^{\alpha-1} \end{pmatrix}.$$
We cancel all the $\begin{pmatrix}n\\2\end{pmatrix}$ $2\times2$-determinants so that $\mathrm{d}f\left(x_{1},\ldots,x_{n}\right)$ and $\mathrm{d}g\left(x_{1},\ldots,x_{n}\right)$ linearly dependant : for all $1\leq k$, $\ell\leq n$ with $k\neq\ell$, $$\begin{vmatrix}x_{1}\ldots x_{k-1}x_{k+1}x_{n} & x_{1}\ldots x_{\ell-1}x_{\ell+1}\ldots x_{n}\\ \alpha x_{k}^{\alpha-1} & \alpha x_{\ell}^{\alpha-1} \end{vmatrix}=\alpha x_{1}\ldots x_{k-1}x_{k+1}\ldots x_{\ell-1}x_{\ell+1}\ldots x_{n}\left(x_{\ell}^{\alpha}-x_{k}^{\alpha}\right)=0 \Longleftrightarrow x_{k}=x_{\ell}\Longrightarrow x_{1}=\left(\frac{1}{n}\right)^{\frac{1}{\alpha}}$$ with the constraint $g\left(x_{1},\ldots,x_{n}\right)=0$. We get $$f\left(\left(\frac{1}{n}\right)^{\frac{1}{\alpha}},\ldots,\left(\frac{1}{n}\right)^{\frac{1}{\alpha}}\right)=\left(\frac{1}{n}\right)^{\frac{n}{\alpha}}$$ and hence $$f_{\mid\mathcal{S}}\left(x_{1},\ldots,x_{n}\right)\leq\left(\frac{1}{n}\right)^{\frac{1}{\alpha}}.$$ We extend this result by homothetie to $\mathbb{R}^{n}$ : for all $\left(x_{1},\ldots,x_{n}\right)\in\mathbb{R}^{n}$, $$x_{1}\ldots x_{n}\leq\left(\frac{x_{1}^{\alpha}+\ldots+x_{n}^{\alpha}}{n}\right)^{\frac{n}{\alpha}}$$ with equality iff $x_{1}=\ldots=x_{n}$.
<br>or<hr>to separate parts of the text. See editing help. – Martin Sleziak May 03 '15 at 07:50