How to prove that $(x^2+y)(y^2+z)(z^2+x)+2xyz \leqslant 10$, where $x,y,z \in \mathbb{R}^+$ and $x^2+y^2+z^2+xyz=4$

Question

$x,y,z \in \mathbb{R}^+$ and $x^2+y^2+z^2+xyz=4$, prove $$(x^2+y)(y^2+z)(z^2+x)+2xyz \leqslant 10$$

I try several trig substitutions but feel hopeless with the cyclic term here. The condition $x^2+y^2+z^2+xyz=4$ made it too difficult to homogenize the inequality. I don't even know how to do brutal force either.

Answer 1

$$ x^2+y^2+z^2+xyz=4;\space\space x,y,z>0\qquad (1)\\ (x^2+y)(y^2+z)(z^2+x)+2xyz \le 10\qquad (2)$$ $(1)\Rightarrow 0<x,y,z\le 2 \text{ and}\space x=2\iff y=z=0$

If $x=2$ then $(2)$ is trivially verified. Put $x=2-\epsilon$ where $0\le \epsilon\le 2$ so $(1)$ becomes $$y^2+z^2+(2-\epsilon)yz=\epsilon(4-\epsilon)\qquad (1')$$ $(1’)$ is the equation of an ellipse whose axes are contained in the diagonals. In fact, changing coordinates by a rotation of $45^{\circ}$, straightforward calculation in $(1’)$ can transforms $(y,z)$ in $(y_1,z_1)$ giving $$\left(\frac{y_1}{\sqrt{2\epsilon}}\right)^2+\left(\frac{z_1}{2(4-\epsilon)}\right)^2=1$$ The concerned points $(y,z)$ are those of the red arc in the figure below corresponding to the value $\epsilon = 0.8$.

From $(2)$ we get $$F(\epsilon,y,z)=((2-\epsilon)^2+y)(y^2+z)(z^2+2-\epsilon)+(4-2\epsilon)yz\qquad (2’)$$ where $$\begin{cases}0\le \epsilon\le 2\\ 0<y,z\le \sqrt{\epsilon(4-\epsilon)}\end{cases}$$ For $\epsilon$ fixed, $(2’)$ is maximum when $y=z$ (@HN_NH exercise for) and this occurs when $y^2=\epsilon$ (easily get from $(1’)$ or from the drawn ellipse). It follows $$F_1(\epsilon)=(2-\epsilon)^3(\epsilon+\sqrt{\epsilon})+(2-\epsilon)^2(\epsilon^2+\epsilon\sqrt{\epsilon})+(2-\epsilon)(3\epsilon+\epsilon\sqrt{\epsilon})+\epsilon^2\sqrt{\epsilon}+\epsilon^2$$ $$F_1(\epsilon)=(2\epsilon^2-6\epsilon+8)\sqrt{\epsilon}+2\epsilon^3-10\epsilon^2+14\epsilon$$ Now, $F_1(\epsilon)$ has in its domain a maximum at $\epsilon=1$ in whose case $F_1(1)=10$. This corresponds to $(x,y,z)=(1,1,1)$ in $(2)$; for the other allowed values the proposed inequality becomes $$(x^2+y)(y^2+z)(z^2+x)+2xyz \lt 10$$

Answer 2

Let $x=\frac{2a}{\sqrt{(a+b)(a+c)}}$ and $y=\frac{2b}{\sqrt{(a+b)(b+c)}},$ where $a$, $b$ and $c$ are positives.

Thus, the condition gives $z=\frac{2c}{\sqrt{(a+c)(b+c)}}$ and we need to prove that $$x^2y^2z^2+3xyz+\sum_{cyc}(x^3y^2+x^3z)\leq10$$ or $$\frac{32a^2b^2c^2}{\prod\limits_{cyc}(a+b)^2}+\frac{12abc}{\prod\limits_{cyc}(a+b)}+\sum_{cyc}\left(\tfrac{16a^3b^2}{\sqrt{(a+b)^5(a+c)^3(b+c)^2}}+\tfrac{8a^3c}{\sqrt{(a+b)^3(a+c)^4(b+c)}}\right)\leq5$$ or $$\sum_{cyc}\left(16a^3b^2(b+c)\sqrt{\tfrac{a+c}{a+b}}+8a^3c(b+c)\sqrt{(b+c)(a+b)}\right)\leq$$ $$\leq5\prod_{cyc}(a+b)^2-32a^2b^2c^2-12abc\prod_{cyc}(a+b).$$ Now, by AM-GM $$\sqrt{(b+c)(a+b)}\leq\frac{1}{2}(2b+a+c)$$ and $$ab\sqrt{\frac{a+c}{a+b}}=\frac{ab\sqrt{(a+c)(a+b)}}{a+b}\leq\frac{\left(\frac{a+b}{2}\right)^2\cdot\frac{1}{2}(2a+b+c)}{a+b}=\frac{1}{8}(a+b)(2a+b+c).$$ Thus, it's enough to prove that: $$\sum_{cyc}\left(16a^3b^3\sqrt{\tfrac{a+c}{a+b}}+2a^2bc(a+b)(2a+b+c)+4a^3c(b+c)(2b+a+c)\right)\leq$$ $$\leq5\prod_{cyc}(a+b)^2-32a^2b^2c^2-12abc\prod_{cyc}(a+b)$$ or $$16\sum_{cyc}a^3b^3\sqrt{\frac{a+c}{a+b}}\leq\sum_{cyc}(5a^4b^2+a^4c^2+6a^3b^3+2a^4bc+4a^3b^2c+2a^3c^2b-4a^2b^2c^2).$$ Now, by C-S $$\sum_{cyc}a^3b^3\sqrt{\frac{a+c}{a+b}}\leq\sqrt{\sum_{cyc}\frac{a^3b^3}{a+b}\sum_{cyc}a^3b^3(a+c)}.$$ Id est, it's enough to prove that: $$256\sum_{cyc}\frac{a^3b^3}{a+b}\sum_{cyc}a^3b^3(a+c)\leq\left(\sum_{cyc}(5a^4b^2+a^4c^2+6a^3b^3+2a^4bc+4a^3b^2c+2a^3c^2b-4a^2b^2c^2)\right)^2,$$ which is obviously true after full expanding.

The last part for you.

Answer 3

$\color{brown}{\textbf{Trigonometrical substitution.}}$

From the given conditions should \begin{cases} x,y,z \in [0,2]\\[4pt] (2z+xy)^2 = (4-x^2)(4-y^2).\tag1 \end{cases}

Taking in account $(1),$ can be applied substitution $$x=2\sin a,\quad y=2\sin b\quad \Rightarrow \quad z = 2\cos(a+b) = 2\sin c,\tag2$$ where $$a\ge 0,\quad b\ge 0,\quad c\ge 0,\quad a+b+c=\dfrac\pi2.\tag3$$

Then the given inequality takes the form of $$4(2\sin^2a+\sin b)(2\sin^2b+\sin c)(2\sin^2c+\sin a) + 8\sin a\sin b\sin c \le 5.\tag4$$

$\color{brown}{\textbf{The proof.}}$

Taking in account $(3),$ one can get \begin{align} &2\sin^2 a + \sin b = \sin^2 a + 1 - \cos^2 a + \cos (c+a)\\[4pt] &= 1 + \cos a (\cos c - \cos a) + \sin a (\sin a -\sin c) = 1 - A \sin (a+\varphi), \end{align} where \begin{align} &A = \sqrt {(\cos c -\cos a)^2 + (\sin a - \sin c)^2} = \sqrt {2 - 2\cos (a-c)} = 2 \sin \dfrac{|a-c|}2,\\[4pt] &\tan \varphi = \dfrac{\cos c -\cos a}{\sin a - \sin c} = \dfrac{2\sin \frac{a-c}2 \sin \frac{a+c}2}{2\sin \frac{a-c}2 \cos \frac{a+c}2} = \tan \frac{a+c}2. \end{align}

Therefore, $$2\sin^2 a + \sin b = 1 - 2 \sin \frac {|a-c|}2 \sin \frac{3a+c}2.$$ Using the symmetry of task by $a,b,c$, should $$2\sin^2 a + \sin b \le 1,\quad 2\sin^2 b + \sin c \le 1,\quad 2\sin^2 c + \sin a \le 1.\tag5$$

On the other hand, is known the identity \begin{align} \sin(a+b+c) = \cos a \cos b \sin c + \cos a \sin b \cos c + \sin a \cos b \cos c - \sin a \sin b \sin c. \end{align}

Taking in account $(3),$ one can get \begin{align} &\sin a \sin b \sin c = \cos a \cos b \sin c + \cos a \sin b \cos c + \sin a \cos b \cos c - \sin(a+b+c)\\[4pt] &= \dfrac12(\cos a \sin(b+c) + \cos c \sin(a+b) + \cos b \sin(c+a)) - 1\\[4pt] &= \dfrac12(\cos^2 a + \cos^2 c + \cos^2 b) - 1 = \dfrac14(\cos 2a + \cos 2b + \cos 2c - 1)\\[4pt] &= \dfrac18 (\cos(a+b)\cos(a-b) + \cos(b+c)\cos(b-c) + \cos(c+a)\cos(c-a) - 2)\\[4pt] &= \dfrac18 (\sin c\cos(a-b) + \sin a\cos(b-c) + \sin b\cos(c-a) - 2), \end{align} $$\sin a \sin b \sin c \le \dfrac18.\tag6$$

Since from $(5),(6)$ should $(4),$ then the given inequality is proved.