Let $x$, $y$ and $z$ be positives and satisfying $x^2+y^2+z^2=2xyz+1$ .
Find a maximum of this expression: $$P=(x-2yz)(y-2zx)(z-2xy).$$
Let $x$, $y$ and $z$ be positives and satisfying $x^2+y^2+z^2=2xyz+1$ .
Find a maximum of this expression: $$P=(x-2yz)(y-2zx)(z-2xy).$$
Set $y=x$ and $z=1$. Then $x^2+y^2+z^2=2x^2+1=2xyz+1$ and $$(x-2yz)(y-2zx)(z-2xy)=(-x)(-x)(1-2x^2)=x^2(1-2x^2)$$ which clearly has no minimum...
Let $x=1$ and $y=z=\frac{1}{2}.$
Hence, $P=\frac{1}{8}$.
We'll prove that it's a maximal value.
Indeed, let $x\geq y\geq z$.
If $x\leq2yz$ then we obtain: $$2xy\geq2xz\geq2yz\geq x\geq y\geq z,$$ which says that $P\leq0$.
Thus, we can assume that $x>2yz$.
In this case the condition gives $y=z$ and $$P=(1-2z^2)z^2=\frac{1}{2}(1-2z^2)2z^2\leq\frac{1}{2}\left(\frac{1-2z^2+2z^2}{2}\right)^2=\frac{1}{8};$$ 2. $x\in(0,1).$
Let $x=\cos\alpha$.
We have here $$(x-2yz)(y-2xz)(z-2xy)=(x-2yz)(yz-2x(y^2+z^2)+4x^2yz)=$$ $$=(x-2yz)(yz-2x(1+2xyz-x^2)+4x^2yz)=(x-2yz)(2x^3-2x+yz)=$$ $$=\frac{1}{2}(x-2yz)(4x^3-4x+2yz)\leq\frac{1}{2}\left(\frac{x-2yz+4x^3-4x+2yz}{2}\right)^2=$$ $$=\frac{1}{8}(4x^3-3x)^2=\frac{1}{8}\cos^23\alpha\leq\frac{1}{8}.$$ 3. $x>1$.
In this case we obtain: $$x^2+y^2+z^2-1-2xyz\geq x^2+2yz-1-2xyz=(x-1)(x-2yz+1)>0,$$ which is a contradiction and this case is just impossible.
Done!