Find all functions $f \colon \mathbb R \to \mathbb R$ such that $f(f(x) - y) = f(f(x)) - 2f(x)y + f(y), \forall x, y \in \mathbb R$.

Question

Find all functions $f \colon \mathbb R \to \mathbb R$ such that $$\large f(f(x) - y) = f(f(x)) - 2f(x)y + f(y), \forall x, y \in \mathbb R$$

It can be deduced that the solutions are $f(x) = 0, \forall x \in \mathbb R$ and $f(x) = x^2, \forall x \in \mathbb R$

Here's an attempt.

Let $P(x, y)$ be the assertion $f(f(x) - y) = f(f(x)) - 2f(x)y + f(y)$.

We have that for $P(0, 0)$, $f(f(0)) = f(f(0)) + f(0) \iff f(0) = 0$.

Furthermore, for $P(x, f(x))$, we have that $$f(f(x) - f(x)) = f(f(x)) - 2[f(x)]^2 + f(f(x)), \forall x, y \in \mathbb R$$

$$\iff f(0) = 2f(f(x)) - 2[f(x)]^2, \forall x, y \in \mathbb R \iff f(f(x)) = [f(x)]^2, \forall x, y \in \mathbb R$$

$(\implies f(f(0)) = [f(0)]^2 = 0)$

$$\iff f(f(x) - y) = [f(x)]^2 - 2f(x)y + f(y), \forall x, y \in \mathbb R$$

$$\iff f(f(x) - y) - (f(x) - y)^2 = f(y) - y^2, \forall x, y \in \mathbb R$$

Replacing $y$ and $f(y)$, we have that $f(f(x) - f(y)) = (f(x) - f(y))^2$

For $P(0, x), f(f(0) - x) = f(f(0)) - 2f(0)x + f(x), \forall x, y \in \mathbb R \iff f(x) = f(-x), \forall x, y \in \mathbb R$

$\iff f(x)$ is an even function.

Then I don't what to do next.

Answer 1

This is a partial solution for the case when $f$ is continuous.

Assume there is a $t\in\mathbb R$ such that $f(t)>0.$ Then, there is a $N$ such that $f(t)>\frac1N.$

I will prove that for any integer $n>0,$ $f(nf(x))=n^2f(x)^2$ inductively. When $n=1,$ this is obviously true. Assume the statement is true for $n.$ For $n+1,$ looking at $P(f(x),-nf(x)),$ we get \begin{equation} \begin{split} f((n+1)x)&=f(f(x)+nf(x))\\ &=f(f(x))+2f(x)nf(x)+f(nf(x))\\ &=f(x)^2+2nf(x)+n^2f(x)^2\\ &=(n+1)^2f(x^2). \end{split} \end{equation}

Now, let $\{x_i\}_{i\ge 0}$ be the sequence defined by $x_0=t$ (defined above) and $x_{n+1}=Nf(x_n).$ Then, $f(x_{n+1})=f(Nf(x_n))=N^2f(x_n)^2.$ I will by induction prove $f(x_n)>N^{n-1}.$ When $n=1,$ $f(x_1)=(Nf(x_0))^2=(Nf(t))^2>1.$ Assume this is true for $n.$ For $n+1,$ $f(x_{n+1})=(Nf(x_n))^2>(NN^{n-1})^2=N^{2n}\ge N^n.$

Thus, the sequence $\{f(x_i)\}_{i\ge 0}$ diverges.

Let $x\in \mathbb R_{>0}$ be arbitrary. Then, since $\{f(x_i)\}_{i\ge 0}$ diverges, for some $i,$ $f(x_i)>x>0=f(0).$ By the Intermediate Value Theorem, there is some $r\in (0,x_i)$ such that $x=f(r).$ Thus, $f(x)=f(f(r))=f(r)^2=x^2.$

When $x\in\mathbb R_{<0},$ one can use $f$ being an even function to prove $f(x)=x^2.$

Thus, the only continuous functions which satisfy the condition are $0$ and $x^2.$

Edit (A solution not using continuity):

Again, assume there exists a $t\in\mathbb R$ such that $f(t)\neq 0$.

For any $x,y\in \mathbb R,$ looking at $P(x,f(y)),$ we obtain $f(f(x)-f(y))=f(f(x))-2f(x)f(y)+f(f(y))=f(x)^2-2f(x)f(y)+f(y)^2=(f(x)-f(y))^2.$ (call this equation *)

Let $x\in\mathbb R$ be arbitrary, let $y=\frac{f(f(t))-\sqrt x}{2f(t)},$ and look at $P(t,y).$ We have $f(f(t)-y)=f(f(t))-2f(t)y+f(y),$ and thus, $f(f(t)-y)-f(y)=f(f(t))-2f(t)y.$ Now, $f(f(f(t)-y)-f(y))=(f(f(t)-y)-f(y))^2=(f(f(t))-2f(t)y)^2=x,$ (I used * on second equals sign) and hence, there is a $r\in\mathbb R$ such that $f(r)=x.$ Now, $f(x)=f(f(r))=f(r)^2=x^2.$