Partial approach. This will define a large class of answers that are not necessarily continuous.
Let $g(x)=f(x)-(x+1).$
Then $f(x)^2=g(x)^2+2(x+1)g(x)+(x+1)^2$ and $f(x^2)=g(x^2)+(x^2+1).$
So:
$$2x = f(x)^2-f(x^2)= g(x)^2+2(x+1)g(x)-g(x^2)+2x$$
Or $$g(x)^2+2(x+1)g(x)=g(x^2).\tag 1$$
So if we define $g(x)$ as a positive function on $(1/4,1/2)$ we can define inductively for $x\in [2^{-2^{k+1}},2^{-2^{k}}],$ as $g(x)=g(\sqrt x)^2+2(x+1)g(\sqrt x).$
And for negative $k$ and $x\in[2^{-2^{k+1}},2^{-2^k}):$
$$g(x)=-(x+1)+\sqrt{(x+1)^2+g(x^2)},\tag2$$
which is gotten by solving the quadratic equation $(1)$ and choosing the positive case.
We can choose either $g(0)=1$ or $g(0)=0.$ Likewise we can choose $g(1)$ from $1,2.$
This defines $g$ on $(0,1).$ We can define $g(0)$ as $0$ or $-1$ and likewise $g(1)$ as $0$ or $-3.$ But we need $g(1)=0$ since $g(-1)^2=g(1).$
Note that these definitions ensure $g(x)$ is always positive.
We can likewise define $g$ as positive on $[2,4)$ and again define inductively to get $g$ defined on all of $(1,\infty).$
Then the problem is the negative cases. But again, we can define the negative cases from $(2)$ again, except the problem is that some $g(x^2)$ might be negative.
So, we can find a large set of such functions if they don't have to be continuous.
I doubt these are ever continuous except when $g(x)$ is everywhere $0,$ but I might be wrong.
And this might not be all the possible examples, since this defines only the cases when $g$ is positive on the positive reals.
Then $f(x)^2=g(x)^2-2(x+1)g(x)+(x+1)^2$ and $f(x^2)=g(x^2)-(x^2+1).$
So:
$$2x = f(x)^2-f(x^2)= g(x)^2-2(x+1)g(x)-g(x^2)+2x$$
Or $$g(x)^2-2(x+1)g(x)=g(x^2).$$
When $x=0$ or $x=1,$ we have $g(x)=0.$
– Thomas Andrews Mar 06 '23 at 16:36$$f(x)^2-f(x^2)=2x$$
to
$$f(x)f(y)-f(xy)=xy$$
is not at all justified. You cannot just freely change some parts of it but not others. However it is true that if a function satisfies the second equation, then it also satisfied the first; indeed as you pointed out we can go from the second to the first by taking $y=x$. This does mean that we can find some solution(s) by solving the second equation. This does not mean, however, that we will find all, as there could (potentially) be functions satisfying the first but not the second equation.
– Lorago Mar 06 '23 at 16:37