You want to solve the equation:
$\sqrt{5-x}=5-x^2$
In your first attempt you squared the equation and got:
$5-x=(5-x^2)^2$
Note, that you have to be precice when you do so.
The LHS is always positiv and not defined (in the real numbers) if $x>5$.
The RHS can be negativ.
When you just square, you create "fake solutions".
In your 2nd attempt it seems like you get
$\frac{\sqrt{5-x}}{\sqrt{5}-x}=1$ this is wrong in general.
Stick with your first try:
$x^4-10x^2+x+20=0$
One can see $x^4-10x^2+x+20=(x^2-x-4)(x^2+x-5)=0$, which reduces the problem to quadratic equations. This can be solved easily.
But seeing this factorisation is not trivial and unclear.
One needs a good eye or a CAS like wolframalpha.
One way would be, to suppose, that there is such a factorisation, and then go like this:
$x^4-10x^2+x+20=(x^2+ax+b)(x^2+cx+d)$
And then compare the coefficients, to get $a, b, c$ and $d$.
You might get this idea, when trying to use long division and search for a linear factor, and go over the divisors of $20$.
Namely $\pm 1, \pm 2, \pm 4,\pm 5, \pm 10,\pm 20$. They all fail to give you a root.
Edit:
Taken from the discussion in the comments, I want to show a little bit more details to the comparasion of the coefficients.
After some thought process scatched before, we might write:
$x^4-10x^2+x+20=(x^2+ax+b)(x^2+cx+d)$
$(x^2+ax+b)(x^2+cx+d)=x^4+(a+c)x^3+(ac+b+d)x^2+(ad+bc)x+bd$
Now we have:
$x^4+\color{red}{(a+c)}x^3+\color{blue}{(ac+b+d)}x^2+\color{orange}{(ad+bc)}x+\color{green}{bd}=x^4+\color{red}{0}x^3-\color{blue}{10}x^2+\color{orange}{1}x+\color{green}{20}$
This leads to the equation system:
$a+c=0$
$ac+b+d=-10$
$ad+bc=1$
$bd=20$