Note that, because
$$
\lfloor x\rfloor \leq x
$$
then, assuming positive values for x:
$$
\lfloor x\rfloor ^2 \leq \lfloor x\rfloor x \leq x^2
$$
The center term is the value they gave us (39), so $\lfloor x\rfloor ^2$ must be some "perfect" or integer square less than that.
Now, there's the temptation to just grab the highest perfect square less than 39 (which would be 36), but we need to be careful; we need to consider the possibility that, in reducing $x$ to $\lfloor x\rfloor$ we could have reduced $\lfloor x\rfloor ^2$ past 36. How can we be sure that $\lfloor x\rfloor ^2$ is the highest perfect square below 39? Well, consider this...
If $\lfloor x\rfloor$ were some integer less than 6, then x would have to be less than 6, so $x^2$ would have to be less than 36, which would violate the right half of $\lfloor x\rfloor ^2 \leq \lfloor x\rfloor x \leq x^2$ (because 39 isn't less than nor equal to 36). You can do this symbolically and show that this principle holds true for any $x \geq -1$, but I didn't want to make things too complicated. Suffice it to say that $\lfloor x\rfloor ^2$ will be the highest perfect square which is less than or equal to the value given for $\lfloor x\rfloor x$.
This yields:
$\lfloor x\rfloor ^2 = 36$, and
$\lfloor y\rfloor ^2 = 64$, and then you're home free.