Given the following trajectory equation with gravitational force $\left(F_G\right)$ and drag force $\left(F_D\right)$:
$$m a = F_G + F_D = m g \hat{y} - b \left( \hat{x} + \hat{y} \right)$$
and letting $k=\frac{b}{m}$ you can seperate the above equation into $x$ and $y$-equations,
$$\begin{split} x''(t)&=-k x'(t)\\ y''(t)&=-g-ky'(t) \end{split}$$
My question is: what is the math to go from $\hat{x}$ and $\hat{y}$ to $x'\left(t\right)$ and $y'\left(t\right)$?
It looks to me like your equation $$ ma=F_G+F_D=mg\hat{y}-b(\hat{x}+\hat{y}) $$ has been badly garbled.
If $\ \mathbf{i}\ $ and $\ \mathbf{j}\ $ are unit vectors in the horizontal direction of increasing $ x\ $ and vertical direction of increasing $\ y\ $ respectively, then the force of gravity on a particle is $$ F_G=-mg\,\mathbf{j} $$ and if its trajectory is $\ x(t)\,\mathbf{i}+y(t)\,\mathbf{j}\ $, its acceleration is $$ a=x''(t)\,\mathbf{i}+y''(t)\,\mathbf{j} $$ Your differential equations would therefore follow from Newton's second law, $$ ma=F_G+F_D $$ if $$ F_D=-b\big(x'(t)\,\mathbf{i}+y'(t)\,\mathbf{j}\big)\ . $$ However, I can see no interpretation of the $\ \hat{x}\ $ and $\ \hat{y}\ $ appearing in your original equation (even allowing for a minor typo) that would lead to this result.
Also, $\ F_D\ $ of this form would be linear drag, not quadratic drag.
