I wanted to post a correct solution for posterity.
Since $y=0$ when $x=0$ and $y=1$ when $x=1$ it is clear that the positive derivative constraint is sufficient to find all valid $a$ and $b$. Basically, the only way $y$ can be less than 0 is if the line goes "down". Similarly, the only way $y$ can be larger than 1 but still end up at 1 eventually is for the line to go "down". This reasoning relies on the curve being continuous, but there may be a discontinuity. To avoid a discontinuity, we must require that the numerator doesn't become zero. It is clear that if $a<0$, there will be a discontinuity, so our first constraint is $a>0$.
Taking the derivative with respect to $x$ and collecting terms, we have
$$ \frac{\mathrm{d}y}{\mathrm{d}x} = \frac{\left(a x \left(- a + 1\right) + a \left(x \left(a - 1\right) + 1\right) - b x \left(x \left(a - 1\right) + 1\right)^{2} + b \left(- x + 1\right) \left(x \left(a - 1\right) + 1\right)^{2}\right)}{\left(x \left(a - 1\right) + 1\right)^{2}} $$
Since the denominator is squared, the sign of the derivative is completely determined by the sign of the numerator, which turns out to be a third order polynomial. While possible, it is not computationally tractable to calculate the roots of this equation analytically. We can, however, employ Budan's theorem to ensure that there are no roots in the interval (0, 1). Basically, we require that the polynomial obtained by setting $x = \frac{1}{x + 1}$ and multiplying by $(x + 1)^3$ has no changes in sign in the coefficients.
This polynomial is:
$$ \left(a + b\right) x^{3}+ \left(2 a b + 3 a - b\right) x^{2}+ \left(a^{2} b - 2 a b + 3 a\right) x + (a - a^{2} b) $$
Requiring no sign changes in the coefficients is equivalent to requiring that each coefficient multiplied by the leading coefficient is positive. This leads to a set of inequalities concerning quadratics, which can be solved relatively easily and shows this constraint region:
