Perhaps to answer my own question now that I have a better understanding of optimization (in particular convex optimization).
Here, I'll define "hardness" as the amount of computation required to reach a global optimum.
The "hardness" of an optimization problem depends less on whether or not there are constraints, but more on the form of the constraint and also the cost function.
A major threshold that makes a problem significantly harder is when the new constraint / cost function converts the program from convex to non-convex (with non-convex problems being significantly harder to solve).
Convex problems in general have very efficient algorithms for reaching global optimums of which the number of constraints plays a rather insignificant role (of course, less constraints makes it easier, but adding more convex constraints does not impact the solve time much).
In the realm of non-convex optimization problems, the problem gets more complicated and usually depends on the cost function and constraints themselves.
Removing constraints obviously can help by making the problem less convex but adding constraints may also help push the solution to a better (i.e. from infeasible to feasible or from higher cost to lower cost) local optima. So the answer is, it depends and YMMA.
Of note, for non-convex problems, providing an initial guess (even if infeasible) can greatly help steer the optimization to a better local optima.