Essentially yes. If you don't vary your input too much then your output should not change with great degree.
In my experience with the Levenberg-Marquardt algorithm this all has to do with how far you are from a solution. The steepest descent method works best far from the solution because it takes "big steps" to get nearer to the solution. The Gauss-Newton method "presumes that the objective function is approximately quadratic in the parameters near the optimal solution". See this link: http://people.duke.edu/~hpgavin/ce281/lm.pdf . So the Gauss-Newton algorithm takes very small steps near the solution. Small steps far from the solution would take a long time to achieve convergence and may not always give the best answer the more complex your equation gets. So this is how the Steepest (Gradient) descent gives stability to the algorithm.
The great thing about the Levenberg-Marquardt algorithm is that the damping parameter (see: http://www.aip.org/cip/pdf/vol_11/iss_1/110_1.pdf) lets the algorithm "know" when to switch between the two methods.
See also papers by James P. Sethna (http://arxiv.org/pdf/1201.5885.pdf) some of his earlier ones give very nice explanations and definitions too.
