What are regular points? (optimization)

Question

I am taking Optimization and my homework question just asked for the definition of a regular point. I have tried googling it. I read multiple definitions but still am confused. Is a regular point just a point that is in the feasible region?

thanks

Cf Constraint qualification and tangential cone vs linearized tangential cone. — user251257, Dec 01 '16 at 19:38
@LinAlg it's usual formulation is only for convex sets and it is global not local... — user251257, Dec 01 '16 at 23:56

score 2 · Answer 1 · answered Mar 16 '17 at 16:55

2

For inequality constraints a regular point is one where the gradients of the active constraints are linearly independent.

answered Mar 16 '17 at 16:55

NSP

21

what if there is only one constraint? So that instead of $h_1, h_2, \cdots, h_m$ it is $h_1$, does that mean that the points are all feasible? – baxx Apr 27 '19 at 21:03
If there is only one constraint and $\nabla h_1 \neq \mathbf{0}$, then $\nabla h_1$ is linear independent, since only $0*\nabla h_1=\mathbf{0}$, which satisfy the definition of linearly dependent. – dawen Aug 02 '21 at 12:57

score 1 · Answer 2 · answered Mar 18 '21 at 23:54

Since I can't comment on baxx response to NSP I'll right my own addition.

"For inequality constraints a regular point is one where the gradients of the active constraints are linearly independent." This also applies to equality constraints. See Bertsekas Nonlinear Programming Third Edition pg.346

and

If there is only one constraint $\nabla h_1 \neq 0$

Directly quoting from Bertsekas Nonlinear Programming Third Edition pg. 346: "... a feasible vector $x$ for which the constraint gradients $\nabla h_1(x),...,h_m(x)$ are linearly independent will be called regular."

score 0 · Answer 3 · answered Apr 20 '23 at 20:06

Regularity is a condition in the gradient of the active constraints at the point/vector, not a feature of the point/vector in question; the difference in semantics is important in understanding the definition. E.g., there could be feasible points which are not regular.

A regular point is a characterization that builds up to its optimality based on a few other components.

Formally put,

"A point $x^\star$ satisfying the constraints $h_1(x^\star), \cdots, h_m(x^\star) = 0$ is said to be a regular point of the constraints if the gradient vectors $\nabla h_1(x^\star), \cdots, \nabla h_m(x^\star)$ are linearly independent"

There are many examples that illustrate the behavior around the regular point in "An Introduction to Optimization" (Chapter 20) by Chong and Żak (from which I also used the definition of a regular point)

It's worth explicitly pinpointing the contribution of regularity in the proof of local optimality in the Lagrange's Theorem, which was based on only showing that the gradient of the objective function at a regular point $x^\star$ belongs to the orthogonal complement of the tangent space, i.e.,

$$\nabla f(x^\star) \in T(x^\star)^\perp$$

What are regular points? (optimization)

3 Answers3