2

The Armijo type line search is to find an $a_k > 0$ such that $$ f(x^k + \alpha_kd^k) \leq f(x^k) + \sigma_1 \alpha_k \nabla f(x^k)^Td^k $$ given $\sigma_1 \in (0, 1/2)$.

We know that for sufficient small positive $\alpha_k$, the inequality holds.

In practise, when the $\alpha_k$ found is too small like $1\text{e}^{-9}$, what's the best choice to deal with $\alpha_k$ in this iteration ?

Saddle Point
  • 141
  • 1
    In practice I would regularize the descent direction. But I've run into cases where this happens even when $d^k$ is the gradient, so I'm interested in a canonical answer (it may just be, "you need higher-order information near $x^k$") – user7530 Dec 25 '14 at 05:44
  • Is $d$ a reasonable descent direction? What is the Hessian like near the point? Are you near a local minimum? What is $\alpha_kd^k$ compared with $x^k$? – copper.hat Dec 25 '14 at 07:45

0 Answers0