0

I try to figure out how coordinate descent works from wiki https://en.wikipedia.org/wiki/Coordinate_descent
From wiki example :
the equation is $5x^2-6xy+5y^2$.

Let $x = -0.5$ and $y =-1$
For the first iteration $y$ is going up graphically and the result of equation is minimizing
then $x$ is going right and the result of equation is minimizing, too
But how do we do this mathmatically?

My guess is:
put $x = -0.5$ an equation becomes $5(-0.5)^2-6(-0.5)y+5y^2$
after partial dererative $-6(-0.5)+10y=0$ and solve for $y$
do it in the same way for $x$ with the new $y$ from above.
Can someone tell me if I am in the right track?

Later
  • 722
  • 2
  • 8
  • 24
Linear Algebra fans
  • 63

1 Answers1

1

Yes, this is right. Fix $x$, get optimal $y$. Then fix that $y$, get new optimal $x$, and so on.

Anyway, the coordinate descent method is in principle used for numerical minimization. This means that you use a numerical algorithm for 1D minimization (such as the Golden section method).

Using the method analytically is a little masochistic as you will have to solve an infinity of 1D minimization problems rather than directly tackle 2D minimization by cancellation of the gradient.

  • I found a pesdu code/Alogrithum form a pdf files Page.5 http://www.optimization-online.org/DB_FILE/2014/12/4679.pdf Alogrithm 1 : We got alpha which is a step also i and e however. It is diiferent from the example in wiki and someone even said this method is deterative free. It makes me very confusing – Linear Algebra fans Mar 22 '19 at 09:34
  • @LinearAlgebrafans: this is a different question, enter it as such. –  Mar 22 '19 at 09:35