1

I have following dicrete-time system $$ x(k+1) = A.x(k) + B.u(k) \\ y(k) = C.x(k) $$

I am supposed to find controller in form of $$ u^* = -K.x $$so that following performance index is minimized $$ \sum\limits_{k=0}^\infty\frac1{2} x^T Q x + u^T R u $$

using Lyapunov equation for discrete time systems

$$ A^T P A + Q = P $$

I think I need to get from Lyapunov to Riccati equation, but I dunno how (I know how to do that for time-continuous systems), any help (also in form of simple literature) is welcome.

Martin G
  • 319

1 Answers1

1

Optimizing your cost function amounts to solving a Bellman equation which, in case of linear systems, amounts to solving an algebraic Ricatti equation. This can be solved iteratively by using the techniques common in dynamic programming such as value or policy iteration.

For example, value iteration (forward in time) will read as:

$$ u_i(x_k) := -( R + B^T P_i B )^{-1}B^T P_i Ax_k \\ P_0:=0 \\ P_{i+1}:= A^T P_i A + Q - A^T P_i B( R + B^T P_i B )^{-1} B^T P_i A$$

Notice that $k$ is your time step index while $i$ is the iteration index.

It has been proven to converge to the optimal solution of the Bellman equation.

Further reading: P. L. Lancaster and Rodman, Algebraic Riccati Equations. London, U.K.: Oxford Univ. Press, 1995.

Rubi Shnol
  • 1,125
  • 8
  • 28