How to solve second order differential optimal control or optimization problem？

Question

From a long time, I meet an optimal control problem, but i don't know how to solve it. Well, to be more specific,

Suppose we have following dynamic system and cost function,
\begin{cases} \ddot{y}=\dot{y}-rv \\ J_{max}=\int_{0}^{\infty}e^{-\rho t}[(1-\frac{v}{2})v-\sigma y]dt,v\in[0,1] \end{cases} where $\rho$ is a discount factor, $r$ are parameters, $v$ is control, is a random variable, my question is how to solve this kind of question, determine y(t) and control v(t), $\sigma$ is a random variable with prabability discrete distribution function, for explicitly, $\sigma$ is a uniform distribution, when $ 0<\sigma<5 $，f($\sigma$)=$\frac{1}{5}$, else, f($\sigma$)= 0.

if dymamic system is first order differential equation, I know how to solve it, normally it can be use PMP or HJB. Does anyone can be give me some suggestions or comments? Thanks a lot.

More details are necessary. What is this random variable? Also, if you have a random variable, then one cannot minimize something that depends on the random variable like that but rather minimize the expectation of this quantity. Also, is unclear whether your cost is lower bounded. — KBS, May 26 '22 at 13:50

score 2 · Answer 1 · edited Jun 02 '22 at 15:41

2

Any higher order ordinary differential equation can be turned into a system of first order ordinary differential equations. For example by choosing each next element of the state vector as the derivative of the previous element.

So in your case $\ddot{y}=\dot{y}-rv$ can also be written as \begin{align} \dot{x}_1 &= x_2, \\ \dot{x}_2 &= x_2 - rv, \end{align}

with $x_1=y$ and $x_2=\dot{y}$. After switching to this formulation one can use the normal approach, such as PMP or HJB.

edited Jun 02 '22 at 15:41

amWhy

209,954

answered May 26 '22 at 15:04

Kwin van der Veen

11,858

Could u give me more detail answer, I am not major in this field. But begin to learn how to use this kind of method. I can solve simple variable. Thanks a lot. – Yang Jul 19 '22 at 13:46
@ЯнХе what do you exactly want to know? Namely, in your question you state that you already know how to solve it using PMP or HJB. – Kwin van der Veen Jul 19 '22 at 17:35
@ЯнХе you can have a look at my answer here. – Kwin van der Veen Jul 19 '22 at 18:57
Thank u so much. I will. – Yang Jul 19 '22 at 19:05

score 0 · Answer 2 · answered May 28 '22 at 18:51

In general, you have two options: Dynamic Programming and Pontryagin Maximum Principle. I would personally go for dynamic programming here since the system is linear and the cost is a quadratic polynomial.

Let $\dot{x}=Ax+Bv$, $y=Cx$ be the system in compact state-space form. Since there is an affine term in the cost, we look for a value function of the form

$$V(x)=x^TPx+2b^Tx+c.$$

The HJB equation is given in this case by (where I am assuming that $\sigma$ is NOT a random variable):

$$-\rho V(x)+\min_v\{V_x(x)(Ax+Bv)+v(1-v/2)\}-\sigma Cx=0.$$

However, the input constraint may be difficult to consider within this framework and the PMP is more suitable when such constraints are involved. There are several version of the PMP available for infinite horizon discounted optimal control problems.

How to solve second order differential optimal control or optimization problem？

2 Answers2