Data-driven policy iteration for nonlinear optimal control problems

The design of optimal control laws for nonlinear systems is tackled without knowledge of the underlying plant and of a functional description of the cost function. The proposed data-driven method is based only on real-time measurements of the state of the plant and of the (instantaneous) value of the reward signal and relies on a combination of ideas borrowed from the theories of optimal and adaptive control problems. As a result, the architecture implements a policy iteration strategy in which, hinging on the use of neural networks, the policy evaluation step and the computation of the relevant information instrumental for the policy improvement step are performed in a purely continuous-time fashion. Furthermore, the desirable features of the design method, including convergence rate and robustness properties, are discussed. Finally, the theory is validated via two benchmark numerical simulations.