Nonlinear Optimal Control Beyond the Hamilton–Jacobi–Bellman Equation

Within the framework of the Linear Quadratic Regulator it is well known that “all roads lead to the Algebraic Riccati Equation”. The deceptive veil of linearity is torn herein by observing that such an algebraic equation generalizes to three different conditions in the nonlinear setting. The first one is obviously the celebrated Hamilton-Jacobi-Bellman partial differential equation arising by relying on Dynamic Programming arguments. However, it is shown that the optimal solution can be equivalently constructed also by characterizing a specific invariant manifold or an invariant distribution of the associated Hamiltonian vector field. While these three strategies reduce to the Algebraic Riccati Equation in the linear case, they instead lead to distinct conditions in the nonlinear setting, with the latter two yielding quasi-linear and linear partial differential equations, respectively, in place of the quadratic Hamilton-Jacobi-Bellman equation.