A Study on Architecture, Algorithms, and Applications of Approximate Dynamic Programming Based Approach to Optimal Control

Penalty function;Reinforcement learning;Neuro-dynamic programming;Model predictive control;Function approximation

Lee, Jong Min ; Chemical Engineering

University:Georgia Institute of Technology

Department:Chemical Engineering

关键词: Penalty function;    Reinforcement learning;    Neuro-dynamic programming;    Model predictive control;    Function approximation;   

Others  :  https://smartech.gatech.edu/bitstream/1853/5048/1/lee_jong_m_200407_phd.pdf

美国|英语

来源: SMARTech Repository

【 摘 要 】

This thesis develops approximate dynamic programming (ADP) strategies suitable for process control problems aimed at overcoming the limitations of MPC, which are the potentially exorbitant on-line computational requirement and the inability to consider the future interplay between uncertainty and estimation in the optimal control calculation. The suggested approach solves the DP only for the state points visited by closed-loop simulations with judiciously chosen control policies.The approach helps us combat a well-known problem of the traditional DP called 'curse-of-dimensionality,' while it allows the user to derive an improved control policy from the initial ones. The critical issue of the suggested method is a proper choice and design of function approximator. A local averager with a penalty term is proposed to guarantee a stably learned control policy as well as acceptable on-line performance.The thesis also demonstrates versatility of the proposed ADP strategy with difficult process control problems. First, a stochastic adaptive control problem is presented. In this application an ADP-based control policy shows an "active" probing property to reduce uncertainties, leading to a better control performance. The second example is a dual-mode controller, which is a supervisory scheme that actively prevents the progression of abnormal situations under a local controller at their onset. Finally, two ADP strategies for controlling nonlinear processes based on input-output data are suggested. They are model-based and model-free approaches, and have the advantage of conveniently incorporating the knowledge of identification data distribution into the control calculation with performance improvement.