Neuro_Dynamic Programming and Reinforcement Learning for Optimal Energy Management of a Series Hydraulic Hybrid Vehicle Considering Engine Transient Emissions.

Neuro Dynamic Programming;Hybrid Vehicle;Supervisory Controller / Power Management;Soot and NOx Model;Optimal Control;Transient Diesel Emission;Mechanical Engineering;Engineering;Mechanical Engineering

Johri, RajitPeng, Huei ;

University of Michigan

关键词: Neuro Dynamic Programming;    Hybrid Vehicle;    Supervisory Controller / Power Management;    Soot and NOx Model;    Optimal Control;    Transient Diesel Emission;    Mechanical Engineering;    Engineering;    Mechanical Engineering;   

Others  :  https://deepblue.lib.umich.edu/bitstream/handle/2027.42/89829/rajit_1.pdf?sequence=1&isAllowed=y

瑞士|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

Sequential decision problems under uncertainty are encountered in various fields such as optimal control and operations research. In this dissertation, Neuro-Dynamic Programming (NDP) and Reinforcement Learning (RL) are applied to address policy optimization problems with multiple objectives and large design state space. Dynamic Programming (DP) is well suited for determining an optimal solution for constrained nonlinear model based systems. However, DP suffers from curse of dimensionality i.e. computational effort grows exponentially with state space. The new algorithms address this problem and enable practical application of DP to a much broader range of problems. The other contribution is to design fast and computationally efficient transient emission models.The power management problem for a hybrid vehicle can be formulated as an infinite time horizon stochastic sequential decision-making problem. In the past, policy optimization has been applied successfully to design optimal supervisory controller for best fuel economy. Static emissions have been considered too but engine research has shown that transient operation can have significant impact on real-world emissions. Modeling transient emissions results in addition of more states. Therefore, the problem with multiple objectives i.e. minimize fuel consumption and transient particulate and NOX emissions, becomes computationally intractable by DP. This research captures the insight with models and brings it into the supervisory controller design.A self-learning supervisory controller is designed based on the principles of NDP and RL. The controller starts ;;naïve” i.e. with no knowledge to control the onboard power but learns to do so in an optimal manner after interacting with the system. The controller tries to minimize multiple objectives and continues to evolve until a global solution is achieved. Virtual sensors for predicting real-time transient particulate and NOX emissions are developed using neuro-fuzzy modeling technique, which utilizes a divide-and-conquer strategy. The highly nonlinear engine operating space is partitioned into smaller subspaces and a separate local model is trained to for each subspace.Finally, the supervisory controller along with virtual emission sensors is implemented and evaluated using the Engine-In-the-Loop (EIL) setup. EIL is a unique facility to systematically evaluate control methodologies through concurrent running of real engine and a virtual hybrid powertrain.