A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Chinese Journal of Mechanical Engineering

Original Article

Hong Chen¹  Liwen Wang²  Yanjun Huang²  Shuo Yang³  Kang Yuan⁴ 

[1]College of Electronic and Information Engineering, Tongji University, 201804, Shanghai, China

[2]School of Automotive Studies, Tongji University, 201804, Shanghai, China

[3]School of Automotive Studies, Tongji University, 201804, Shanghai, China

[4]Shanghai Institute of Intelligent Science and Technology, Tongji University, 201804, Shanghai, China

[5]Shanghai Institute of Intelligent Science and Technology, Tongji University, 201804, Shanghai, China

[6]College of Electronic and Information Engineering, Tongji University, 201804, Shanghai, China

关键词: Model predictive control;    Reinforcement learning;    Autonomous vehicles;   

DOI  :  10.1186/s10033-023-00904-7

 received in 2022-12-20, accepted in 2023-06-08,  发布年份 2023

来源: Springer

【 摘 要 】

Model predictive control is widely used in the design of autonomous driving algorithms. However, its parameters are sensitive to dynamically varying driving conditions, making it difficult to be implemented into practice. As a result, this study presents a self-learning algorithm based on reinforcement learning to tune a model predictive controller. Specifically, the proposed algorithm is used to extract features of dynamic traffic scenes and adjust the weight coefficients of the model predictive controller. In this method, a risk threshold model is proposed to classify the risk level of the scenes based on the scene features, and aid in the design of the reinforcement learning reward function and ultimately improve the adaptability of the model predictive controller to real-world scenarios. The proposed algorithm is compared to a pure model predictive controller in car-following case. According to the results, the proposed method enables autonomous vehicles to adjust the priority of performance indices reasonably in different scenarios according to risk variations, showing a good scenario adaptability with safety guaranteed.

【 授权许可】

CC BY   
© The Author(s) 2023

【 图 表 】

40798_2023_601_Article_IEq5.gif

40798_2023_601_Article_IEq3.gif

40798_2023_601_Article_IEq2.gif

Fig. 4

Fig. 3

Fig. 2

Fig. 3

Fig. 1

Fig. 3

Fig. 3

Fig. 4

41239_2023_389_Article_IEq13.gif

Fig. 1

Fig. 1

Fig. 3

41239_2023_389_Article_IEq8.gif

Fig. 2

Fig. 2

Fig. 1

40708_2023_197_Article_IEq87.gif

40708_2023_197_Article_IEq86.gif

40708_2023_197_Article_IEq84.gif

Fig. 1

40708_2023_197_Article_IEq81.gif

40708_2023_197_Article_IEq83.gif

40708_2023_197_Article_IEq82.gif

Fig. 4

40708_2023_197_Article_IEq79.gif

40708_2023_197_Article_IEq78.gif

Fig. 3

40708_2023_197_Article_IEq75.gif

40708_2023_197_Article_IEq70.gif

Fig. 1

40708_2023_197_Article_IEq68.gif

Fig. 2

Fig. 4

40708_2023_197_Article_IEq62.gif

Fig. 1

40708_2023_197_Article_IEq60.gif

40708_2023_197_Article_IEq59.gif

Fig. 1

40708_2023_197_Article_IEq55.gif

40708_2023_197_Article_IEq54.gif

Fig.9

Fig. 5

40708_2023_197_Article_IEq51.gif

40708_2023_197_Article_IEq50.gif

40708_2023_197_Article_IEq48.gif

40708_2023_197_Article_IEq47.gif

40708_2023_197_Article_IEq46.gif

Fig. 1

Fig. 2

Fig. 1

Fig.8

Fig. 1

Fig. 1

40708_2023_197_Article_IEq35.gif

40798_2023_598_Article_IEq30.gif

40798_2023_598_Article_IEq29.gif

40798_2023_598_Article_IEq28.gif

40798_2023_598_Article_IEq27.gif

Fig. 2

40798_2023_598_Article_IEq26.gif

Fig. 1

40798_2023_598_Article_IEq10.gif

40798_2023_598_Article_IEq9.gif

40798_2023_598_Article_IEq8.gif

40798_2023_598_Article_IEq7.gif

40798_2023_598_Article_IEq6.gif

40798_2023_598_Article_IEq5.gif

12938_2023_1124_Article_IEq44.gif

Fig. 3

Fig. 1

Fig. 6

Fig. 1

Fig. 5

Fig. 4

42004_2023_934_Article_IEq6.gif

Fig. 3

Fig. 7

Fig. 2

Fig. 7

Fig. 11

42004_2023_944_Article_IEq9.gif

40854_2023_486_Article_IEq44.gif

Fig. 4

Fig. 9

Fig. 5

Fig. 4

Fig. 4

Fig. 1