Recognition of visual-related non-driving activities using a dual-camera monitoring system

PATTERN RECOGNITION

卷:116

Article

Yang, Lichao¹  Dong, Kuo²  Ding, Yan³  Brighton, James¹  Zhan, Zhenfei²  Zhao, Yifan¹ 

[1] Cranfield Univ, Sch Aerosp Transport & Mfg, Cranfield, Beds, England

[2] Chongqing Univ, Chongqing Automot Collaborat Innovat Ctr, 174 Shazheng St, Chongqing 400044, Peoples R China

[3] Beijing Inst Technol, Sch Aerosp Engn, Key Lab Dynam & Control Flight Vehicle, Minist Educ, Beijing 100081, Peoples R China

关键词: Driver behaviour;    Level 3 automation;    Computer vision;    Non-driving related task;    activities identification;   

DOI  :  10.1016/j.patcog.2021.107955

来源: Elsevier

【 摘 要 】

For a Level 3 automated vehicle, according to the SAE International Automation Levels definition (J3016), the identification of non-driving activities (NDAs) that the driver is engaging with is of great importance in the design of an intelligent take-over interface. Much of the existing literature focuses on the driver take-over strategy with associated Human-Machine Interaction design. This paper proposes a dual-camera based framework to identify and track NDAs that require visual attention. This is achieved by mapping the driver's gaze using a nonlinear system identification approach, on the object scene, recognised by a deep learning algorithm. A novel gaze-based region of interest (ROI) selection module is introduced and contributes about a 30% improvement in average success rate and about a 60% reduction in average pro-cessing time compared to the results without this module. This framework has been successfully demon-strated to identify five types of NDA required visual attention with an average success rate of 86.18%. The outcome of this research could be applicable to the identification of other NDAs and the tracking of NDAs within a certain time window could potentially be used to evaluate the driver's attention level for both automated and human-driving vehicles. (c) 2021 Elsevier Ltd. All rights reserved.

【 授权许可】

Free