End-to-End Multiview Gesture Recognition for Autonomous Car Parking System

deep learning;video classification;dynamic hand gesture recognition;embedded platform;automotive;vehicle self-parking

Ben Amara, Hasseneadvisor:Karray, Fakhreddine ; affiliation1:Faculty of Engineering ; Karray, Fakhreddine ;

University of Waterloo

关键词: deep learning;    embedded platform;    Master Thesis;    video classification;    vehicle self-parking;    automotive;    dynamic hand gesture recognition;   

Others  :  https://uwspace.uwaterloo.ca/bitstream/10012/14657/3/BenAmara_Hassene.pdf

瑞士|英语

来源: UWSPACE Waterloo Institutional Repository

【 摘 要 】

The use of hand gestures can be the most intuitive human-machine interaction medium.The early approaches for hand gesture recognition used device-based methods. Thesemethods use mechanical or optical sensors attached to a glove or markers, which hindersthe natural human-machine communication. On the other hand, vision-based methods arenot restrictive and allow for a more spontaneous communication without the need of anintermediary between human and machine. Therefore, vision gesture recognition has beena popular area of research for the past thirty years.Hand gesture recognition finds its application in many areas, particularly the automotiveindustry where advanced automotive human-machine interface (HMI) designers areusing gesture recognition to improve driver and vehicle safety. However, technology advancesgo beyond active/passive safety and into convenience and comfort. In this context,one of America’s big three automakers has partnered with the Centre of Pattern Analysisand Machine Intelligence (CPAMI) at the University of Waterloo to investigate expandingtheir product segment through machine learning to provide an increased driver convenienceand comfort with the particular application of hand gesture recognition for autonomouscar parking.In this thesis, we leverage the state-of-the-art deep learning and optimization techniquesto develop a vision-based multiview dynamic hand gesture recognizer for self-parking system.We propose a 3DCNN gesture model architecture that we train on a publicly availablehand gesture database. We apply transfer learning methods to fine-tune the pre-trainedgesture model on a custom-made data, which significantly improved the proposed systemperformance in real world environment. We adapt the architecture of the end-to-end solutionto expand the state of the art video classifier from a single image as input (fed bymonocular camera) to a multiview 360 feed, offered by a six cameras module. Finally, weoptimize the proposed solution to work on a limited resources embedded platform (NvidiaJetson TX2) that is used by automakers for vehicle-based features, without sacrificing theaccuracy robustness and real time functionality of the system.