Mobile robot 3D trajectory estimation on a multilevel surface with multimodal fusion of 2D camera features and a 3D light detection and ranging point cloud

International Journal of Advanced Robotic Systems

article

Vinicio Rosas-Cervantes¹  Quoc-Dong Hoang¹  Sooho Woo¹  Soon-Geul Lee¹ 

[1] Integrated Education Institute for Frontier Science and Technology ,(BK21Four), Kyung Hee University;Department of Mechanical Engineering, Kyung Hee University

关键词: Mobile robot;    feature recognition;    odometry and mapping;    3D localization;   

DOI  :  10.1177/17298806221089198

学科分类：社会科学、人文和艺术（综合）

来源: InTech

【 摘 要 】

Nowadays, multi-sensor fusion is a popular tool for feature recognition and object detection. Integrating various sensors allows us to obtain reliable information about the environment. This article proposes a 3D robot trajectory estimation based on a multimodal fusion of 2D features extracted from color images and 3D features from 3D point clouds. First, a set of images was collected using a monocular camera, and we trained a Faster Region Convolutional Neural Network. Using the Faster Region Convolutional Neural Network, the robot detects 2D features from camera input and 3D features using the point’s normal distribution on the 3D point cloud. Then, by matching 2D image features to a 3D point cloud, the robot estimates its position. To validate our results, we compared the trained neural network with similar convolutional neural networks. Then, we evaluated their response for the mobile robot trajectory estimation.

【 授权许可】

CC BY