Deep-Learning-Incorporated Augmented Reality Application for Engineering Lab Training

Applied Sciences

Sidike Paheding¹  John Estrada²  Quamar Niyaz²  Xiaoli Yang² 

[1]Department of Applied Computing, Michigan Technological University, Houghton, MI 49931, USA

[2]Department of Electrical and Computer Engineering, Purdue University Northwest, Hammond, IN 46323, USA

关键词: artificial intelligence;    augmented reality;    machine learning;    object detection;    computer in education;    lab equipment tutorial;   

DOI  :  10.3390/app12105159

来源: DOAJ

【 摘 要 】

Deep learning (DL) algorithms have achieved significantly high performance in object detection tasks. At the same time, augmented reality (AR) techniques are transforming the ways that we work and connect with people. With the increasing popularity of online and hybrid learning, we propose a new framework for improving students’ learning experiences with electrical engineering lab equipment by incorporating the abovementioned technologies. The DL powered automatic object detection component integrated into the AR application is designed to recognize equipment such as multimeter, oscilloscope, wave generator, and power supply. A deep neural network model, namely MobileNet-SSD v2, is implemented for equipment detection using TensorFlow’s object detection API. When a piece of equipment is detected, the corresponding AR-based tutorial will be displayed on the screen. The mean average precision (mAP) of the developed equipment detection model is $81.4\%$, while the average recall of the model is $85.3\%$. Furthermore, to demonstrate practical application of the proposed framework, we develop a multimeter tutorial where virtual models are superimposed on real multimeters. The tutorial includes images and web links as well to help users learn more effectively. The Unity3D game engine is used as the primary development tool for this tutorial to integrate DL and AR frameworks and create immersive scenarios. The proposed framework can be a useful foundation for AR and machine-learning-based frameworks for industrial and educational training.

【 授权许可】

Unknown