Telemedicine supported by Augmented Reality: an interactive guide for untrained people in performing an ECG test

BioMedical Engineering OnLine

Maria Romano³  Mario Cesarelli³  Pasquale Ambruosi³  Raffaele Vertucci²  Fabio Narducci¹  Paolo Bifulco³ 

[1]VRLab, University of Salerno, Salerno, Italy

[2]Selex ES, Giugliano, Italy

[3]Department of Electrical Engineering and Information Technology, University of Naples “Federico II”, Naples, Italy

关键词: Electrode placement;    ECG device operation;    Untrained user;    Augmented Reality;   

Others  :  1084216
DOI  :  10.1186/1475-925X-13-153

 received in 2014-03-27, accepted in 2014-10-16,  发布年份 2014

【 摘 要 】

Background

In many telemedicine applications, the correct use of medical device at the point of need is essential to provide an appropriate service. Some applications may require untrained people to interact with medical devices and patients: care delivery in transportation, military actions, home care and telemedicine training.

Appropriate operation of medical device and correct connection with patient’s body are crucial. In these scenarios, tailored applications of Augmented Reality can offer a valid support by guiding untrained people at the point of need. This study aims to explore the feasibility of using Augmented Reality in telemedicine applications, by facilitating an acceptable use of biomedical equipment by any unskilled person. In particular, a prototype system was built in order to estimate how untrained users, with limited or no knowledge, can effectively interact with an ECG device and properly placing ECG electrodes on patient’s chest.

Methods

An Augmented Reality application was built to support untrained users in performing an ECG test. Simple markers attached to the ECG device and onto patient’s thorax allow camera calibration. Once objects and their pose in the space are recognized, the video of the current scene is enriched, in real-time, with additional pointers, text boxes and audio that help the untrained operator to perform the appropriate sequence of operations. All the buttons, switches, ports of the ECG device together with the location of precordial leads were coded and indicated. Some user’s voice commands were also included to improve usability.

Results

Ten untrained volunteers, supported by the augmented reality, were able to carry out a complete ECG test first on a mannequin and then on a real patient in a reasonable time (about 8 minutes on average). Average positioning errors of precordial electrodes resulted less than 3 mm for the mannequin and less than 7 mm for the real patient. These preliminary findings suggest the effectiveness of the developed application and the validity of clinical ECG recordings.

Conclusion

This application can be adapted to support the use of other medical equipment as well as other telemedicine tasks and it could be performed with a Tablet or a Smartphone.

【 授权许可】

   
2014 Bifulco et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

【 参考文献 】

• [1]Wootton R, Craig J, Patterson V: Introduction to telemedicine. 2nd edition. London: The Royal Society of Medicine Press Ltd; 2006.
• [2]Norris AC: Essentials of Telemedicine and Telecare. John Wiley & Sons, Ltd. Published Online: 23 April 2002; Print ISBN: 97804715315; Online ISBN: 9780470846346; doi:10.1002/0470846348
• [3]Bracale M, Monteagudo JL, Rossing N, Hakansson S, Reponen J, Bifulco P: Telemedicine in Europe - the TELEPLANS project. J Telemed Telecare 2002, 8(Suppl 5):308-310.
• [4]Weinlich M, Nieuwkamp N, Stueben U, Marzi I, Walcher F: Telemedical assistance for in-flight emergencies on intercontinental commercial aircraft. J Telemed Telecare 2009, 15(Suppl 8):409-413.
• [5]Beltrame F, Boddy K, Maryni P: Adopting telemedicine services in the airline framework. IEEE Trans Inf Technol Biomed June 2001, 5(Suppl 2):171-177.
• [6]Ferrer-Roca O, Díaz De León RD, de Latorre FJ, Suárez-Delgado M, Di Persia L, Cordo M: Aviation medicine: challenges for telemedicine. J Telemed Telecare 2002, 8(Suppl 1):1-4.
• [7]Amenta F, Dauri A, Rizzo N: Telemedicine and medical care to ships without a doctor on board. J Telemed Telecare 1998, 4(Suppl 1):44-45.
• [8]Gargiulo P, Helgason T, Bifulco P, Bracale M: Telemedicine applications for seamen in the North Atlantic. In Proceed. of The 12th Nordic Baltic Conference on Biomedical Engineering & Medical Physics. Reykjavik, Iceland; 2002.
• [9]Wilson KL, Doswell JT, Fashola OS, Debeatham W, Darko N, Walker TM, Danner OK, Matthews LR, Weaver WL: Using augmented reality as a clinical support tool to assist combat medics in the treatment of tension pneumothoraces. Mil Med 2013, 178(Suppl 9):981-985.
• [10]Bracale M, Cesarelli M, Bifulco P: Telemedicine services for two islands in the Bay of Naples. J Telemed Telecare 2002, 8(Suppl 1):5-10.
• [11]Martiniuk A, Negin J, Hersch F, Dalipanda T, Jagilli R, Houasia P, Gorringe L, Christie A: Telemedicine in the Solomon Islands: 2006 to 2009. J Telemed Telecare 2011, 17(Suppl 5):251-256.
• [12]Drew BJ, Sommargren CE, Schindler DM, Zegre J, Benedict K, Krucoff MW: Novel electrocardiogram configurations and transmission procedures in the prehospital setting: effect on ischemia and arrhythmia determination. J Electrocardiol October 2006, 39(Suppl 4):157-160.
• [13]Oldenburg M, Baur X, Schlaich C: Assessment of three conventional automated external defibrillators in seafaring telemedicine. Occup Med (Lond) March 2012, 62(Suppl 2):117-122.
• [14]Silver HJ, Wellman NS, Galindo-Ciocon D, Johnson P: Family caregivers of older adults on home enteral nutrition have multiple unmet task-related training needs and low overall preparedness for caregiving. J Am Diet Assoc January 2004, 104(Suppl 1):43-50.
• [15]Silver HJ, Wellman NS: Family caregiver training is needed to improve outcomes for older adults using home care technologies. J Am Diet Assoc June 2002, 102(Suppl 6):831-836.
• [16]Romano M, Cesarelli M, D’Addio G, Mazzoleni MC, Bifulco P, Ferrara N, Rengo F: Telemedicine fetal phonocardiography surveillance: An Italian satisfactory experience. Stud Health Technol Inform 2010, 155:176-181.
• [17]Shah MN, McDermott R, Gillespie SM, Philbrick EB, Nelson D: Potential of telemedicine to provide acute medical care for adults in senior living communities. Acad Emerg Med February 2013, 20(Suppl 2):162-168.
• [18]Shah MN, Morris D, Jones CM, Gillespie SM, Nelson DL, McConnochie KM, Dozier A: A qualitative evaluation of a telemedicine-enhanced emergency care program for older adults. J Am Geriatr Soc 2013, 61(Suppl 4):571-576.
• [19]Thomas B, Close B, Donoghue J, Squires J, De Bondi P, Morris M, Piekarski W: ARQuake: An Outdoor/Indoor Augmented Reality First Person Application. In Wearable Computers, The Fourth International Symposium on: 16-17 October 2000. Atlanta, Georgia, USA; 2000:139-146.
• [20]Van Krevelen DWF, Poelman R: A Survey of Augmented Reality Technologies, Applications and Limitations. The Int J Virtual Reality 2010, 9(Suppl 2):1-20.
• [21]Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B: Recent Advances in Augmented Reality. IEEE Computer Graphics Applications November 2001, 21(Suppl 6):34-47.
• [22]Sutherland C, Hashtrudi-Zaad K, Sellens R, Abolmaesumi P, Mousavi P: An augmented reality haptic training simulator for spinal needle procedures. IEEE Trans Biomed Eng November 2013, 60(Suppl 11):3009-3018.
• [23]Alberto Borghese N, Frosio I: Compact tracking of surgical instruments through structured markers. Med Biol Eng Comput July 2013, 51(Suppl 7):823-833.
• [24]State A, Livingston MA, Hirota G, Garrett WF, Whitton MC, Fuchs H, Pisano ED: Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques (SIGGRAPH 96) 1996, 439-446.
• [25]Fuchs H, Livingston MA, Raskar R, State A, Crawford JR, Rademacher P, Drake SH, Meyer AA: Augmented Reality Visualization for Laparoscopic Surgery. In Proceedings of the First International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 98). Heidelberg, Germany: Springer; 1998:934-943.
• [26]Navab N, Traub J, Sielhorst T, Feuerstein M, Bichlmeier C: Action- and workflow-driven augmented reality for computer-aided medical procedures. IEEE Comput Graph Appl 2007, 27(Suppl 5):10-14.
• [27]Feuerstein M, Mussack T, Heining SM, Navab N: Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 2008, 27(Suppl 1):355-369.
• [28]Vogt S, Khamene A, Sauer F: Reality augmentation for medical procedures: System architecture, single camera marker tracking, and system evaluation. Int J Computer Vision 2006, 70(Suppl 2):179-190.
• [29]Burdea GC: Virtual rehabilitation - benefits and challenges. Methods Inf Med 2003, 42(Suppl 5):519-523.
• [30]Villiger M, Bohli D, Kiper D, Pyk P, Spillmann J, Meilick B, Curt A, Hepp-Reymond MC, Hotz-Boendermaker S, Eng K: Virtual reality-augmented neurorehabilitation improves motor function and reduces neuropathic pain in patients with incomplete spinal cord injury. Neurorehabil Neural Repair 2013, 27(Suppl8):675-683.
• [31]Kotranza A, Lok B: Virtual human + tangible interface = mixed reality human an initial exploration with a virtual breast exam patient. In Proceedings of Virtual Reality Conference, 2008. VR’08. IEEE. Reno, Nevada, USA; 2008:99-106.
• [32]State A, Chen DT, Tector C, Brandt A, Chen H, Ohbuchi R, Bajura M, Fuchs H: Case Study: Observing a Volume Rendered Fetus within a Pregnant Patient. In Proceedings of IEEE Conference on Visualization’94: 17-21 October 1994. Washington DC, USA; 1994:364-368.
• [33]Sielhorst T, Obst T, Burgkart R, Riener R, Navab N: An Augmented Reality Delivery Simulator for medical training. In Proceedings of the Workshop AMI-ARCS 2004 Augmented Environments for Medical Imaging: 30 September 2004. Rennes, France: Rennes, France; 2004:11-20.
• [34]Masero V, Sanchez FM, Uson J: A telemedicine system for enabling teaching activities. J Telemed Telecare 2000, 6:86-88.
• [35]Graschew G, Rakowsky S, Balanou P, Schlag PM: Interactive telemedicine in the operating theatre of the future. J Telemed Telecare 2000, 6:20-24.
• [36]Green M, White L: The Cave-let: a low-cost projective immersive display. J Telemed Telecare 2000, 6:24-26.
• [37]Van Dijk H, Hermens HJ: Distance training for the restoration of motor function. J Telemed Telecare 2004, 10:63-71.
• [38]Henderson S, Feiner S: Exploring the benefits of augmented reality documentation for maintenance and repair. Visualization Computer Graphics, IEEE Transactions on 2011, 17(10):1355-1368.
• [39]McCann K, Holdgate A, Mahammad R, Waddington A: Accuracy of ECG electrode placement by emergency department clinicians. Emerg Med Australas 2007, 19(5):442-448.
• [40]Rudiger A, Hellermann JP, Mukherjeec R, Follath F, Turina J: Electrocardiographic artifacs due to electrode misplacement and their frequency in different clinical settings. Am J Emer Med 2007, 25:174-178.
• [41]Wenger W, Kligfield P: Variability of precordial electrode placement during routine electrocardiography. J Electrocardiol 1996, 29:179-184.
• [42]Harrigan RA, Chan TC, Brady WJ: Electrocardiographic electrode misplacement, misconnection, and artifact. J Emerg Med December 2012, 43(Suppl 6):1038-1044.
• [43]Rudiger A, Hellermann JP, Mukherjee R, Follath F, Turina J: Electrocardiographic artifacts due to electrode misplacement and their frequency in different clinical settings. Am J Emerg Med 2007, 25(Suppl 2):174-178.
• [44]ARtoolkit official website [http://www.hitl.washington.edu/artoolkit/ webcite]
• [45]Abate AF, Narducci F, Ricciardi S: Mixed reality environment for mission critical systems servicing and repair. In 5th International Conference, VAMR 2013, Held as Part of HCI International 2013, Proceedings, Part II: 21-26 July 2013. Las Vegas, Nevada, USA: Springer Berlin Heidelberg, 2013; 2013:201-210.
• [46]Yoon JH, Park JS, Kim C: Increasing camera pose estimation accuracy using multiple markers. In Advances in Artificial Reality and Tele-Existence. Berlin Heidelberg: Springer; 2006:239-248.
• [47]Abawi DF, Bienwald J, Dorner R: Accuracy in optical tracking with fiducial markers: An accuracy function for ARToolKit. In Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality 2004. IEEE Computer Society; 260-261.
• [48]Kato H, Billinghurst M: Marker tracking and HDM calibration for a video-based augmented reality conferencing system. In Augmented Reality, 1999. (IWAR’99) Proceedings. 2nd IEEE and ACM International Workshop. IEEE: IEEE; 1999:85-94.
• [49]Lundsgaard C, Van Slyke DD: Studies of lung volume. I relation between thorax size and lung volume in normal adults. J Exper Med 1918, 27(1):65-86.
• [50]Wilson F, Kossmann C, Burch G, Goldberger E, Graybiel A, Hecht H, Johnston F, Lepeschkin E, Myers G: Recommendations for standardization of electrocardiographic and vector cardiographic leads. Circulation 1954, 10:564-573.
• [51]Kania M, Rix H, Fereniec M, Zavala-Fernandez H, Janusek D, Mroczka T, Maniewski R: The effect of precordial lead displacement on ECG morphology. Med Biol Eng Comput 2014, 52(2):109-119.
• [52]Rajaganeshan R, Ludlam CL, Francis DP, Parasramka SV, Sutton R: Accuracy in ECG lead placement among technicians, nurses, general physicians and cardiologists. Int J Clin Pract 2008, 62(Supp1):65-70.
• [53]Farbiz F, Cheok AD, Liu W, Zhou ZY, Xu K, Prince S, Billinghurst M, Live KH: Three-dimensional content for augmented reality. IEEE Trans Multimedia 2005, 7(3):514-523. doi:10.1109/TMM.2005.846787
• [54]Pati S, Erat O, Wang L, Weidert S, Euler E, Navab N, Fallavollita P: Accurate pose estimation using single marker single camera calibration system. Proc. SPIE 8671, Medical Imaging 2013: Image-Guided Procedures, Robotic Interventions, and Modeling, 867126 2013. doi:10.1117/12.2006776