Automatic detection and classification of coastal Mediterranean fish from underwater images: Good practices for robust training

Frontiers in Marine Science

Marine Science

Jacopo Aguzzi¹  Ignacio A. Catalán²  Amaya Álvarez-Ellacuría²  Hilmar Hinz²  Josep Alós²  Marco Signarioli²  Miquel Palmer²  Guillermo Vizoso²  José-Luis Lisani³  Josep Sánchez³  Antoni Enric Heinrichs-Maquilón³  Marco Francescangeli⁴ 

[1] Department of Renewable Marine Resources, Institut de Ciències del Mar (ICM-Spanish National Research Council), Passeig Maritim de la Barceloneta, Barcelona, Spain;Department of Research Infrastructures for Marine Biological Resources, Anton Dohrn Zoological Sation, Naples, Italy;Marine Ecology Department, Mediterranean Institute for Advanced Studies (IMEDEA) Spanish National Research Council-University of the Balearic Islands (CSIC-UIB), Esporles, Spain;Mathematics and Computer Science Department, University of the Balearic Islands (UIB), Palma, Spain;SARTI Research Group, Electronics Department, Universitat Politècnica de Catalunya (UPC), Vilanova i la Geltrú, Spain;

关键词: deep learning;    mediterranean;    fish;    pre-treatment;    YOLOv5;    EfficientNet;    faster RCNN;   

DOI  :  10.3389/fmars.2023.1151758

 received in 2023-01-26, accepted in 2023-03-20,  发布年份 2023

来源: Frontiers

【 摘 要 】

Further investigation is needed to improve the identification and classification of fish in underwater images using artificial intelligence, specifically deep learning. Questions that need to be explored include the importance of using diverse backgrounds, the effect of (not) labeling small fish on precision, the number of images needed for successful classification, and whether they should be randomly selected. To address these questions, a new labeled dataset was created with over 18,400 recorded Mediterranean fish from 20 species from over 1,600 underwater images with different backgrounds. Two state-of-the-art object detectors/classifiers, YOLOv5m and Faster RCNN, were compared for the detection of the ‘fish’ category in different datasets. YOLOv5m performed better and was thus selected for classifying an increasing number of species in six combinations of labeled datasets varying in background types, balanced or unbalanced number of fishes per background, number of labeled fish, and quality of labeling. Results showed that i) it is cost-efficient to work with a reduced labeled set (a few hundred labeled objects per category) if images are carefully selected, ii) the usefulness of the trained model for classifying unseen datasets improves with the use of different backgrounds in the training dataset, and iii) avoiding training with low-quality labels (e.g., small relative size or incomplete silhouettes) yields better classification metrics. These results and dataset will help select and label images in the most effective way to improve the use of deep learning in studying underwater organisms.

【 授权许可】

Unknown   
Copyright © 2023 Catalán, Álvarez-Ellacuría, Lisani, Sánchez, Vizoso, Heinrichs-Maquilón, Hinz, Alós, Signarioli, Aguzzi, Francescangeli and Palmer

【 图 表 】

FMATS_fmats-2023-1230097_wc_tfx14.tif

FBUIL_fbuil-2023-1174147_wc_tfx24.tif

fdmed-04-1120394-i005.tif

fendo-14-1201198-i004.tif

fbuil-08-1066338-fx1.tif

fmars-10-1151758-i016.tif

fmars-10-1151758-i015.tif

fendo-13-1003017-i002.tif

fphar-14-1189241-fx9.tif

fmars-10-1151758-i012.tif

FMOLB_fmolb-2023-1216192_wc_tfx2.tif

fendo-14-1154561-i003.tif

fchem-11-1174895-fx17.tif

FEART_feart-2023-1169258_wc_tfx3.tif

fchem-11-1174895-fx14.tif

fchem-11-1174895-fx13.tif

FPHAR_fphar-2023-1249041_wc_tfx35.tif

FPHAR_fphar-2023-1249041_wc_tfx17.tif

fmars-10-1151758-i003.tif

fendo-14-1235501-i014.tif

fcimb-13-1200923-i002.tif