Edge computing at sea: high-throughput classification of in-situ plankton imagery for adaptive sampling

Frontiers in Marine Science

Marine Science

Charles Cousin¹  Cedric Guigand¹  Dominic Daprano²  Malhar M. Damle²  Christopher M. Sullivan³  Robert K. Cowen⁴  Moritz S. Schmid⁴  Su Sponaugle⁵ 

[1] Bellamare LLC, San Diego, CA, United States;Center for Quantitative and Life Sciences, Oregon State University, Corvallis, OR, United States;Center for Quantitative and Life Sciences, Oregon State University, Corvallis, OR, United States;College of Earth, Ocean, and Atmospheric Sciences, Oregon State University, Corvallis, OR, United States;Hatfield Marine Science Center, Oregon State University, Newport, OR, United States;Hatfield Marine Science Center, Oregon State University, Newport, OR, United States;Department of Integrative Biology, Oregon State University, Corvallis, OR, United States;

关键词: adaptive sampling;    edge computing;    ocean technology;    underwater imaging;    plankton ecology;    machine learning;    data visualization;    California Current;   

DOI  :  10.3389/fmars.2023.1187771

 received in 2023-03-16, accepted in 2023-05-15,  发布年份 2023

来源: Frontiers

【 摘 要 】

The small sizes of most marine plankton necessitate that plankton sampling occur on fine spatial scales, yet our questions often span large spatial areas. Underwater imaging can provide a solution to this sampling conundrum but collects large quantities of data that require an automated approach to image analysis. Machine learning for plankton classification, and high-performance computing (HPC) infrastructure, are critical to rapid image processing; however, these assets, especially HPC infrastructure, are only available post-cruise leading to an ‘after-the-fact’ view of plankton community structure. To be responsive to the often-ephemeral nature of oceanographic features and species assemblages in highly dynamic current systems, real-time data are key for adaptive oceanographic sampling. Here we used the new In-situ Ichthyoplankton Imaging System-3 (ISIIS-3) in the Northern California Current (NCC) in conjunction with an edge server to classify imaged plankton in real-time into 170 classes. This capability together with data visualization in a heavy.ai dashboard makes adaptive real-time decision-making and sampling at sea possible. Dual ISIIS-Deep-focus Particle Imager (DPI) cameras sample 180 L s-1, leading to >10 GB of video per min. Imaged organisms are in the size range of 250 µm to 15 cm and include abundant crustaceans, fragile taxa (e.g., hydromedusae, salps), faster swimmers (e.g., krill), and rarer taxa (e.g., larval fishes). A deep learning pipeline deployed on the edge server used multithreaded CPU-based segmentation and GPU-based classification to process the imagery. AVI videos contain 50 sec of data and can contain between 23,000 - 225,000 particle and plankton segments. Processing one AVI through segmentation and classification takes on average 3.75 mins, depending on biological productivity. A heavyDB database monitors for newly processed data and is linked to a heavy.ai dashboard for interactive data visualization. We describe several examples where imaging, AI, and data visualization enable adaptive sampling that can have a transformative effect on oceanography. We envision AI-enabled adaptive sampling to have a high impact on our ability to resolve biological responses to important oceanographic features in the NCC, such as oxygen minimum zones, or harmful algal bloom thin layers, which affect the health of the ecosystem, fisheries, and local communities.

【 授权许可】

Unknown   
Copyright © 2023 Schmid, Daprano, Damle, Sullivan, Sponaugle, Cousin, Guigand and Cowen