Confronting Biodiversity Challenges in Marine Video Monitoring

Introduction

Our current generation is facing the challenges of climate change and global warming. The series of feedback events due to the constant degradation of the environment resulted in anthropogenic defaunation and declination in the quality of the entire ecosystem. For centuries, humans have been keeping a watch on species using various methods like drawings, writings and paintings. Then photographs and videos were introduced to study and track various species. The introduction of video tracking played an impactful role in studying marine biodiversity. Oceans cover 71% of the Earth, which is around 361 million sq km. Monitoring marine species has been a challenge for researchers for decades. Ocean provides a home to a wide range of species, some are even yet to be discovered.

Video surveys are making things easy for the researchers. They have the capability to monitor underwater activities with much greater accuracy and precision. These surveys can search for species and monitor their movements at a depth much deeper than the limit of any diver. Hence the automatic data generation by video devices has proved to be very helpful to the researchers. However, they are gathering information at a rate much greater than any human expert can process. Modern ecologists are collecting data at a previously unimaginable rate, but they themselves are facing data processing bottlenecks. The revolution of deep learning is helping to solve this issue. The current deep-learning techniques are capable of detecting and classifying animal species automatically. 

An accurate classification by any DL model is based on two premises, (1) a rich dataset of each species and (2) a balanced dataset. However, DL models fail to consider the rules of biodiversity, especially the distribution of species abundance, species rarity and ecosystem openness. These rules imply three challenges for DL applications, the imbalance of datasets biases the training of the model, scarce data degrades the performance of the model and ecosystem openness introduces the issue of incorrect classification.

Distribution of Species Abundance   

Classification of any species is not an easy task. The distribution and abundance of species follow a universal rule, which must be accounted for in order to develop an accurate DL model. Numerous studies of world ecosystems have clearly shown that the species abundance is highly skewed, which means a few species are very abundant while many species are present in relatively low numbers. This will automatically generate imbalanced data and the DL model to work efficiently clearly needs balanced data. This issue of imbalance in an abundance of species is sometimes regarded as a "long-tail dataset issue". This is specifically proven in communities of coral reef fishes, where out of a hundred species only a few are abundant and are available at a single location.   

Long tail datasets will not lead to an unbiased model. Classes with more samples in the training dataset will have more impact on the model. In other words, the model will not train enough for species with less abundance. This will eventually affect the model predictions on unknown data. It is impossible to collect a high amount of data on these "rare" species, especially in marine ecosystems. Hence a possible solution to this issue is data augmentation.

 

Data augmentation is a technique used to boost the influence of classes with limited data in image datasets during model training. Various methods are employed for expanding image datasets, including resampling, geometric transformations, kernel filters (e.g., sharpness, colour adjustments, and blurring), and feature space augmentation. Unlike resampling, which duplicates the same image for training, data augmentation involves altering existing images in the dataset to simulate different conditions, thereby preventing deep learning models from overfitting.

Species Rarity

This issue is linked to the first universal rule. This rule states that the species are most abundant in the middle of their geographic range or ecological niche and become scarcer towards the edges. This rarity is a fundamental aspect of biodiversity, where many communities consist mainly of rare species. This rarity means that there are limited training images available for a significant portion of species, making it challenging to gather sufficient data, often referred to as the "scarce data issue".

The possible solution to this issue is Free-Shot Learning (FSL). FSL is an approach where the model is trained to make accurate classifications when it has access to very limited labelled data. Typically, in traditional machine learning, models require a large amount of data for training, but in few-shot learning, the goal is to generalize and adapt to new, unseen categories with just a few examples or "shots" per category. The samples usually range from 1 to 20 images per class. 

The approach is mainly based on the phase of meta-learning. It is often referred to as "learning to learn". Here instead of focusing on solving specific tasks, the algorithm learns a general initialization that can be fine-tuned rapidly for new, similar tasks. For example, to train a model capable of distinguishing five classes from a pool of 64 potential classes, during each training iteration, five classes were randomly chosen. This process helped the model develop the ability to quickly adapt to new tasks, essentially "learning to learn." After this meta-training phase, the model became proficient at handling new tasks, even when provided with very limited images for those tasks.

To date, most FSL algorithms are not able to discriminate more than 20 classes. However, the applications of FSL in marine ecosystem is still under development and constant research is undergoing to increase the number of classes. One approach to overcome this is the Many Classes Few Shots approach. 

Ecosystem Openness

Ecosystem openness in the context of a marine ecosystem refers to the degree of connectivity and exchange of energy, nutrients, and organisms between the marine ecosystem and its surrounding environments. Ecosystem openness applies to deep learning when dealing with a broader range of classes in real-world applications than in the training data. In biodiversity monitoring, this occurs because ecological systems are dynamic and interconnected, making it challenging to predict all possible species or conditions in advance. 

This "open world problem" means that deep learning models need to adapt to new, unseen classes in real-world datasets, which may not have been part of the initial training data. This challenge arises due to the continuous flow of new data and species in the environment, making it impractical to create a closed-world model that accounts for all possibilities. Unfortunately, it is not possible to predict the behaviour of a DL model when facing objects unseen during the training phase. 

An approach to this issue is Open Set Recognition. OSR is a concept that deals with the scenario where a model encounters classes during testing that it hasn't seen during training. Unlike traditional classifiers that assume all possible classes are known in advance, open-set recognition aims to correctly identify known classes while also flagging or rejecting unknown classes. 

Open-set recognition strategies are built on three key principles:

1. Optimizing Classification Space: OSR aims to create distinct clusters for known classes, maximizing the separation between classes (inter-class margin) and minimizing overlap within the same class (intra-class spaces).
2. Distance Metrics: OSR uses carefully chosen or learned distance metrics in machine learning to measure how new images relate to these class clusters. This helps determine the similarity between an image and known classes.
3. Threshold Selection: Machine learning techniques are employed to set thresholds that distinguish between "known classes" and "new classes" based on the calculated distances. The idea is to ensure that images of new, unseen classes are classified as such, preventing them from being misclassified into known class clusters.

Conclusion

To conclude, we can say that to effectively address real-world challenges, deep learning must evolve beyond its initial task of classifying a limited number of classes with abundant and balanced data. Instead, it needs to adapt to the more practical scenarios of imbalanced data, limited data availability, and open-ended data distributions. DL applications are in their early stage in marine ecosystems. The methods discussed above have the potential to overcome the real-world issues faced by current DL models. In fact, the amount of research to face these challenges has increased in the last few years, suggesting a positive growth in the domain.