This study contributes to advancing the field of automatic fish event recognition in natural underwater videos, addressing the current gap in studying fish interaction and competition, including predator-prey relationships and mating behaviors. We used the corkwing wrasse (Symphodus melops) as a model, a marine species of commercial importance that reproduces in sea-weed nests built and cared for by a single male. These nests attract a wide range of visitors and are the focal point for behavior such as spawning, chasing, and maintenance. We propose a deep learning methodology to analyze the movement trajectories of the nesting male and classify the associated events observed in their natural habitat. Our approach leverages unsupervised pre-training based on diffusion models, leading to improved feature learning. Additionally, we introduce a dataset comprising 16,937 trajectories across 12 event classes, making it the largest in terms of event class diversity. Our results demonstrate the superior performance of our method compared to several deep architectures. The code for the proposed method and the trajectories can be found at https://github.com/NoeCanovi/Fish_Behaviors_Generative_Models.

Trajectory-based fish event classification through pre-training with diffusion models

Cigdem Beyan
2024-01-01

Abstract

This study contributes to advancing the field of automatic fish event recognition in natural underwater videos, addressing the current gap in studying fish interaction and competition, including predator-prey relationships and mating behaviors. We used the corkwing wrasse (Symphodus melops) as a model, a marine species of commercial importance that reproduces in sea-weed nests built and cared for by a single male. These nests attract a wide range of visitors and are the focal point for behavior such as spawning, chasing, and maintenance. We propose a deep learning methodology to analyze the movement trajectories of the nesting male and classify the associated events observed in their natural habitat. Our approach leverages unsupervised pre-training based on diffusion models, leading to improved feature learning. Additionally, we introduce a dataset comprising 16,937 trajectories across 12 event classes, making it the largest in terms of event class diversity. Our results demonstrate the superior performance of our method compared to several deep architectures. The code for the proposed method and the trajectories can be found at https://github.com/NoeCanovi/Fish_Behaviors_Generative_Models.
2024
Fish behaviorUnderwater videosEvent recognitionTrajectoryGenerative modelsAutoencoderDiffusion modelCorkwing wrasse
File in questo prodotto:
File Dimensione Formato  
IJ24_Trajectory-based fish event classification.pdf

accesso aperto

Tipologia: Versione dell'editore
Licenza: Creative commons
Dimensione 6.65 MB
Formato Adobe PDF
6.65 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11562/1132653
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? 0
social impact