Special issue on “Fine-grained categorization in ecological multimedia”

In the last decades, the extensive research on recognition/classification has mainly focused on distinguishing classes, which are somehow dissimilar. Recently, the computer vision, machine learning and multimedia scientific communities have addressed with increasing interest the problem of fine-grained categorization: it refers to a subordinate level of classification, such as identifying different species of animals, plants, fruits etc. in different types of multimedia (e.g., audio, images, videos). Of course, this task represents a harder challenge than the “basic” classification one, because the discriminative features among the classes are more subtle and difficult to identify. Automated systems performing such tasks might provide significant support to many applications, especially those requiring specialized human knowledge as in the ecology domain: indeed, it is really complex (if not impossible) for non-expert people to distinguish between plant types or animal species, where inter-class similarity might be very high. Moreover, especially for the ecological context, the need for such automatic tools has become even greater due to technological advances leading to a massive collection of multimedia content (images, videos and audios) whose analysis still requires the employment of expert human operators. This special issue specifically addresses the development of methods for extracting distinctive features employed for categorizing animal and plant species from images, videos and audios as well as approaches for classifying ecology time-series. – Fine-grained classification of animal species. Atanboria et al. in “Automatic classification of flying bird species using computer vision techniques” tackle the problem of in-flight bird species identification in videos using a set of visual and motion cues. In particular, the authors devised novel motion features, such as curvature of flight trajectories and wing beat frequency, to improve classification performance. Indeed, using the rich set of extracted features with a Normal Bayes classifier, the reported classification results are of about 92%, which significantly outperform state-of-the-art ones. Venkitasubramaniana et al. in “Wildlife Recognition in Nature Documentaries with Weak Supervision from Subtitles and External Data” propose a weakly supervised framework to annotate objects such as animals in documentary videos. More specifically, the devised method starts from classifiers trained on general imagery (e.g., ImageNet), which are then iteratively improved using text features extracted from subtitles. Experiments on a challenging dataset of wildlife documentaries reported an accuracy (measured as F1 metrics) of about 70%, which outperforms classifiers (even deep-learning based ones) trained on ImageNet. – Plant categorization and phenotyping. Champ et al. in “Categorizing plant images at the variety level: did you say fine-grained?” address the problem of categorizing plant images at the variety level by introducing two image datasets (one of grape leaf and the other one of rice seed) related to different scenarios on large-scale plant resources. The two datasets greatly differ, in terms of visual appearance, provide a valuable resources for testing the generalization capabilities of image-based plant categorization methods. The extensive experimental results showed that recent state-of-the-art classification methods (including convolutional neural networks) achieve good performance over the rice seed dataset while fail to tackle the grape leaf one. Minervini et al. in “Finely-grained annotated datasets for image-based plant phenotyping” also propose a collection of benchmark datasets of raw and annotated top-view color images of rosette plants. These datasets contain annotations for testing several computer vision tasks from plant detection to plant segmentation to leaf detection and counting. Almeida et al. in “Phenological Visual Rhythms: Compact Representations for Fine-Grained Plant Species Identification” propose an approach based on visual rhythm for plant species identification. Experimental results reported high accuracy of the developed approaches compared to existing methods. – Time-series classification for environmental monitoring. Faria et al. in “Time Series-based Classifier Fusion for Fine-Grained Plant Species Recognition” start from the assumption that it is almost impossible to perform fine-grained categorization of image plants by using only visual cues and only single classifier. To address this issue, the authors combined and fused multiple classifiers (suitably selected using diversity principles) to categorize time-series extracted by analyzing temporal variation of RGB colors in hemispherical lens camera images. The employed framework showed satisfying performance achieving better results than state-of-the-art methods. Fortuna et al. in “A new fine-grained classification strategy for solar daily radiation patterns” develop a classifier able to distinguish between solar radiation daily patterns. In particular, the authors devised a new set of descriptors, namely, area ratio and intermittency, that allowed for a better classification with respect to traditional approaches based on neural networks that use classic features such as solar altitude, albedo, atmospheric transparency and cloudiness. We would like to thank, first, the authors for their innovative contribution to this special issue, then, all the reviewers for the great effort and the valuable feedback provided to the submitted manuscripts. We also would like to extend our thanks to the Editor in Chief, Gabriella Sanniti di Baja, and the whole editorial staff of Pattern Recognition Letters for recognizing the value and the contribution that this special issue may provide to future research on the interdisciplinary research area between pattern recognition and ecology. We believe that advances on this new research area will provide great benefits to safeguard the world we live in.