Fine-Grained Image Analysis With Deep Learning: A Survey

  title={Fine-Grained Image Analysis With Deep Learning: A Survey},
  author={Xiu-Shen Wei and Yi-Zhe Song and Oisin Mac Aodha and Jianxin Wu and Yuxin Peng and Jinhui Tang and Jian Yang and Serge J. Belongie},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
Fine-grained image analysis (FGIA) is a longstanding and fundamental problem in computer vision and pattern recognition, and underpins a diverse set of real-world applications. The task of FGIA targets analyzing visual objects from subordinate categories, e.g., species of birds or models of cars. The small inter-class and large intra-class variation inherent to fine-grained image analysis makes it a challenging problem. Capitalizing on advances in deep learning, in recent years we have… 

Explored An Effective Methodology for Fine-Grained Snake Recognition

A strong multimodal backbone is designed to utilize various meta-information to assist in fine-grained identification and new loss functions to solve the long tail distribution with dataset are provided.

Dual Attention Networks for Few-Shot Fine-Grained Recognition

To generate fine-grained tailored representations for few-shot recognition, a Dual Attention Network (Dual Att-Net) consisting of two dual branches of both hard- and soft-attentions is proposed, which outperforms other existing state-of-the-art methods.

Multi-View Active Fine-Grained Recognition

Comprehensive experiments demonstrate that the proposed method delivers a better performance-efficient trade-off than previous FGVC methods and advanced neural networks.

SR-GNN: Spatial Relation-Aware Graph Neural Network for Fine-Grained Image Categorization

This approach is inspired by the recent advancement in self-attention and graph neural networks (GNNs) approaches to include a simple yet effective relation-aware feature transformation and its refinement using a context-aware attention mechanism to boost the discriminability of the transformed feature in an end-to-end learning process.

Fine-grained Object Categorization for Service Robots

This work proposes a novel deep mixed multi-modality approach based on Vision Transformer and Convolutional Neural Network to improve the performance of FGVC and generates two synthetic fine-grained RGB-D datasets.

Improving Fine-Grained Visual Recognition in Low Data Regimes via Self-Boosting Attention Mechanism

The self-boosting attention mechanism is proposed, a novel method for regularizing the network to focus on the key regions shared across samples and classes to significantly improve fine-grained visual recognition performance on low data regimes and can be incorporated into existing network architectures.

Detecting Fine-Grained Airplanes in SAR Images With Sparse Attention-Guided Pyramid and Class-Balanced Data Augmentation

Airplane detection in synthetic aperture radar (SAR) images has drawn much attention owing to the success of deep learning methods. However, the development of fine-grained airplane detection in SAR

Webly-Supervised Fine-Grained Recognition with Partial Label Learning

This paper utilizes a pre-trained deep model to perform deep descriptor transformation to estimate the positive correlation between these web images, and detects the open-set noises based on the correlation values, and develops a top-k recall optimization loss for firstly assigning a label set towards each web image to reduce the impact of hard label assignment for closed- set noises.

Boosting Few-shot Fine-grained Recognition with Background Suppression and Foreground Alignment

A two-stage background suppression and foreground alignment framework, which is composed of a background activation suppression module, a foreground object alignment (FOA) module, and a local to local (L2L) similarity metric is proposed to enable the proposed method to have the ability to capture subtle differences in confused samples.

SEMICON: A Learning-to-hash Solution for Large-scale Fine-grained Image Retrieval

This paper proposes a suppression-enhancing mask (SEM) based attention and interactive channel transformation (ICON) to learn binary hash codes for dealing with large-scale fine-grained image retrieval tasks and demonstrates superiority over competing methods.



Hyper-class augmented and regularized deep learning for fine-grained image classification

A systematic framework of learning a deep CNN that addresses the challenges from two new perspectives by identifying easily annotated hyper-classes inherent in the fine-grained data and acquiring a large number of hyper-class-labeled images from readily available external sources is proposed.

A Survey of Fine-Grained Image Categorization

This paper reviews the recent progress in fine-grained image categorization, and elaborate different algorithms from strongly supervised learning and weakly supervised learning, and compare their performances on four publicly available benchmarks.

Fine-grained Image Classification by Visual-Semantic Embedding

A visual-semanticembedding model which explores semanticembedding from knowledge bases and text, and further trains a novel end-to-end CNN framework to linearly map image features to a rich semantic embedding space is proposed.

Picking Deep Filter Responses for Fine-Grained Image Recognition

This paper proposes an automatic fine-grained recognition approach which is free of any object / part annotation at both training and testing stages, and conditionally pick deep filter responses to encode them into the final representation, which considers the importance of filter responses themselves.

Fine-Grained Image Classification via Combining Vision and Language

  • Xiangteng HeYuxin Peng
  • Computer Science
    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
The two-stream model combing vision and language (CVL) for learning latent semantic representations is proposed, which demonstrates the CVL approach achieves the best performance on the widely used CUB-200-2011 dataset.

Meta-Reinforced Synthetic Data for One-Shot Fine-Grained Visual Recognition

A meta-learning framework to reinforce the generated images by original images so that these images can facilitate one-shot learning and a Meta Image Reinforcing Network (MetaIRNet) is proposed to conduct one- shot fine-grained recognition as well as image reinforcement.

The application of two-level attention models in deep convolutional neural network for fine-grained image classification

This paper proposes to apply visual attention to fine-grained classification task using deep neural network and achieves the best accuracy under the weakest supervision condition, and is competitive against other methods that rely on additional annotations.

Selective Sparse Sampling for Fine-Grained Image Recognition

A simple yet effective framework, called Selective Sparse Sampling, to capture diverse and fine-grained details and outperforms the state-of-the-art methods on challenging benchmarks including CUB-200-2011, FGVC-Aircraft, and Stanford Cars.

Fine-Grained Recognition as HSnet Search for Informative Image Parts

This work addresses fine-grained image classification by forming the problem as a sequential search for informative parts over a deep feature map produced by a deep Convolutional Neural Network (CNN).

Piecewise Classifier Mappings: Learning Fine-Grained Learners for Novel Categories With Few Examples

An end-to-end trainable deep network inspired by the state-of-the-art fine-grained recognition model and is tailored for the FSFG task is proposed, which generates the decision boundary via learning a set of more attainable sub-classifiers in a more parameter-economic way.