Deep Cropping via Attention Box Prediction and Aesthetics Assessment

@article{Wang2017DeepCV,
  title={Deep Cropping via Attention Box Prediction and Aesthetics Assessment},
  author={Wenguan Wang and Jianbing Shen},
  journal={2017 IEEE International Conference on Computer Vision (ICCV)},
  year={2017},
  pages={2205-2213}
}
  • Wenguan Wang, Jianbing Shen
  • Published 1 October 2017
  • Computer Science
  • 2017 IEEE International Conference on Computer Vision (ICCV)
We model the photo cropping problem as a cascade of attention box regression and aesthetic quality classification, based on deep learning. A neural network is designed that has two branches for predicting attention bounding box and analyzing aesthetics, respectively. The predicted attention box is treated as an initial crop window where a set of cropping candidates are generated around it, without missing important information. Then, aesthetics assessment is employed to select the final crop as… 
View on IEEE
arxiv.org
74 Citations
Highly Influential Citations
8
Background Citations
37
Methods Citations
18
Results Citations
1

Figures and Tables from this paper

74 Citations

A Deep Network Solution for Attention and Aesthetics Aware Photo Cropping
TLDR
A neural network that has two branches for attention box prediction (ABP) and aesthetics assessment (AA) that produces high-quality cropping results, even with the limited availability of training data for photo cropping.
Image Cropping with Composition and Saliency Aware Aesthetic Score Map
TLDR
This paper proposes an interpretable image cropping model that can be used to localize aesthetically important regions in an image, which sheds light on the composition rules learned by the model.
A2-RL: Aesthetics Aware Reinforcement Learning for Image Cropping
TLDR
This work forms the aesthetic image cropping as a sequential decision-making process and proposes a weakly supervised Aesthetics Aware Reinforcement Learning (A2-RL) framework to address this problem.
An End-to-End Neural Network for Image Cropping by Learning Composition from Aesthetic Photos
TLDR
A deep learning based framework to learn the objects composition from photos with high aesthetic qualities is proposed, where an anchor region is detected through a convolutional neural network (CNN) with the Gaussian kernel to maintain the interested objects' integrity.
Weakly Supervised Real-time Image Cropping based on Aesthetic Distributions
TLDR
A weakly supervised cropping framework is proposed, where the distribution dissimilarity between high quality images and cropped images is used to guide the coordinate predictor's training and the ground truths of cropping windows are not required by the proposed method.
Fast A3RL: Aesthetics-Aware Adversarial Reinforcement Learning for Image Cropping
TLDR
This work forms image cropping as a sequential decision-making process and proposes a reinforcement learning-based framework to address this problem, namely, Fast Aesthetics-Aware Adversarial Reinforcement Learning (Fast A3RL).
A Lightweight Auto-Crop Based on Deep Reinforcement Learning
TLDR
This work proposes a real-time and lightweight framework based on deep reinforcement learning algorithm, name advantage actor critic(A2C), to achieve fast and automatic cropping, and shows that parallel actor-learners, have an e-cient learning on image cropping.
Aesthetic Photo Collage with Deep Reinforcement Learning
TLDR
A novel pipeline for automatic generation of aspect ratio specified collage is proposed and the reinforcement learning technique is introduced in collage for the first time and the results outperform others in aesthetic quality evaluation.
Image aesthetic assessment using a saliency symbiosis network
TLDR
This work reports on a method named saliency symbiosis network, which gets very close to the human visual mechanism after adding saliency features in CNN, and proposes an effective mean Huber loss function, which becomes less sensitive to outliers and can quickly train the model to being optimal.
Multi-Scale Attention Network for Image Cropping
TLDR
A multi-scale attention network (MSANet) is proposed to address the issue of global compositional relationships among different regions of a given image and achieves state-of-the-art performance with less time and lighter weights.
...
...

References

SHOWING 1-10 OF 49 REFERENCES
Composition-Preserving Deep Photo Aesthetics Assessment
TLDR
This paper presents a composition-preserving deep Con-vNet method that directly learns aesthetics features from the original input images without any image transformations, and adds an adaptive spatial pooling layer upon the regular convolution and pooling layers to directly handle input images with original sizes and aspect ratios.
Rating Image Aesthetics Using Deep Learning
TLDR
A novel deep neural network approach is adopted to allow unified feature learning and classifier training to estimate image aesthetics, and a double-column deep convolutional neural network is developed to support heterogeneous inputs.
RAPID: Rating Pictorial Aesthetics using Deep Learning
TLDR
The RAPID (RAting PIctorial aesthetics using Deep learning) system is presented, which adopts a novel deep neural network approach to enable automatic feature learning and style attributes of images to help improve the aesthetic quality categorization accuracy.
Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation
TLDR
The proposed deep multi-patch aggregation network integrates shared feature learning and aggregation function learning into a unified framework and significantly outperformed the state of the art in all three applications.
Photo Aesthetics Ranking Network with Attributes and Content Adaptation
TLDR
This work proposes to learn a deep convolutional neural network to rank photo aesthetics in which the relative ranking of photo aesthetics are directly modeled in the loss function.
Automatic Image Cropping using Visual Composition, Boundary Simplicity and Content Preservation Models
TLDR
This paper proposes a new, aesthetic photo cropping system which combines three models: visual composition, boundary simplicity, and content preservation, and achieves significant improvement over recent cropping methods in both quantitative and qualitative evaluation.
Shallow and Deep Convolutional Networks for Saliency Prediction
TLDR
This paper addresses the problem with a completely data-driven approach by training a convolutional neural network (convnet) and proposes two designs: a shallow convnet trained from scratch, and a another deeper solution whose first three layers are adapted from another network trained for classification.
Scenic photo quality assessment with bag of aesthetics-preserving features
TLDR
A bottom-up approach is developed to construct an aesthetic library with bag-of-aesthetics preserving features instead of top-down methods that implement the heuristic guidelines (rule-specific features) listed in the photography literature.
High level describable attributes for predicting aesthetics and interestingness
TLDR
This paper demonstrates a simple, yet powerful method to automatically select high aesthetic quality images from large image collections and demonstrates that an aesthetics classifier trained on describable attributes can provide a significant improvement over baseline methods for predicting human quality judgments.
Image Aesthetic Assessment: An experimental survey
TLDR
This article systematically evaluates recent deep-learning settings that are useful for developing a robust deep model for aesthetic scoring and summarizes these approaches based on visual feature types (hand-crafted features and deep features) and evaluation criteria (data set characteristics and evaluation metrics).
...
...