Insights From A Large-Scale Database of Material Depictions In Paintings

  title={Insights From A Large-Scale Database of Material Depictions In Paintings},
  author={Hubert Lin and Mitchell van Zuijlen and Maarten W. A. Wijntjes and Sylvia C. Pont and Kavita Bala},
  booktitle={ICPR Workshops},
Deep learning has paved the way for strong recognition systems which are often both trained on and applied to natural images. In this paper, we examine the give-and-take relationship between such visual recognition systems and the rich information available in the fine arts. First, we find that visual recognition systems designed for natural images can work surprisingly well on paintings. In particular, we find that interactive segmentation tools can be used to cleanly annotate polygonal… Expand

Figures and Tables from this paper

Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision
The dataset of painterly material depictions is a rich source for gaining insights into the depiction and perception of materials across multiple disciplines and is demonstrated to have cross-disciplinary utility by presenting novel findings across art history, human perception, and computer vision. Expand
Understanding and Creating Art with AI: Review and Outlook
A comprehensive overview of artwork datasets and recent works that address a variety of tasks such as classification, object detection, similarity retrieval, multimodal representations, computational aesthetics, etc. Expand


The State of the Art: Object Retrieval in Paintings using Discriminative Regions
This work draws upon recent work in mid-level discriminative patches to develop a novel method for reranking paintings based on their spatial consistency with natural images of an object category, which combines both class based and instance based retrieval in a single framework. Expand
Deep Learning for Fine-Grained Image Analysis: A Survey
This survey aims to give a survey on recent advances of deep learning based FGIA techniques in a systematic way, and organizes the existing studies of FGia techniques into three major categories: fine-grained image recognition, fine- grained image retrieval and fine-Grained image generation. Expand
Material recognition in the wild with the Materials in Context Database
A new, large-scale, open dataset of materials in the wild, the Materials in Context Database (MINC), is introduced, and convolutional neural networks are trained for two tasks: classifying materials from patches, and simultaneous material recognition and segmentation in full images. Expand
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
It is shown that ImageNet-trained CNNs are strongly biased towards recognising textures rather than shapes, which is in stark contrast to human behavioural evidence and reveals fundamentally different classification strategies. Expand
Understanding gloss perception through the lens of art: Combining perception, image analysis, and painting recipes of 17th century painted grapes.
The pictorial shortcuts used by 17th century painters to imitate the optical phenomenon of specular reflections when depicting grapes were analyzed, suggesting that painting practice embeds knowledge about key image features that trigger specific material percepts. Expand
OpenSurfaces: a richly annotated catalog of surface appearance
This work uses human annotations and presents a new methodology for segmenting and annotating materials in Internet photo collections suitable for crowdsourcing (e.g., through Amazon's Mechanical Turk), and designs a multi-stage set of annotation tasks with quality checks and validation. Expand
Microsoft COCO: Common Objects in Context
We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of sceneExpand
Deeper, Broader and Artier Domain Generalization
This paper builds upon the favorable domain shift-robust properties of deep learning methods, and develops a low-rank parameterized CNN model for end-to-end DG learning that outperforms existing DG alternatives. Expand
StreetStyle: Exploring world-wide clothing styles from millions of photos
This paper introduces a large-scale dataset of photos of people annotated with clothing attributes, and uses this dataset to train attribute classifiers via deep learning, and presents a method for discovering visually consistent style clusters that capture useful visual correlations in this massive dataset. Expand
A survey on deep learning techniques for image and video semantic segmentation
A review on deep learning methods for semantic segmentation applied to various application areas and points out a set of promising future works to help researchers decide which are the ones that best suit their needs and goals. Expand