Understanding Compositional Structures in Art Historical Images using Pose and Gaze Priors

@article{Madhu2020UnderstandingCS,
  title={Understanding Compositional Structures in Art Historical Images using Pose and Gaze Priors},
  author={Prathmesh Madhu and Tilman Marquart and Ronak Kosti and Peter Bell and Andreas K. Maier and Vincent Christlein},
  journal={ArXiv},
  year={2020},
  volume={abs/2009.03807}
}
Image compositions as a tool for analysis of artworks is of extreme significance for art historians. These compositions are useful in analyzing the interactions in an image to study artists and their artworks. Max Imdahl in his work called Ikonik, along with other prominent art historians of the 20th century, underlined the aesthetic and semantic importance of the structural composition of an image. Understanding underlying compositional structures within images is challenging and a time… 
[PDF] Semantic Reader
10 Citations
Highly Influential Citations
1
Background Citations
4
Methods Citations
4
Results Citations
1

10 Citations

ICC++: Explainable Image Retrieval for Art Historical Corpora using Image Composition Canvas

A novel approach called Image Composition Canvas (ICC++) to compare and retrieve images having similar compositional elements and outperforms the best deep learning-based method, opening the research direction for explainable machine learning for digital humanities.

Semi-supervised Human Pose Estimation in Art-historical Images

This paper proposes a novel approach to estimate human poses in art-historical images that achieves significantly better results than methods that use pre-trained models or style transfer, and introduces a novel domain-specific art data set that includes both bounding box and keypoint annotations of human figures.

Understanding and Creating Art with AI: Review and Outlook

A comprehensive overview of artwork datasets and recent works that address a variety of tasks such as classification, object detection, similarity retrieval, multimodal representations, computational aesthetics, etc.

Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History

Throughout the history of art, the pose—as the holistic abstraction of the human body’s expression—has proven to be a constant in numerous studies. However, due to the enormous amount of data that so…

What to do with 2.000.000 Historical Press Photos? The Challenges and Opportunities of Applying a Scene Detection Algorithm to a Digitised Press Photo Collection

The aim is not only the digitisation and publication of all 2,000,000 photo negatives of Fotopersbureau De Boer but also to explore how artificial intelligence can enrich this collection, benefiting both users of the archive and cultural historians studying historical photographs.

Artificial Intelligence and Art History : A Necessary Debate ?

Neural networks can be applied to retrieve images for which a network produces a similar output, i.e. which the network “sees” in a similar way, and provides a new way to look for images, bypassing the language bottleneck.

NAPA: Neural Art Human Pose Amplifier

An end-to-end system that uses neural style transfer for pose regression and a 2d-induced bone map from which pose is lifted to target human pose estimation in artistic images is designed.

ICC++: Explainable Feature Learning for Art History using Image Compositions

Automatic Analysis of Human Body Representations in Western Art

. The way the human body is depicted in classical and modern paintings is relevant for art historical analyses. Each artist has certain themes and concerns, resulting in different poses being used…

POSE-ID-on - A Novel Framework for Artwork Pose Clustering

This work developed a modified version of the K-Medians algorithm to cluster similar poses and to find a limited number of poses that are representative of the whole dataset, and assessed the validity of the proposed framework, which is named POSE-ID-on.

References

SHOWING 1-10 OF 29 REFERENCES

How to Read Paintings: Semantic Art Understanding with Multi-Modal Retrieval

SemArt is a collection of fine-art painting images in which each image is associated to a number of attributes and a textual artistic comment, such as those that appear in art catalogues or museum collections, and several models for encoding visual and textual artistic representations into a common semantic space are proposed.

Linking Art through Human Poses

It is experimentally show that explicit human pose matching is superior to standard content-based image retrieval methods on a manually annotated art composition transfer dataset.

Context-Aware Embeddings for Automatic Art Analysis

This work proposes to enhance visual representations from neural networks with contextual artistic information by designing context-aware embeddings, which encodes relationships between artistic attributes.

Recognizing Characters in Art History Using Deep Learning

A new technique is introduced that generates more data with similar styles, effectively creating data in the similar domain, and shows that the model trained on domain related data gives the best performance for recognizing character.

Artpedia: A New Visual-Semantic Dataset with Visual and Contextual Sentences in the Artistic Domain

A visual-semantic model is devised that jointly addresses the problem of cross-modal retrieval of images and sentences coming from the artistic domain by exploiting the latent alignment between visual and textual chunks.

Weakly Supervised Object Detection in Artworks

This work proposes a method for the weakly supervised detection of objects in paintings, and introduces a new database, IconArt, on which it performs detection experiments on classes that could not be learned on photographs, such as Jesus Child or Saint Sebastian.

Unsupervised Disentanglement of Pose, Appearance and Background from Images and Videos

The proposed factorization results in landmarks that are focused on the foreground object of interest when measured against ground-truth foreground masks, and the rendered background quality is improved as ill-suited landmarks are no longer forced to model this content.

Microsoft COCO: Common Objects in Context

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene…

Unsupervised Part-Based Disentangling of Object Shape and Appearance

This work presents an unsupervised approach for disentangling appearance and shape by learning parts consistently over all instances of a category by simultaneously exploiting invariance and equivariance constraints between synthetically transformed images.

Ikonographie und Interaktion. Computergestützte Analyse von Posen in Bildern der Heilsgeschichte

Abstract The last few years have seen an explosion of medieval images in digital form, chiefly as a result of photo-library and manuscript digitisation projects. An entire corpus of images, even…