Knowledge-driven understanding of images in comic books

@article{Rigaud2015KnowledgedrivenUO,
  title={Knowledge-driven understanding of images in comic books},
  author={Christophe Rigaud and Cl{\'e}ment Gu{\'e}rin and Dimosthenis Karatzas and Jean-Christophe Burie and Jean-Marc Ogier},
  journal={International Journal on Document Analysis and Recognition (IJDAR)},
  year={2015},
  volume={18},
  pages={199 - 221}
}
Document analysis is an active field of research, which can attain a complete understanding of the semantics of a given document. One example of the document understanding process is enabling a computer to identify the key elements of a comic book story and arrange them according to a predefined domain knowledge. In this study, we propose a knowledge-driven system that can interact with bottom-up and top-down information to progressively understand the content of a document. We model the comic… 
View on Springer
hal.archives-ouvertes.fr

35 Citations

Retrieval of comic book images using context relevance information

It is shown that the relationship between the computed similarity score versus panel order can help locating and grouping panels with similar content, or to detect the changing between "scenes", which eventually help to retrieve more relevant results.

An Overview of Comics Research in Computer Science

This paper proposes in this paper an overview about the previous research about comics in computer science, which will state what have been done and which are the main outlooks of current research.

Multi-task Model for Comic Book Image Analysis

The Comic MTL model can detect the association between balloons and their speakers (comic characters) and handle other tasks like panels, characters detection and balloons segmentation with promising results.

An ontology-based framework for the automated analysis and interpretation of comic books' images

Segmentation and indexation of complex objects in comic book images. (Segmentation et indexation d'objets complexes dans les images de bandes dessinées)

The first public comic book image dataset and ground truth is provided to the community along with an overall experimental comparison of all the proposed methods and some of the state-of-the-art methods.

Estimating Comic Content from the Book Cover Information Using Fine-Tuned VGG Model for Comic Search

This research estimates the age and cultural background of comics expressed by clothes and belongings written on the cover of comics from the reasoning model which performed fine-tuning from the VGG-16 model and associated comics with each other based on the obtained semantic vectors and tags.

Digital Comics Image Indexing Based on Deep Learning

This study shows how to combine state-of-the-art image analysis methods to encode and index images into an XML-like text file, which can be used to automatically split comic book images into sub-images corresponding to panels easily indexable with relevant information about their respective content.

Semi-automatic Text and Graphics Extraction of Manga Using Eye Tracking Information

Experimental results show that the fixations and saccades indeed form consistent patterns among readers, and can be used for manga textual and graphical analysis.

Comic MTL: optimized multi-task learning for comic book image analysis

The Comic MTL model can detect the associations between balloons and their speakers (comic characters) and handle other tasks like panel and character detection and also balloons segmentation with promising results.

A survey of comics research in computer science

The previous research about comics in computer science is reviewed, to state what have been done and to give some insights about the main outlooks.

References

SHOWING 1-10 OF 65 REFERENCES

Robust Frame and Text Extraction from Comic Books

This paper proposes to rely on this particularity of comic books to automatically extract frame and text using a connected-component labeling analysis and compared with some existing methods found in the literature.

Automatic Text Localisation in Scanned Comic Books

A novel approach for the automatic text localization in scanned comics book pages, an essential step towards a fully automatic comics book understanding, focuses on speech text as it is semantically important and represents the majority of the text present in comics.

Analysis and Interpretation of Graphical Documents

This chapter provides an overview of how analysis strategies have contributed to constructing pipelines, how specific domain knowledge is integrated in these analyses, and which interpretation contexts have been contributed to successful approaches.

Specific Comic Character Detection Using Local Feature Matching

This paper focuses on stable features regarding the possible transformations and proposed a framework to detect them, and some discriminative features are selected as detectors for characterizing characters, on the basis of a training dataset.

Color Descriptor for Content-Based Drawing Retrieval

A color-based approach for comics character retrieval using content-based drawing retrieval and color palette is presented, which is an essential step towards a fully automatic comic book understanding.

An ontology approach to object-based image retrieval

The proposed approach bridges the gap between keyword-based approaches, which assume the existence of rich image captions or require manual evaluation and annotation of every image of the collection, and query-by-example approaches,Which assume that the user queries for images similar to one that already is at his disposal.

eBDtheque: A Representative Database of Comics

eBDtheque, a database of various comic book images and their ground truth for panels, balloons and text lines plus semantic annotations is presented, and the piece of software used to establish the ground truth and a tool to validate results against this ground truth are presented.

Panel and Speech Balloon Extraction from Comic Books

A method based on region growing and mathematical morphology to extract automatically the panels of a comic page and a method to detect speech balloons is proposed and compared with other methods find in the literature.

An Active Contour Model for Speech Balloon Detection in Comics

This work detail a novel approach for closed and non-closed speech balloon localization in scanned comic book pages, an essential step towards a fully automatic comic book understanding.

Layout Analysis of Tree-Structured Scene Frames in Comic Images

A method for detecting the scene frame division in comic images using the density gradient after filling the quadrangle regions in each image with black is proposed and results show that 80 percent of 672 pages in four print comic booklets are successfully divided into scene frames by the proposed method.
...