Annotation-free Learning of Deep Representations for Word Spotting using Synthetic Data and Self Labeling

  title={Annotation-free Learning of Deep Representations for Word Spotting using Synthetic Data and Self Labeling},
  author={Fabian Wolf and Gernot A. Fink},
  booktitle={International Workshop on Document Analysis Systems},
  • Fabian WolfG. Fink
  • Published in
    International Workshop on…
    4 March 2020
  • Computer Science
Word spotting is a popular tool for supporting the first exploration of historic, handwritten document collections. Today, the best performing methods rely on machine learning techniques, which require a high amount of annotated training material. As training data is usually not available in the application scenario, annotation-free methods aim at solving the retrieval task without representative training samples. In this work, we present an annotation-free method that still employs machine… 

Z-Transform-Based Profile Matching to Develop a Learning-Free Keyword Spotting Method for Handwritten Document Images

This work introduces a new way of profile matching to compare the query word profiles with the target words’ profiles and achieves satisfactory performance compared to state-of-the-art learning-free methods when applied to four publicly available standard datasets.

Improving Handwritten Word Synthesis for Annotation-free Word Spotting

This work shows that an annotation-free word spotting method benefits from an adapted synthesis procedure, and investigates the influence of the choice of the underlying vocabulary and the combination of synthesis and data augmentation.

The Role of Synthetic Data in Improving Neural Network Algorithms

  • Andrey N. RabchevskyL. Yasnitsky
  • Computer Science
    2022 4th International Conference on Control Systems, Mathematical Modeling, Automation and Energy Efficiency (SUMMA)
  • 2022
Using examples, the important role of synthetic data in the improvement of neural network algorithms and the development of artificial intelligence is shown.

Self-Training of Handwritten Word Recognition for Synthetic-to-Real Adaptation

  • Fabian WolfG. Fink
  • Computer Science
    2022 26th International Conference on Pattern Recognition (ICPR)
  • 2022
This work proposes a self-training approach to train a HTR model solely on synthetic samples and unlabeled data and shows its effectiveness on reducing the gap to a model trained in a fully-supervised manner.

Pho(SC)-CTC—a hybrid approach towards zero-shot word image recognition

A hybrid model based on the CTC framework (Pho( SC)-CTC) that takes advantage of the rich features learned by Pho(SC)Net followed by a “con-nectionist temporal classification” (CTT) framework to perform the final classi-cation in order to recognize unseen/out-of-lexicon words in historical document images.

A Review of Deep Learning Techniques in Document Image Word Spotting

This study covers recent deep learning technique role in word spotting and future scope of word spotting with deep learning and an experimental comparison for the research community to evaluate algorithmic advances along with benchmarked datasets, and future challenges in this field.

Pho(SC)Net: An Approach Towards Zero-shot Word Image Recognition in Historical Documents

A hybrid representation that considers the character’s shape appearance to differentiate between two different words and has shown to be more effective in recognizing unseen words is proposed.

Benchmarking Annotation Procedures for Multi-channel Time Series HAR Dataset

The semi-automated annotation consists of predictions from a temporal convolutional neural-network, and manual revisions for generating high-quality and fine-grained annotations for Human Activity Recognition.

One Step Is Not Enough: A Multi-Step Procedure for Building the Training Set of a Query by String Keyword Spotting System to Assist the Transcription of Historical Document

A multi-step procedure that exploits a Keyword Spotting system and human validation for building up a training set in a time shorter than the one required by a fully manual procedure is proposed.



Exploring Confidence Measures for Word Spotting in Heterogeneous Datasets

This paper investigates four different metrics for quantifying the confidence of a CNN in its predictions, specifically on the retrieval problem of word spotting and shows that there exists a direct relation between the proposed confidence measures and the quality of an estimated attribute representation.

Learning Deep Representations for Word Spotting under Weak Supervision

This work introduces a method to drastically reduce the manual annotation effort while retaining the high performance of a CNN for word spotting in handwritten documents and achieves results highly competitive to the state-of-the-art in word spotting with shorter training times and a fraction of the annotation effort.

Training-Free and Segmentation-Free Word Spotting using Feature Matching and Query Expansion

A training-free and segmentation-free word spotting approach that can be applied in unconstrained scenarios and uses a combination of different keypoint detectors and Fourier-based descriptors to obtain a sufficient degree of relaxed matching.

ICDAR2015 Competition on Keyword Spotting for Handwritten Documents

The principal goal of the Competition on Keyword Spotting for Handwritten Documents was to promote different approaches used in the field of Keyword Spotting and to fairly compare them using uniform

Semantic and Verbatim Word Spotting Using Deep Neural Networks

  • T. WilkinsonAnders Brun
  • Computer Science
    2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR)
  • 2016
A word spotting system based on convolutional neural networks that outperforms the previous state-of-the-art for word spotting on standard datasets and can perform word spotting using both query- by-string and query-by-example in a variety of word embedding spaces.

Word spotting for historical documents

  • T. RathR. Manmatha
  • Computer Science
    International Journal of Document Analysis and Recognition (IJDAR)
  • 2006
It is shown in a subset of the George Washington collection that such a word spotting technique can outperform a Hidden Markov Model word-based recognition technique in terms of word error rates.

A Probabilistic Retrieval Model for Word Spotting Based on Direct Attribute Prediction

This work presents a new approach for ranking retrieval lists originally proposed for zero-shot learning where attribute representations play an important role, and shows that this probabilistic ranking improves word spotting performance, especially in the query-by-string scenario.

A survey on semi-supervised learning

This survey aims to provide researchers and practitioners new to the field as well as more advanced readers with a solid understanding of the main approaches and algorithms developed over the past two decades, with an emphasis on the most prominent and currently relevant work.

Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study

This paper provides a survey of self-labeled methods for semi-supervised classification and proposes a taxonomy based on the main characteristics presented in them, aiming to measure their performance in terms of transductive and inductive classification capabilities.

Making Two Vast Historical Manuscript Collections Searchable and Extracting Meaningful Textual Features Through Large-Scale Probabilistic Indexing

Besides allowing effortless information retrieval, it will be shown that probabilistic indices can also be used to estimate textual features of the indexed but otherwise untranscribed collections, such as running words and Zipf's curves.