• Corpus ID: 15413758

Information Extraction with HMM Structures Learned by Stochastic Optimization

  title={Information Extraction with HMM Structures Learned by Stochastic Optimization},
  author={Dayne Freitag and Andrew McCallum},
Recent research has demonstrated the strong performance of hidden Markov models applied to information extraction—the task of populating database slots with corresponding phrases from text documents. A remaining problem, however, is the selection of state-transition structure for the model. This paper demonstrates that extraction accuracy strongly depends on the selection of structure, and presents an algorithm for automatically finding good structures by stochastic optimization. Our algorithm… 

Figures and Tables from this paper

Hidden Markov Models for Information Extraction

This paper uses a machine learning approach to examine the effectiveness of HMMs on extracting information of varying levels of structure and a stochastic optimization procedure is used to find the optimal structure for a given task.

Conditional Estimation of HMMs for Information Extraction

Comparing joint and conditional likelihood helps to explain the empirical finding that, for IE, HMMs with fixed structures tend to outperform those with more flexible structures: fixed structures constrain EM to better optimize conditional likelihood.

Role Identification from Free Text Using Hidden Markov Models

The overall conclusion is that the performance of HMMs is still worse than an equivalent manually constructed system, however, clear paths for improvement of the method are shown, aiming at a method, which is easily adaptable to new domains.

Using hidden Markov model for information extraction based on multiple templates

This work presents a new algorithm using hidden Markov model for information extraction based on multiple templates, which first clusters the training data into multiple templates based on the format, then learns model structure parameters from the clustered training data and model emission probabilities from the initial training data.

Optimization of Hidden Markov Model by a Genetic Algorithm for Web Information Extraction

This paper demonstrates a new training method based on GA and Baum-Welch algorithms to obtain an HMM model with optimized number of states in the HMM models and its model parameters for web

Learning Hidden Markov Model Topology Based on KL Divergence for Information Extraction

A state-merging method is adopted for learning the topology with the use of a localized Kullback Leibler (KL) divergence for domain-specific job advertisements and preliminary experiments show promising results.

Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling

By using simulated annealing in place of Viterbi decoding in sequence models such as HMMs, CMMs, and CRFs, it is possible to incorporate non-local structure while preserving tractable inference.

Learning for Sequence Extraction Tasks

A generic sequence model is introduced and it is shown how it can be used for dealing with different closed-query tasks and taking into account the sequential nature of texts allows for a liner analysis than what is usually done in IR with static text representations.

Web Information Extraction Based on Hybrid Conditional Model

A hybrid conditional model to combine maximum entropy and maximum entropy Markov model is put forward for Web information extraction, and experimental results show that the new approach improves the performance in precision and recall over traditional hidden Markov models.



Learning Hidden Markov Model Structure for Information Extraction

It is demonstrated that a manually-constructed model that contains multiple states per extraction field outperforms a model with one state per field, and the use of distantly-labeled data to set model parameters provides a significant improvement in extraction accuracy.

Relational learning techniques for natural language information extraction

A novel rule representation speci c to natural language and a learning system, Rapier, which learns information extraction rules, and initial results on a small corpus of computer-related job postings with a preliminary version of Rapier are presented.

Relational learning techniques for natural language information extraction

Experimental results show that the number of examples required to achieve a given level of performance can be significantly reduced by this method, demonstrating the superiority of relational learning for some information extraction tasks.

Information Extraction from HTML: Application of a General Machine Learning Approach

This work shows how information extraction can be cast as a standard machine learning problem, and argues for the suitability of relational learning in solving it, and the implementation of a general-purpose relational learner for information extraction, SRV.

Learning Stochastic Regular Grammars by Means of a State Merging Method

We propose a new algorithm which allows for the identification of any stochastic deterministic regular language as well as the determination of the probabilities of the strings in the language. The

Hidden Markov model topology estimation to characterize the dynamic structure of repetitive lifting data

  • R. VaskoA. El-JaroudiJ. R. BostonT. Rudy
  • Computer Science
    Proceedings of the 19th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 'Magnificent Milestones and Emerging Opportunities in Medical Engineering' (Cat. No.97CH36136)
  • 1997
An algorithm that estimates the topology of an HMM for a given set of time series data is developed and applied to estimate the dynamic structure of human body motion data from a repetitive lifting task.

Nymble: a High-Performance Learning Name-finder

This paper presents a statistical, learned approach to finding names and other nonrecursive entities in text (as per the MUC-6 definition of the NE task), using a variant of the standard hidden

A tutorial on hidden Markov models and selected applications in speech recognition

The fabric comprises a novel type of netting which will have particular utility in screening out mosquitoes and like insects and pests. The fabric is defined of voids having depth as well as width

Information Extraction Using Hidden Markov Models

This thesis shows how to design and tune a hidden Markov model to extract factual information from a corpus of machine-readable English prose and presents a HMM that classifies and parses natural language assertions about genes being located at particular positions on chromosomes.

Information extraction using hmms and shrinkage. In Papers from the AAAI-99 Workshop on Machine Learning for Information Extration

  • AAAI Technical Report WS-99-11
  • 1999