• Corpus ID: 11231900

Log-linear models and conditional random fields

@inproceedings{Elkan2007LoglinearMA,
  title={Log-linear models and conditional random fields},
  author={Charles Peter Elkan},
  year={2007}
}
This document describes log-linear models, which are a far-reaching extension of logistic regression, and conditional random fields (CRFs), which are a special case of log-linear models. Section 1 explains what a log-linear model is, and introduces feature functions. Section 2 then presents linear-chain CRFs as an example of log-linear models, and Section 3 explains the special algorithms that make inference tractable for these CRFs. Section 4 gives a general derivation of the gradient of a log… 
cseweb.ucsd.edu
32 Citations
Highly Influential Citations
2
Background Citations
11
Methods Citations
10

32 Citations

Log-Linear RNNs: Towards Recurrent Neural Networks with Flexible Prior Knowledge

It is argued that the log-linear and the neural-network components contribute complementary strengths to the LL-RNN: the LL aspect allows the model to incorporate rich prior knowledge, while the NN aspect, according to the "representation learning" paradigm, allows themodel to discover novel combination of characteristics.

Conditional Random Fields for Word Hyphenation

The experiment results show that Collins Perceptron is the most efficient method for training linear-chain CRFs and ABE is a better feature representation scheme that outperforms RBE by 7.9% accuracy and the design is reasonable by comparing it to the state-of-the-art [2].

Analyzing Sequence Data Based on Conditional Random Fields with Co-training

A novel model is proposed named Conditional Random Fields with Co-training (Co-CRF), which can produce a more accurate analysis than the traditional CRF, especially with very limited training data.

A Conditional Random Field Model for Context Aware Cloud Detection in Sky Images

It is shown that very high cloud detection accuracy can be achieved by combining a discriminative classifier and a higher order clique potential in a CRF framework.

On accelerating iterative algorithms with CUDA: A case study on Conditional Random Fields training algorithm for biological sequence alignment

The parallelized algorithm for the Gradient Ascent based CRF training methods for biological sequence alignment is designed, which flexibly parallelized the different iterative computation patterns, and the according optimization methods are presented.

Features we trust!

This work investigates the problem of image classification within a supervised learning framework that exploits implicit mutual information in different visual features and their associated classifiers using a fully connected conditional random fields (CRF) over clusters.

Chinese Lexical Analysis with Deep Bi-GRU-CRF Network

A deep Bi-GRU-CRF network that jointly models word segmentation, part-of-speech tagging and named entity recognition tasks and achieves a 95.5% accuracy on the test set, roughly 13% relative error reduction over the best Chinese lexical analysis tool.

Natural language processing: an introduction

The historical evolution of NLP is described, and common NLP sub-problems in this extensive field are summarized, and possible future directions for NLP are considered.

A social recommendation framework based on multi-scale continuous conditional random fields

A novel model, Multi-scale Continuous Conditional Random Fields (MCCRF), is proposed as a framework to solve above problems for social recommendations, where relational dependency within predictions is modeled by the Markov property, thus predictions are generated simultaneously and can help each other.

Hyphenation with Conditional Random Field

In this project, we approach the problem of English-word hyphenation using a linear-chain conditional random field model. We measure the effectiveness of different feature combinations and two…

16 References

Efficiently Inducing Features of Conditional Random Fields

This paper presents an efficient feature induction method for CRFs founded on the principle of iteratively constructing feature conjunctions that would significantly increase conditional log-likelihood if added to the model.

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.

Training conditional random fields via gradient tree boosting

This paper describes a new method for training CRFs by applying Friedman's (1999) gradient tree boosting method, which scales linearly in the order of the Markov model and in the Order of the feature interactions, rather than exponentially like previous algorithms based on iterative scaling and gradient descent.

An Introduction to Conditional Random Fields for Relational Learning

A solution to this problem is to directly model the conditional distribution p(y|x), which is sufficient for classification, and this is the approach taken by conditional random fields.

Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms

Experimental results on part-of-speech tagging and base noun phrase chunking are given, in both cases showing improvements over results for a maximum-entropy tagger.

Discriminative random fields: a discriminative framework for contextual interaction in classification

  • Sanjiv KumarM. Hebert
  • Computer Science
    Proceedings Ninth IEEE International Conference on Computer Vision
  • 2003
This work presents discriminative random fields (DRFs), a discrim inative framework for the classification of image regions by incorporating neighborhood interactions in the labels as well as the observed data that offers several advantages over the conventional Markov random field framework.

Large Margin Methods for Structured and Interdependent Output Variables

This paper proposes to appropriately generalize the well-known notion of a separation margin and derive a corresponding maximum-margin formulation and presents a cutting plane algorithm that solves the optimization problem in polynomial time for a large class of problems.

Training Conditional Random Fields for Maximum Labelwise Accuracy

This work gives a gradient-based procedure for minimizing an arbitrarily accurate approximation of the empirical risk under a Hamming loss function.

Max-Margin Markov Networks

Maximum margin Markov (M3) networks incorporate both kernels, which efficiently deal with high-dimensional features, and the ability to capture correlations in structured data, and a new theoretical bound for generalization in structured domains is provided.

Conditional Random Fields: An Introduction

The task of assigning label sequences to a set of observation sequences arises in many fields, including bioinformatics, computational linguistics and speech recognition [6, 9, 12]. For example,…