# Log-linear models and conditional random fields

@inproceedings{Elkan2007LoglinearMA, title={Log-linear models and conditional random fields}, author={C. Elkan}, year={2007} }

This document describes log-linear models, which are a far-reaching extension of logistic regression, and conditional random fields (CRFs), which are a special case of log-linear models. Section 1 explains what a log-linear model is, and introduces feature functions. Section 2 then presents linear-chain CRFs as an example of log-linear models, and Section 3 explains the special algorithms that make inference tractable for these CRFs. Section 4 gives a general derivation of the gradient of a log… Expand

#### Supplemental Video

#### Topics from this paper

#### 29 Citations

Log-Linear RNNs: Towards Recurrent Neural Networks with Flexible Prior Knowledge

- Computer Science
- ArXiv
- 2016

It is argued that the log-linear and the neural-network components contribute complementary strengths to the LL-RNN: the LL aspect allows the model to incorporate rich prior knowledge, while the NN aspect, according to the "representation learning" paradigm, allows themodel to discover novel combination of characteristics. Expand

Conditional Random Fields for Word Hyphenation

- 2013

Word hyphenation is an important problem which has many practical applications. The problem is challenging because of the vast amount of English words. We use linear-chain Conditional Random Fields… Expand

Analyzing Sequence Data Based on Conditional Random Fields with Co-training

- Computer Science
- 2012 Eighth International Conference on Computational Intelligence and Security
- 2012

A novel model is proposed named Conditional Random Fields with Co-training (Co-CRF), which can produce a more accurate analysis than the traditional CRF, especially with very limited training data. Expand

A Conditional Random Field Model for Context Aware Cloud Detection in Sky Images

- Computer Science, Engineering
- ArXiv
- 2019

It is shown that very high cloud detection accuracy can be achieved by combining a discriminative classifier and a higher order clique potential in a CRF framework. Expand

On accelerating iterative algorithms with CUDA: A case study on Conditional Random Fields training algorithm for biological sequence alignment

- Computer Science
- 2010 IEEE International Conference on Bioinformatics and Biomedicine Workshops (BIBMW)
- 2010

The parallelized algorithm for the Gradient Ascent based CRF training methods for biological sequence alignment is designed, which flexibly parallelized the different iterative computation patterns, and the according optimization methods are presented. Expand

Features we trust!

- Computer Science
- 2015 IEEE International Conference on Image Processing (ICIP)
- 2015

This work investigates the problem of image classification within a supervised learning framework that exploits implicit mutual information in different visual features and their associated classifiers using a fully connected conditional random fields (CRF) over clusters. Expand

Chinese Lexical Analysis with Deep Bi-GRU-CRF Network

- Computer Science
- ArXiv
- 2018

A deep Bi-GRU-CRF network that jointly models word segmentation, part-of-speech tagging and named entity recognition tasks and achieves a 95.5% accuracy on the test set, roughly 13% relative error reduction over the best Chinese lexical analysis tool. Expand

Natural language processing: an introduction

- Computer Science, Medicine
- J. Am. Medical Informatics Assoc.
- 2011

The historical evolution of NLP is described, and common NLP sub-problems in this extensive field are summarized, and possible future directions for NLP are considered. Expand

A social recommendation framework based on multi-scale continuous conditional random fields

- Computer Science
- CIKM
- 2009

A novel model, Multi-scale Continuous Conditional Random Fields (MCCRF), is proposed as a framework to solve above problems for social recommendations, where relational dependency within predictions is modeled by the Markov property, thus predictions are generated simultaneously and can help each other. Expand

Hyphenation with Conditional Random Field

- 2012

In this project, we approach the problem of English-word hyphenation using a linear-chain conditional random field model. We measure the effectiveness of different feature combinations and two… Expand

#### References

SHOWING 1-10 OF 16 REFERENCES

Efficiently Inducing Features of Conditional Random Fields

- Computer Science, Mathematics
- UAI
- 2003

This paper presents an efficient feature induction method for CRFs founded on the principle of iteratively constructing feature conjunctions that would significantly increase conditional log-likelihood if added to the model. Expand

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

- Computer Science
- ICML
- 2001

This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data. Expand

Training conditional random fields via gradient tree boosting

- Mathematics, Computer Science
- ICML
- 2004

This paper describes a new method for training CRFs by applying Friedman's (1999) gradient tree boosting method, which scales linearly in the order of the Markov model and in the Order of the feature interactions, rather than exponentially like previous algorithms based on iterative scaling and gradient descent. Expand

An Introduction to Conditional Random Fields for Relational Learning

- Computer Science
- 2007

A solution to this problem is to directly model the conditional distribution p(y|x), which is sufficient for classification, and this is the approach taken by conditional random fields. Expand

Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms

- Computer Science
- EMNLP
- 2002

Experimental results on part-of-speech tagging and base noun phrase chunking are given, in both cases showing improvements over results for a maximum-entropy tagger. Expand

Discriminative random fields: a discriminative framework for contextual interaction in classification

- Mathematics, Computer Science
- Proceedings Ninth IEEE International Conference on Computer Vision
- 2003

This work presents discriminative random fields (DRFs), a discrim inative framework for the classification of image regions by incorporating neighborhood interactions in the labels as well as the observed data that offers several advantages over the conventional Markov random field framework. Expand

Large Margin Methods for Structured and Interdependent Output Variables

- Computer Science
- J. Mach. Learn. Res.
- 2005

This paper proposes to appropriately generalize the well-known notion of a separation margin and derive a corresponding maximum-margin formulation and presents a cutting plane algorithm that solves the optimization problem in polynomial time for a large class of problems. Expand

Training Conditional Random Fields for Maximum Labelwise Accuracy

- Computer Science
- NIPS
- 2006

This work gives a gradient-based procedure for minimizing an arbitrarily accurate approximation of the empirical risk under a Hamming loss function. Expand

Max-Margin Markov Networks

- Computer Science
- NIPS
- 2003

Maximum margin Markov (M3) networks incorporate both kernels, which efficiently deal with high-dimensional features, and the ability to capture correlations in structured data, and a new theoretical bound for generalization in structured domains is provided. Expand

Conditional Random Fields: An Introduction

- Computer Science
- 2004

The task of assigning label sequences to a set of observation sequences arises in many fields, including bioinformatics, computational linguistics and speech recognition [6, 9, 12]. For example,… Expand