• Corpus ID: 22457893

Sequence to Sequence Learning for Event Prediction

@inproceedings{Nguyen2017SequenceTS,
  title={Sequence to Sequence Learning for Event Prediction},
  author={Dai Quoc Nguyen and Dat Quoc Nguyen and Cuong Xuan Chu and Stefan Thater and Manfred Pinkal},
  booktitle={IJCNLP},
  year={2017}
}
This paper presents an approach to the task of predicting an event description from a preceding sentence in a text. Our approach explores sequence-to-sequence learning using a bidirectional multi-layer recurrent neural network. Our approach substantially outperforms previous work in terms of the BLEU score on two datasets derived from WikiHow and DeScript respectively. Since the BLEU score is not easy to interpret as a measure of event prediction, we complement our study with a second… 

Figures and Tables from this paper

Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction

This paper proposes to incorporate a conditional variational autoencoder (CVAE) into seq2seq for its ability to represent diverse next events as a probabilistic distribution and extends existing evaluation datasets by tying each current event to multiple next events.

Future is not One-dimensional: Graph Modeling based Complex Event Schema Induction for Event Prediction

The concept of Temporal Complex Event Schema is introduced: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations and a Temporal Event Graph Model that models the emergence of event instances following the temporal complex event schema.

Reasoning about Goals, Steps, and Temporal Ordering with WikiHow

This work proposes a suite of reasoning tasks on two types of relations between procedural events: goal-step relations and step-step temporal relations, and introduces a dataset targeting these two relations based on wikiHow, a website of instructional how-to articles.

1 Grapheme-to-Phoneme Conversion with 2 Convolutional Neural Networks 3

A novel CNN-based sequence-to-sequence (seq2seq) architecture for G2P conversion that approaches the accuracy of previous state-of-the-art results in terms of phoneme error rate.

TransSent: Towards Generation of Structured Sentences with Discourse Marker

This paper focuses on the task of generating long structured sentences with explicit discourse markers, by proposing a new task Sentence Transfer and a novel model architecture TransSent, which models relations between two discourses by interpreting them as transferring from one discourse to the other in the embedding space.

Grapheme-to-Phoneme Conversion with Convolutional Neural Networks

This paper proposes a novel CNN-based sequence-to-sequence (seq2seq) architecture for G2P conversion that approaches the accuracy of previous state-of-the-art results in terms of phoneme error rate.

Automatic Post-Editing for Vietnamese

This paper constructs the first large-scale dataset of 5M Vietnamese translated and corrected sentence pairs and applies strong neural MT models to handle the APE task, using the constructed dataset.

Generating Convincing Harmony Parts with Bidirectional Long Short-Term Memory Networks

This paper explores systems where a user provides a sequence of notes, and a neural network model responds with an accompanying sequence of equal length, and sees a preference for the sequences generated by the bidirectional model as well as an indication that these sequences sound more human.

SeqST-GAN

Citywide crowd flow data are ubiquitous nowadays, and forecasting the flow of crowds is of great importance to many real applications such as traffic management and mobility-on-demand (MOD) services.

Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks

This paper explores systems where a user provides a sequence of notes, and a neural network model responds with an accompanying sequence of equal length, and sees a preference for the sequences generated by the bidirectional model as well as an indication that these sequences sound more human.

References

SHOWING 1-10 OF 22 REFERENCES

Using Sentence-Level LSTM Language Models for Script Inference

This work compares systems that operate on structured verb-argument events produced by an NLP pipeline with recent Recurrent Neural Net models that directly operate on raw tokens to predict sentences, finding the latter to be roughly comparable to the former in terms of predicting missing events in documents.

Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond

This work proposes several novel models that address critical problems in summarization that are not adequately modeled by the basic architecture, such as modeling key-words, capturing the hierarchy of sentence-to-word structure, and emitting words that are rare or unseen at training time.

Skip-Thought Vectors

We describe an approach for unsupervised learning of a generic, distributed sentence encoder. Using the continuity of text from books, we train an encoder-decoder model that tries to reconstruct the

Sequence to Sequence Learning with Neural Networks

This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.

Language to Logical Form with Neural Attention

This paper presents a general method based on an attention-enhanced encoder-decoder model that encode input utterances into vector representations, and generate their logical forms by conditioning the output sequences or trees on the encoding vectors.

Multi-task Sequence to Sequence Learning

The results show that training on a small amount of parsing and image caption data can improve the translation quality between English and German by up to 1.5 BLEU points over strong single-task baselines on the WMT benchmarks, and reveal interesting properties of the two unsupervised learning objectives, autoencoder and skip-thought, in the MTL context.

Grammar as a Foreign Language

The domain agnostic attention-enhanced sequence-to-sequence model achieves state-of-the-art results on the most widely used syntactic constituency parsing dataset, when trained on a large synthetic corpus that was annotated using existing parsers.

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Qualitatively, the proposed RNN Encoder‐Decoder model learns a semantically and syntactically meaningful representation of linguistic phrases.

Distributed Representations of Words and Phrases and their Compositionality

This paper presents a simple method for finding phrases in text, and shows that learning good vector representations for millions of phrases is possible and describes a simple alternative to the hierarchical softmax called negative sampling.

Neural Machine Translation by Jointly Learning to Align and Translate

It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.