Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

  title={Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation},
  author={Albert Gatt and Emiel J. Krahmer},
This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past two decades, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures… 

Figures and Tables from this paper

A Survey of Natural Language Generation

This survey aims to give the latest synthesis of deep learning research on the NLG core tasks, as well as the architectures adopted in the field, and draw attention to the challenges in NLG evaluation.

A Survey of Knowledge-Enhanced Text Generation

A comprehensive review of the research on knowledge-enhanced text generation over the past five years is presented, which includes two parts: (i) general methods and architectures for integrating knowledge into text generation; (ii) specific techniques and applications according to different forms of knowledge data.

Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods

This survey provides a systematic overview of the research progress on the faithfulness problem of NLG, including problem analysis, evaluation metrics and optimization methods, and organizes the evaluation and optimized methods for different tasks into a unified taxonomy to facilitate comparison and learning across tasks.

Evaluation of Text Generation: A Survey

This paper surveys evaluation methods of natural language generation (NLG) systems that have been developed in the last few years, with a focus on the evaluation of recently proposed NLG tasks and neural NLG models.

The Amazing World of Neural Language Generation

This tutorial will present an overview of the current state of the art in neural network architectures, and how they shaped recent research directions in text generation, and discuss how and why these models succeed/fail at generating coherent text, and provide insights on several applications.

Why is constrained neural language generation particularly challenging?

An extensive survey on the emerging topic of constrained neural language generation is presented in which it is formally defined and categorize the problems of natural language generation by distinguishing between conditions and constraints, and existing methods and evaluation metrics for constrained text generation are reviewed.

A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions

This work provides a comprehensive review towards building open domain dialogue systems, an important application of natural language generation, and finds that, predominantly, the approaches for building dialogue systems use seq2seq or language models architecture.

Advances in Natural Language Generation: Generating Varied Outputs from Semantic Inputs

This thesis focused on the automatic generation of more varied output texts, based on various semantic input representations, and introduced an end-to-end approach,based on neural networks, which generates varied referring expressions to a discourse entity, deciding on its referential form and content in one shot without explicit feature extraction.

Domain Adaptation for Natural Language Generation Thesis Proposal

This work aims to develop NLG systems that can perform well in the domains with the lack of available training data, using large pretrained neural language models to facilitate domain adaptation ofNLG systems.

Semi-Supervised Neural Text Generation by Joint Learning of Natural Language Generation and Natural Language Understanding Models

A semi-supervised deep learning scheme that can learn from non-annotated data and annotated data when available and achieve very competitive results while not using any pre-processing or re-scoring tricks is proposed.



Evaluating Natural Language Processing Systems: An Analysis and Review

This comprehensive state-of-the-art book is the first devoted to the important and timely issue of evaluating NLP systems, and provides a wide-ranging and careful analysis of evaluation concepts, reinforced with extensive illustrations.

Evaluation in the context of natural language generation

What is involved in natural language generation, and how evaluation has figured in work in this area to date is described; a particular text generation application is examined and the issues that are raised in assessing its performance on a variety of dimensions are looked at.

Evaluation in Natural Language Generation: Lessons from Referring Expression Generation

Several problems that arise in designing evaluation for this task are explored, and general considerations that need to be met in evaluating Natural Language Generation subtasks are identified.

Surface Realisation from Knowledge-Bases

This thesis presents two novel approaches to surface realisation from knowledge bases: a supervised approach and a weakly supervised approach, and explores a method for surfacerealisation from KB data which uses a supplied lexicon but does not require a parallel corpus.

Natural Language Generation through Character-based RNNs with Finite-state Prior Knowledge

In order to avoid generating non-words and inventing information not present in the input, this work proposes a method for incorporating prior knowledge into the RNN in the form of a weighted finite-state automaton over character sequences.


Today’s NLG efforts should be compared against actual human performance, which is fluent and varies randomly and with context. Consequently, evaluations should not be done against a fixed ‘gold

An Investigation into the Validity of Some Metrics for Automatically Evaluating Natural Language Generation Systems

The results of two studies of how well some metrics which are popular in other areas of NLP correlate with human judgments in the domain of computer-generated weather forecasts suggest that, at least in this domain, metrics may provide a useful measure of language quality, although the evidence for this is not as strong as one would ideally like to see.


The experiments show that the off‐the‐shelf planners tested are unusably slow for nontrivial NLG problem instances, and offer the domains and experiences tested as challenges for the planning community.

Natural Language Generation in Interactive Systems

This guide serves to introduce graduate students and new researchers to the field of natural language processing and artificial intelligence, while inspiring them with ideas for future research on natural language generation and interactive systems.

Stochastic Language Generation in Dialogue using Factored Language Models

Bagel is presented, a fully data-driven generation method that treats the language generation task as a search for the most likely sequence of semantic concepts and realization phrases, according to Factored Language Models (FLMs).