Constrained Language Models Yield Few-Shot Semantic Parsers

@article{Shin2021ConstrainedLM,
  title={Constrained Language Models Yield Few-Shot Semantic Parsers},
  author={Richard Shin and C. H. Lin and Sam Thomson and Charles C. Chen and Subhro Roy and Emmanouil Antonios Platanios and Adam Pauls and Dan Klein and Jas' Eisner and Benjamin Van Durme},
  journal={ArXiv},
  year={2021},
  volume={abs/2104.08768}
}
We explore the use of large pretrained language models as few-shot semantic parsers. The goal in semantic parsing is to generate a structured meaning representation given a natural language input. However, language models are trained to generate natural language. To bridge the gap, we use language models to paraphrase inputs into a controlled sublanguage resembling English that can be automatically mapped to a target meaning representation. Our results demonstrate that with only a small amount… 

Figures and Tables from this paper

Few-Shot Semantic Parsing with Language Models Trained On Code
TLDR
OpenAI Codex is tested and it is found that unlike GPT-3, Codex performs similarly when targeting meaning representations directly, perhaps as meaning representations used in semantic parsing are structured similar to code.
Training Naturalized Semantic Parsers with Very Little Data
TLDR
This work introduces an automated methodology that delivers very signif-icant additional improvements by utilizing modest amounts of unannotated data, which is typically easy to obtain, and delivers new SOTA few-shot performance on the Overnight dataset, particularly in very low-resource settings.
On The Ingredients of an Effective Zero-shot Semantic Parser
TLDR
This paper analyzes zero-shot parsers through the lenses of the language and logical gaps (Herzig and Berant, 2019), which quantify the discrepancy of language and programmatic patterns between the synthetic canonical examples and real-world user-issued ones.
The Power of Prompt Tuning for Low-Resource Semantic Parsing
TLDR
This work investigates prompt tuning for semantic parsing—the task of mapping natural language utterances onto formal meaning representations—and finds that a prompt tuned T5-xl significantly outperforms its prompt tuned counterpart, as well as strong GPT-3 and BART baselines.
Compositional Task-Oriented Parsing as Abstractive Question Answering
TLDR
This work continues to explore naturalized semantic parsing by pre-senting a general reduction of TOP to abstractive question answering that overcomes some limitations of canonical paraphrasing.
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
TLDR
A survey of recent work that uses large, pre-trained transformer-based language models to solve NLP tasks via pre-training then fine-tuning, prompting, or text generation approaches.
Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning
TLDR
This work studies pre-trained language models that generate explanation graphs in an end-to-end manner and analyzes their ability to learn the structural constraints and semantics of such graphs and proposes simple yet effective ways of graph perturbations via node and edge edit operations that lead to structurally and semantically positive and negative graphs.
Learning To Retrieve Prompts for In-Context Learning
TLDR
An efficient method for retrieving prompts for in-context learning using annotated data and a LM is proposed, which substantially outperforms prior work and multiple baselines across the board.
Synchromesh: Reliable code generation from pre-trained language models
TLDR
This paper proposes SYNCHROMESH: a framework for substantially improving the reliability of pre-trained models for code generation, and observes substantial complementary gains from CSD and TST in prediction accuracy and in effectively preventing run-time errors.
Weakly Supervised Mapping of Natural Language to SQL through Question Decomposition
TLDR
This work uses the recently proposed question decomposition representation called QDMR, an intermediate between NL and formal query languages, and uses NL-QDMR pairs, along with the question answers, as supervision for automatically synthesizing SQL queries.
...
1
2
3
...

References

SHOWING 1-10 OF 66 REFERENCES
Language to Logical Form with Neural Attention
TLDR
This paper presents a general method based on an attention-enhanced encoder-decoder model that encode input utterances into vector representations, and generate their logical forms by conditioning the output sequences or trees on the encoding vectors.
Semantic Parsing via Paraphrasing
TLDR
This paper presents two simple paraphrase models, an association model and a vector space model, and trains them jointly from question-answer pairs, improving state-of-the-art accuracies on two recently released question-answering datasets.
Building a Semantic Parser Overnight
TLDR
A new methodology is introduced that uses a simple grammar to generate logical forms paired with canonical utterances that are meant to cover the desired set of compositional operators and uses crowdsourcing to paraphrase these canonical utterance into natural utterances.
Learning for Semantic Parsing with Statistical Machine Translation
TLDR
It is shown that WASP performs favorably in terms of both accuracy and coverage compared to existing learning methods requiring similar amount of supervision, and shows better robustness to variations in task complexity and word order.
Semantic Parsing as Machine Translation
TLDR
This work approaches semantic parsing as a straightforward machine translation task, and demonstrates that standard machine translation components can be adapted into a semantic parser that is competitive with the state of the art, and achieves higher accuracy than recently proposed purpose-built systems.
Neural Semantic Parsing with Type Constraints for Semi-Structured Tables
TLDR
A new semantic parsing model for answering compositional questions on semi-structured Wikipedia tables with a state-of-the-art accuracy and type constraints and entity linking are valuable components to incorporate in neural semantic parsers.
Data Recombination for Neural Semantic Parsing
TLDR
Data recombination improves the accuracy of the RNN model on three semantic parsing datasets, leading to new state-of-the-art performance on the standard GeoQuery dataset for models with comparable supervision.
Genie: a generator of natural language semantic parsers for virtual assistant commands
TLDR
A methodology and the Genie toolkit that can handle new compound commands with significantly less manual effort are presented and design principles that make VAPL languages amenable to natural language translation are proposed.
Unnatural Language Processing: Bridging the Gap Between Synthetic and Natural Language Data
TLDR
The results suggest that simulation-to-real transfer is a practical framework for developing NLP applications, and that improved models for transfer might provide wide-ranging improvements in downstream tasks.
Semantic Parsing with Dual Learning
TLDR
This work develops a semantic parsing framework with the dual learning algorithm, which enables a semantic parser to make full use of data through a dual-learning game.
...
1
2
3
4
5
...