Evaluating NLP Models via Contrast Sets
- Matt Gardner, Yoav Artzi, Ben Zhou
- Computer ScienceArXiv
- 6 April 2020
A new annotation paradigm for NLP is proposed that helps to close systematic gaps in the test data, and it is recommended that after a dataset is constructed, the dataset authors manually perturb the test instances in small but meaningful ways that change the gold label, creating contrast sets.
Joint Transition-Based Models for Morpho-Syntactic Parsing: Parsing Strategies for MRLs and a Case Study from Modern Hebrew
- A. More, Amit Seker, Victoria Basmova, Reut Tsarfaty
- Computer Science, LinguisticsInternational Conference on Topology, Algebra and…
- 1 March 2019
This work proposes a joint morphosyntactic transition-based framework which formally unifies two distinct transition systems, morphological and syntactic, into a single transition- based system with joint training and joint inference, and empirically shows that MA&D results obtained in the joint settings outperform MA& D results obtained by the respective standalone components.