Evaluating Models’ Local Decision Boundaries via Contrast Sets
- Matt Gardner, Yoav Artzi, Ben Zhou
- Computer ScienceFindings
- 6 April 2020
A more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data, and recommends that the dataset authors manually perturb the test instances in small but meaningful ways that (typically) change the gold label, creating contrast sets.
Seeing Things from a Different Angle:Discovering Diverse Perspectives about Claims
- Sihao Chen, Daniel Khashabi, Wenpeng Yin, Chris Callison-Burch, D. Roth
- Computer ScienceNorth American Chapter of the Association for…
- 8 June 2019
A thorough analysis of the dataset is provided to highlight key underlying language understanding challenges, and it is shown that human baselines across multiple subtasks far outperform ma-chine baselines built upon state-of-the-art NLP techniques.
Evaluating NLP Models via Contrast Sets
- Matt Gardner, Yoav Artzi, Ben Zhou
- Computer ScienceArXiv
- 6 April 2020
A new annotation paradigm for NLP is proposed that helps to close systematic gaps in the test data, and it is recommended that after a dataset is constructed, the dataset authors manually perturb the test instances in small but meaningful ways that change the gold label, creating contrast sets.
PerspectroScope: A Window to the World of Diverse Perspectives
- Sihao Chen, Daniel Khashabi, Chris Callison-Burch, D. Roth
- Computer ScienceAnnual Meeting of the Association for…
- 11 June 2019
This work presents PerspectroScope, a web-based system which lets users query a discussion-worthy natural language claim, and extract and visualize various perspectives in support or against the…
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters
- Tal Schuster, Sihao Chen, S. Buthpitiya, Alex Fabrikant, Donald Metzler
- Computer ScienceArXiv
- 15 April 2022
This work further explores the direct zero-shot applicability of NLI models to real applications, beyond the sentence-pair setting they were trained on, and develops new aggregation methods to allow operating over full documents, reaching state-of-the-art performance on the ContractNLI dataset.
Design Challenges for a Multi-Perspective Search Engine
- Sihao Chen, Siyi Liu, Xander Uyttendaele, Yi Zhang, William F. Bruno, D. Roth
- Computer ScienceNAACL-HLT
- 15 December 2021
This work proposes and proposes a practical prototype pipeline system, and uses the prototype system to conduct a user survey in order to assess the utility of the paradigm, as well as understanding the user information needs when issuing controversial and open-ended queries to a search engine.
Understanding the Impact of Bots on Developers Sentiment and Project Progress
- Anze Gao, Sihao Chen, Tao Wang, Jinsheng Deng
- Computer ScienceIEEE 13th International Conference on Software…
- 21 October 2022
Whether the adoption of bots has an impact on developer sentiment and projects progress is explored and it is found that human users had significantly reduced positive sentiment in bot-created issues and PRs.
PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition
- Sihao Chen, S. Buthpitiya, Alex Fabrikant, D. Roth, Tal Schuster
- Philosophy, Computer ScienceArXiv
- 21 December 2022
This work proposes P ROP S EGM E NT, a corpus of over 35 K propositions annotated by expert human raters, and demonstrates that its conceptual framework is potentially useful for understanding and ex-plaining the compositionality of NLI labels.
USFP: An unbalanced severe typhoon formation prediction framework based on transfer learning
- Xiaotian Pan, Xiang Wang, Sihao Chen
- Computer ScienceFrontiers in Marine Science
- 1 February 2023
An unbalanced severe typhoon formation prediction (USFP) framework based on transfer learning is proposed which outperforms the numerical model IFS of ECMWF and existing machine learning methods.