Pre-Training With Whole Word Masking for Chinese BERT
- Yiming Cui, Wanxiang Che, Guoping Hu
- Computer ScienceIEEE/ACM Transactions on Audio Speech and…
- 19 June 2019
The whole word masking (wwm) strategy for Chinese BERT is introduced, along with a series of Chinese pre-trained language models, and a simple but effective model called MacBERT is proposed, which improves upon RoBERTa in several ways.
Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency
- Shuhuai Ren, Yihe Deng, Kun He, Wanxiang Che
- Computer ScienceAnnual Meeting of the Association for…
- 1 July 2019
A new word replacement order determined by both the wordsaliency and the classification probability is introduced, and a greedy algorithm called probability weighted word saliency (PWWS) is proposed for text adversarial attack.
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding
- Yang Xu, Yiheng Xu, Lidong Zhou
- Computer ScienceAnnual Meeting of the Association for…
- 29 December 2020
LayoutLMv2 architecture with new pre-training tasks to model the interaction among text, layout, and image in a single multi-modal framework and achieves new state-of-the-art results on a wide variety of downstream visually-rich document understanding tasks.
LTP: A Chinese Language Technology Platform
- Wanxiang Che, Zhenghua Li, Ting Liu
- Computer ScienceInternational Conference on Computational…
- 23 August 2010
LTP (Language Technology Platform) is an integrated Chinese processing platform which includes a suite of high performance natural language processing modules and relevant corpora that achieved good results in some relevant evaluations, such as CoNLL and SemEval.
Revisiting Pre-Trained Models for Chinese Natural Language Processing
- Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, Guoping Hu
- Computer ScienceFindings
- 1 April 2020
Experimental results show that MacBERT could achieve state-of-the-art performances on many NLP tasks, and it is proposed that this model improves upon RoBERTa in several ways, especially the masking strategy that adopts MLM as correction (Mac).
Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation
- Wanxiang Che, Yijia Liu, Yuxuan Wang, Bo Zheng, Ting Liu
- Computer ScienceConference on Computational Natural Language…
- 9 July 2018
This paper describes the system (HIT-SCIR) submitted to the CoNLL 2018 shared task on Multilingual Parsing from Raw Text to Universal Dependencies, which was ranked first according to LAS and outperformed the other systems by a large margin.
A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding
- Libo Qin, Wanxiang Che, Yangming Li, Haoyang Wen, Ting Liu
- Computer ScienceConference on Empirical Methods in Natural…
- 1 September 2019
A novel framework for SLU to better incorporate the intent information, which further guiding the slot filling is proposed, which achieves the state-of-the-art performance and outperforms other previous methods by a large margin.
A Span-Extraction Dataset for Chinese Machine Reading Comprehension
- Yiming Cui, Ting Liu, Guoping Hu
- Computer ScienceEMNLP-IJCNLP
- 2019
This paper introduces a Span-Extraction dataset for Chinese machine reading comprehension to add language diversities in this area and hosted the Second Evaluation Workshop on Chinese Machine Reading Comprehension (CMRC 2018).
Learning Semantic Hierarchies via Word Embeddings
- Ruiji Fu, Jiang Guo, Bing Qin, Wanxiang Che, Haifeng Wang, Ting Liu
- Computer ScienceAnnual Meeting of the Association for…
- 1 June 2014
This paper proposes a novel and effective method for the construction of semantic hierarchies based on word embeddings, which can be used to measure the semantic relationship between words.
Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network
- Yutai Hou, Wanxiang Che, Ting Liu
- Computer ScienceAnnual Meeting of the Association for…
- 10 June 2020
A Label-enhanced Task-Adaptive Projection Network (L-TapNet) based on the state-of-the-art few-shot classification model – TapNet is proposed, by leveraging label name semantics in representing labels.
...
...