A Linguistic Investigation of Machine Learning based Contradiction Detection Models: An Empirical Analysis and Future Perspectives

@article{Pielka2022ALI,
  title={A Linguistic Investigation of Machine Learning based Contradiction Detection Models: An Empirical Analysis and Future Perspectives},
  author={Maren Pielka and F Rode and Lisa Pucknat and Tobias Deu{\ss}er and Rafet Sifa},
  journal={2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA)},
  year={2022},
  pages={1649-1653}
}
  • Maren PielkaF. Rode R. Sifa
  • Published 19 October 2022
  • Computer Science
  • 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA)
We analyze two Natural Language Inference data sets with respect to their linguistic features. The goal is to identify those syntactic and semantic properties that are particularly hard to comprehend for a machine learning model. To this end, we also investigate the differences between a crowd-sourced, machine-translated data set (SNLI) and a collection of text pairs from internet sources. Our main findings are, that the model has difficulty recognizing the semantic importance of prepositions… 
[PDF] Semantic Reader
1 Citations

Tables from this paper

One Citation

Deeper Bidirectional Neural Networks with Generalized Non-Vanishing Hidden Neurons

The new NoVa hidden neurons have outperformed ReLU hidden neurons in deep classifiers on some large image test sets and are compared to ReLU and several other hidden neurons on large imageTest sets that include CIFAR-100 and Caltech-256.

References

SHOWING 1-10 OF 13 REFERENCES

Detecting Contradictions in German Text: A Comparative Study

Four state-of-the-art model paradigms are being compared with respect to their performance on a machine-translated version of the well-known Stanford Natural Language Inference data set (SNLI), as well as on the German test split of the Cross-Lingual NLI corpus (XNLI).

Towards Contradiction Detection in German: a Translation-Driven Approach

This paper presents the first large-scale assessment for this problem in German, and a validation of machine translation as a data generation method, and presents a novel approach to learn sentence embeddings by exploiting the hidden states of an encoder-decoder Sequence-To-Sequence RNN trained for autoencoding or translation.

A large annotated corpus for learning natural language inference

The Stanford Natural Language Inference corpus is introduced, a new, freely available collection of labeled sentence pairs, written by humans doing a novel grounded task based on image captioning, which allows a neural network-based model to perform competitively on natural language inference benchmarks for the first time.

Finding Contradictions in Text

It is demonstrated that a system for contradiction needs to make more fine-grained distinctions than the common systems for entailment, and is argued for the centrality of event coreference and therefore incorporate such a component based on topicality.

Contradiction Detection with Contradiction-Specific Word Embedding

A tailored neural network to learn contradiction-specific word embedding (CWE), which can separate antonyms in the opposite ends of a spectrum and performs comparably with the top-performing system in accuracy of three-category classification.

Unsupervised Cross-lingual Representation Learning at Scale

It is shown that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks, and the possibility of multilingual modeling without sacrificing per-language performance is shown for the first time.

LIMIT-BERT : Linguistics Informed Multi-Task BERT

LIMIT-BERT outperforms the strong baseline Whole Word Masking BERT on both dependency and constituent syntactic/semantic parsing, GLUE benchmark, and SNLI task and is able to release a well pre-trained model for multi-purpose of natural language processing tasks once for all.

KPI-BERT: A Joint Named Entity Recognition and Relation Extraction Model for Financial Reports

An end-to-end trainable architecture that is based on Bidirectional Encoder Representations from Transformers combining a recurrent neural network (RNN) with conditional label masking to sequentially tag entities before it classifies their relations is introduced.

Informed Pre-Training on Prior Knowledge

It is shown that informed pre-training on such knowledge prototypes speeds up the learning processes, improves generalization capabilities in the regime where not enough training data is available, and increases model robustness.

RoBERTa: A Robustly Optimized BERT Pretraining Approach

It is found that BERT was significantly undertrained, and can match or exceed the performance of every model published after it, and the best model achieves state-of-the-art results on GLUE, RACE and SQuAD.