NutCracker at WNUT-2020 Task 2: Robustly Identifying Informative COVID-19 Tweets using Ensembling and Adversarial Training

@inproceedings{Kumar2020NutCrackerAW,
  title={NutCracker at WNUT-2020 Task 2: Robustly Identifying Informative COVID-19 Tweets using Ensembling and Adversarial Training},
  author={Priyanshu Kumar and Aadarsh Singh},
  booktitle={WNUT},
  year={2020}
}
We experiment with COVID-Twitter-BERT and RoBERTa models to identify informative COVID-19 tweets. We further experiment with adversarial training to make our models robust. The ensemble of COVID-Twitter-BERT and RoBERTa obtains a F1-score of 0.9096 (on the positive class) on the test data of WNUT-2020 Task 2 and ranks 1st on the leaderboard. The ensemble of the models trained using adversarial training also produces similar result. 

Tables from this paper

WNUT-2020 Task 2: Identification of Informative COVID-19 English Tweets
TLDR
An overview of the WNUT-2020 shared task on the identification of informative COVID-19 English Tweets is provided, finding that many systems obtain very high performance and fine-tuning pre-trained language models on relevant language data followed by supervised training performs well in this task.
UPB at SemEval-2021 Task 5: Virtual Adversarial Training for Toxic Spans Detection
TLDR
This model considers applying Virtual Adversarial Training in a semi-supervised setting during the fine-tuning process of several Transformer-based models (i.e., BERT and RoBERTa), in combination with Conditional Random Fields, leading to performance improvements and more robust models.
Finding the needle in a haystack: Extraction of Informative COVID-19 Danish Tweets
TLDR
This work introduces a new dataset of 5,000 tweets for finding informative COVID-19 tweets for Danish and model the problem by keeping its natural distribution, and examines how well a simple probabilistic model and a convolutional neural network perform on this task.
The Surprising Performance of Simple Baselines for Misinformation Detection
TLDR
This paper examines the performance of a broad set of modern transformer-based language models and shows that with basic fine-tuning, these models are competitive with and can even significantly outperform recently proposed state-of-the-art methods.
Classification aware neural topic model for COVID-19 disinformation categorisation.
TLDR
A corpus containing what is currently the largest available set of manually annotated COVID-19 disinformation categories is presented, and a classification-aware neural topic model (CANTM) designed for COVID -19 disinformation category classification and topic discovery is presented.
Fake News Detection System using XLNet model with Topic Distributions: CONSTRAINT@AAAI2021 Shared Task
TLDR
The team introduced an approach to combine topical distributions from Latent Dirichlet Allocation (LDA) with contextualized representations from XLNet, and compared the method with existing baselines to show that XLNet \(+\) Topic Distributions outperforms other approaches by attaining an F1-score of 0.967.
g2tmn at Constraint@AAAI2021: Exploiting CT-BERT and Ensembling Learning for COVID-19 Fake News Detection
TLDR
This paper proposes the approach using the transformer-based ensemble of COVID-Twitter-BerT (CT-BERT) models, and describes the models used, the ways of text preprocessing and adding extra data, and the best model achieved the weighted F1-score of 98.69 on the test set of this shared task.

References

SHOWING 1-10 OF 23 REFERENCES
WNUT-2020 Task 2: Identification of Informative COVID-19 English Tweets
TLDR
An overview of the WNUT-2020 shared task on the identification of informative COVID-19 English Tweets is provided, finding that many systems obtain very high performance and fine-tuning pre-trained language models on relevant language data followed by supervised training performs well in this task.
COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter
TLDR
COVID-Twitter-BerT (CT-BERT), a transformer-based model, pretrained on a large corpus of Twitter messages on the topic of COVID-19, shows a 10-30% marginal improvement compared to its base model, BERT-Large, on five different classification datasets.
Fine-tuned BERT Model for Multi-Label Tweets Classification
TLDR
The approach to classify disaster-related tweets into multilabel information types (i.e, labels) and submitted to the TREC-IS 2019 challenge showed that the approach outperforms the F1-score of median score in identifying actionable information.
Adversarial Training Methods for Semi-Supervised Text Classification
TLDR
This work extends adversarial and virtual adversarial training to the text domain by applying perturbations to the word embeddings in a recurrent neural network rather than to the original input itself.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
TLDR
It is found that BERT was significantly undertrained, and can match or exceed the performance of every model published after it, and the best model achieves state-of-the-art results on GLUE, RACE and SQuAD.
A first look at COVID-19 information and misinformation sharing on Twitter
TLDR
It is suggested that a meaningful spatio-temporal relationship exists between information flow and new cases of COVID-19, and while discussions about myths and links to poor quality information exist, their presence is less dominant than other crisis specific themes.
Deep Neural Networks versus Naive Bayes Classifiers for Identifying Informative Tweets during Disasters
TLDR
This paper designs various feature sets (based on tweet content, user details and polarity clues) and uses these feature sets individually or in various combinations, with Naïve Bayes classifiers, and develops neural models based on Convolutional Neural Networks and Recurrent Neural Networks with handcrafted architectures.
Classification Approaches to Identify Informative Tweets
TLDR
The challenge of classifying a social post as news or not is tackled and a new manually annotated dataset containing 2,992 tweets from 5 different topical categories is provided, which shows that the best classifiers are the SVM and BERT models.
Classification and Summarization for Informative Tweets
  • S. Roy, Sumit Mishra, Rakesh Matam
  • Computer Science
    2020 IEEE International Students' Conference on Electrical,Electronics and Computer Science (SCEECS)
  • 2020
TLDR
A disaster dataset (Fani Cyclone dataset) is considered, which consists of the tweets related to a Cyclone named "Fani" and the tweets are pre-processed and then classified into two categories – informative and non-informative.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TLDR
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
...
1
2
3
...