Corpus ID: 218973958

Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020

@inproceedings{Mishra2020MultilingualJF,
  title={Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020},
  author={Sudhanshu Mishra and S. Prasad and Shubhanshu Mishra},
  booktitle={TRAC},
  year={2020}
}
We present our team ‘3Idiots’ (referred as ‘sdhanshu’ in the official rankings) approach for the Trolling, Aggression and Cyberbullying (TRAC) 2020 shared tasks. Our approach relies on fine-tuning various Transformer models on the different datasets. We also investigated the utility of task label marginalization, joint label classification, and joint training on multilingual datasets as possible improvements to our models. Our team came second in English sub-task A, a close fourth in the… Expand
Exploring multi-task multi-lingual learning of transformer models for hate speech and offensive speech identification in social media
TLDR
A multi-task and multilingual approach based on recently proposed Transformer Neural Networks to solve three sub-tasks for hate speech to show that it is possible to to utilize different combined approaches to obtain models that can generalize easily on different languages and tasks, while trading off slight accuracy for a much reduced inference time compute cost. Expand
Non-neural Structured Prediction for Event Detection from News in Indian Languages
TLDR
This method is based on structured prediction using only word n-gram and regex features and does not rely on any latest deep learning or neural network methods, and was the best performing system on all tasks and languages. Expand
Evaluating Aggression Identification in Social Media
TLDR
The report and findings of the Shared Task on Aggression and Gendered Aggression Identification organised as part of the Second Workshop on Trolling,Aggression and Cyberbullying (TRAC - 2) at LREC 2020 are presented. Expand
Information extraction from digital social trace data with applications to social media and scholarly communication data
Shubhanshu is a Machine Learning Researcher at Twitter working on the Content Understanding Research team. He finished his Ph.D. at the iSchool, University of Illinois at Urbana-Champaign, where heExpand

References

SHOWING 1-10 OF 15 REFERENCES
3Idiots at HASOC 2019: Fine-tuning Transformer Neural Networks for Hate Speech Identification in Indo-European Languages
TLDR
The team 3Idiots’s approach for participating in the 2019 shared task on hate speech and offensive content (HASOC) identification in Indo-European languages relies on finetuning pre-trained monolingual and multilingual transformer (BERT) based neural network models and investigates an approach based on labels joined from all sub-tasks. Expand
Unsupervised Cross-lingual Representation Learning at Scale
TLDR
It is shown that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks, and the possibility of multilingual modeling without sacrificing per-language performance is shown for the first time. Expand
Overview of the HASOC track at FIRE 2019: Hate Speech and Offensive Content Identification in Indo-European Languages
TLDR
The HASOC track intends to stimulate development in Hate Speech for Hindi, German and English by identifying Hate Speech in Social Media using LSTM networks processing word embedding input. Expand
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TLDR
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks. Expand
Multi-dataset-multi-task Neural Sequence Tagging for Information Extraction from Tweets
TLDR
Effectiveness of multi-dataset-multi-task learning in training neural models for four sequence tagging tasks for Twitter data, namely, part of speech tagging, chunking, super sense tagging, and named entity recognition is studied. Expand
Sentiment Analysis with Incremental Human-in-the-Loop Learning and Lexical Resource Customization
TLDR
This work provides a free, open and GUI-based sentiment analysis tool that allows for a) relabeling predictions and/or adding labeled instances to retrain the weights of a given model, and b) customizing lexical resources to account for false positives and false negatives in sentiment lexicons. Expand
HuggingFace's Transformers: State-of-the-art Natural Language Processing
TLDR
The \textit{Transformers} library is an open-source library that consists of carefully engineered state-of-the art Transformer architectures under a unified API and a curated collection of pretrained models made by and available for the community. Expand
Evaluating Aggression Identification in Social Media
TLDR
The report and findings of the Shared Task on Aggression and Gendered Aggression Identification organised as part of the Second Workshop on Trolling,Aggression and Cyberbullying (TRAC - 2) at LREC 2020 are presented. Expand
RoBERTa: A Robustly Optimized BERT Pretraining Approach
TLDR
It is found that BERT was significantly undertrained, and can match or exceed the performance of every model published after it, and the best model achieves state-of-the-art results on GLUE, RACE and SQuAD. Expand
Challenges and frontiers in abusive content detection
Online abusive content detection is an inherently difficult task. It has received considerable attention from academia, particularly within the computational linguistics community, and performanceExpand
...
1
2
...