Multi-dataset-multi-task Neural Sequence Tagging for Information Extraction from Tweets

@article{Mishra2019MultidatasetmultitaskNS,
  title={Multi-dataset-multi-task Neural Sequence Tagging for Information Extraction from Tweets},
  author={Shubhanshu Mishra},
  journal={Proceedings of the 30th ACM Conference on Hypertext and Social Media},
  year={2019}
}
  • Shubhanshu Mishra
  • Published 2019
  • Computer Science
  • Proceedings of the 30th ACM Conference on Hypertext and Social Media
Multi-task learning is effective in reducing the required data for learning a task, while ensuring competitive accuracy with respect to single task learning. We study effectiveness of multi-dataset-multi-task learning in training neural models for four sequence tagging tasks for Twitter data, namely, part of speech (POS) tagging, chunking, super sense tagging, and named entity recognition (NER). We utilize -- 7 POS, 10 NER, 1 Chunking, and 2 super sense -- tagged publicly available datasets. We… Expand
Multi-Task Supervised Pretraining for Neural Domain Adaptation
TLDR
This paper proposes a new approach which takes advantage from both approaches by learning a hierarchical model trained across multiple tasks from a source domain, and is then fine-tuned on multiple tasks of the target domain. Expand
Exploring multi-task multi-lingual learning of transformer models for hate speech and offensive speech identification in social media
TLDR
A multi-task and multilingual approach based on recently proposed Transformer Neural Networks to solve three sub-tasks for hate speech to show that it is possible to to utilize different combined approaches to obtain models that can generalize easily on different languages and tasks, while trading off slight accuracy for a much reduced inference time compute cost. Expand
Non-neural Structured Prediction for Event Detection from News in Indian Languages
TLDR
This method is based on structured prediction using only word n-gram and regex features and does not rely on any latest deep learning or neural network methods, and was the best performing system on all tasks and languages. Expand
Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units
TLDR
This paper proposes to augment the pre-trained model with normalised, weighted and randomly initialised units that foster a better adaptation while maintaining the valuable source knowledge in the standard fine-tuning scheme of neural domain adaptation. Expand
Assessing Demographic Bias in Named Entity Recognition
TLDR
This work assesses the bias in various Named Entity Recognition systems for English across different demographic groups with synthetically generated corpora to shed light on potential biases in automated KB generation due to systematic exclusion of named entities belonging to certain demographics. Expand
Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020
TLDR
The multilingual joint training approach to be the best trade-off between computational efficiency of model deployment and model’s evaluation performance and the utility of task label marginalization, joint label classification, and joint training on multilingual datasets. Expand
3Idiots at HASOC 2019: Fine-tuning Transformer Neural Networks for Hate Speech Identification in Indo-European Languages
TLDR
The team 3Idiots’s approach for participating in the 2019 shared task on hate speech and offensive content (HASOC) identification in Indo-European languages relies on finetuning pre-trained monolingual and multilingual transformer (BERT) based neural network models and investigates an approach based on labels joined from all sub-tasks. Expand
ALBERT-based fine-tuning model for cyberbullying analysis
TLDR
This paper explores some of the recent approaches to the difficulties of contextual understanding and proposes an ALBERT-based fine-tuned model that achieves state-of-the-art results. Expand
SocialNLP 2020 The Eighth International Workshop on Natural Language Processing for Social Media Proceedings of the Conference
  • 2020
Usage of presuppositions in social media and news discourse can be a powerful way to influence the readers as they usually tend to not examine the truth value of the hidden or indirectly expressedExpand
Information extraction from digital social trace data with applications to social media and scholarly communication data
Shubhanshu is a Machine Learning Researcher at Twitter working on the Content Understanding Research team. He finished his Ph.D. at the iSchool, University of Illinois at Urbana-Champaign, where heExpand
...
1
2
...

References

SHOWING 1-10 OF 13 REFERENCES
Twitter Part-of-Speech Tagging for All: Overcoming Sparse and Noisy Data
TLDR
A novel approach to system combination for the case where available taggers use different tagsets, based on voteconstrained bootstrapping with unlabeled data, reaches 88.7% tagging accuracy, a new high in PTB-compatible tweet part-of-speech tagging. Expand
Semi-supervised Named Entity Recognition in noisy-text
TLDR
The models described in this paper are based on linear chain conditional random fields (CRFs), use the BIEOU encoding scheme, and leverage random feature dropout for up-sampling the training data, and include word clusters and pre-trained distributed word representations, updated gazetteer features, and global context predictions. Expand
Named Entity Recognition in Tweets: An Experimental Study
TLDR
The novel T-ner system doubles F1 score compared with the Stanford NER system, and leverages the redundancy inherent in tweets to achieve this performance, using LabeledLDA to exploit Freebase dictionaries as a source of distant supervision. Expand
Microblog-genre noise and impact on semantic annotation accuracy
TLDR
The results demonstrate the importance of making approaches specific to the genre, and indicate a diminishing returns effect that reduces the effectiveness of complex text normalisation in semantic annotation of tweets. Expand
Detecting the Correlation between Sentiment and User-level as well as Text-Level Meta-data from Benchmark Corpora
TLDR
This paper analyzes six popular benchmark datasets where tweets are annotated with sentiment labels to identify patterns and correlations of meta-data features of tweets and users with potential to improve sentiment analysis applications on social media data. Expand
Natural Language Processing (Almost) from Scratch
We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including part-of-speech tagging, chunking, named entityExpand
When is multitask learning effective? Semantic sequence prediction under varying data conditions
TLDR
This paper evaluates a range of semantic sequence labeling tasks in a MTL setup, and shows that MTL is not always effective, because significant improvements are obtained only for 1 out of 5 tasks. Expand
Deep Contextualized Word Representations
TLDR
A new type of deep contextualized word representation is introduced that models both complex characteristics of word use and how these uses vary across linguistic contexts, allowing downstream models to mix different types of semi-supervision signals. Expand
Sentiment Analysis with Incremental Human-in-the-Loop Learning and Lexical Resource Customization
TLDR
This work provides a free, open and GUI-based sentiment analysis tool that allows for a) relabeling predictions and/or adding labeled instances to retrain the weights of a given model, and b) customizing lexical resources to account for false positives and false negatives in sentiment lexicons. Expand
Multitask Learning: A Knowledge-Based Source of Inductive Bias
This paper suggests that it may be easier to learn several hard tasks at one time than to learn these same tasks separately. In effect, the information provided by the training signal for each taskExpand
...
1
2
...