Developing a successful SemEval task in sentiment analysis of Twitter and other social media texts

@article{Nakov2016DevelopingAS,
  title={Developing a successful SemEval task in sentiment analysis of Twitter and other social media texts},
  author={Preslav Nakov and Sara Rosenthal and Svetlana Kiritchenko and Saif M. Mohammad and Zornitsa Kozareva and Alan Ritter and Veselin Stoyanov and Xiao-Dan Zhu},
  journal={Language Resources and Evaluation},
  year={2016},
  volume={50},
  pages={35-65}
}
We present the development and evaluation of a semantic analysis task that lies at the intersection of two very trendy lines of research in contemporary computational linguistics: (1) sentiment analysis, and (2) natural language processing of social media text. [...] Key Result We hope that the long-lasting role of this task and the accompanying datasets will be to serve as a test bed for comparing different approaches, thus facilitating research.Expand
Sentiment Polarity Classification at EVALITA: Lessons Learned and Open Challenges
TLDR
The experience in the organization of SENTIment POLarity Classification Task (SENTIPOLC), a shared task on sentiment classification of Italian tweets, proposed for the first time in 2014 within the Evalita evaluation campaign is reported on.
Sentiment Analysis in Twitter for Macedonian
TLDR
A corpus of tweets annotated with tweet- level sentiment polarity (positive, negative, and neutral), as well as with phrase-level sentiment, which was developed and made freely available for research purposes and bootstrapped several large-scale sentiment lexicons for Macedonian.
#europehappinessmap: A Framework for Multi-Lingual Sentiment Analysis via Social Media Big Data (A Twitter Case Study)
TLDR
A novel multilingual social media sentiment analysis framework is proposed to compute European Gross National Happiness for countries and change the way of OECD type organizations’ survey and interview methodology.
National happiness index monitoring using Twitter for bilanguages
TLDR
Experiments show that the proposed improved LDA for sentiment analysis yields the best results, and also validate the proposed system for national sentiment monitoring in Abu Dhabi using twitter.
Odi et Amo. Creating, Evaluating and Extending Sentiment Lexicons for Latin.
TLDR
The research community is provided with a set of sentiment lexicons built by taking advantage of manually-curated resources belonging to the long tradition of Latin corpora and lexicons creation, including a gold standard developed by two Latin language and culture experts.
The Multifaceted Concept of Context in Sentiment Analysis
TLDR
A strength–weakness–opportunity–threat (SWOT) matrix is made to demonstrate the effectiveness of context-based sentiment analysis and the concept of ‘context in use’ is described by exemplifying the types of context.
Sentiment Analysis in Twitter: A SemEval Perspective
TLDR
The recent rise of social media has greatly democratized content creation, with Twitter being especially popular for research due to its scale, representativeness, variety of topics discussed, as well as ease of public access to its messages.
SemEval-2016 Task 4: Sentiment Analysis in Twitter
TLDR
The fourth year of the SemEval-2016 Task 4 comprises five subtasks, three of which represent a significant departure from previous editions, and the task continues to be very popular, attracting a total of 43 teams.
FinnSentiment - A Finnish Social Media Corpus for Sentiment Polarity Annotation
TLDR
This publications introduces a 27,000 sentence data set annotated independently with sentiment polarity by three native annotators for Finnish, and analyses their inter-annotator agreement and provides two baselines to validate the usefulness of the data set.
Annotated news corpora and a lexicon for sentiment analysis in Slovene
TLDR
This study introduces Slovene web-crawled news corpora with sentiment annotation on three levels of granularity: sentence, paragraph and document levels, and constructs a Slovene sentiment lexicon, the first of its kind for Slovene.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 97 REFERENCES
SemEval-2013 Task 2: Sentiment Analysis in Twitter
TLDR
Crowdourcing on Amazon Mechanical Turk was used to label a large Twitter training dataset along with additional test sets of Twitter and SMS messages for both subtasks, which included two subtasks: A, an expression-level subtask, and B, a message level subtask.
SemEval-2014 Task 9: Sentiment Analysis in Twitter
TLDR
The Sentiment Analysis in Twitter task is described, a continuation of the last year’s task that ran successfully as part of SemEval2013 and introduced three new test sets: regular tweets, sarcastic tweets, and LiveJournal sentences.
Sentiment Analysis in Twitter for Macedonian
TLDR
A corpus of tweets annotated with tweet- level sentiment polarity (positive, negative, and neutral), as well as with phrase-level sentiment, which was developed and made freely available for research purposes and bootstrapped several large-scale sentiment lexicons for Macedonian.
NRC-Canada-2014: Recent Improvements in the Sentiment Analysis of Tweets
TLDR
This paper describes state-of-the-art statistical systems for automatic sentiment analysis of tweets and creates separate tweet-specific sentiment lexicons for terms in affirmative contexts and in negated contexts.
SemEval-2015 Task 11: Sentiment Analysis of Figurative Language in Twitter
This report summarizes the objectives and evaluation of the SemEval 2015 task on the sentiment analysis of figurative language on Twitter (Task 11). This is the first sentiment analysis task wholly
SemEval-2015 Task 10: Sentiment Analysis in Twitter
TLDR
The 2015 iteration of the SemEval shared task on Sentiment Analysis in Twitter was the most popular sentiment analysis shared task to date with more than 40 teams participating in each of the last three years.
Sentiment Analysis of Short Informal Texts
We describe a state-of-the-art sentiment analysis system that detects (a) the sentiment of short informal textual messages such as tweets and SMS (message-level task) and (b) the sentiment of a word
AVAYA: Sentiment Analysis on Twitter with Self-Training and Polarity Lexicon Expansion
TLDR
Results show that expanding thePolarity lexicon and augmenting the training data with unlabeled tweets can yield improvements in precision and recall in classifying the polarity of non-neutral messages and contexts.
Semi-supervised recognition of sarcastic sentences in Twitter and Amazon
TLDR
This paper experiments with semi-supervised sarcasm identification on two very different data sets: a collection of 5.9 million tweets collected from Twitter, and aCollection of 66000 product reviews from Amazon.
Semi-Supervised Recognition of Sarcasm in Twitter and Amazon
TLDR
This paper experiments with semisupervised sarcasm identification on two very different data sets: a collection of 5.9 million tweets collected from Twitter, and aCollection of 66000 product reviews from Amazon.
...
1
2
3
4
5
...