Overview of the CLEF-2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News

@article{Nakov2021OverviewOT,
  title={Overview of the CLEF-2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News},
  author={Preslav Nakov and Giovanni Da San Martino and Tamer Elsayed and Alberto Barr{\'o}n-Cede{\~n}o and Rub{\'e}n M{\'i}guez and Shaden Shaar and Firoj Alam and Fatima Haouari and Maram Hasanain and Watheq Mansour and Bayan Hamdan and Zien Sheikh Ali and Nikolay Babulkov and Alex Nikolov and Gautam Kishore Shahi and Julia Maria Stru{\ss} and Thomas Mandl and Mucahid Kutlu and Yavuz Selim Kartal},
  journal={ArXiv},
  year={2021},
  volume={abs/2109.12987}
}
We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics (in all five languages). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked… Expand

Figures and Tables from this paper

UPV at CheckThat! 2021: Mitigating Cultural Differences for Identifying Multilingual Check-worthy Claims
TLDR
Joint training of language identification and check-worthy claim detection tasks can provide performance gains for some of the selected languages and is proposed as an auxiliary task to mitigate unintended bias. Expand
Accenture at CheckThat! 2021: Interesting claim identification and ranking with contextually sensitive lexical training data augmentation
TLDR
The architecture and data augmentation pipeline produced the best submitted system for Arabic, and performance scales according to the quantity of provided training data for English, Spanish, Turkish, and Bulgarian. Expand
NITK_NLP at CheckThat! 2021: Ensemble Transformer Model for Fake News Classification
Social media has become an inevitable part of our life as we are primarily dependent on them to get most of the news around us. However, the amount of false information propagated through it is muchExpand
CIVIC-UPM at CheckThat! 2021: Integration of Transformers in Misinformation Detection and Topic Classification
TLDR
A Natural Language Processing (NLP) approach that uses Doc2Vec and different state-of-the-art transformer-based models for the CLEF2021 Checkthat! lab Task 3 and the results show that it is possible to achieve 41.43% macro-average F1-score in the misinformation detection and 67.65% in the topic classification. Expand
BlackOps at CheckThat! 2021: User Profiles Analyze of Intelligent Detection on Fake Tweets Notebook for PAN
An expensive task is fake news detection for recent trends among the concept of misinformation or rumors. In everywhere most of the times information lead or play emergent preface but forthwithExpand
Nkovachevich at CheckThat! 2021: BERT fine-tuning approach to fake news detection
TLDR
The experiments, which include BERT fine-tuning, together with a couple of simpler algorithms, produce average results, and hardly overcome model and data size limitations, but show a potential to be successful if implemented properly. Expand
Classifier for fake news detection and Topical Domain of News Articles
TLDR
Machine learning algorithms might be beneficial in classifying articles based on labels provided by experts in order to combat the spread of fake news. Expand
WhatTheWikiFact: Fact-Checking Claims Against Wikipedia
TLDR
WhatTheWikiFact, a system for automatic claim verification using Wikipedia, can predict the veracity of an input claim, and it further shows the evidence it has retrieved as part of the verification process. Expand
MEDIC: A Multi-Task Learning Dataset for Disaster Image Classification
TLDR
This work proposes MEDIC1, which is the largest social media image classification dataset for humanitarian response consisting of 71,198 images to address four different tasks in a multitask learning setup and has high potential to contribute research on multi-task learning, which recently receives much interest from the machine learning community. Expand
Overview of the CLEF-2021 CheckThat! Lab: Task 3 on Fake News Detection
TLDR
All data sets from the lab as well as the evaluation scripts, which should enable further research in automatic classification of news articles with respect to their correctness and topical domain, are released to the research community. Expand
...
1
2
...

References

SHOWING 1-10 OF 108 REFERENCES
The CLEF-2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News
We describe the fourth edition of the CheckThat! Lab, part of the 2021 Cross-Language Evaluation Forum (CLEF). The lab evaluates technology supporting various tasks related to factuality, and it isExpand
Overview of the CLEF-2021 CheckThat! Lab Task 2 on Detecting Previously Fact-Checked Claims in Tweets and Political Debates
We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting three tasks related to factuality,Expand
Overview of the CLEF-2021 CheckThat! Lab Task 1 on Check-Worthiness Estimation in Tweets and Political Debates
We present an overview of Task 1 of the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The task asks to predict which posts in a TwitterExpand
Overview of the CLEF-2019 CheckThat! Lab: Automatic Identification and Verification of Claims. Task 1: Check-Worthiness
We present an overview of the 2nd edition of the CheckThat! Lab, part of CLEF 2019, with focus on Task 1: Check-Worthiness in political debates. The task asks to predict which claims in a politicalExpand
Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness
We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims, with focus on Task 1: Check-Worthiness. The task asks to predict which claimsExpand
Overview of CheckThat 2020: Automatic Identification and Verification of Claims in Social Media
We present an overview of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured five tasks in two different languages: English and Arabic. The first four tasks compose the fullExpand
CheckThat! at CLEF 2019: Automatic Identification and Verification of Claims
We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in aExpand
Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 2: Factuality
We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims, with focus on Task 2: Factuality. The task asked to assess whether a givenExpand
Overview of the CLEF-2019 CheckThat!: Automatic Identification and Verification of Claims
We present an overview of the second edition of the CheckThat! Lab at CLEF 2019. The lab featured two tasks in two different languages: English and Arabic. Task 1 (English) challenged theExpand
Overview of CheckThat! 2020i Arabic: Automatic Identification and Verification of Claims in Social Media
In this paper, we present an overview of the Arabic tasks of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured three Arabic tasks over social media (and the Web): Task 1 onExpand
...
1
2
3
4
5
...