Skip to search formSkip to main contentSkip to account menu
Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definitions
TLDR
Due to a pervasive lack of clarity in reports and extreme diversity in approaches, human evaluation in NLG presents as extremely confused in 2020, and that the field is in urgent need of standard methods and terminology.
View on ACL
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
TLDR
GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics, is introduced and the description of the data for the 2021 shared task at the associated GEM Workshop is described.
View on ACL
I Stand With You: Using Emojis to Study Solidarity in Crisis Events
TLDR
Emojis are a powerful indicator of sociolinguistic behaviors that are exhibited on social media as the crisis events unfold and are used to characterize human behavior in online social networks, through the temporal and geospatial diffusion of emojis.
View PDF on arXiv
Can You Verifi This? Studying Uncertainty and Decision-Making About Misinformation Using Visual Analytics
TLDR
It is revealed that the presence of conflicting information, presented to users in the form of cues, impacts the ability to judge the veracity of news in systematic ways and has the potential to inform the design of visual analytics systems so that they may be used to mitigate the effects of cognitive biases and stymie the spread of misinformation on social media.
View Paper
Vulnerable to misinformation?: Verifi!
TLDR
Interviews with experts in digital media, communications, education, and psychology who study misinformation highlight the complexity of the problem of combating misinformation and show promising potential for Verifi2 as an educational tool on misinformation.
View on ACM
Towards Best Experiment Design for Evaluating Dialogue System Output
TLDR
Through a systematic study with 40 crowdsourced workers in each task, it is found that using continuous scales achieves more consistent ratings than Likert scale or ranking-based experiment design and that factors such as time taken to complete the task and no prior experience of participating in similar studies of rating dialogue system output positively impact consistency and agreement amongst raters.
View on ACL
Emotional Neural Language Generation Grounded in Situational Contexts
TLDR
This work develops a language modeling approach that generates affective content when the dialogue is situated in a given context and outperforms the state-of-the-art method on the perplexity metric and achieves a higher BLEU metric score.
View on ACL
Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation
TLDR
This paper presents a human annotation setup to identify three different response types: responses that are factually consistent with respect to the input knowledge, responses that contain hallucinated knowledge, and non-verifiable chitchat style responses.
View PDF on arXiv
Anchored in a Data Storm: How Anchoring Bias Can Affect User Strategy, Confidence, and Decisions in Visual Analytics
TLDR
The findings reveal that strategy cues and visual anchors can significantly affect user activity, speed, confidence, and, under certain circumstances, accuracy.
View PDF on arXiv
The Anchoring Effect in Decision-Making with Visual Analytics
TLDR
This paper presents a novel, systematic study and resulting analyses that investigate the effects of anchoring effect on human decision-making using visual analytic systems, and describes detailed analyses of users’ interaction logs which reveal the impact of Anchoring bias on the visual representation preferred and paths of analysis.
View on IEEE
...
...
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy, Terms of Service, and Dataset License