All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text
- Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, Noah A. Smith
- PsychologyAnnual Meeting of the Association for…
- 30 June 2021
The role untrained human evaluations play in NLG evaluation is examined and three approaches for quickly training evaluators to better identify GPT3-authored text are explored and it is found that while evaluation accuracy improved up to 55%, it did not significantly improve across the three domains.
Generating Scientific Definitions with Controllable Complexity
- Tal August, Katharina Reinecke, Noah A. Smith
- Computer ScienceAnnual Meeting of the Association for…
- 2022
A novel reranking approach is introduced and it is found in human evaluations that it offers superior fluency while also controlling complexity, compared to several controllable generation baselines.
Explain like I am a Scientist: The Linguistic Barriers of Entry to r/science
- Tal August, Dallas Card, Gary Hsieh, Noah A. Smith, Katharina Reinecke
- EducationInternational Conference on Human Factors in…
- 21 April 2020
It is shown that r/science uses a specialized language that is distinct from other subreddits, and it is suggested that the specialized language used in r/ science has a gatekeeping effect, preventing participation by people whose language does not align with that used inr/science.
A Case for Design Localization : Diversity of Website Designs in 44 Countries
- Manuel Nordhoff, Tal August, Nigini Oliveira, Katharina Reinecke
- Computer Science
- 2018
The first large-scale analysis of 80,901 website designs across 44 countries, made available via an interactive web-based design catalog, finds significant differences between several design aspects, such as a website’s colorfulness, visual complexity, the number of text areas and the average saturation of colors.
Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing
- Tal August, Lucy Lu Wang, Jonathan Bragg, Marti A. Hearst, Andrew Head, Kyle Lo
- Computer ScienceACM Transactions on Computer-Human Interaction
- 28 February 2022
The study results suggest that guiding readers to relevant passages and providing plain language summaries alongside the original paper content can make reading medical papers easier and give readers more confidence to approach these papers.
The Effect of Moderation on Online Mental Health Conversations
- David Wadden, Tal August, Qisheng Li, Tim Althoff
- PsychologyInternational Conference on Web and Social Media
- 19 May 2020
It is found that participation in group mental health discussions led to improvements in psychological perspective, and that these improvements were larger in moderated conversations, suggesting that moderation can serve as a valuable tool to improve the efficacy and safety of online mental health conversations.
Writing Strategies for Science Communication: Data and Computational Analysis
- Tal August, Lauren Kim, Katharina Reinecke, Noah A. Smith
- EducationConference on Empirical Methods in Natural…
- 1 November 2020
This work compiles a set of writing strategies drawn from a wide range of prescriptive sources and develops an annotation scheme allowing humans to recognize them, and collects and annotates a subset of this corpus.
Keep it Simple: How Visual Complexity and Preferences Impact Search Efficiency on Websites
- Amanda Baughan, Tal August, Naomi Yamashita, Katharina Reinecke
- Computer ScienceInternational Conference on Human Factors in…
- 21 April 2020
It is confirmed that the visual complexity of a website has a significant negative effect on search efficiency and information recall, however, the search efficiency of those who preferred simple websites was more negatively affected by highly complex websites than those who prefer high visual complexity.
Pay Attention, Please: Formal Language Improves Attention in Volunteer and Paid Online Experiments
- Tal August, Katharina Reinecke
- PsychologyInternational Conference on Human Factors in…
- 2 May 2019
This work explores how formality influences engagement, measured by attention, dropout, time spent on the study and participant performance, in an online study with 369 participants on Mechanical Turk (paid) and LabintheWild (volunteer).
A Case for Design Localization: Diversity of Website Aesthetics in 44 Countries
- Manuel Nordhoff, Tal August, Nigini Oliveira, Katharina Reinecke
- Computer ScienceInternational Conference on Human Factors in…
- 21 April 2018
This work contributes the first large-scale analysis of 80,901 website designs across 44 countries, made available via an interactive web-based design catalog, showing that the design of websites with a global reach are more homogenized compared to local websites between countries.
...
...