What Evidence We Would Miss If We Do Not Use Grey Literature?

  title={What Evidence We Would Miss If We Do Not Use Grey Literature?},
  author={Fernando Kenji Kamei and Gustavo Henrique Lima Pinto and Igor Scaliante Wiese and M{\'a}rcio Ribeiro and S{\'e}rgio Castelo Branco Soares},
  journal={Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)},
  • F. Kamei, G. Pinto, S. Soares
  • Published 13 July 2021
  • Psychology
  • Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)
Context: Multivocal Literature Reviews (MLR) search for evidence in both Traditional Literature (TL) and Grey Literature (GL). Despite the growing interest in MLR-based studies, the literature assessing how GL has contributed to MLR studies is still scarce. Objective: This research aims to assess how the use of GL contributed to MLR studies. By contributing, we mean, understanding to what extent GL is providing evidence that is indeed used by an MLR to answer its research question. Method: We… 

Figures and Tables from this paper

Generated abstracts: evaluating automatic text summarization for blog posts in gray literature studies
An analysis of the use of automatic summarization to support the inclusion assessment in gray literature studies discussing when this solution is viable and indicates that some inclusion criteria, which might be positively determined by few sentences, is susceptible to be misclassified when using summaries.
When is Good Good Enough? Context Factors for Good Remote Work of Agile Software Development Teams. The Otto Case
The results show, that the agile software development team members to their health are affected by the switch to remote work as agile methods setting communication and collaboration in focus, and most the team members value the gamification approach to put more focus on physical activities and the health well-being.


Multivocal literature reviews in software engineering: Preliminary findings from a tertiary study
The motivation of the researchers to include GL in review studies is investigated, and how GL was included in the studies is examined, to provide preliminary work about the current research involving MLR studies.
On Using Grey Literature and Google Scholar in Systematic Literature Reviews in Software Engineering
A systematic mapping study is conducted to measure the extent of GL usage in SE SLRs as well as to measureThe feasibility of finding primary studies using Google Scholar and to explore if it is feasible to use only Google Scholar for finding scholarly articles for academic research.
The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature
It is found that grey literature can give substantial benefits in certain areas of SE, and that the inclusion of grey literature brings forward certain challenges as evidence in them is often experience and opinion based.
An Evidence-Based Inquiry into the Use of Grey Literature in Software Engineering
A conceptual model for how GL works in SE research lifecycle is proposed and can provide a panorama of the state-of-the-art of using GL in SE for the follow-up research, as to determine the important position of GL inSE research.
Gray literature: An important resource in systematic reviews
  • Arsenio Paez
  • Psychology
    Journal of evidence-based medicine
  • 2017
Gray literature, or evidence not published in commercial publications, can make important contributions to a systematic review, and reduce publication bias, increase reviews’ comprehensiveness and timeliness, and foster a balanced picture of available evidence.
The impact of limited search procedures for systematic literature reviews — A participant-observer case study
Researchers undertaking SLRs may be justified in using targeted manual searches if they intend to omit low quality papers; if publication bias is not an issue; or if they are assessing research trends in research methodologies.
On the Use of Grey Literature: A Survey with the Brazilian Software Engineering Research Community
The findings contribute to form a body of knowledge on the use of GL by SE researchers, by discussing novel (some contradictory) results and providing a set of lessons learned to both SE researchers and practitioners.
Using reasoning markers to select the more rigorous software practitioners' online content when searching for grey literature
A set of discourse markers that can be used to indicate the presence of reasoning in a blog article, and therefore provides an indication of the more rigorous blog article content is provided.
Systematic Reviews in the Social Sciences: A Practical Guide
Such diverse thinkers as Lao-Tze, Confucius, and U.S. Defense Secretary Donald Rumsfeld have all pointed out that we need to be able to tell the difference between real and assumed knowledge. The
Is grey literature really grey or a hidden glory to showcase the sleeping beauty
Purpose The study aims to showcase the developmental perspective of “grey literature” and its importance to different sectors of the society. Furthermore, issues, challenges and possibilities