What Evidence We Would Miss If We Do Not Use Grey Literature?

  title={What Evidence We Would Miss If We Do Not Use Grey Literature?},
  author={Fernando Kenji Kamei and Gustavo Henrique Lima Pinto and Igor Scaliante Wiese and M{\'a}rcio Ribeiro and S{\'e}rgio Castelo Branco Soares},
  journal={Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)},
  • F. Kamei, G. Pinto, S. Soares
  • Published 13 July 2021
  • Psychology
  • Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)
Context: Multivocal Literature Reviews (MLR) search for evidence in both Traditional Literature (TL) and Grey Literature (GL). Despite the growing interest in MLR-based studies, the literature assessing how GL has contributed to MLR studies is still scarce. Objective: This research aims to assess how the use of GL contributed to MLR studies. By contributing, we mean, understanding to what extent GL is providing evidence that is indeed used by an MLR to answer its research question. Method: We… 

Figures and Tables from this paper

Generated abstracts: evaluating automatic text summarization for blog posts in gray literature studies

An analysis of the use of automatic summarization to support the inclusion assessment in gray literature studies discussing when this solution is viable and indicates that some inclusion criteria, which might be positively determined by few sentences, is susceptible to be misclassified when using summaries.

When is Good Good Enough? Context Factors for Good Remote Work of Agile Software Development Teams. The Otto Case

The results show, that the agile software development team members to their health are affected by the switch to remote work as agile methods setting communication and collaboration in focus, and most the team members value the gamification approach to put more focus on physical activities and the health well-being.

Characterising the digital transformation of IT consulting services–Results from a systematic mapping study

The digital transformation is multifaceted and characterised through many aspects and it is concluded that digital IT consulting services are primarily explored from business perspective and research of stakeholders' needs, technical requirements, and designs of relevant artefacts is scarce.



Multivocal literature reviews in software engineering: Preliminary findings from a tertiary study

The motivation of the researchers to include GL in review studies is investigated, and how GL was included in the studies is examined, to provide preliminary work about the current research involving MLR studies.

The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature

It is found that grey literature can give substantial benefits in certain areas of SE, and that the inclusion of grey literature brings forward certain challenges as evidence in them is often experience and opinion based.

An Evidence-Based Inquiry into the Use of Grey Literature in Software Engineering

A conceptual model for how GL works in SE research lifecycle is proposed and can provide a panorama of the state-of-the-art of using GL in SE for the follow-up research, as to determine the important position of GL inSE research.

Gray literature: An important resource in systematic reviews

  • Arsenio Paez
  • Psychology
    Journal of evidence-based medicine
  • 2017
Gray literature, or evidence not published in commercial publications, can make important contributions to a systematic review, and reduce publication bias, increase reviews’ comprehensiveness and timeliness, and foster a balanced picture of available evidence.

The impact of limited search procedures for systematic literature reviews — A participant-observer case study

Researchers undertaking SLRs may be justified in using targeted manual searches if they intend to omit low quality papers; if publication bias is not an issue; or if they are assessing research trends in research methodologies.

On the Use of Grey Literature: A Survey with the Brazilian Software Engineering Research Community

The findings contribute to form a body of knowledge on the use of GL by SE researchers, by discussing novel (some contradictory) results and providing a set of lessons learned to both SE researchers and practitioners.

Using reasoning markers to select the more rigorous software practitioners' online content when searching for grey literature

A set of discourse markers that can be used to indicate the presence of reasoning in a blog article, and therefore provides an indication of the more rigorous blog article content is provided.

Systematic Reviews in the Social Sciences: A Practical Guide

Such diverse thinkers as Lao-Tze, Confucius, and U.S. Defense Secretary Donald Rumsfeld have all pointed out that we need to be able to tell the difference between real and assumed knowledge. The

Is grey literature really grey or a hidden glory to showcase the sleeping beauty

Purpose The study aims to showcase the developmental perspective of “grey literature” and its importance to different sectors of the society. Furthermore, issues, challenges and possibilities

Toward the use of blog articles as a source of evidence for software engineering research

It is decided that blog articles need to be rigorous, relevant, well written and experience based for them to be considered credible to researchers.