A systematic assessment of rigor in information systems ranking-type Delphi studies

@article{Par2013ASA,
  title={A systematic assessment of rigor in information systems ranking-type Delphi studies},
  author={Guy Par{\'e} and Ann Frances Cameron and Placide Poba-Nzaou and Mathieu Templier},
  journal={Inf. Manag.},
  year={2013},
  volume={50},
  pages={207-217}
}
A B S T R A C T The ranking-type Delphi method is well suited as a means for consensus-building by using a series of questionnaires to collect data from a panel of geographically dispersed participants. This method allows a group of experts to systematically approach a particular task or problem. While information systems researchers have been using this method for almost three decades, no research to date has attempted to assess the extent to which Delphi studies have been rigorously conducted… 
Design and evaluation of ranking-type Delphi studies using best-worst-scaling
  • A. Strasser
  • Computer Science
    Technol. Anal. Strateg. Manag.
  • 2019
TLDR
The approach enables to design a ranking with up to 22 items in a manner that participants can reasonably rank, avoids and overcomes shortcomings of common ranking approaches, and offers a statistical procedure for an unambiguous evaluation.
The Delphi Method in Information Systems Research (2004‑2017)
TLDR
It is concluded that IS researchers have applied the Delphi method’s most important phases and the procedural recommendations to promote rigor were considered in the majority of the analised studies.
Contextualizing the twin concepts of systematicity and transparency in information systems literature reviews
TLDR
It is argued that a systematic approach, accompanied by transparent reporting, is essential for positivist as well as interpretivist reviews, regardless of their specific type, scope and methods.
Reflections on the application of the Delphi method: lessons from a case in public transport research
  • Fabio Hirschhorn
  • Computer Science
    International Journal of Social Research Methodology
  • 2018
TLDR
The article contributes to the methodological debate by reflecting on the introduction of novel practices that can help overcome some typical pitfalls of the Delphi: a dedicated blog supporting the survey, safety-net questions, and a constant-sum type question.
Delphi Method Variants in Information Systems Research: Taxonomy Development and Application
TLDR
The study identifies different variants of Delphi and determines their characteristics, critically reflects to what extent a clear distinction between these variants exists, develops a proposed taxonomy ofDelphi Method variants, and develops and applies this taxonomy.
Exploring the application of the Delphi method as a forecasting tool in Information Systems and Technologies research
TLDR
A guideline for applying the Delphi method in IS/IT studies is proposed based on a literature review of high impactIS/IT papers that apply Delphi with a view to answering forecasting questions and defines the most relevant aspects which should be considered when applying this method.
A Framework for Guiding and Evaluating Literature Reviews
TLDR
A set of guidelines for guiding and evaluating IS literature reviews and specify to which review types they apply are presented and grouped according to the generic phases and steps of the review process.
Paving the Way for Real-Time Delphi in Information Systems Research: A Synthesis of Survey Instrument Designs and Feedback Mechanisms
TLDR
This research reviews extant literature on Real-time Delphi and provides a synthesis of employed survey instrument designs and feedback mechanisms that contributes to the methodological knowledge base of information systems research by serving as an implementation reference and foundation on which researchers can build to establish Real- time Delphi as a novel and promising addition to the information systemsResearch methods repertoire.
Critical challenges associated with the adoption of social media: A Delphi of a panel of Canadian human resources managers
Although still young, social media platforms (SM) have already attracted over a quarter of the world's population rendering SM very attractive for organizations. SM adoption presents challenges that
Taxonomy Research in Information Systems: a Systematic Assessment
TLDR
It is revealed that the development process often remained opaque and that taxonomies were hardly evaluated, and this study proposes stimulating questions for future research and contributes to the IS community’s progress towards methodologically well-founded taxonomicies.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 71 REFERENCES
MANAGING DELPHI SURVEYS USING NONPARAMETRIC STATISTICAL TECHNIQUES
TLDR
This paper presents a method, based on nonparametric statistical techniques, to conduct ranking-type Delphi surveys, perform analysis, and report results and illustrates the use of the method in a manner affording comparison between it and previous practice.
The Delphi method as a research tool: an example, design considerations and applications
TLDR
Detailed principles for making design choices during the process of selecting appropriate experts for the Delphi study are given and suggestions for theoretical applications are made.
Identifying Software Project Risks: An International Delphi Study
TLDR
A rigorous data collection method called a "ranking-type" Delphi survey is deployed to produce a rank-order list of risk factors, which is compared with other published risk factor lists for completeness and variation.
Building Consensus Using the Policy Delphi Method
TLDR
The interquartile deviation is presented as one way of measuring consensus, and the McNemar test is described as a way to quantify the degree of shift in responses from the first to second stage.
Validation in Information Systems Research: A State-of-the-Art Assessment
TLDR
If and how the field has advanced in instrument validation is determined, and approaches are suggested for reinvigorating the quest for validation in IS research via content/construct validity, reliability, and manipulation validity.
Rigor in Information Systems Positivist Case Research: Current Practices
TLDR
The level of methodological rigor in positivist IS case research conducted over the past decade has experienced modest progress with respect to some specific attributes but the overall assessed rigor is somewhat equivocal and there are still significant areas for improvement.
Theory and applications of the Delphi technique: A bibliography (1975–1994)
Abstract Delphi is a popular, long-range, qualitative forecasting technique that has been extensively applied to a wide variety of problems in different domains. Since the method was conceived in the
The delphi technique: an exposition and application
The article provides a n overview of the Delphi technique and its related adaptations, as well as practical hints on procedures gained from its use to forecast changes in the market analysis
The decision Delphi
If Delphi is used as a prognostic tool in the social sciences, it runs the danger of producing a self-fulfilling prophecy. In a Delphi application conducted at the Austrian Academy of Sciences about
Minimizing Non-Response in The Delphi Process: How to Respond to Non-Response
TLDR
Possible options to achieve and maintain a desirable response rate when engaged in a Delphi research project are discussed and the importance of seeking help from well recognized experts or endorsed individuals is focused on.
...
1
2
3
4
5
...