Elsevier

Information & Management

Volume 42, Issue 1, December 2004, Pages 15-29
Information & Management

The Delphi method as a research tool: an example, design considerations and applications

https://doi.org/10.1016/j.im.2003.11.002Get rights and content

Abstract

The Delphi method has proven a popular tool in information systems research for identifying and prioritizing issues for managerial decision-making. However, many past studies have not adopted a systematic approach to conduct a Delphi study. This article provides rigorous guidelines for the process of selecting appropriate experts for the study and gives detailed principles for making design choices during the process that ensure a valid study. A detailed example of a study to identify key factors affecting the diffusion of e-commerce in Sub-Saharan Africa illustrates the design choices that may be involved. We conclude with suggestions for theoretical applications.

Introduction

The Delphi method has proven a popular tool in information systems (IS) research [4], [6], [13], [14], [16], [24], [25], [35]. Citing “a lack of a definitive method for conducting the research and a lack of statistical support for the conclusions drawn,” Schmidt [34] presented a step-wise methodology for conducting such studies. Building on the framework that Schmidt developed, we offer two contributions towards increasing the value of Delphi studies in investigating research questions. First, we fill in many details in the context of Schmidt’s framework by providing guidelines on how to conduct a rigorous Delphi study that identifies the most important issues of interest by soliciting qualified experts. Second, we demonstrate how to use a Delphi survey as a research tool to serve a variety of different purposes in the theorizing process. Increasing the rigor will increase the confidence with which researchers can use the results in subsequent studies and managers can make decisions based on information gathered using these methods.

Bricolage is a French term that means “to use whatever resources and repertoire one has to perform whatever task one faces” [40]. Characterizations of the research process as bricolage and the researcher as bricoleur [10] serve to remind us of the improvisation and opportunism inherent in the research process and the need to put our research tools to multiple use. A third goal, then, is to encourage researchers to incorporate the Delphi method into their research repertoire and to suggest some of the various ways they could apply the method in the theorizing process.

Section snippets

Overview of the Delphi method

The Delphi method originated in a series of studies that the RAND Corporation conducted in the 1950s. The objective was to develop a technique to obtain the most reliable consensus of a group of experts [8]. While researchers have developed variations of the method since its introduction, Linstone and Turoff [17] captured common characteristics in this description:

Delphi may be characterized as a method for structuring a group communication process so that the process is effective in allowing a

Example research study design using the Delphi method

Schmidt presented a guideline focusing on the major phases of the process and on analysis issues. However, the example we present in this paper focuses on perhaps the most important yet most neglected aspect of the Delphi method—choosing appropriate experts. This neglect is problematic, considering that most Delphi researchers characterize the technique as a method for soliciting information from experts. We based our guidelines primarily on those initially developed by Delbecq et al. [9].

Conclusion: uses of the Delphi method for theory building

The Delphi method is a versatile research tool that researchers can employ at various points in their research. Use of the Delphi method for forecasting and issue identification/prioritization can be valuable in the early stages, particularly in selecting the topic and defining the research question(s). A major contribution of the MISRC/SIM studies of key managerial IS concerns, published in MIS Quarterly, [5], for example, was to provide IS researchers with an understanding of the most

Acknowledgements

We thank Victor Mbarika for his help in the e-commerce study that we used as an illustrative framework for this paper. We also thank Casey Cegielski for his valuable references that gave us a literature base to study the Delphi method.

Chitu Okoli is an assistant professor in the Department of Decision Sciences and Management Information Systems of the John Molson School of Business at Concordia University, Montréal, Canada. He obtained a PhD in 2003 from the Department of Information Systems and Decision Sciences at Louisiana State University, USA. He primarily researches applications of the Internet in developing countries, and strategically using the Internet for competitive advantage. In addition to the Delphi method,

References (41)

  • C.G. Cegielski, A Model of the Factors that Affect the Integration of Emerging Information Technology Into Corporate...
  • M.R. Czinkota et al.

    International business and trade in the next decade: report from a Delphi study

    Journal of International Business Studies

    (1997)
  • N. Dalkey et al.

    An experimental application of the Delphi method to the use of experts

    Management Science

    (1963)
  • A.L. Delbecq, A.H. Van de Ven, D.H. Gustafson, Group Techniques for Program Planning: A Guide to Nominal Group and...
  • N. Denzin, Y. Lincoln, Entering the field of qualitative research, in: N. Denzin, Y. Lincoln (Eds.), The Landscape of...
  • D.A. Dillman, Mail and Internet Surveys: The Tailored Design Method, Wiley, New York,...
  • A. Dutta

    The physical infrastructure for electronic commerce in developing nations: historical trends and the impact of privatization

    International Journal of Electronic Commerce

    (1997)
  • P. Holsapple et al.

    Knowledge manipulation activities: results of a Delphi study

    Information & Management

    (2002)
  • J.E. Kendall et al.

    SEER: a divergent methodology applied to forecasting the future roles of the systems analyst

    Human Systems Management

    (1992)
  • V. Lai et al.

    Managing international data communications

    Information & Management

    (2002)
  • Cited by (2894)

    View all citing articles on Scopus

    Chitu Okoli is an assistant professor in the Department of Decision Sciences and Management Information Systems of the John Molson School of Business at Concordia University, Montréal, Canada. He obtained a PhD in 2003 from the Department of Information Systems and Decision Sciences at Louisiana State University, USA. He primarily researches applications of the Internet in developing countries, and strategically using the Internet for competitive advantage. In addition to the Delphi method, Chitu Okoli also uses quantitative surveys and stochastic simulation.

    Suzanne Pawlowski is an assistant professor in the Department of Information Systems and Decision Sciences at Louisiana State University. She holds a PhD degree in Computer Information Systems from Georgia State University and MBA and BA degrees from the University of California, Berkeley. Research interests include enterprise systems and organizational learning, retention of IT professionals, and collaborative visualization technologies. Scholarly publications include papers in ACM Computing Surveys, Communications of the ACM, IEEE Transactions on Software Engineering, and Communications of AIS. Her previous work experience includes a 20-year career in IT at Lawrence Livermore National Laboratory, including manager of application development.

    View full text