The Delphi Technique
- Pages: 8
- Word count: 1758
- Category: College Example Technology
A limited time offer! Get a custom sample essay written according to your requirements urgent 3h delivery guaranteed
Order Now
The Delphi Method was developed at the RAND corporation in the early 1950s as a spin-off of an Air Force-sponsored research project, “Project Delphi.” Since that time it has been refined further and applied to gain information in a wide range of fields. These fields are as diverse as regional economic development, health care policy, sociology, environmental risks, prediction of fruit prices, tourism and recreation, forestry and advanced manufacturing techniques. The Delphi technique may be particularly useful in situations where strictly objective data are scarce. The original project, however, was designed to anticipate an optimal targeting of U.S. industries by a hypothetical Soviet strategic planner. Delphi was first brought before a wider audience in a 1963 RAND study “Report on a Long-Range Forecasting Study,” by Olaf Helmer and T. J. Gordon (Linstone and Turoff, 2002). It is undoubtedly the best-known method of eliciting and synthesizing expert opinion.
In the middle 1960s and early 1970s the Delphi method found a wide variety of applications, and by 1974 the number of Delphi studies had exceeded 10,000 (Arkansas Administrative Office of the Courts, 2002). Although most applications are concerned with technology forecasting, the method has also been applied to many types of policy analysis.
Policy Delphi’s differ from technology forecasting Delphi’s with respect to both purpose and method. In technology forecasting, the team conducting the Delphi study seeks experts who are most knowledgable on the issues in question, and seeks to achieve a high degree of consensus regarding predicted developments. Policy Delphi’s on the other hand seek to incorporate the views of the entire spectrum of “stakeholders” and seek to communicate the spread of their opinions to the decision maker. We shall be chiefly concerned with the forecasting type of Delphi’s in this brief discussion.
The Delphi method has undergone substantial evolution and diversification, but the basic approach may be described as follows. A monitoring team defines a set of issues and selects a set of respondents who are experts on the issues in question. A respondent generally does not know who the other respondents are, and the responses are anonymous. A preliminary questionnaire is sent to the respondents for comments, which are then used to establish a definitive questionnaire (Sharp, 2006). Typical questions take the form “In what year will suchand-such take place?”
“This questionnaire is then sent to the respondents and their answers are analyzed by the monitoring team. The set of responses is then sent back to the respondents, together with the median answer and the interquartile range, the range containing all but the lower 25% and the upper 25% of the responses. The respondents are asked if they wish to revise the initial predictions. Those whose answers remain outside the interquartile range for a given item are asked to give arguments for their prediction on this item” (Winston, 1999).
The revised predictions are then processed in the same way as the first responses, and arguments for outliers are summarized. This information is then sent back to the respondents, and the whole process is iterated. A Delphi exercise typically involves three or four rounds.
The responses on the final round generally show a smaller spread than the responses on the first round, and this is taken to indicate that the experts have reached a degree of consensus. The median values on the final round are taken as the best predictions (Ali, 2005).
As was mentioned before the Delphi method has undergone many variations. One of the most important variations involved letting the experts indicate their own expertise for each question. Only the opinions of the experts claiming the most expertise for a given item are used to determine the distribution of opinion for that item. This method is claimed to improve accuracy. It has been found that women consistently rate themselves lower than men (Linstone and Turoff, 2002).
The claim of improved accuracy resulting from self-ratings was challenged in an extensive study by Brockhoff (1975). It was found that self-ratings of participants did not coincide with “objective expertise” as measured by relative deviation from the true value on fact-finding and forecasting tasks. Brockhoff also compared the results of Delphi groups with groups using “face-to-face” confrontation, but could not draw firm conclusions, owing partly to a large number of dropouts (Brockhoff, 1975).
The Delphi method was developed by mathematicians and engineers, and enjoyed considerable popularity among research managers, policy analysts, andcorporate planners in the late 1960s and early 1970s. By the middle 1970s psychometricians, people trained in conducting controlled experiments with humans, began taking a serious look at the Delphi methods and results (Ali, 2005).
Perhaps the most significant study in this regard is Sackman Delphi Critique (1975). Sackman approaches the Delphi exercises as psychometric experiments (which, strictly speaking, they are not) and concludes that the Delphi method violates several essential methodological rules of sound experimental science. For one thing, he notes that the questionnaire items are often so vague that it would be impossible to determine when, if ever, they occurred. Furthermore, the respondents are not treated equally. People whose predictions fall inside the interquartile band are “rewarded” with a reduced workload in returning the questionnaires, whereas those whose predictions fall outside this band are “punished” and must produce arguments. Moreover, in many Delphi exercises there seem to be a significant number of dropouts–people who simply don’t return their forms. Delphi exercises do not publish the number of dropouts, they make no attempt to discover the reason for dropping out, and do not assess the influence of this negative selection on the results. This all raises the possibility that the Delphi convergence may owe more to boredom than to consensus. Finally, Sackman argues that experts and nonexperts generally produce comparable results in Delphi exercises (Sackman, 1975).
Evidently, the Delphi method is designed to elicit estimates from experts within a group or panel without allowing interaction between individuals on the panel, thus avoiding problems with dominant members. Experts do, however, have the ability to revise their estimates on the basis of group views. Such an option is not available using the traditional survey method (Ali, 2005).
This technique proceeds through a series of data collection rounds. In a classic Delphi survey, the first round is unstructured, allowing panellists to identify freely and elaborate on the issues that they consider important. These are consolidated into a single set by the monitors, who then produce a structured questionnaire designed to elicit the views, opinions and judgements of the panellists in a quantitative form. The consolidated list of scenarios is presented to the panellists in the second round, at which time they place estimates on key variables, such as the time an event will occur. These responses are then summarized and the summary information is presented to the panellists, who are invited to reassess their original opinions in the light of anonymous individual responses. In addition, if panellists’ assessments fall outside the upper or lower quartiles, they may be asked to provide justification of why they consider their estimates are more accurate than the median values. Further rounds of collection of estimates, compiling summary information and inviting revisions continue until there is no further convergence of expert opinion. Experience reveals this usually occurs after two rounds, or at the most four rounds (Sharp, 2006).There are a number of variants on the classic Delphi method.
When the issues are well defined, a clearly specified scenario can be developed by the monitoring team. In such circumstances, it is common to replace the unstructured first round with a highly structured set of questions through which specific estimates of parameters are obtained. A statistical summary of all responses is then provided to the panel for the second round, rather than the third. In such cases, it is common for the Delphi method to include only one or two iterations. The classic Delphi method is conducted through a combination of a polling procedure and a conference. However, communication between conference panellists is restricted and undertaken through the monitoring team. Even though panellists are at the same physical location, there is no face-to-face contact. A variant is the ‘paper’ Delphi (sometimes also known as a ‘paper-and-pencil Delphi poll’) that is conducted entirely by mail. Another variant is the ‘real time’ Delphi in which feedback is provided by computer and final results are usually available at the end of the session. The quality of forecasts provided by the Delphi method (and other forecasting techniques) very much depends on how the technique is applied (Parente and Anderson-Parente, 1987).
In summary, Delphi Method is a technique for determining the likelihood of future events based upon past experience. The Delphi method assembles a panel of experts from different disciplines to comment upon the research of others in their own and different fields. It is typically used to arrive at high-level predictions, especially in relation to economics and politics. The aim is to account for the complex factors that affect long-range forecasting, or situations in which unknowns might play a major part, by generating a wide range of possible future scenarios. The method also claims to safeguard against the tendency of group discussions on these kinds of matters to arrive at a consensus. With the Delphi method experts respond to questionnaires at a distance.
References
Ali, A. K. (2005). Using the Delphi technique to search for empirical measures of local planning agency power. The Qualitative Report, 10(4), 718-744. Retrieved 6 March 2006 from http://www.nova.edu/ssss/QR/QR10-4/ali.pdf
Benchmarks & Bar Charts: Arkansas Court Statistics Research. (2002, Fall). Arkansas Administrative Office of the Courts, 2 (1). Retrieved 7 March 2006 from courts.state.ar.us/benchmarks/newsletterv2n1fall2002.pdf
Brockhoff, K. (1975). The Performance of Forecasting Groups in Computer Dialogue and Face to Face Discussion. In H. A. Linstone and M. Turoff (eds.), The Delphi Method, Techniques and Applications, Addison Wesley, Reading, Mass., pp. 291-321.
Linstone, Harold A. and Turoff, Murray. (2002). The Delphi Method: Techniques and applications. Retrieved 6 March 2006 from http://www.is.njit.edu/pubs/delphibook/ch4a.html
Parente, F. J., and Anderson-Parente, J. K. (1987). Delphi Inquiry Systems. In G. Wright and P. Ayton (eds.), Judgmental Forecasting. Wiley, Chichester.
Sharp, Arthur G. (2006). Delphi Technique. Encyclopedia of Business, 2nd ed. Retrieved 7 March 2006 from http://www.referenceforbusiness.com/encyclopedia/Cos-Des/Delphi-Technique.html
Sackman, H. (1975). Delphi Critique, Expert Opinion, Forecasting and Group Processes. Lexington Books, Lexington, Mass. p. 73.
Winston, Wayne. (1999). Using analytical hierarchy process to select a job. Financial Models Using Simulation and Optimization. Retrieved 6 March 2006 from http://www.seas.gwu.edu/~dorpjr/EMSE388/Session%209/Session%209.pdf.