Kris McGill of the systematic review team explains what the Delphi Method is and how it is useful to researchers. Posted by Kris McGill on Thursday 29th July 2021 What is the Delphi Method? A Delphi is a consensus based methodology developed during the Cold War to forecast the impact of technology on warfare. The method works with the principle that the best consensus can be reached by using a structured group of experts. The method asks experts to rate a range of items (e.g. potential bombing targets during war) in terms of their priority/importance. All ratings are then combined, the lowest scoring items are removed, and the experts are provided feedback. This process repeats, reducing the number of items over multiple rounds, until consensus is reached. The original Delphi concluded that the steel industry was the most vulnerable during the Cold War. However, the method is now frequently used to determine: what interventions to use for specific disorders, what outcome measures to use for research, future research priorities, and many other purposes. How do you conduct an Delphi? The Delphi uses three or four distinct steps: Define the problem Defining the problem is often the easiest step because typically you know what question you want answered before you select what method to use. Some example questions that are often covered using a Delphi are mentioned above. Identify the experts Depending on the area, you typically want experienced researchers, experienced clinicians, and people with lived experience. There is no rule for how many people to include in the Delphi but you need to bear in mind that you will have dropout. In my experience you will lose about 10% of people between rounds. Repeated rounds of controlled feedback Before launching the Delphi, you need to decide what items are going to be rated. You can either extract them from literature or ask your experts to give you a long list of what to include. Once you have all your items you present them to your experts via an online questionnaire and ask them to rate their importance. You should start with as many items and possible and allow the multiple rounds to narrow them down. You need to use a rating scale for the questionnaire (often referred to as a Likert Scale). After your experts have rated each item you tally up the results and determine what makes the cut for the next round. There is no agreed way to do this for a Delphi but I use a 70% or greater positive rating to qualify for the next round. After you have tallied the results you provide a report of how all items have performed to you experts. You then repeat this process until you have a consensus or you have a list small enough to consider in a consensus meeting. If you decide to have the consensus meeting your Delphi becomes a modified Delphi. An optional final consensus meeting If you decide to have a final consensus meeting, you invite your experts to attend, discuss your final list of items, and come to an agreed consensus. You can use formal voting techniques during your meeting (such as Nominal Group Technique) or have a less structured discussion to reach your final consensus. How did we use the modified Delphi method at ACRC? At the Advanced Care Research Centre (ACRC) we used a one-round modified Delphi to decide what systematic review questions that we will take on over the next year. We started by asked everyone at ACRC to suggest any review questions that they think would be helpful. We then had a one round modified Delphi where we asked all ACRC members to rank each question’s importance. We used a final consensus meeting to consider the top eight reviews (all scored above 70% positive). We adopted a simple voting system at the meeting to prioritise the review questions and this gave us our top four. The final review questions are: What are the facilitators and barriers to implementing new technologies to support paid and unpaid care for older adults? Which digital technologies are being used to enable older people to remain in their living environments and delay or avoid increases in need for social care support? What do we know about the networks older adults receiving care are embedded within and the forms of value across these? In older people, in high-income countries, which community-based health and social care organisational interventions are most effective at improving or maintaining quality of life and independent living? See more from the ACRC blog Publication date 22 Jul, 2021
