Methods for conducting international Delphi surveys to optimise global participation in core outcome set development: a case study in gastric cancer informed by a comprehensive literature review

Core outcome sets (COS) should be relevant to key stakeholders and widely applicable and usable. Ideally, they are developed for international use to allow optimal data synthesis from trials. Electronic Delphi surveys are commonly used to facilitate global participation; however, this has limitations. It is common for these surveys to be conducted in a single language potentially excluding those not fluent in that tongue. The aim of this study is to summarise current approaches for optimising international participation in Delphi studies and make recommendations for future practice.

A comprehensive literature review of current approaches to translating Delphi surveys for COS development was undertaken. A standardised methodology adapted from international guidance derived from 12 major sets of translation guidelines in the field of outcome reporting was developed. As a case study, this was applied to a COS project for surgical trials in gastric cancer to translate a Delphi survey into 7 target languages from regions active in gastric cancer research.

Three hundred thirty-two abstracts were screened and four studies addressing COS development in rheumatoid and osteoarthritis, vascular malformations and polypharmacy were eligible for inclusion. There was wide variation in methodological approaches to translation, including the number of forward translations, the inclusion of back translation, the employment of cognitive debriefing and how discrepancies and disagreements were handled. Important considerations were identified during the development of the gastric cancer survey including establishing translation groups, timelines, understanding financial implications, strategies to maximise recruitment and regulatory approvals. The methodological approach to translating the Delphi surveys was easily reproducible by local collaborators and resulted in an additional 637 participants to the 315 recruited to complete the source language survey. Ninety-nine per cent of patients and 97% of healthcare professionals from non-English-speaking regions used translated surveys.

Consideration of the issues described will improve planning by other COS developers and can be used to widen international participation from both patients and healthcare professionals.


Bilal Alkhaffaf, Jane M. Blazeby, Aleksandra Metryka, Anne-Marie Glenny, Ademola Adeyeye, Paulo Matos Costa, Ismael Diez del Val, Suzanne S. Gisbertz, Ali Guner, Simon Law, Hyuk-Joon Lee, Ziyu Li, Koji Nakada, Rafael Mauricio Restrepo Nuñez, Daniel Reim, John V. Reynolds, Peter Vorwald, Daniela Zanotti, William Allum, M. Asif Chaudry, Ewen Griffiths, Paula R. Williamson & Iain A. Bruce on behalf of the GASTROS International Working Group


Journal: Trials
Volume: 22
Pages: -
Year: 2021
DOI: 10.1186/s13063-021-05338-x

Further Study Information

Current Stage: Completed
Funding source(s):

Health Area

Disease Category: Cancer, Gastroenterology

Disease Name: Gastric Cancer

Target Population

Age Range: 18 - 100

Sex: Either

Nature of Intervention: Surgery

Stakeholders Involved

Study Type

- COS methods research


- Literature review