A walk through core outcome sets – useful references for core outcome set developers
The COMET (Core Outcome Measures in Effectiveness Trials) Initiative brings together researchers interested in the development, application and promotion of COS, derived using rigorous consensus methods, for effectiveness trials. COMET aims to collate and stimulate the development of relevant resources, both applied and methodological, to facilitate exchange of ideas and information, to work with patients, the public and their representatives to develop material to improve health service user engagement, and to foster methodological research in the area of COS. Data on relevant individual studies, both published and ongoing, are being included in a free, publically available internet-based resource. This is a unique resource, which is updated periodically, and which should serve to minimize duplication of effort in the development of COS. A systematic review to identify studies which sought to determine which outcomes/domains to measure in all clinical trials in a specific condition has been completed. This systematic review identified many health areas where a COS has been developed, but also highlights important gaps. It is a further step towards a comprehensive, up-to-date database of COS.
In an interesting commentary, Mike Clarke asks some important questions: Why do we need such initiatives? What's the problem? And are these and other initiatives the solution? This paper provides a good overview of the problems with outcomes in trials.
Accumulating work in this area has identified the need for general guidance on the development of core outcome sets, and this is ongoing. Williamson et al suggest key issues to consider in the development of a core outcome set including its scope, the stakeholder groups to involve, choice of consensus method and the achievement of a consensus. There is also a useful review of studies using the Delphi technique to determine which outcomes to measure in trials, that also provides guidance about using this technique to determine core outcome sets.
One of the earliest examples of an attempt to standardise outcomes is an initiative by the World Health Organisation in the 1970s, relating to cancer trials. More than 30 representatives from groups doing trials in cancer came together, the results of which was a WHO handbook of guidelines recommending the minimum requirements for data collection in cancer trials. Another notable example is OMERACT (Outcome Measures in Rheumatology); an independent initiative of international health professionals interested in outcome measures in rheumatology. The first OMERACT conference on rheumatoid arthritis was held in Maastricht, the Netherlands in 1992. The motivation for this were discussions between 2 of the executive members, comparing the outcomes of rheumatoid arthritis used in European clinical trials with that of North American clinical trials, and noting that they used different endpoints. This made it extremely difficult to compare and combine in meta-analyses. Over the last 20 years, OMERACT has served a critical role in the development and validation of clinical and radiographic outcome measures in rheumatoid arthritis, osteoarthritis, psoriatic arthritis, fibromyalgia, and other rheumatic diseases. OMERACT strives to improve endpoint outcome measurement through a data driven, iterative consensus process involving relevant stakeholder groups. A powerful and important aspect of OMERACT is the integration of patients at each stage of the OMERACT process. This patient input along with clinical trialist insight, epidemiologist assessment, and industry perspective, has led OMERACT to be a unique decision making group in developing outcome measures for all types of clinical trials and observational research. In Trials, Tugwell et al describe the need for, and success of, this initiative. OMERACT have now developed COS for many rheumatologic conditions, and have described a conceptual framework that could be considered as a template in other areas of health care.
In another example, a group of researchers at the University of Liverpool were interested to know if a core outcome set already existed for trials of regular paediatric asthma, so a systematic review of studies that aimed to determine which outcomes to measure in clinical trials in children was carried out. The work of 13 collaborations was included in the review, one of which represented paediatric asthma. In this study, 14 clinicians and researchers indicated, by questionnaire, which outcomes they felt were most appropriate for a variety of clinical, public health, and research settings related to asthma; so this was by no means a consensus core outcome set. As this didn’t exist, they then asked was this because there was already consistency amongst trials undertaken to date, so an implicit core outcome set? In another systematic review, they wanted to identify which outcomes and domains were measured in randomised controlled trials (RCTs) of inhaled corticosteroids for children with asthma. They found that RCTs in children with asthma almost always assess the effects of therapies on short term disease activity, but none consider the effects on long term progression of disease. Quality of life and functional status are measured infrequently. While there were similarities between studies, particularly in the selection of primary outcomes that measure disease activity, other outcomes showed wide variability. As a result, Sinha et al set out to determine the most appropriate and important outcomes to measure in these trials.
Once a COS is agreed, it is then important to determine how the outcomes included in the set should be defined and measured. Often a variety of measurement instruments exists to measure a given outcome, with varying quality (i.e. reliability and validity). Important tools for selecting a measurement instrument for a COS are systematic reviews of measurement instruments. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative collects all systematic reviews of measurement properties of all available measurement instruments that intend to measure (aspects of) health status or (health-related) quality of life. An overview of these reviews and guidelines for performing such reviews can be found on the COSMIN website http://www.cosmin.nl/systematic-reviews-of-measurement-properties-_5_0.html.
The uptake and implementation of a core outcome set also needs to be considered. An observational review was carried out of 350 randomised trials for the treatment of rheumatoid arthritis identified through The Cochrane Library (up to and including September 2012 issue). Reports of these trials were evaluated to determine whether or not there were trends in the proportion of trials reporting on the full set of core outcomes over time. This observational review suggests that a higher percentage of trialists conducting trials in rheumatoid arthritis are now measuring the rheumatoid arthritis core outcome set. Core outcome sets have the potential to improve the evidence base for health care, but consideration must be given to the methods for disseminating their availability amongst the relevant communities.
- Clarke (2007) - http://www.trialsjournal.com/content/8/1/39
- Williamson et al (2012) - http://www.trialsjournal.com/content/13/1/132
- Review of studies using the Delphi technique (2011) - http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1000393
- WHO Handbook (1981) - http://www.ncbi.nlm.nih.gov/pubmed/7459811
- Tugwell et al (2007) - http://www.omeract.org/pdf/OMERACT%20Tugwell%20Trials.pdf
- Conceptual framework (2014) - http://www.omeract.org/pdf/JCE%20FILTER%202.0%20FINAL.pdf
- Review of studies that aim to determine which outcomes to measure in clinical trials in children (2008) - http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.0050096
- Review of outcomes in trials of inhaled corticosteroids for children with asthma (2009) - http://www.plosone.org/article/related/info:doi/10.1371/journal.pone.0006276
- Sinha et al (2012) - http://www.trialsjournal.com/content/13/1/103
An observational review of RA trials (2013) - www.trialsjournal.com/content/14/1/324