|
Advertisement |
|
|
||
 |
|
|||||
|
|||||
|
 |
||||||
|
||||||
|
||||||
|
Journal of Clinical Oncology, Vol 26, No 22 (August 1), 2008: pp. 3721-3726 © 2008 American Society of Clinical Oncology. DOI: 10.1200/JCO.2007.14.1192 Survival End Point Reporting in Randomized Cancer Clinical Trials: A Review of Major Journals
From the Clinical Research Department, Institut Bergonié, Regional Comprehensive Cancer Center, Bordeaux; Biostatistics Unit, Centre Régional de Lutte Contre le Cancer Val d'Aurelle, Regional Comprehensive Cancer Center, Montpellier; Biostatistics and Epidemiological Unit, Regional Comprehensive Cancer Center; and Université de Bourgogne (EA 4184), Dijon, France Corresponding author: S. Mathoulin-Pélissier, MD, PhD, Clinical Research Department, Institut Bergonié, Regional Comprehensive Cancer Center, Bordeaux, France; e-mail: mathoulin{at}bergonie.org
Purpose Several publications showed that the standards for reporting randomized clinical trials (RCTs) might not be entirely suitable. Our aim was to evaluate the reporting of survival end points in cancer RCTs. Methods A search in MEDLINE databases identified 274 cancer RCTs published in 2004 in four general medical journals and four clinical oncology journals. Eligible articles were those that reported primary analyses of RCT with survival end points. Methodologists reviewed and scored the articles according to seven key points: prevalence of complete definition of survival end points (time of origin, survival events, censoring events) and relevant information about their analyses (estimation or effect size, precision, number of events, patients at risk). Concordance of key points was evaluated from a random subsample.
Results After screening, 125 articles were selected; 104 trials were phase III (83%) and 98 publications (78%) were obtained from oncology journals. Among these RCTs, a total of 267 survival end points were recorded, and overall survival (OS) was the most frequent outcome (118 terms, 44%). Survival terms were totally defined for 113 end points (42%) in 65 articles (52%). Accurate information about analysis was retrieved for 73 end points (27%) in 40 articles (32%). The less well-defined information was the number of patients at risk (55%). The reliability was good ( Conclusion A majority of articles failed to provide a complete reporting of survival end points, thus adding another source of uncontrolled variability.
For more than 20 years, several publications emphasized the adoption of guidelines to reporting randomized clinical trials (RCTs) and showed that the standards for reporting RCTs might not be entirely suitable.1,2 Recent publications about quality pointed out that RCTs failed to use adequate methods for allocation concealment or for reporting complete outcomes.3-5 In a review of 130 articles, Altman6 reported the low quality of survival analyses published in cancer journals. The publications of the CONSORT (Consolidated Standards of Reporting Trials) statement7,8 encouraged the reporting of clearly defined primary and secondary outcome measures: "All outcome measures, whether primary or secondary, should be identified and completely defined" (item 6). Overall survival (OS) or its validated surrogates are the main primary end points for technology and cancer treatment assessment. Reporting results of a time-dependent end point implies definition of the starting point, the events considered or censored, their assessment (tests performed, frequency of examination) and the duration of follow-up.9 The generic name for these time-to-event outcomes is "survival time" or "survival." Patients who have not (yet) experienced failure at the time of the analysis constitute so-called censored observations (right censoring). Until now, there have been no international consensus standards for the definition of survival end points used in clinical trials. The following terms are used most often: disease-free survival (DFS), relapse-free survival (RFS), or progression-free survival (PFS) depending on the setting. Taking into account the use of OS surrogate end points, two recent articles focusing on colon and breast cancer10,11 proposed some definitions in the adjuvant setting. The US Food and Drug Administration underlined the importance of DFS as well as response rates or time-to-progression definitions and a complete RCT report12 to approve new cancer drug applications. The purpose of this study was to evaluate the reporting of time-to-event end points in RCTs related to cancer in eight major medical journals (the term "survival end points" will be used to represent all the end points). On the basis of this review, we discuss potential improvements required for a high-quality RCT report.
Search Strategy and Studies Selection We conducted a computerized search of MEDLINE databases to identify cancer RCTs published in 2004 in four general (Lancet, British Medical Journal, Journal of the American Medical Association, and New England Journal of Medicine) and four cancer-related (Journal of Clinical Oncology, Cancer, British Journal of Cancer, and Journal of the National Cancer Institute) journals. We used the following strategy to identify these publications: journals box (journals cited before), date (01/2004-12/2004), human domain, cancer topic, and RCT as articles types. Following these criteria, a total of 274 articles were identified. Selected articles were cancer RCTs that reported primary analyses of survival end points as primary or secondary objectives.
Outcome Reporting
Data Analyses Effect size was assessed by either the hazard ratio or the difference in median survival times. Information concerning follow-up was noted but not retained as a key point because we feel there is no consensus as yet about this term and relevant information to evaluate it in RCTs.13 We first present a general description of selected publications, trial phase, cancer site, setting and terms of survival end points. Thereafter, we report the results for each key point and then for elements related to survival definitions, survival analyses, and overall.
The analysis units were each survival end point as well as each publication (ie, all end points included in one article). Clustering of end points was not taken into account in the analysis. For all analyses "unclear" and "no" responses were grouped into the same category. The degree of agreement concerning the seven key points between two reviewers (A.K. and one of three others) was determined with the
After eligibility screening, 149 articles were not included: not an RCT (n = 16), no survival end point (n = 87), not cancer as main disease (n = 39). Details for exclusion are described in Figure 1.
Among the included articles (n = 125), 104 trials were phase III (83%) and 98 (78%) were extracted from the cancer journals. The median number of patients was 348 (range, 38 to 5,758 patients) in phase III trials and 108 (range, 40 to 241 patients) in phase II trials. According to cancer sites, RCT distribution reflects incidence in the general population: 24 breast articles (19%), 20 colorectal (16%), 11 gynecologic (9%), and 11 prostate (9%). The other sites were 16 lung (13%), 16 hematology (13%), head and neck (6%), melanoma (3%), pancreas (3%), and stomach (2%). Sixty articles (48%) concerned studies in metastatic disease and 41 in the adjuvant setting (33%); 24 other contexts (19%) concern advanced cancers, neoadjuvant or palliative settings. Among the 125 articles, a total of 267 survival end points, (86 of which were primary end points) were recorded with a median number of two (range, one to five). OS was the most frequent survival term (118 terms, 44% in 94% of articles). Other used terms were 33 PFS (26%), 27 time to progression (TTP; 22%), 21 DFS (17%), 10 RFS (8%) and 12 event-free survival (EFS; 10%). Others end points were cited such as time to relapse, local-RFS, metastasis-RFS, time to treatment discontinuation, and time to recurrence. When analyzed separately on the basis of the phase of the trials being conducted, the following end points were found to be reported in phase III trials at the following frequencies (Table 1): OS (85%), EFS (100%), DFS (86%), PFS (82%), and TTP (59%). EFS and OS had a similar distribution (50%) in articles on metastatic and adjuvant settings and 89% of TTP and 70% of PFS were in metastatic settings, whereas 9% of DFS were in metastatic settings.
In the Methods section, using the three key points to define survival, we observed that the "starting point" and the "event of interest" were clearly defined for 203 (76%) and 187 end points (70%) respectively; the information on censoring was the less well defined (n = 125, 47%; Table 2). The method for "event of interest" assessment was defined for 136 end points (51%). Regarding articles, we noticed a similar distribution. Survival terms were totally defined in the methods section for 113 end points (42%) in 65 articles (52%).
In the results section, using the four key points, survival estimates were clearly identified for 248 end points (93%), precision for 163 (61%), and the number of events for 172 (65%). Less well-defined information concerned the number of patients at risk, which was described for 119 end points (45%). Complete information on survival analysis was retrieved for 73 end points (27%) in 40 articles (32%). Finally, we observed 33 survival terms but only 10 articles (8%) with all key points reported. The median number of terms among these selected articles was two (range, one to four), and seven articles had only two terms. The definitions of each end point reported in these articles are presented in Table 3.
As an exploratory analysis, we found some significant differences in seven key points according to type of journal (20% general v 10% cancer), but no significant difference according to phase (14% phase III v 5% phase II) and primary and secondary end points (16% v 11%, respectively).
To assess reliability, a total of 33 articles were randomly selected. The Information related to general methods of RCTs was provided: ITT principle (89 articles, 71%) and sample size justification (98 articles, 78%). Survival analysis was cited with Kaplan-Meier method in a majority of articles (92%, n = 115) and cumulative incidence for 5% (n = 6). The treatment effect was reported in (n = 91) 73% of articles (hazard ratio in 65 and difference of median survival time in 57 articles). The Cox model was used in 64 articles (51%), and five articles took into account competing risks. Follow-up was expressed in the results section for 71 articles (57%) by the median follow-up without specifying the method of calculation (two articles used the reverse Kaplan-Meier method). An interim analysis was noted in 34 articles (27%), and only 24% (n = 30) mentioned the statistical software used, more often in articles with multivariate analysis (33% v 13% respectively). A member of a statistics or clinical epidemiology department was not identified as a coauthor for 57 articles (46%).
Our results show that a majority of RCTs used OS as an end point, followed by PFS, TTP, and DFS. As outcomes, EFS and OS were similarly distributed in the metastatic and adjuvant setting, but PFS and TTP were more widely used in metastatic settings. More than 48% of articles published in major journals failed to provide a clear definition of the survival end point, and 68% reported insufficient information of the survival analysis. Furthermore, the number of patients at risk was reported for only 66 articles (53%). Only 33 survival end points resulting in 10 articles reported the seven key points. In 1995, these points were underlined by Altman6 and particularly for the specification of the time origin or the censoring events. They highlighted that 62% of articles reported at least one end point not clearly defined, and almost half of articles did not report a summary of the length of follow-up. Despite the time frame of publications, our study showed similar results. A trend in improvement could be underlined, but we selected only RCT, limiting the comparisons with Altman's article. Although there is an increasing interest about quality assessment of RCT reports, few data in medical subspecialties are available.10,15 To our knowledge, our research is the first to investigate survival end point reports related to cancer RCTs since the first publication on the CONSORT statement and Altman.6,8 The latter performed a systematic review of 132 articles in five clinical oncology journals reporting at least some information related to survival, and 18 articles were RCT. We have reviewed 125 selected articles that reported primary analyses of RCTs with survival end points as primary or secondary objectives. On the one hand, our selection of articles seems more discriminatory than those of Altman. On the other hand, the quality of articles has been checked according to more aspects than in our study (follow-up, explanatory variables, multivariate analyses, etc). We have retained seven key points to accurately interpret results of survival in RCT: definition of survival end point clearly reported (time of origin, survival, censoring events) and availability of information of survival analyses and results (survival estimation or effect size, precision of the survival probability, number of events, number of patients at risk). As others have suggested,6,16 follow-up was an important key point. First of all, data analysis cutoff dates and schedule of assessment17 have an impact on the probability of observing events related to the time frame. Secondly, follow-up variability could bias cross comparisons of survival results between arms, particularly when a large majority of patients are alive and free of events. We have observed that the follow-up was expressed in the results section for only 71 articles (57%) by the median follow-up without defining methods of calculation; furthermore it was nearly always reported irrespective of the treatment arm. However, information concerning the follow-up data was not considered in the key points because of lack of consensus about its definition and the relevant data needed to assess it in RCTs.13 Nevertheless, we could propose three definitions for follow-up. "Theoretical follow-up" would be defined as the time interval between the origin (random assignment, inclusion) and the cutoff date for analysis. "Observed follow-up" for survivors would be defined as the time interval between the date of origin and the date of last follow-up irrespective of the analysis cutoff date. The reverse Kaplan-Meier method defined as the time interval between the origin and last follow-up using Kaplan-Meier estimation for patients alive as the event and deaths as censored observations.16 Our overall results were discordant with a review of RCTs in Hodgkin's lymphoma (1966 through 2002) that reported recent improvements in some items of the CONSORT checklist; for instance, in this article, the frequency of reporting of primary outcomes increased from 15% to 70%.18 Although other groups have recently proposed standardized definitions,10,11,15 we could argue that, for many years, hematology international groups debated and disseminated guidelines for reporting outcomes.19 Moreover, we chose to present our review focused on seven key points that extracted more detailed survival information. Our study has other limitation. The quality of statistical reports may well differ from the quality of the evaluated trial publications. We did not include the 22 reporting items listed in the CONSORT checklist, thus limiting the comparison in reporting related survival end points and the overall quality of the trial. We observed slight differences in the type of journal with a better reporting of key points in general journals compared with cancer-specific journals. This may be a result of the fact that cancer-specific journals have been used to analyzing survival data longer and thus may take more things for granted in reporting results. Our primary intent was not to assess whether these trials were adequately powered, although one issue could be that poorly designed trials are also poorly reported. Nevertheless, we showed that ITT principles and sample size justification were largely reported. Finally, report of survival and survival definitions are another source of variability that is currently not controlled for in published articles. This point has been recently underlined for different cancer sites regarding which events should be censored and how they are taken into account, especially for TTP.10,11,20,21 However little information was provided regarding the definition of the time to event or time to censored events, which is another source of confusion. As an example, patients alive without progression could be censored at the date of last follow-up news or at the date of the last clinical follow-up visit. Although OS was better defined, the increasing use of alternative end points as surrogates should be associated with a clear and standardized definition. Furthermore, researchers need to choose the surrogate carefully by checking that these outcomes are validate surrogates for OS and by checking the survival end point definition. The events that should be taken into account for each end point and events definition have an impact on the power of the trial and could bias the reporting of significant results as well as cross comparison between studies. As outlined10 in the PETACC-3 (Pan-European Trials in Adjuvant Colon Cancer 3), the study conclusion could be associated with a significant or nonsignificant difference by including or not including second primary tumor as events in the definition of DFS. Considering survival end point definitions and their impact on the study power, we suggest that editors should systematically require this information in the published article. The journals that we have reviewed all emphasized the CONSORT statement. They claimed general considerations about reports and study conduct; for example: "For clinical trials, authors should clearly define and explain the purpose of the study, study design, numbers of patients ... time points for evaluation of response, duration of follow-up, end points used (eg, overall survival, disease-free survival), specific outcomes assessed, and methods of assessment."22 However, one could argue that limits of the article length could explain the lack of precision in the published articles. Authors have to balance which relevant information should be reported for clinical communication. Because the publication of RCTs concerned firstly clinical results, the clinical point of view was probably more supported by authors, as well as the editorial committee. To take into account this binding limitation related to article's length, we suggest that authors should stipulate in the RCT publication that survival definitions were provided in an international registration of clinical trials, considering that all trials are currently entered into such registers.23 The prespecified survival end point definitions in the protocol would preserve the integrity of results. In conclusion, the lack of standardized survival end point definitions did not allow an accurate cross comparison of results without a clear definition of end points, and could mitigate the statistical significance of the results. Consequently, readers should critically assess the reported terms used in survival analysis because the lack of these elements can lead to a misinterpretation of the overall RCT results. We believe that rational uniform terminology related to survival in the reported literature is ever the more necessary to avoid misinterpretation of results. We propose to develop standardized definitions related to survival end points of events, time to events, censored events, time to censored events, and follow-up on the basis of an international consensus among a large panel of biostatisticians and clinicians (using explicit consensus method as CONSORT statement). These definitions should be proposed according to each cancer site and according to the therapeutic setting. Harmonizing these survival end point definitions between different cancer sites can only lead to an improvement in the reporting of survival results in cancer clinical trials.
The author(s) indicated no potential conflicts of interest.
Conception and design: Simone Mathoulin-Pelissier, Andrew Kramar Financial support: Simone Mathoulin-Pelissier, Sophie Gourgou-Bourgade, Franck Bonnetain Administrative support: Simone Mathoulin-Pelissier Provision of study materials or patients: Simone Mathoulin-Pelissier, Sophie Gourgou-Bourgade Collection and assembly of data: Simone Mathoulin-Pelissier, Sophie Gourgou-Bourgade, Franck Bonnetain Data analysis and interpretation: Simone Mathoulin-Pelissier, Sophie Gourgou-Bourgade, Franck Bonnetain, Andrew Kramar Manuscript writing: Simone Mathoulin-Pelissier, Sophie Gourgou-Bourgade, Franck Bonnetain Final approval of manuscript: Simone Mathoulin-Pelissier, Sophie Gourgou-Bourgade, Franck Bonnetain, Andrew Kramar
We thank Nadine Jaulin (Institut Bergonié, Regional Comprehensive Cancer Center Bordeaux, France) for helpful research in Medline. A list the 125 selected articles is available on request from S.M.P.
Presented in part at the 43rd Annual Meeting of the American Society of Clinical Oncology, June 1-5, 2007, Chicago, IL. Authors’ disclosures of potential conflicts of interest and author contributions are found at the end of this article.
1. Zelen M: Guidelines for publishing papers on cancer clinical trials: Responsibilities of editors and authors. J Clin Oncol 1:164-169, 1983[Abstract] 2. DerSimonian R, Charette LJ, McPeek B, et al: Reporting on methods in clinical trials. N Engl J Med 306:1332-1337, 1982[Abstract] 3. Chan AW, Altman DG: Identifying outcome reporting bias in randomised trials on PubMed: Review of publications and survey of authors. BMJ 330:753, 2005 4. Altman DG: Endorsement of the CONSORT statement by high impact medical journals: Survey of instructions for authors. BMJ 330:1056-1057, 2005 5. Hewitt C, Hahn S, Torgerson DJ, et al: Adequacy and reporting of allocation concealment: Review of recent trials published in four general medical journals. BMJ 330:1057-1058, 2005 6. Altman DG, De Stavola BL, Love SB, et al: Review of survival analyses published in cancer journals. Br J Cancer 72:511-518, 1995[Medline] 7. Moher D, Jones A, Lepage L: Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA 285:1992-1995, 2001 8. Begg C, Cho M, Eastwood S, et al: Improving the quality of reporting of randomized controlled trials: The CONSORT statement. JAMA 276:637-639, 1996 9. Clark TG, Bradburn MJ, Love SB, et al: Survival analysis part I: Basic concepts and first analyses. Br J Cancer 89:232-238, 2003[CrossRef][Medline] 10. Punt CJ, Buyse M, Kohne CH, et al: Endpoints in adjuvant treatment trials: A systematic review of the literature in colon cancer and proposed definitions for future trials. J Natl Cancer Inst 99:998-1003, 2007 11. Hudis CA, Barlow WE, Costantino JP, et al: Proposal for standardized definitions for efficacy end points in adjuvant breast cancer trials: The STEEP system. J Clin Oncol 25:2127-2132, 2007 12. Johnson JR, Williams G, Pazdur R: End points and United States Food and Drug Administration approval of oncology drugs. J Clin Oncol 21:1404-1411, 2003 13. Clark TG, Altman DG, De Stavola BL: Quantification of the completeness of follow-up. Lancet 359:1309-1310, 2002[CrossRef][Medline] 14. Cohen J: Statistical Power Analysis for the Behavioral Sciences. New York, NY, Academic Press, 1977 15. Glynne-Jones R, Mawdsley S, Pearce T, et al: Alternative clinical end points in rectal cancer: Are we getting closer? Ann Oncol 17:1239-1248, 2006 16. Shuster JJ: Median follow-up in clinical trials. J Clin Oncol 9:191-192, 1991 17. Panageas KS, Ben-Porat L, Dickler MN, et al: When you look matters: The effect of assessment schedule on progression-free survival. J Natl Cancer Inst 99:428-432, 2007 18. Kober T, Trelle S, Engert A: Reporting of randomized controlled trials in Hodgkin lymphoma in biomedical journals. J Natl Cancer Inst 98:620-625, 2006 19. Dixon DO, McLaughlin P, Hagemeister FB, et al: Reporting outcomes in Hodgkin's disease and lymphoma. J Clin Oncol 5:1670-1672, 1987 20. Chua YJ, Sargent D, Cunningham D: Definition of disease-free survival: This is my truth-show me yours. Ann Oncol 16:1719-1721, 2005 21. Cookson MS, Aus G, Burnett AL, et al: Variation in the definition of biochemical recurrence in patients treated for localized prostate cancer: The American Urological Association Prostate Guidelines for Localized Prostate Cancer Update Panel report and recommendations for a standard in the reporting of surgical outcomes. J Urol 177:540-545, 2007[CrossRef][Medline] 22. Journal of the National Cancer Institute: Instructions to Authors. http://www.oxfordjournals.org/our_journals/jnci/for_authors/index.html 23. De Angelis C, Drazen JM, Frizelle FA, et al: Clinical trial registration: A statement from the International Committee of Medical Journal Editors. N Engl J Med 351:1250-1251, 2004 Submitted August 22, 2007; accepted April 11, 2008.
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||
|
|||||||||||
|
|
Copyright © 2008 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|
TOP
ABSTRACT
INTRODUCTION
= 0.72). Finally, according to the key points, optimal reporting was found in 33 end points (12%) or 10 publications. 
2 or Fisher's exact test (P < .05 was considerer significant). We also present data related to the methods of RCT conduct and to survival without regards to their relevant use (methods, analyses, software) and the implication of the statistician as an author. Analyses were performed using Stata software (version 9.1; StataCorp, College Station, TX). 
