|
Advertisement |
|
|
||
 |
|
|||||
|
|||||
|
 |
||||||
|
||||||
|
||||||
|
Journal of Clinical Oncology, Vol 26, No 13 (May 1), 2008: pp. 2205-2211 © 2008 American Society of Clinical Oncology. DOI: 10.1200/JCO.2007.14.6795 Consistency of Phase III Clinical Trial Abstracts Presented at an Annual Meeting of the American Society of Clinical Oncology Compared With Their Subsequent Full-Text Publications
From the Department of Medicine, McMaster University; and the Department of Oncology, Juravinski Cancer Centre, Hamilton, Ontario, Canada Corresponding author: Sebastien J. Hotte, MD, Juravinski Cancer Centre, 699 Concession St, Hamilton, Ontario, Canada, L8V 5C2; e-mail: sebastien.hotte{at}hrcc.on.ca
Purpose This study aimed to determine the consistency of phase III clinical trial abstracts presented at American Society of Clinical Oncology (ASCO) Annual Meetings compared with their subsequent full-text publications. Methods We identified abstracts describing phase III clinical trials of chemotherapy, chemoradiotherapy, immunotherapy, and hormone therapy presented at the 36th ASCO Annual Meeting in May 2000. We searched MEDLINE and PubMed for all corresponding publications. Data were extracted from the abstracts and publications that met our inclusion criteria. Results A total of 192 abstracts were identified. Seventy-four abstracts met our inclusion criteria. Six years after the 2000 ASCO Meeting, 74% of abstracts had corresponding publications. The primary end point was stated in 34% of abstracts and 100% of published papers. The primary end point result differed by more than 5% between the abstract and publication in 42% of comparisons. The statistical significance of the primary end point and study conclusions were consistent between abstracts and subsequent publications in 89% and 91% of the comparisons, respectively. Abstracts selected as plenary or oral presentations were significantly more likely to be published. No factors predicted consistency for primary end point significance and overall conclusion between ASCO abstracts and their journal publications. Conclusion When carefully selected, ASCO Annual Meeting abstracts of phase III trials consistently reflect final published results, but some differences were observed that warrant caution in using abstract results to shape treatment decisions before full publication.
Phase III clinical trials significantly impact patient care because they provide the highest level of evidence in treatment decisions. The results of these trials are frequently first presented at medical conferences, such as the Annual Meeting of the American Society of Clinical Oncology (ASCO), one of the largest meetings of oncology professionals in the world. In addition to a review of the literature from medical journals, authors of some oncology practice guidelines and evidence summaries incorporate evidence from conference abstracts.1,2 Because there is often insufficient information provided in the meeting abstracts to determine study validity and less rigorous peer reviewing of abstracts in meeting proceedings compared with journal publications, the methodologic quality of the study and evidence provided by these abstracts is uncertain.3,4 Studies comparing abstracts with their full-text publications from other medical specialty meetings have found that primary end point results were consistent in 59% to 78% of published papers, whereas final conclusions were consistent in 70% to 93% of comparisons.5-8 In oncology, the topic of consistency between conference abstracts and their subsequent publications has not been fully addressed. One previous study examined the fate of a random sample of abstracts presented at the 1984 ASCO Annual Meeting. A portion of their study addressed the consistency of five variables in a random sample of phase III clinical trial abstracts compared with their subsequent publications. Subjectively, there was good to moderate correlation between the conclusion of the article and the abstract in 15 of the 18 comparisons.9 In the present study, we systematically examined abstracts describing phase III clinical trials from an ASCO Annual Meeting to determine their consistency with the subsequent full-text publications. We also examined the quality of the abstracts and attempted to determine factors predicting full publication of an ASCO abstract, and factors predicting consistency between the abstract and the subsequent publication.
We used the ASCO Web site's Abstract Advanced Search function10 and the index of the programs and proceedings of the 2000 ASCO Annual Meeting11 to identify abstracts describing phase III clinical trials presented at the 36th ASCO Annual Meeting (May 20-23, 2000, New Orleans, LA). We defined a phase III clinical trial as any trial explicitly identifying itself as a phase III clinical trial within the abstract or a clinical trial designed to compare one treatment to placebo or another treatment for a given cancer type and stage with "survival" or "disease-/progression-free survival" as a primary end point. The abstracts identified through the search were screened for our inclusion and exclusion criteria. We included abstracts describing phase III clinical trials of chemotherapy, chemoradiotherapy, immunotherapy, and hormone therapy. Our exclusion criteria were systematic reviews, combined analysis, abstracts where the main focus of the analysis was post hoc or where the patients used in the study were originally randomized to another phase III trial, and reanalysis of data that had been previously published or presented at a previous Annual Meeting of ASCO. We searched MEDLINE and PubMed from May 2000 to May 2006 for all English-language publications corresponding to the included ASCO abstracts. The search was performed using the first, second, and last authors’ names and key words from the abstract title. The resulting citations were screened to determine whether any corresponded to the ASCO abstract. The authors of abstracts without a corresponding publication were contacted by e-mail to determine whether the study results were truly not published and the authors were asked for reasons explaining lack of publication. A second e-mail was sent to any authors who did not respond to the first e-mail within 30 days. All of the ASCO abstracts and any corresponding journal publications were obtained and examined in detail. Data were subsequently extracted to a computer database by one of the authors (V.C.T.). To determine interobserver variability, 25% of the abstracts and publications were randomly selected and evaluated by the second author (S.J.H.). Any discrepancies were resolved through discussion.
Statistical Analysis
The screening and selection process of the ASCO abstracts is summarized in Figure 1. Seventy-four abstracts were included in our study. The MEDLINE and PubMed search found 55 corresponding journal publications. The publication rate of phase III clinical trial ASCO abstracts 6 years after the ASCO Annual Meeting was 74%. After e-mailing the authors of the 19 unpublished abstracts, four abstracts were found to have been published more than 6 years after presentation at the ASCO Annual Meeting (ie, after May 2006). One author informed us that a manuscript corresponding to the ASCO abstract was in its final stages of preparation for submission. Seven studies were confirmed to have remained unpublished. The predominant reason for lack of publication was rejection of the manuscript by several journals. The remaining seven authors failed to reply to our e-mails.
The median time to publication was 29 months and the majority of phase III clinical trial abstracts were published within 2 to 3 years of presentation at the ASCO Annual Meeting (Fig 2). Forty-four percent of the phase III clinical trial abstracts were published in the Journal of Clinical Oncology (Fig 3).
Twenty-five percent of the unique abstracts identified (n = 48) were screened by the second author (S.J.H.) for the inclusion and exclusion criteria, and there was 100% agreement with respect to the abstracts included and excluded. Twenty-five percent of the included abstracts (n = 19) and subsequent publications (n = 14) were also reviewed by the second author, and there was also 100% agreement with respect to the data abstracted by the first author (V.C.T.). Table 1 summarizes the characteristics of the abstracts and journal publications. Abstracts without a subsequent publication were significantly more likely to have been listed in abstract form only in the meeting proceedings, whereas the abstracts that were followed by full publications were more likely to have been presented at the ASCO Annual Meeting in the form of a poster or oral presentation (P = .04). The sample size of the published abstracts was significantly larger than that of unpublished abstracts (P = .02). Eighty-five percent of phase III clinical trials in oncology presented at the 2000 ASCO Annual Meeting were conducted in Europe and North America. The source of funding was not clearly stated in 95% of the ASCO abstracts.
Krzyzanowska et al4 suggested five quality characteristics that are felt to be most important for submission of scientific oncology abstracts. For our series, these quality characteristics are described in Table 2. All of the abstracts contained a brief description of the intervention. Statistical significance of the primary end point and clear conclusions were reported in 64% and 67% of the abstracts, respectively. The number of participating centers, 95% CI or P value, and the funding source were rarely reported.
A comparison of the phase III clinical trial abstracts and their corresponding publications is summarized in Table 3. The data reiterate the poor quality of the ASCO abstracts because primary end points, secondary end points, statistical analysis, results of the primary end point measure, statistical significance of the primary end point measure, and conclusions were frequently not clearly described in the abstract. For the subgroup of abstracts clearly describing these study characteristics, the primary end points are identical between the abstract and journal publication in 90% of the comparisons. The 10% that were not consistent were generally a result of the identification of an additional primary end point or elimination of a primary end point in the journal publication. In one study, time to progression was a primary end point added to overall survival in the journal publication, but this was not clearly stated in the abstract. Another study identified disease-free survival and overall survival as primary end points in the abstract, but survival rate was the only primary end point in the journal publication. The results of the primary end point measure stated in the abstract were within 5% when compared with the journal publication in 58% of the comparisons. The statistical significance of the primary end point and the conclusions were identical in 89% and 91% of the comparisons, respectively.
Abstracts that were selected as plenary or oral presentations compared with those selected from publication only in the proceedings were significantly more likely to be published (P = .03). There was a trend toward publication of abstracts selected as posters compared with publication in the proceedings alone, as well as those with an explicit definition of the primary end point and with statistically significant primary end points.
Logistic regression analysis did not identify any factors predicting consistent primary end point significance or overall conclusions for the ASCO abstracts and their journal publications. A trend toward consistent conclusions was observed for abstracts with explicit definitions of their primary end point.
Seventy-four phase III clinical trial abstracts presented at the ASCO Annual Meeting in 2000 were systematically reviewed and compared with their subsequent journal publications when possible. Our publication rate of 74% is higher than the 63% quoted by a Cochrane review paper describing abstracts of randomized trials and their full publications.12 The likely explanation is that we included only phase III clinical trials rather than all randomized trials. We would have expected that abstracts selected for oral presentation and describing "positive" results would be associated with full publication, as described in the Cochrane review by Scherer.12 Although our analysis did indicate that abstracts selected for plenary or oral presentations were more likely to be published, we found a trend toward subsequent publication only for abstracts describing statistically significant primary end points. Previous studies have reported on the suboptimal quality of ASCO abstracts describing randomized trials,3,4 and our study supports this finding in ASCO abstracts describing phase III clinical trials. The data we have presented, however, also currently provide the best evidence of consistency between phase III clinical trial ASCO abstracts and their subsequent journal publications. Because of the previously reported poor quality of ASCO abstracts,3 beginning in 2004, authors submitting abstracts describing randomized controlled trials were required to have improved structure. The format of clinical trial abstracts were standardized to include information under the headings Background, Methods, Results, Conclusions.13 However, there are no strict guidelines requiring description of primary and secondary end points, statistical methods, and sources of funding. A future study comparing the quality of ASCO abstracts before and after 2004 would be justified. When clearly stated, the primary end points are consistent between the ASCO abstract and the journal publication in 90% of comparisons. The magnitude of the primary end point result was within 5% in only 58% of comparisons. We would therefore advise caution in directly quoting the magnitude of primary end points presented in ASCO abstracts for oncology practice guidelines, evidence summaries, and meta-analyses. The large number of discrepancies may be accounted for by the fact that 44% of the abstracts with subsequent publications clearly indicated they were presenting interim results. Among the subgroup of abstracts that presented interim results, 89% of the reported primary end point magnitudes differed by greater than 5% in the subsequent publication. Inadequate follow-up time could certainly affect survival data and disease-free survival, which were the primary end points in 91% of the phase III clinical trials we examined. The statistical significance of the primary end point and the final conclusion were consistent in 89% and 91% of the comparisons, respectively. These two results are essentially equal given that a statistically significant primary end point favoring the efficacy of one treatment over another would generally correlate with the conclusion that one treatment is superior to another. However, if significant toxicities are discovered, the final conclusion may be discordant with the statistical signifi-cance of the primary end point. Our results are comparable to a previous nonsystematic study of ASCO abstracts describing phase III oncology trials. The authors found "good agreement" of conclusions between the ASCO abstracts and their subsequent publications in 83% of the comparisons.9 We have shown that primary end points, statistical significance of the primary end point, and final conclusions are usually consistent between the ASCO abstracts and their subsequent publications. However, this correlation may not apply to the results and conclusions of unpublished ASCO abstracts compared with their final study results and conclusions. Because selection as a plenary or oral presentation has been shown to predict subsequent publication, we would suggest that our finding of consistency be applied largely to this group of ASCO abstracts. Our underlying assumption has been that ASCO abstracts reflect the study results presented at ASCO Annual Meetings. There is a concern that the constraints of space and time prevent authors from including a comprehensive description of the study methods and results in their abstracts. The actual presentation at the meeting may be more representative of the study results and, therefore, more consistent with the final publication. However, until recently, it was difficult to determine the data presented at a meeting without having physically attended the meeting. Beginning with the 35th ASCO Annual Meeting (May 15-18, 1999, Atlanta, GA), a portion of the presentation slides and videos of the presentations were made available on the ASCO Web site under the heading "Virtual Meeting."14 However, it was not until more recently that a larger proportion of the Annual Meeting presentations have been made available through Virtual Meeting. To determine whether the data presented at an ASCO Annual Meeting is consistent with the subsequent publications, future studies dealing with this topic should examine the consistency between the abstracts, Virtual Meeting presentations, and subsequent journal publications. Another limitation of our study is that we searched MEDLINE and PubMed only for corresponding journal publications to the abstracts. We may have missed journals not indexed by these two databases. Contacting the authors of the abstracts for which we were unable to identify journal publications should have eliminated this limitation; however, we did not receive responses from seven authors. The responses that we did receive from the authors indicated that we had not missed any of their publications as a result of lack of indexing in MEDLINE or PubMed. It should also be noted that we systematically analyzed the phase III clinical trial abstracts from only one ASCO Annual Meeting. The ability to apply our findings to the ASCO abstracts of subsequent years, especially to those presented at ASCO meetings after 2004, is uncertain, but lag time to publication prevented us from addressing this limitation. When carefully selected, abstracts of phase III clinical trials from the ASCO Annual Meetings seem to consistently reflect final published results in terms of the statistical significance of the primary end point and conclusions. These abstracts can likely be used as evidence for practice guidelines, but we recommend that studies presented at oral or plenary sessions be primarily used for this purpose given that occasional discrepancies underline the need for caution in completely adopting these results before full publication.
Although all authors completed the disclosure declaration, the following author(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a "U" are those for which no compensation was received; those relationships marked with a "C" were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors. Employment or Leadership Position: None Consultant or Advisory Role: None Stock Ownership: None Honoraria: None Research Funding: None Expert Testimony: None Other Remuneration: Vincent C. Tam, ASCO Foundation Merit Award
Conception and design: Vincent C. Tam, Sebastien J. Hotte Collection and assembly of data: Vincent C. Tam, Sebastien J. Hotte Data analysis and interpretation: Vincent C. Tam, Sebastien J. Hotte Manuscript writing: Vincent C. Tam, Sebastien J. Hotte Final approval of manuscript: Vincent C. Tam, Sebastien J. Hotte
We thank Cyndy Mackenzie for her assistance in obtaining copies of all the published papers and Wenjie Cai for his statistical support.
Presented in part at the 43rd Annual Meeting of the American Society of Clinical Oncology, Chicago, IL, June 1, 2007. Authors’ disclosures of potential conflicts of interest and author contributions are found at the end of this article.
1. Winer EP, Hudis C, Burstein HJ, et al: American Society of Clinical Oncology technology assessment on the use of aromatase inhibitors as adjuvant therapy for postmenopausal women with hormone receptor-positive breast cancer: Status report 2004. J Clin Oncol 23:619-629, 2005 2. Jonker D, Rumble RB, Maroun J, et al: The role of oxaliplatin combined with 5-fluorouracil and folinic acid in the first and second-line treatment of advanced colorectal cancer: A clinical practice guideline. http://www.cancercare.on.ca/index_gastrointestinalCancerguidelines.htm 3. Hopewell S, Clarke M: Abstracts presented at the American Society of Clinical Oncology conference: How completely are trials reported? Clinical Trials 2:265-268, 2005[CrossRef][Medline] 4. Krzyzanowska MK, Pintilie M, Brezden-Masley C, et al: Quality of abstracts describing randomized trials in the proceedings of American Society of Clinical Oncology meetings: Guidelines for improved reporting. J Clin Oncol 22:1993-1999, 2004 5. Bhandari M, Devereaux PJ, Guyatt GH, et al: An observational study of orthopaedic abstracts and subsequent full-text publications. Bone Joint Surg Am 84-A:615-621, 2002 6. Preston CF, Bhandari M, Fulkerson E, et al: The consistency between scientific papers presented at the Orthopaedic Trauma Association and their subsequent full-text publication. J Orthop Trauma 20:129-133, 2006[CrossRef][Medline] 7. Toma M, McAlister FA, Bialy L, et al: Transition from meeting abstract to full-length journal article for randomized controlled trials. JAMA 295:1281-1287, 2006 8. Weintraub WH: Are published manuscripts representative of the surgical meeting abstracts? An objective appraisal. J Pediatr Surg 22:11-13, 1987[Medline] 9. De Bellefeuille C, Morrison CA, Tannock IF: The fate of abstracts submitted to a cancer meeting: Factors which influence presentation and subsequent publication. Ann Oncol 3:187-191, 1992 10. American Society of Clinical Oncology: Abstracts search. http://www.asco.org/portal/site/ASCO/menuitem.2433254f3861a98ff7b1f4106e37a01d/?vgnextoid=22228cf0807cdc010VgnVCM100000f2730ad1RCRD&searchAbst=Y 11. American Society of Clinical Oncology: Indexes. Proc Am Soc Clin Oncol 19:669a-789a, 2000 12. Scherer RW, Langenberg P, von Elm E: Full publication of results initially presented in abstracts. Cochrane Database of Systematic Reviews, MR000005, 2007 13. ASCO News & Forum: 2007 Annual Meeting Call for Abstracts. http://www.asconews.org/portal/site/anf/menuitem.8f6d1d4a064dae4107078010ee37a01d/?vgnextoid=ba7fdd0f572a0110VgnVCM100000ed730ad1RCRD&cpsextcurrchannel=1 14. American Society of Clinical Oncology: Virtual Meeting. http://www.asco.org/portal/site/ASCO/menuitem.64cfbd0f85cb37b2eda2be0aee37a01d/?vgnextoid=09f8201eb61a7010VgnVCM100000ed730ad1RCRD&vgnextfmt=default Submitted October 3, 2007; accepted January 4, 2008.
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||
|
|||||||||||
|
|
Copyright © 2008 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|
TOP
ABSTRACT
INTRODUCTION
2 statistics were used to make appropriate comparisons. A P value less than .05 was considered significant. Where appropriate, 95% CIs were calculated. Univariate and multivariate logistic regression was performed to determine factors predicting publication, consistency between the ASCO abstracts and subsequent publications for magnitude of primary end point, statistical primary end point significance, and conclusion. 
