|
Advertisement |
|
|
||
 |
|
|||||
|
|||||
|
 |
||||||
|
||||||
|
||||||
|
© 2003 American Society for Clinical Oncology Trends in Clinical Trials Reports in Common Cancers Between 1989 and 2000From the Department of Internal Medicine, the Outcomes Research Institute, and the Massey Cancer Center, Virginia Commonwealth University, Richmond, VA. Address reprint requests to Bruce E. Hillner, MD, Department of Internal Medicine, the Outcomes Research Institute, and the Massey Cancer Center, Virginia Commonwealth University, Richmond, VA 23298-0170; email: Hillner{at}hsc.vcu.edu.
Purpose: Cancer clinical trials can be dichotomized into pilot (phase I and phase II) and randomized controlled clinical trials (RCTs). The best data source for evidence-based medicine is from RCTs. However, many patients prefer to enroll onto pilot trials, and many investigators prefer to conduct or refer their patients to pilot trials. This exploratory study sought to describe the epidemiologic patterns of clinical trial reports in common cancers. Methods: A structured review was conducted of MEDLINE citations of all English clinical trials reports published between 1989 and 2000 in breast, lung, colorectal, prostate, and female genital cancers, plus leukemias and lymphomas. Each report was classified by design (RCT, pilot, or other) and country. The abstracts of RCTs were reviewed for sample size. Reports addressing screening or prevention were excluded. Results: A total of 12,035 reports, of which 3,560 were from RCTs, were found. The annual growth in RCT reports in breast, colorectal, and prostate cancer was significant (range, 4.8% to 8.5% per year) but was insignificant in leukemias, lymphomas, and female genital and lung cancers (range, 0.1% to 4.3% per year). Within each cancer, the average sample size per report did not change during the 12 years. Nonrandomized trial reports increased on average 15.1% per year (range, 10.1% to 23.2%). The United States accounted for 30% of all RCT reports and 45% of pilot trial reports. Conclusion: The faster growth in nonrandomized compared with RCT reports may reflect rapid advances in cancer biology or different structural, commercial, and financial incentives, especially in the United States compared with Europe. Unless additional studies show evidence of an increase in their quality, the modest growth in RCT reports may limit future evidence-based cancer care.
IN ALL areas of medicine, increasing attention is being directed at practicing evidence-based medicine.1 There is universal consensus that the best evidence is derived from randomized controlled trials (RCTs), ideally from multiple centers involving sufficient patients to have high statistical power. RCTs are usually the culmination of a translation research chain of pilot studies that assess the feasibility and safety of a new approach.2 The volume and type of clinical research activity directed at a specific condition will vary depending on the burden of illness, the perception of ongoing advances in effective treatments, patient advocacy, and availability of financial support. Oncology has a long tradition of clinical trials organized by the United States National Cancer Institute–supported cooperative groups and similar groups in Europe. In the United States, cancer organizations have successfully encouraged expanded participation, access, and insurance payment for cancer clinical trials that are both publicly and industry-funded.3 Whether cancer patients and providers have been equally willing to participate in RCTs versus pilot trials cannot be readily measured because no comprehensive registries are maintained of initiated trials. The author’s impression was that the oncology specialty peer-reviewed literature was becoming dominated by noncomparative pilot reports evaluating new approaches, drug or other biologic therapies, or combinations. Whether this impression that there has been a substantial shift to pilot trials away from RCTs was true, and whether the impression applied to the total universe of oncology clinical trials, was uncertain. The National Library of Medicine’s structured database, MEDLINE, codifies more than 4,600 journals using a controlled vocabulary for subject matter and methodology that has been continuously evolving and refining how the medical literature is indexed and codified. Using terms available in MEDLINE since 1989, I identified and categorized articles of the seven most common types of cancers. The project’s goals were to determine the number and growth rates of reports in each cancer type of RCTs, multicenter trials, pilot trials, their country of origin, and the sample sizes in the RCTs.
Identification, Selection, and Categorization of Studies A series of computerized searches of MEDLINE (United States National Library of Medicine) were performed using the Knowledge Finder search engine (Aries Systems Corp, North Andover, MA). The search strategy targeted the seven most common categories of cancer: lung, breast, colorectal, prostate, female genital, lymphomas, and leukemias. For each condition, the search started in 1989, because this was the first year that MEDLINE separately codified multicenter studies as a distinct publication type, end continued through 2000. The search considered all types of clinical trial reports for these cancers including prevention, diagnosis, treatment, and prognosis. The inclusion criteria and classification scheme used are shown in Fig 1  . The literature search was subsequently downloaded into Reference Manager (version 9.5, ISI Researchsoft Inc, Philadelphia, PA) and converted into a delimited text file.
Before each article was classified, a second round of exclusions was made. Because articles in journal supplements rarely are peer reviewed or represent the primary report, these articles were identified and deleted. Because the focus was on known cancers, articles that addressed screening or prevention were excluded. This was done by excluding articles with the subject heading of mass screening and reviewing the abstracts of all articles with screening or prevention in the title; these articles were excluded if appropriate. A primary goal was to classify clinical trial reports for each cancer into one of three categories: a pilot trial, an RCT, or an unknown design trial (see Appendix for exact MEDLINE definitions). Pilot trial reports were defined as an aggregate of all reports including one or more of the following medical subject heading (MeSH) classifications: a phase I clinical trial (publication type), a phase II clinical trial (publication type), a feasibility study (MeSH subject term), or pilot projects (MeSH subject term). Because these terms have been consistently used only since 1993, a free-text search of each article’s title and abstract for these terms was done for publications between 1989 and 1994. If the words phase I, phase II, pilot, or feasibility study were found, the report was classified as a pilot report. A pilot trial involving two or more cancer types could lead to two or more citations. Randomized controlled clinical trials were identified by MeSH publication type. A Boolean search excluded articles with the pilot report terms. For example, a report of a randomized controlled phase II trial would have been classified as pilot trial because of its phase II design. By exclusion, a report classified as a clinical trial that did not include the pilot or RCT definitions was categorized as other design. Examples include case-control studies and retrospective case series from other trials. In addition to the design categorization, reports were categorized using other MEDLINE classifications of a multicenter study (publication type), the address of corresponding author, and acknowledged United States government financial support. The MEDLINE address field was converted from text to columns, sorted, and categorized by country. On the basis of their relative frequency, articles were aggregated by country or region into 10 groups. Non–United States government financial support reported by MEDLINE was not useful because it does not discriminate between for-profit and nonprofit businesses or non–United States governments. Because the full reports were not reviewed, no other attempt was made to address any design or statistical characteristics of the reports other than sample size. In addition, all RCT reports were considered equal. That is, reports that represented the primary report, a secondary objective, a correlative science study such as a genetic or pharmacokinetic evaluation, a retrospective review of patients from an RCT, or a combination of two or more trials not classified as a meta-analysis were all included and reviewed for sample size. The only further exclusion was made in the review of female genital neoplasm reports, for which articles addressing leiomyoma (a benign tumor) were excluded.
Sample Size Analysis
Statistical Analysis
All Trial Reports Table 1 shows the breakdown by design and cancer type of the over 12,000 clinical trial reports identified and classified. Report number per condition ranged from 1,087 to 2,634. Breast cancer had the most total and most RCT reports and the highest percentage of reports that were from RCTs. Lung cancer had the most reports and the highest percentage of reports that were from pilot trials.
RCT Reports Table 2 shows the number of RCT reports by cancer type stratified by year. During the 12 years included in the review, there were 3,560 RCTs reports in these seven cancers. From 1996 to 2000, there were approximately 100 breast cancer RCT reports per year. In contrast, prostate cancer had the fewest reports at 293 and fewer than 20 RCT reports per year from 1989 to 1995.
The number of reports in leukemia, lung cancer, lymphomas, and female genital cancers did not statistically significantly increase per year during the 12 years. The annual number of lung cancer reports varied widely from year to year. In lymphoma, trial reports steadily increased until 1998 but markedly declined in 1999 and 2000. Variation in the progressive growth of reports was lowest in breast, colorectal, and prostate cancer. During the 12 years, the average annual increase in RCT reports ranged from 0.1% to 8.5% per year. Reports in prostate cancer (8.5%), colorectal cancer (5.4%), and breast cancer (4.9%) increased at a significant rate. Reports on leukemia (0.1%), lung cancer (2.3%), lymphomas (4.3%), and female genital cancer (2.3%) had growth rates that were on average positive but were not statistically significant.
Sample Size in RCT Reports
When aggregated by year, there was no evidence of a trend to an increase in the average sample size per trial report in any of the cancers. There was no difference in the sample size from the United States compared to all other countries in six of the cancers, but sample size was larger in prostate RCTs (United States, 292 [95% CI, 228 to 356] v non–United States, 177 [95% CI, 138 to 215]).
Because larger trials are more likely to be sufficiently powered to change evidence-based practice, an alternative approach assessed sample size trends using a specific predefined minimal threshold size (see Methods). The right-hand columns of Table 3 Regression models assessed the contribution of country or region of origin and acknowledged United States government support, multicenter classification, and publication year as factors in sample size. United States government support was a significant factor in predicting sample size in all cancers with the exception of colorectal and prostate cancer (data not shown). Multicenter classification was a predictive factor in all cancers with the exception of lymphoma. If the report was from France about lymphoma or from the United Kingdom about leukemia, lung, or prostate cancer, it was predictive of a greater sample size.
Multicenter RCTs
Pilot and All Nonrandomized Trials Figure 2 shows the number of pilot trials per year by cancer. In each of the cancers in 1989, there were fewer than 10 pilot trial reports. During the early 1990s, the number of identified pilot trials increased rapidly in all cancers. This is likely explained by a combination of improved coding, greater use of the phase terminology, and new active compounds. From 1993 to 2000, the number of prostate, lung, and breast cancer pilot reports continued to increase, approximately doubling in each condition. Growth in pilot trial reports was slower in lymphoma and in female genital and colorectal cancer. Leukemia was the only condition with no increase in pilot reports over the period.
Because the most uncertainty occurred in distinguishing between pilot and other non-RCT reports, Table 5 shows the temporal trends of all non-RCTs per year. The growth in nonrandomized trial reports averaged 15.1% per year and ranged from a low of 10% in leukemia to 23% in prostate cancer.
Country Figure 3 shows the relative contribution from the 10 largest countries of RCT and pilot trial reports. The United States accounted for 30% and Western Europe combined accounted for 58% of all RCT reports. The United Kingdom, Scandinavia (Sweden, Norway, Denmark, and Finland), and Italy each contributed more than 10% of all RCT reports.
The United States was the source for just less than one half of all pilot trials (45%). The only other country or region contributing 10% of more was Italy. The differences in the relative contribution by country are highlighted in Fig 3  . The United States accounted for about 50% more of the pilot trial reports than RCT reports. The United Kingdom had a two-fold and Scandinavia a four-fold greater percentile of RCT reports compared with non-RCT reports. Each of the other countries had about an equal balance of RCT and pilot reports.
Table 6
Patients with cancer who choose to participate in clinical research are a treasured limited resource in the quest to provide better cancer therapy. Clinical cancer research has multiple goals, including finding better treatments and defining new standards of care using treatment tools already available. Although the individual patient rarely has to choose between participating in comparative RCTs (defining best available care) and making new discoveries in a pilot trial, individual investigators, their organizations, and their countries may have a choice in setting priorities and financial resources. On the basis of the number of clinical trial reports, the evidence indicates that the duel aims of discovery- and evidence-based goals of cancer clinical research are being met in a different manner. Evidence from published reports of all clinical trials in the most common cancers increased by 150% from 1990 to 2000. The growth in trial reports was predominantly in pilot or nonrandomized RCTs. In four of the seven cancers, there was no evidence of a significant increase in the number of RCT reports. In all seven cancers, the average sample size in RCTs did not change over time. Excluding breast cancer, there was an average of only about five reports per year per condition from RCTs with sufficient power to detect a minimal absolute difference of 10% between interventions. Several limitations are present in this study. One concern is whether it is valid to equate the coverage of MEDLINE with international activity in clinical cancer research. The answer is a qualified yes. Although MEDLINE is maintained by a United States government organization, it captures nearly all English-language published reports independent of country of origin. A review of the MEDLINE database found that 45% to 49% of journals indexed in MEDLINE with the words cancer or oncology in their title were non–United States publications (S. Tybaert, personal correspondence, November 2002). A second concern is the completeness and accuracy of MEDLINE in coding a specific publication. One of the early findings of the Cochrane Collaboration5 was that many randomized trial reports were not indexed as such in MEDLINE, especially before 1980. Using a database of known randomized clinical trials in ophthalmology, Dickersin et al6 found that for the year 1988, MEDLINE identified only 87% of the publications as RCTs. Therefore, part of the increase in cancer RCTs since 1989 may partly reflect the quality of indexing by MEDLINE and may not reflect any real increase in the number of reports. A third concern is that this study only considered the number but not the design quality of the clinical trial reports. Other studies of RCT reports have addressed the types and frequency of methodological flaws and reporting errors.7 In this evaluation of RCT reports, the only potential quality indicators assessed were the average sample size, sample size thresholds sufficient to detect a 15% (175 patients) and 10% (376 patients) absolute differences between group, and if the report was classified by MEDLINE as a multicenter report. There was no evidence of an increase in the average sample size, but there was a modest 6% annual increase in the reports exceeding these size thresholds in the six nonbreast cancers. Multicenter RCT reports increased by 11% per year. However, this may solely reflect undercoding of multicenter trials by MEDLINE indexers, especially in the initial years after the term was introduced. The sample size threshold was arbitrary; it certainly is too high for pharmacology studies using a cross-over design. A marked difference in sample sizes was found between RCT reports in breast cancer and the other six cancers. This is predominantly because of the greater frequency of adjuvant versus advanced disease trials in breast cancer. Sixty percent of RCT reports that provided the sample size had fewer than 175 patients, which was the minimum needed for detecting a large 15% absolute difference in the primary end point. An inherent limitation of this assessment is that it only considered published reports. The focus on publication versus identification of ongoing or completed trials was necessary because there are no comprehensive trial registers maintained of initiated studies. Several efforts have been started that have begun to bridge this gap. At present, all active United States government–funded clinical trials should be indexed and accessible on the internet (http://www.clinicaltrials.gov). In the United Kingdom, all trials funded by the United Kingdom National Health Service must be registered in a voluntary metaregister of controlled trials (http://www.controlled-trials.com). However, clinical trials sponsored by pharmaceutical companies or pilot trials performed by individual cancer centers have no requirement or incentive to register their trials in these registries. There are also hundreds of drug trial registries existing without a standardized content. A recent review of online registers of cancer drugs being evaluated in phase III trials shows that only three of 12 prostate cancer drugs and eight of 20 colon cancer drugs did not appear in any of the online registers searched.7 This work may inaccurately reflect the actual number of RCTs per cancer. A potential reason for the slow growth in RCT reports could be publication bias against submitting or publishing negative or insignificant results. Another reason could be that fewer articles are misclassified as RCTs since the 1996 publication of the Consolidated Standard of Reporting Trials.8 Counting individual reports may lead to an overestimation of the RCT evidence base if there are substantial secondary publications. In addition, the primary objective of each publication may not be directed at the primary end point of the RCT. Although MEDLINE uses a rigorous definition for the RCT publication type, it cannot differentiate between reports of nonrandomized data collected within the setting of an RCT and the primary end point. Examples include correlative science studies, subgroup analysis, prognostic factors, and case-control studies within an RCT. Other reports, particularly in leukemia, combined results from previously reported RCTs without performing a formal meta-analysis, which may falsely increase the number and sample size estimate. Finally, the frequency of multiple publications of the same trial results is unknown. The number of cancer target journals has markedly increased during this period. In 2002, there were 192 journals with either cancer or oncology in their titles that were indexed by MEDLINE. The current project did not attempt to track the number of publications per clinical trial across journal or years. There has clearly been an sudden increase in nonrandomized clinical trial reports, predominantly of pilot trials, in each cancer. The likely reasons include a disproportionate growth in the scientific understanding of cancer, a reflection of the lack of progress in identifying improved therapies (eg, lung cancer), the predominant use of short-term end points, an expansion in research funding, and the preference of patients and investigators. More may not be better if most of these pilot trials have "me too" (nearly identical) designs and are not precursors to a comparison to standard treatment.10 Financial and cultural issues are likely to be important if not readily quantifiable. It should not be surprising that the United States accounted for almost one half of the world’s pilot trial reports. Financial and academic advancement incentives to United States investigators to enroll patients in pilot projects are well known. An evaluation of the role played by the pharmaceutical and biomedical device industry in supporting clinical research of all types will require a direct review of each report. At present, MEDLINE does not distinguish between acknowledged support from United States–based nonprofit organizations (eg, American Cancer Society), for-profit companies, and non–United States governments. Although authors are increasingly expected to acknowledge potential conflicts of interest, the requirement for acknowledging financial support is lax. The evidence herein indicates that there is a wide variation about how different countries use their patient population for cancer research. The distribution of RCT reports by country is likely to have multiple influences, including local for-profit companies, tradition of entrepreneurship, extent of centralization of cancer services, and overall economic richness. Slevin11 commented that patients participating in RCTs may view themselves as either volunteers or a victims. These divergent attitudes may be predominantly cultural. The roles played by the United States and Europe in the overall spectrum of clinical research appears to be complimentary; the United States primarily performs pilot investigations, and northern Europeans chiefly perform comparative RCTs. The contrast between the United States and Scandinavia is striking in the amount of resources directed at pilot trials versus RCTs. Scandinavian countries, with nearly the same level of economic affluence, reported five times as many RCT reports as pilot trials. In addition, a count of trial reports may not be an accurate representation of the overall financial commitment to clinical research for a specific condition. The expense is much greater for large, comparative RCTs using survival or other hard event end points than for small pilot trials using changes in biologic end points. Therefore, if one assumes equal trial sizes, the financial commitment to expanding the foundation of evidenced-based cancer care is greatest in Scandinavia and the Netherlands, which have highest number of RCT reports per population. In conclusion, this review of published clinical trial reports in the seven most common cancers found that cancer clinical trial reports are increasing. RCT reports are increasing at a modest rate per year and are increasingly multicenter in design; however, the trials are not larger, and most are underpowered to detect clinically meaningful differences. The faster growth in pilot reports may reflect their shorter time to completion and also investigator, patient, sponsor, and cultural preferences. To expand evidence-based oncology care, larger, multicenter RCTs are needed without a reduction in participation in pilot investigations.
1. Sackett DL, Haynes RB: Evidence base of clinical diagnosis: The architecture of diagnostic research. Br Med J 324:539–541, 2002 2. Baumann M, Bentzen SM, Doerr W, et al: The translational research chain: Is it delivering the goods? Int J Radiat Oncol Biol Phys 49:345–351, 2001[CrossRef][Medline]
3. Arnold K, Vastag B: Medicare to cover routine care costs in clinical trials. J Natl Cancer Inst 92:1032, 2000
4. Nicolucci A, Grilli R, Alexanian AA, et al: Quality, evolution, and clinical implications of randomized, controlled trials on the treatment of lung cancer: A lost opportunity for meta-analysis. J Am Med Assoc 262:2101–2107, 1989
5. Dickersin K, Manheimer E, Wieland S, et al: Development of the Cochrane Collaboration’s CENTRAL Register of controlled clinical trials. Eval Health Prof 25:38–64, 2002
6. Dickersin K, Scherer R, Lefebvre C: Identifying relevant studies for systematic reviews. Br Med J 309:1286–1291, 1994
7. Manheimer E, Anderson D: Survey of public information about ongoing clinical trials funded by industry: Evaluation of completeness and accessibility. Br Med J 325:528–531, 2002
8. Djulbegovic B, Adams JR, Lyman GH, et al: Evaluation and appraisal of randomized controlled trials in myeloma. Ann Oncol 12:1611–1617, 2001
9. Begg C, Cho M, Eastwood S, et al: Improving the quality of reporting of randomized controlled trials: The CONSORT statement. J Am Med Assoc 276:637–639, 1996 10. Tannock IF: Collaborative clinical trials: Quality or quantity? Int J Radiat Oncol Biol Phys 49:339–343, 2001[CrossRef][Medline] 11. Slevin M, Mossman J, Bowling A, et al: Volunteers or victims: Patients’ views of randomised cancer clinical trials. Br J Cancer 71:1270–1274, 1995[Medline] Submitted August 23, 2002; accepted February 3, 2003.
This article has been cited by other articles:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||
|
|||||||||||
|
|
Copyright © 2003 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|
TOP
ABSTRACT
INTRODUCTION
2 tests were used for categorical variables using SPSS 10.1 (SPSS Inc, Chicago, IL). After the number of reports per year was aggregated, regression curves for annual change in trial reports were explored. The best fit of the data was consistently the growth regression model, the equation of which is Y = eb0 + (b1 * t), where Y is the number of reports, e is the natural logarithm, b is the beta coefficient or growth, and t is time. An analysis of variance using the year as the categorical variable and the number of reports per cancer was used to test whether the beta coefficient and the standardized regression coefficient were statistically significantly different from zero. Regression analyses assessed the effect of country, publication year, multicenter classification, and cancer types associated with RCT sample size (linear) and RCT sample sizes greater than 174 or 375 patients (logistic). 
