|
Advertisement |
|
|
||
 |
|
|||||
|
|||||
|
 |
||||||
|
||||||
|
||||||
|
Originally published as JCO Early Release 10.1200/JCO.2004.06.924 on August 23 2004 © 2004 American Society of Clinical Oncology.
Meta-Analyses Based on Abstracted Data: A Step in the Right Direction, but Only a First Step
Department of Medical Oncology, Henri Mondor Hospital, Assistance Publique Hôpitaux de Paris, France "Only what is exhaustive is of interest. The truth comes from an accumulation of details." Thomas Mann (1875-1955) Despite the development of new drugs, platinum compounds remain a major component of most chemotherapy regimens in patients with advanced non–small-cell lung cancer (NSCLC). In this issue of the Journal of Clinical Oncology, Hotta et al1 use a meta-analytic approach to compare cisplatin and carboplatin in NSCLC. In their analysis of eight trials and 3,000 patients, they find cisplatin to be superior to carboplatin in terms of tumor response rates (odds ratio = 1.36; 95% CI, 1.15 to 1.61; P < .001), but not in terms of survival (hazard ratio = 1.05; 95% CI, 0.91 to 1.22; P = .51). In a subgroup analysis of five trials, they report that—when combined with taxanes or gemcitabine—cisplatin may be superior to carboplatin in terms of overall survival (hazard ratio = 1.11; 95% CI, 1.01 to 1.22; P = .039). NSCLC is one of the most common cancers in the world, and the choice of platinum agents deserves particular attention, since the substitution of cisplatin by carboplatin may have important implications, not only in terms of drug administration, but also in cost. To address this very important issue, Hotta et al1 have performed a meta-analysis based on abstracted data, without retrieval of individual patient data (IPD). In other words, their work is based on a re-analysis of published data, tables, and survival curves rather than on an exhaustive collection, checking, and analysis of data of individual patients included in all trials. In addition to not restricting the trial search to full papers, the approach they used, which consists of extracting data from publications, has severe limitations. By definition, meta-analyses based on abstracted data must deal with already-performed analyses. The reliability of the randomization methods cannot be evaluated, the data cannot be checked, the original analyses cannot be redone, and additional analyses cannot be performed. Authors of meta-analyses based on abstracted data often assess the quality of individual studies included in their analyses, and weigh each trial in the meta-analysis in proportion to its quality. However, no quality assessment system can substitute for the use of individual patient data in guaranteeing the absence of bias in the original analyses.2,3 Deliberate errors are quite rare, especially in multicenter randomized clinical trials, but random errors and biases in the reported results are not.4 Randomization procedure and absence of patient exclusion are key features for the reliability of trial results. In the set of trials considered by Hotta et al, for example, 5% of the patients were excluded from the tumor response analysis, but this proportion reached 15% in one trial, and in four of eight trials, survival analyses were not based on intent to treat. Exclusion of patients from analyses may be related to patient ineligibility (which creates no bias), but may also be related to treatment outcome (in which case bias may be present). Such bias can be more serious when a report does not even mention randomized but not treated or ineligible patients who have been excluded from all analyses. By contrast, the collection of IPD allows including all randomly assigned patients in the analyses. IPD meta-analyses may also reveal inadequate randomization procedures, such as nonrandomized patients included in the analysis of a randomized trial, a practice that has been documented in some IPD meta-analyses.3 Meta-analyses based on abstracted data do not allow updating of patient outcomes, which are frozen in time. Such an update, usually done in IPD meta-analyses, is crucial in the adjuvant setting, but can also be important in advanced stage studies when a small but clinically and statistically significant survival difference may matter.5 Analyses based on IPD represent both the only reliable and the most powerful method to calculate and compare times to tumor recurrence or death. Extrapolations based on published survival curves are simulated substitutes for proper analyses of events. Whenever substantial heterogeneity is present in the results of the analyses, regardless of the method used, further investigation (to the extent possible) is warranted to explain the source of variability. In Figure 2 of the Hotta et al1 paper, for example, the results of one trial appear so different from all others that the difference is unlikely due to chance alone. Data from this trial should be scrutinized to identify the source of heterogeneity; if a source was found, the estimate of the overall hazard ratio could be closer to its true value after exclusion of this particular trial, which may add bias to random error in the overall estimation procedure. The data collection and the meetings held to discuss a meta-analysis and to review the analyses are unique opportunities to bring together the principal investigators of each trial. In our experience, the discussions between those who perform the meta-analysis and the investigators are quite productive, and lead to better data analyses than a mere literature search. The effort made by Hotta et al to contact investigators and to confirm or update the published results are unusual in typical meta-analyses based on abstracted data, and can certainly be commended. However, the authors do not document whether this approach allowed a better assessment of treatment effects within each trial. The superiority of IPD meta-analyses over other methods is so evident (Table 1) that one can question why abstracted meta-analyses are so popular. The answer is, of course, because they are much easier to perform. The limiting factor of IPD meta-analyses is the data collection, which can take between 1 and 5 years, if not more. In some circumstances, such data collection appears impossible. This can happen when trials are too old for data to be available, when a collaborative group does not agree to partake in the meta-analysis, or when a drug sponsor refuses to give access to the IPD of a trial. Regulatory authorities should play an important role in promoting IPD meta-analyses, whenever appropriate, to approve new drugs or to re-evaluate the effectiveness of registered drugs. Peer-reviewed journals should act likewise.
In the case at hand, previously published papers show that IPD meta-analyses addressing the management of lung cancer are quite feasible. At least five IPD meta-analyses have been reported in the last 10 years in lung cancer. Two analyses addressed treatment of small-cell lung cancer, either with cranial irradiation for patients in complete remission after chemotherapy,6 or with thoracic irradiation7; the others addressed treatment of NSCLC, either with postoperative chemotherapy,8 radiotherapy,9 or concomitant radiochemotherapy.10 The value of adjuvant chemotherapy in NSCLC was addressed in a well-conducted IPD meta-analysis almost 10 years ago, and it is quite surprising that the same issue is now revisited using abstracted data of a subset of trials, which provide far less informative and trustworthy analyses.11 Although IPD meta-analyses are considered a gold standard, and the only truly reliable way of pooling data from different trials, we are aware that they also need time and funding, and we understand the temptation of performing meta-analyses based on abstracted data. For important questions in need of a definite answer, this temptation should be resisted, and meta-analyses based on abstracted data should be seen only as a first step toward generating hypotheses to be tested in full-fledged IPD meta-analyses. Authors' Disclosures of Potential Conflicts of Interest The authors indicated no potential conflicts of interest.
REFERENCES
1. Hotta K, Matsuo K, Ueoka H, et al: Meta-analysis of randomized clinical trials comparing cisplatin to carboplatin in patients with advanced non–small-cell lung cancer. J Clin Oncol 22:3852-3859, 2004 2. Buyse M, Piedbois P, Piedbois Y, et al: Meta-analysis: Methods, strengths, and weaknesses. Oncology 14:437-443, 2000[Medline]
3. Clarke MJ, Stewart LA: Systematic reviews: Obtaining data from randomized controlled trials: How much do we need for reliable and informative meta-analyses. BMJ 309:1007-1010, 1994 4. Buyse M, George SL, Evans S, et al: The role of biostatistics in the prevention, detection and treatment of fraud in clinical trials. Stat Med 18:3435-3452, 1999[CrossRef][Medline] 5. Piedbois P, Michiels S, for the Meta-Analysis Group In Cancer: Survival benefit of 5FU/LV over 5FU bolus in patients with advanced colorectal cancer: An updated meta-analysis based on 2,751 patients. Proc Am Soc Clin Oncol 22:294, 2003 (abstr 1180) 6. Auperin A, Arriagada R, Pignon JP, et al: Prophylactic cranial irradiation for patients with small-cell lung cancer in complete remission. Prophylactic Cranial Irradiation Overview Collaborative Group. N Engl J Med 12:476-484, 1999 7. Pignon JP, Arriagada R, Ihde DC, et al: A meta-analysis of thoracic radiotherapy for small-cell lung cancer. N Engl J Med 3:1618-1624, 1992 8. Chemotherapy in non-small cell lung cancer: A meta-analysis using updated data on individual patients from 52 randomised clinical trials. Non-Small Cell Lung Cancer Collaborative Group. BMJ 311:899-909, 1995 9. Postoperative radiotherapy in non-small-cell lung cancer: Systematic review and meta-analysis of individual patient data from nine randomised controlled trials. PORT Meta-analysis Trialists Group. Lancet 352:257-263, 1998[CrossRef][Medline] 10. Auperin A, Le Péchoux C: Meta-analysis of randomized trials evaluating cisplatin or carboplatin-based concomitant chemoradiation versus radiotherapy alone in locally advanced non-small cell lung cancer (NSCLC). Lung Cancer 41:S69, 2003 (suppl 2)
11. Hotta K, Matsuo K, Ueoka H, et al: Role of adjuvant chemotherapy in patients with resected non–small-cell lung cancer: Reappraisal with a meta-analysis of randomized controlled trials. J Clin Oncol 22:3860-3867, 2004
Related Articles
Related Correspondence
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||
|
|||||||||||
|
|
Copyright © 2004 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|
