This Article Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Save to my personal folders Download to citation manager Rights & Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Tannock, I. F. Search for Related Content PubMed PubMed Citation Articles by Tannock, I. F. Social Bookmarking                            What's this?

Commentary on "Cytoreduction Nephrectomy in Metastatic Renal Cancer: The Results of Southwest Oncology Group Trial 8949"

From the Department of Medical Oncology and Hematology, Princess Margaret Hospital and University of Toronto, Toronto, Ontario, Canada.

Address reprint requests to Ian F. Tannock, MD, PhD, Department of Medical Oncology and Hematology, Princess Margaret Hospital and University of Toronto, 610 University Ave, Toronto, Ontario M5G 2M9, Canada; email ian.tannock{at}uhn.on.ca

SOME IMPORTANT QUESTIONS ABOUT THE STUDY

THE ESSENTIAL features of the Southwest Oncology Group (SWOG) 8949 study¹ presented by R.C. Flanigan, MD, are summarized in Table 1. In evaluating the results and impact of this study, it is important to ask several questions, which are summarized below.

Was the Primary End Point Well Defined?
Again, the answer is "yes," since the primary end point was survival. The authors of the study also used some secondary end points and undertook subgroup analysis, which can be misleading because the study was not powered to address secondary end points. Results for secondary end points and of subgroup analysis must be regarded as hypothesis-generating and not definitive.

Were All Patients Accounted for and an Intent-To-Treat Analysis Performed?
The answer to this question is again "yes." Although not all patients who were randomized to the surgical arm underwent nephrectomy, it is appropriate that the trial should reflect what might be achieved in clinical practice and, hence, use an intent-to-treat principle.

Are the Results Generalizable?
Here there are more concerns about the study. Although the subjects had a common disease, the study suffered from slow accrual: 246 patients were recruited from 80 institutions in more than 7 years, which averages to about one patient per institution every 2 years. The reasons for this reluctance to enter patients onto this important study are unknown, but such a slow accrual raises the question that there may have been innate bias in the type of patient who was recruited. The trial was limited to patients with PS 0 or 1, and certainly the results must not be applied to patients with a lower PS.

Are the Results Biologically Plausible?
The reason for undertaking this study was apparently to test the hypothesis that patients would have a better response to interferon alfa-2b in the presence of minimal disease. However, since the study showed only a 3% response rate to interferon alfa in both arms, it seems unlikely that this is the mechanism underlying the improved survival. Of course, it is possible that interferon alfa could influence the survival of patients without them achieving a complete or partial response. This seems unlikely, however, because if interferon alfa were more active in the surgery arm of the trial, one would expect more objective responses in the surgical arm, and this was not found. It seems more likely that the survival difference, assuming that it is not due to chance, occurred because of decreased morbidity and mortality from local problems caused by the primary tumor.

HOW STRONG IS THE EVIDENCE PROVIDED BY THE SWOG 8949 TRIAL?

The P value for the difference between survival curves was reported to be .025, using a stratified log-rank test, one-sided. The investigators should be criticized for using one-sided statistics. Although they present an argument that they are only interested in a result that shows better outcome for the experimental nephrectomy group, at least two counterarguments can be advanced in favor of two-sided statistics. First, nephrectomy could have been harmful, so a result in the opposite direction was distinctly possible. Second, in general, we need more, rather than less, stringent statistics to decrease the probability of finding false-positive results, as will be discussed below.

The two-sided P value for the difference in the survival curves obtained is .05. Given this relatively borderline P value, it is important to address the probability that the SWOG 8949 trial might be a false-positive study.

Some of the causes of false-positive studies are listed in Table 2. ² At least three of these are relevant to the SWOG 8949 study.

Low Prior Probability of a Positive Result
In general, true advances in oncology that lead to improvements in survival are quite rare, so the prior probability of expecting a positive result in any given trial is quite small. Although there are some differences between clinical trials and diagnostic tests to which Bayes’ theorem is more usually applied,³ a single positive result on a background of low prior probability raises the probability that the positive result is true by a relatively small amount.^2,4 This can be understood by considering a concrete example proposed by Mahesh Parmar, MD, of the Medical Research Council (MRC), London, United Kingdom (personal communication).

Suppose that 200 trials are conducted, with a positive result to be declared if the P value is < .05 and there is a power of 90%, and only 20 trials (10%) are testing strategies where there is a true difference in outcome. Then, if one-sided statistics are used, as in the SWOG 8949 trial, there will be 18 true-positive trials reported (ie, 90% of 20), but there will be nine false-positive trials reported (ie, .05 x 180).

Thus, in this hypothetical example, one in three trials reported as positive (ie, with a reported P value < .05), no matter how well done, would be a false-positive trial because of the low prior probability of a positive result. The prior expectation of a survival difference from the use of nephrectomy in patients with metastatic renal cancer is a matter of opinion. However, the "encouraging " results from prior phase II studies need to be counterbalanced by the rarity with which improvements in survival have been demonstrated in phase III trials for patients with this type of cancer.

Publication or Presentation Bias
Trials that seem to be positive are inherently more interesting to clinicians and scientists than trials that seem to be negative.^5-8 Thus, it is not surprising that the Program Committee of the American Society of Clinical Oncology Annual Meeting is likely to select more positive trials for presentation at the Plenary Session than negative trials.

In order to illustrate this effect, I undertook a brief review of randomized clinical trials with more than 100 patients that were presented at the American Society of Clinical Oncology Annual Meeting in 1998. Trials were classified according to whether they were positive (ie, they showed a statistically significant difference in their primary end point), negative (they did not show such a difference), or equivocal (there were some positive and some negative results and the primary end point was not clear). These data are listed in Table 3 and demonstrate the bias to select positive trials for presentation at oral sessions, particularly the Plenary Session.

WHAT TO RECOMMEND TO PATIENTS

In view of the results from two trials, it is now appropriate to recommend nephrectomy as an option for selected patients with metastatic renal cancer and high PS. The treatment is relatively safe when performed by an experienced surgeon on patients with good PS (although one may expect greater morbidity and mortality when the treatment is performed more commonly in the community). There is absolutely no basis for recommending surgery to patients of lower PS.

A second important question is whether interferon alfa should be used as standard systemic treatment. There are two randomized controlled trials in the setting of advanced disease, one performed by the British MRC and the second by a group in Finland. The MRC trial compared interferon alfa alone with no additional treatment,¹⁰ whereas the Finnish study compared vinblastine plus interferon alfa-2a with vinblastine alone.¹¹ The doses of interferon alfa used in these studies were quite high. Both of these studies showed an improvement in survival, albeit with considerable toxicity. The MRC study, and others, has shown that treatment with interferon alfa has a considerable impact to decrease quality of life.^12,13 Thus, the improvement in survival is counterbalanced by toxic and unpleasant side effects that reduce quality of life.

Of particular relevance to the SWOG trial presented here, three randomized controlled trials, thus far presented only in abstract form, have evaluated the use of adjuvant interferon alfa after nephrectomy in patients at high risk for subsequent metastatic disease.^14-16 All three of these trials randomized more than 250 patients, and all have failed to show an improvement in survival (one shows a trend to poorer survival in the group who received interferon alfa). This is a flagrant example of publication bias, in which three large negative studies have not been published: if these trials had been positive, one would have expected early publication in prominent journals.

In view of the above experience, I do not consider interferon alfa to be standard treatment for patients with metastatic renal cancer. The benefits are limited and the drug has toxicity. The same argument could be advanced for treatment with interleukin 2, and at Princess Margaret Hospital, we consider supportive care to be the standard and encourage the inclusion of patients onto trials that evaluate new therapies. New agents and new approaches to therapy are desperately needed in this common disease, which is so resistant to all types of currently available systemic therapy.

REFERENCES

1. Flanigan RC, Blumenstein BA, Salmon S, et al: Cytoreduction nephrectomy in metastatic renal cancer: The results of Southwest Oncology Group trial 8949. Proc Am Soc Clin Oncol 19: 2a, 2000 (abstr 3)

2. Tannock IF: False-positive results in clinical trials: Multiple significance tests and the problem of unreported comparisons. J Natl Cancer Inst 88: 206-207, 1996

3. Pater JL, Willan AR: Clinical trials as diagnostic tests. Control Clin Trials 5: 107-113, 1984[Medline]

4. Parmar MKB, Ungerleider RS, Simon R: Assessing whether to perform a confirmatory randomized clinical trial. J Natl Cancer Inst 88: 1645-1651, 1996[Abstract/Free Full Text]

5. Simes RJ: Publication bias: The case for an international registry of clinical trials. J Clin Oncol 4: 1529-1541, 1986[Abstract/Free Full Text]

6. Begg CB, Berlin JA: Publication bias and dissemination of clinical research. J Natl Cancer Inst 81: 107-115, 1989[Abstract/Free Full Text]

7. Easterbrook PJ, Berlin JA, Gopalan R, et al: Publication bias in clinical research. Lancet 337: 867-872, 1991[Medline]

8. Angell M: Negative studies. N Engl J Med 321: 464-466, 1989[Medline]

9. Mickisch GH, Garin A, Madej M, et al: Tumor nephrectomy plus interferon alpha is superior to interferon alpha alone in metastatic renal cell carcinoma. J Urol 163: 176, 2000 (abstr, suppl 4)

10. Interferon-alpha and survival in metastatic renal carcinoma: Early results of a randomised controlled trial—Medical Research Council Renal Cancer Collaborators. Lancet 353: 14-17, 1999[Medline]

11. Pyrh|onen S, Salminen E, Ruutu M, et al: Prospective randomized trial of interferon alfa-2a plus vinblastine versus vinblastine alone in patients with advanced renal cell cancer. J Clin Oncol 17: 2859-2867, 1999[Abstract/Free Full Text]

12. Trask PC, Esper P, Riba M, et al: Psychiatric side effects of interferon therapy: Prevalence, proposed mechanisms and future directions. J Clin Oncol 18: 2316-2326, 2000[Abstract/Free Full Text]

13. Hancock B, Griffiths G, Ritchie A, et al: Updated results of the MRC randomised controlled trial of alpha interferon vs MPA in patients with metastatic renal cancer. Proc Am Soc Clin Oncol 19: 340a, 2000 (abstr 1336)

14. Porzsolt F: Adjuvant therapy of renal cell cancer with interferon alfa-2A. Proc Am Soc Clin Oncol 11: 202, 1992 (abstr 622)

15. Trump DL, Elson P, Propert K, et al: Randomized controlled trial of adjuvant therapy with lymphoblastoid interferon in resected, high-risk renal cell carcinoma. Proc Am Soc Clin Oncol 15: 253, 1996 (abstr 648)

16. Giorgio P, Piva L, Boracchi P, et al: Adjuvant interferon (IFN) to radical nephrectomy in Robson’s stage II and III renal cell carcinoma (RCC): Contradictory results in pN0 and in pN2-3. Proc Am Soc Clin Oncol 18: 332a, 1999 (abstr 1276)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Facebook    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				W. J. Urba and W. G. Alvord Are All Hypotheses Generated Before Data Analysis Prospective? J. Clin. Oncol., March 15, 2002; 20(6): 1431 - 1433. [Full Text] [PDF]

This Article Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Save to my personal folders Download to citation manager Rights & Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Tannock, I. F. Search for Related Content PubMed PubMed Citation Articles by Tannock, I. F. Social Bookmarking                            What's this?