Originally published as JCO Early Release 10.1200/JCO.2008.18.8011 on December 1 2008

This Article Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Save to my personal folders Download to citation manager Rights & Permissions Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Carter, R. E. Articles by Huang, P. Search for Related Content PubMed PubMed Citation Articles by Carter, R. E. Articles by Huang, P. Social Bookmarking                            What's this?

Cautionary Note Regarding the Use of CIs Obtained From Kaplan-Meier Survival Curves

Rickey E. Carter

Department of Biostatistics, Bioinformatics and Epidemiology, Medical University of South Carolina, Charleston, SC

Peng Huang

Oncology Biostatistics, Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, MD

Time to event or survival analyses are a staple of the medical literature and are thoroughly discussed in statistical texts^1,2 and in methodologic series published by clinical journals.^3-7 The motivation for this brief cautionary note stems from the original findings on the utility of computed tomography (CT) screening for the early detection of stage I lung cancer published by The International Early Lung Cancer Action Program (I-ELCAP)⁸ and the precipitating discussion in the literature.^9-11 While the subsequent commentary raised many valid concerns regarding the findings, there is one concern, a statistical concern, that has not been expressed and could shed some additional insight into the interpretation of the data. Accordingly, this cautionary note will explore the validity of the reported CI for the estimated 10-year survival rate and offer a suggestion to mitigate the overinterpretation of the estimated survival rate and associated CI.

Before a more formal examination of the considerations that may arise when analyzing survival data, it is necessary to examine the motivating example in more detail. The I-ELCAP study screened approximately 32,000 asymptomatic persons at risk for developing lung cancer between 1993 through 2005 using low-dose CT. A total of 484 diagnoses of lung cancer were made, and the median duration of follow-up was 40 months (range, 1 to 123 months). A total of 302 patients underwent surgical resection for the disease, and the 10-year survival rate in these patients was estimated to be 92% (95% CI, 88% to 95%).⁸ As this cautionary note will explain, this CI is likely not appropriate.

The I-ELCAP study used the nonparametric Kaplan-Meier estimator to derive an empirical estimate of the survival function since the data were right censored. In particular, the median follow-up period was 40 months, and only approximately 10% of the subjects at risk at baseline were still included in the risk set after 6 years of follow-up.⁸ While not specifically mentioned in the methodology section of the I-ELCAP report, it is reasonable to assume the Greenwood estimator for the SE of the survival function was computed since it is the default formula in PROC LIFETEST (SAS Institute, Cary, NC), the procedure to calculate the Kaplan-Meier survival curve in SAS (SAS Institute).¹² The Kaplan-Meier and Greenwood estimators share a common attribute adjustments to the estimates only occur where there is an event (a death in the case of the I-ELCAP study). More plainly, the estimated survival function and associated SE remain at the value associated with the most recent event until the next event occurs. A subsequent event causes a decrease in the estimated survival function thereby giving the Kaplan-Meier survival curve its familiar step-down appearance.

When interpreting an estimated survival curve, one should ensure that the size of risk set is not too small. A small risk set size could result in large variation in the estimation and difficulty in interpretation. Moreover, if a treatment is expected to delay the event onset time, caution should be exercised to ensure that adequate follow-up time has been obtained. That is, one should cautiously extrapolate the estimated survival probability and associated CI far beyond the last observed event time, particularly when the risk set is small and majority of treatment subjects were censored.

A very simple illustration of this caution can be drawn from the I-ELCAP study. In particular, only one of the 302 at-risk patients was followed for 10 years (123 months), but if this patient had an event, just 1 month later, the estimated survival probability at 124 months would be zero (according to the Kaplan-Meier product limit estimator, S (124) = 0.92 x (1-1)/1 = 0) and the associated CI would be from 0 to 0 since the SE would also be estimated to be zero. These estimates are in stark contrast to the reported 10-year survival estimates.⁸ Clearly, this sensitivity to single observation must be taken into account when the data are interpreted.

In light of the potential for a misleading interpretation of estimated survival rate and associated CI, a general recommendation can be made. One should cautiously interpret estimates obtained from a Kaplan-Meier survival analysis when the risk set is small. Pocock et al¹³ recommend, in the context of the graphical display of the survival curve, ensuring that the risk set size is at least 10% to 20% of the participants that have not yet experienced the event. This suggestion would appear appropriate in this case as well. In addition, caution should be exercised in general with both the Kaplan-Meier and Greenwood estimators when the risk set is small (say, lower than 30) since they both rely on some asymptotic theory that requires large risk set size. When the risk set becomes small, the asymptotic theory used to derive the formulas may not be valid, so the obtained estimates may be questionable.

In the context of the patients undergoing surgical resection in the I-ELCAP study, limited data is available to estimate the 10-year survival rate given the median length of follow-up for the entire cohort was only 40 months and only one of 302 patients was observed for more than 10 years. The risk set size dramatically decreases for each year beyond year 5. Accordingly, the quoted 10-year survival estimate of 92% (CI, 88% to 95%) is in fact a better representation of the 4- or 5-year survival rate and a longer follow-up period yielding a larger risk set is needed to better understand the impact of the resection on the 10-year survival rate.

AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The author(s) indicated no potential conflicts of interest.

AUTHOR CONTRIBUTIONS

Conception and design: Rickey E. Carter, Peng Huang

Manuscript writing: Rickey E. Carter, Peng Huang

Final approval of manuscript: Rickey E. Carter, Peng Huang

NOTES

published online ahead of print at www.jco.org on December 1, 2008

REFERENCES

1. Hosmer DW, Lemeshow S, May S: Applied Survival Analysis: Regression Modeling of Time-to-Event Data (ed 2). Hoboken, NJ, John Wiley & Sons Inc, 2008

2. Klein JP, Moeschberger ML: Survival Analysis: Techniques for Censored and Truncated Data. New York, NY, Springer-Verlag Inc, 1997

3. Altman DG, Bland JM: Time to event (survival) data. BMJ 317:468-469, 1998[Free Full Text]

4. Bradburn MJ, Clark TG, Love SB, et al: Survival analysis part II: Multivariate data analysis—an introduction to concepts and methods. Br J Cancer 89:431-436, 2003[CrossRef][Medline]

5. Bradburn MJ, Clark TG, Love SB, et al: Survival analysis part III: Multivariate data analysis—choosing a model and assessing its adequacy and fit. Br J Cancer 89:605-611, 2003[CrossRef][Medline]

6. Clark TG, Bradburn MJ, Love SB, et al: Survival analysis part I: Basic concepts and first analyses. Br J Cancer 89:232-238, 2003[CrossRef][Medline]

7. Clark TG, Bradburn MJ, Love SB, et al: Survival analysis part IV: Further concepts and methods in survival analysis. Br J Cancer 89:781-786, 2003[CrossRef][Medline]

8. Henschke CI, Yankelevitz DF, Libby DM, et al: Survival of patients with stage I lung cancer detected on CT screening. N Engl J Med 355:1763-1771, 2006[Abstract/Free Full Text]

9. Lock M, Rodrigues G: Computed tomographic screening for lung cancer. Can Fam Physician 53:1334-1336, 2007[Free Full Text]

10. Unger M: A pause, progress, and reassessment in lung cancer screening. N Engl J Med 355:1822-1824, 2006[Free Full Text]

11. Black W, Baron J: CT screening for lung cancer: Spiraling into confusion? JAMA 297:995-997, 2007[Free Full Text]

12. SAS Institute Inc: SAS/STAT User's Guide, Version 8. Cary, NC, SAS Institute Inc, 1999

13. Pocock SJ, Clayton TC, Altman DG: Survival plots of time-to-event outcomes in clinical trials: Good practice and pitfalls. Lancet 359:1686-1689, 2002[CrossRef][Medline]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Facebook    Reddit    Technorati    Twitter    What's this?

This Article Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Save to my personal folders Download to citation manager Rights & Permissions Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Carter, R. E. Articles by Huang, P. Search for Related Content PubMed PubMed Citation Articles by Carter, R. E. Articles by Huang, P. Social Bookmarking                            What's this?