|
Advertisement |
|
|
||
 |
|
|||||
|
|||||
|
 |
||||||
|
||||||
|
||||||
|
Journal of Clinical Oncology, Vol 26, No 8 (March 10), 2008: pp. 1324-1330 © 2008 American Society of Clinical Oncology. DOI: 10.1200/JCO.2007.14.0673 Nomograms to Predict Serious Adverse Events in Phase II Clinical Trials of Molecularly Targeted Agents
From the Princess Margaret Hospital Phase II Consortium, Toronto, Canada Corresponding author: Lillian L. Siu, MD, FRCPC, Department of Medical Oncology and Hematology, Princess Margaret Hospital, University Health Network, 610 University Ave, Ste 5-718, Toronto, Ontario, M5G 2M9, Canada; e-mail: lillian.siu{at}uhn.on.ca
Purpose A tool that quantifies the risk of treatment-related toxicity based on individual patient characteristics can augment the informed consent process and safety monitoring in the setting of phase II cancer treatment trials of molecularly targeted agents (MTAs). Methods A regression model was constructed to predict the risk of a serious adverse event (SAE) with an MTA and presented as a nomogram. Estimation of risk can be performed by integrating risk estimates from the nomogram and from a reference or average patient. Internal validation was performed using bootstrapping techniques. Results A total of 578 patients were treated with one of 14 MTAs given alone or in combination on one of 27 clinical trials performed by the Princess Margaret Hospital Drug Development Program between 2001 and 2006. Approximately 50% and 24% of patients experienced an SAE and an attributable SAE (SAEatt) during cycle 1, respectively. Albumin, lactate dehydrogenase (LDH), number of target lesions, prior radiotherapy, Charlson score, age, and performance status were included in the optimal model as predictors of a cycle 1 SAE, whereas the number of prior chemotherapy regimens, baseline creatinine, LDH, prior radiotherapy, Charlson score, body-surface area, and performance status were included as predictors of an SAEatt. Moderate-good internal validity was demonstrated, with area under the curve estimates ranging from 56.7% to 86.1% for all SAEs and 63.0% to 89.7% for SAEatts. Conclusion A regression model was constructed that predicts the SAE and SAEatt risk for an individual patient during cycle 1 of phase II trial treatment with moderate to good internal validity. External validation is still required.
The ethical conduct of a clinical trial is a complex process requiring risk assessments by physicians who administer the therapy prescribed by the protocol, by patients who receive such therapy, and by independent bodies such as regulatory agencies and institutional review boards. Physicians, patients, and institutional review board members have been demonstrated to have different risk tolerances when evaluating potential patient enrollment in a cancer treatment trial.1 What physicians consider as potentially too toxic, a patient may consider as acceptable; however, for most clinical trials, risk can only be evaluated in general terms. With early-phase trials, there are often only limited data available to provide estimates of risk for an individual patient. By evaluating each patient's clinical characteristics and all available information about the trial therapy, physicians subjectively quantify risk and determine whether they believe the patient is suitable for the trial. Although most phase II trials are preceded by phase I trials, considerable uncertainty remains regarding individual risks for phase II trial participants. There may be little safety information available, sometimes from only a few patients enrolled at the recommended dose level in a previous phase I trial. From the efficacy view point, in contrast with classic phase I trials,2 phase II trial participants may expect a certain potential for therapeutic benefit. Furthermore, patients considering enrollment into phase II clinical trials generally can choose to receive conventional treatment outside of the trial setting, unlike phase I trial participants who typically have exhausted standard therapeutic options. A tool that quantifies risk objectively could greatly improve the ability of a physician to assess risk and provide patients additional information about treatment risks so they can weigh both the expectations of benefit and risk when considering enrollment onto phase II trials. The development of molecularly targeted agents (MTAs) has emerged as a key strategy in cancer therapeutics. Substantial literature has been published on novel clinical trial designs and end points that may better evaluate the antitumor activity of MTAs than those used for conventional cytotoxic agents.3,4 Theoretically, MTAs are more specific in their effects on cancer targets and, consequently, should lead to fewer toxicities and serious adverse events (SAEs). The evaluation of risk factors or markers of susceptibility to adverse treatment effects in early-phase cancer clinical trials has been described by Rogatko et al.5 In this study, which analyzed data from 23 therapeutic phase I and II trials involving primarily cytotoxic agents, certain pretreatment characteristics were identified as predictors of toxicity. Similar assessments of individual risk in early-phase trials of MTAs have not been reported. This study was thus performed to investigate the individual risk of patients enrolling onto phase II clinical trials of MTA-based regimens. To allow for greater application of our results, a user-friendly tool was developed using nomograms, which enable a visual interpretation of the statistical model. Clinicians can use these nomograms to calculate individual risks directly using a pen, paper, and ruler. Ultimately, it is hoped that this method could lead to a better understanding of an individual's risk before enrollment onto an MTA-based phase II cancer clinical trial.
This study was approved by the University Health Network Research Ethics Board. Data for all clinical trial patients administered through the Drug Development Program at Princess Margaret Hospital (PMH) are included in a common database. These include data from all patients enrolled onto trials conducted by the PMH phase II Consortium as the lead group in the National Cancer Institute/Cancer Therapy Evaluation Program–sponsored N01 phase II trials contract. The database is Oracle-based (Redwood Shores, CA), password protected, and allows users to enter data via a web interface. Data monitoring procedures are performed based on PMH Drug Development Program's standard operating procedures, and a number of built-in data checks are performed to comply with data submission policies. All patients who received treatment as part of a phase II trial, or the phase II portion of a phase I/II trial, evaluating an MTA alone or in combination for a solid tumor from December 2001 until September 2006 were included in this analysis. Patient characteristics at baseline, including selected laboratory and biochemical data, were collected, along with adverse event data. Adverse events were graded according to the National Cancer Institute Common Toxicity Criteria version 2 or Common Terminology Criteria for Adverse Event version 3, depending on the version in use at the time of study initiation. Attribution of each adverse event to MTA was assigned by the treating physician or designated based on standard operating procedures. An SAE was defined as any grade 3 or higher nonhematologic adverse event or any grade 4 or higher hematologic adverse event of any attribution. The primary aim of this retrospective study was to create a prediction model for whether a patient treated on a phase II clinical trial of an MTA-based regimen in solid tumors would experience an SAE during cycle 1. Nomograms were constructed based on these models to make them clinically useful and allow a simple, visual interpretation. A secondary aim was to create a similar prediction model for SAEs deemed at least possibly attributable (SAEatt) to the MTA under study.
Statistical Methods Generalized estimating equations (GEE)8 and logistic regression were used to investigate whether selected characteristics were predictive of an SAE. GEEs account for the correlation that occurs between patients treated with the same treatment. In other words, one can hypothesize that a patient will be more likely to experience an SAE if one knows that a different patient treated with the identical treatment regimen also experienced an SAE. This correlation between patients treated with the same regimen is estimated by GEE models using a compound symmetry structure, which means that the correlation between any two patients treated with a given treatment is equivalent to the correlation between any two other patients treated with the same treatment. A forward stepwise selection process was used to construct an optimal model of predictors and an entry/removal criterion of P = .20. The process was repeated using only patients for which complete data were available yielding similar results, thus only results from the full data set are reported. Potential interactions between variables included in the optimal model were also investigated. Nomograms9 were constructed for visual interpretation of the model. Clinicians can use an estimated baseline risk (based on all prior data available) for a reference patient and calculate the estimated odds ratio via the nomogram for a patient considering enrollment onto a clinical trial. Multiplying the a priori odds estimate for the reference patient by the increased/decreased odds ratio from the nomogram for the potential clinical trial patient gives the posterior odds for the individual patient experiencing an SAE, from which one can obtain the final risk estimate in terms of probabilities. Validation of the model was performed using bootstrapping methods.10 By bootstrapping, one simulates additional data sets as if the data sets had been collected repeatedly by future similar studies. For each simulated data set, the optimal model was applied and the area under the curve (AUC) calculated. The mean AUC across simulated models was calculated along with 95% bias-corrected and accelerated (Bca) CIs and this estimates the predictability of the model, with 0.50 indicating no better than chance prediction and 1.00 indicating perfect predictability. For this analysis, 1,000 bootstrap samples were taken, and bootstrapping was performed on the largest trials (subjectively restricted to trials with 35 or more patients treated), to permit sufficient numbers of patients with and without SAEs and, therefore, reasonable estimation of the AUC. All statistical tests were two-sided.
Five-hundred seventy-eight patients were treated with one of 27 MTA-based phase II trials (Table 1) for solid tumors between 2001 and September 2006. The median age of patients was 60 years (range, 23 to 84 years), and 317 patients (54.8%) were male. The most commonly treated tumors were head and neck (150 patients, or 26%), ovarian (89 patients, or 15%), and colorectal (84 patients, or 15%). Patient characteristics are listed in Table 2.
Just under half (288 patients, or 49.8%) experienced a cycle 1 SAE, and 141 patients (24.4%) experienced a cycle 1 SAEatt. Albumin, LDH, number of target lesions, whether the patient had undergone prior radiotherapy, Charlson comorbidity score, age, and ECOG performance status were all included in the optimal model as statistically significant predictors of cycle 1 SAE on multivariate analysis (Table 3). The nomogram that illustrates this model visually is shown in Fig 1A. For SAEatt, the number of prior chemotherapy regimens, baseline creatinine as a factor of ULN, LDH as a factor of ULN, whether the patient had undergone prior radiotherapy, Charlson comorbidity score, body-surface area, and ECOG performance status were all included in the optimal model as predictors on multivariate analysis (Table 3), and the associated nomogram is shown in Fig 1B. No interaction term was found to significantly increase the predictability of the model.
To read the nomogram, one must obtain the value of each clinical factor for the patient of interest. For example, for Fig 1A, which relates to SAEs of all attributions, one must obtain the albumin value for the patient and find it on the horizontal line labeled as "albumin ULN" (albumin as a factor of ULN). Draw a straight line up until this intersects the line labeled as "points," that value at the point of intersection denotes the number of points for the albumin factor. By repeating this process for each factor, one will have a points score for each factor, and by summing these, one has the total points for the patient of interest. By locating the value of the total points on the horizontal line labeled as "total points" and drawing a straight line down, one can then obtain the estimated odds ratio for the patient by locating the position where this line intersects the "odds ratio" line. To interpret the odds ratio, one must estimate a baseline risk score for a reference patient who has odds ratios of approximately 1.0. A list of theoretical reference patients who have odds ratios of approximately 1 is shown in Table 4 for all SAE and in Table 5 for SAEatt. To estimate baseline risk, one must subjectively evaluate the probability that a reference patient will have an adverse event. This depends on the data available, including whether a maximally tolerated dose was obtained in previous phase I trials. With more information, baseline risk estimation will be improved. With little information, one could choose a range of possible baseline risks (eg, 5%, 20%, 50%, and 75%) and evaluate the individual risk under each scenario. By multiplying the odds ratio for the clinical trial patient with the estimated risk of the reference patient, one can obtain the estimated risk for the clinical trial patient (see examples).
Four trials in the database with 35 or more patients (sample sizes of 46, 38, 36, and 39 patients) were used for bootstrapping purposes. The estimated mean 95% Bca Cl AUCs for these four trials across bootstrap samples were 67.7% (95% Bca CI, 52.5% to 83.5%), 86.1% (95% Bca CI, 70.1% to 97.7%), 71.9% (95% Bca CI, 52.5% to 90.2%), and 56.7% (95% Bca CI, 34.6% to 79.4%) for predicting a cycle 1 SAE. For predicting SAEatt, the estimated mean (95% Bca CI) AUCs were 89.7% (95% Bca CI, 76.3% to 97.9%), 77.0% (95% Bca CI, 62.2% to 91.3%), 63.0% (95% Bca CI, 42.5% to 83.3%), and 78.4% (95% Bca CI, 59.5% to 83.3%). The actual AUCs for these trials were 68%, 86%, 72%, and 57% for predicting a cycle 1 SAE and 90%, 77%, 63%, and 78% for predicting cycle 1 attributable SAE.
Algorithm for Estimating Risk
Hypothetical Examples (2) Using the nomogram in Fig 1A, a patient might calculate his/her odds ratio to be 3.0 (3) An individual patient's risk is obtained by multiplying the odds ratio by the baseline risk (written as odds), that is 3.0 multiplied by 1:2, which is (3 x 1):2 = 3:2. This can be interpreted as following: if five (three + two) identical patients with the same risk factors were enrolled, then three patients would experience an SAE and two patients would not; the odds are 3:2 for the patient experiencing an SAE, or 60%. Example 2. If the baseline risk was estimated to be 25% (equals odds of 1:3), and a patient calculated his/her odds ratio to be 0.4, then his/her individual risk would be (0.4 x 1):3 = 0.4:3. In terms of percentage risk, this patient would then have 0.4/3.4 = 11.8% risk. To provide more robust estimates for the potential trial patient, one might consider examining a range of estimated risks for the reference patient, particularly for the first few patients, when the baseline risk might be based on little prior information. For example, one might think the baseline risk is 33% (odds of 1:2), but because of lack of prior knowledge, one might consider the baseline risk to be as low as 10% (odds of 1:9) or as high as 50% (1:1). If a patient calculated their nomogram-derived odds ratio to be 3.0, then his/her estimated individual risk is (3.0 x 1):2 = 3:2 or 60%, but could be as low as (3.0 x 1):9 = 3:9 or 25% or as high as (3.0 x 1):1 = 3:1 or 75%.
Patients considering enrollment onto a clinical trial have many factors to consider, including the likelihood of the treatment being effective, the time and cost involved, other alternatives, and the risk of treatment-induced toxicity. Quantifying the risk of toxicity related to the study treatment can be exceedingly difficult, particularly given the paucity of data available for early-phase cancer clinical trials. Treatments evaluated in the phase II trial setting may have been administered to only a handful of patients previously, thus when asked about the risk-to-benefit ratio, the clinician often cannot provide much data-driven guidance. In our study, the likelihood of experiencing an SAE during the first cycle of treatment for patients enrolled onto a phase II clinical trial of an MTA-based regimen is quantified. On the basis of the patient-specific odds ratio derived from the nomogram and an estimation of the average risk of a reference case, one could calculate an objective estimate of the individual risk. Given the uncertainty surrounding the derivation of the average risk, a range of estimated risks can be used to yield a more robust appraisal of the true risk. This risk can also be put into context by estimating the risk of an SAE for the patient if he/she was to receive the standard treatment, for which data are more readily available. This understanding of increased/decreased risk could greatly enhance the informed consent process. Additionally, although stringent patient safety monitoring for all patients is expected in clinical trials, the information of some participants being at increased risk for adverse events would raise further awareness for both physicians and patients. This analysis has its limitations. Because this was a retrospective analysis, the regression estimates are optimal for this particular data set, and it is unknown how well the results will hold when evaluated with external data. Bootstrapping was performed to internally validate the model, with moderate to good results; however, it is still possible that data from trials of different MTAs will give different results. Application of the results from our study to trial patients should await external validation. It is our aim to perform a future analysis, possibly with the collaboration of a cooperative group with large patient databases. One must also be careful about extrapolating the results from this study. Data for this analysis were based on patients who entered a phase II clinical trial, and because of selection bias, trial patients are typically fitter and in better condition than the average patient. Further, our results may not be generalizable to unselected patients treated off-study, because there may be characteristics that distinguish trial participants from other patients. Although the length of cycle 1 varied between trials from 21 to 28 days, adverse events attributable to treatment generally occur relatively early in a cycle, and the effect of this difference on our findings is therefore likely negligible. During the 5 years over which these trials occurred, the toxicity criteria used also became revised and updated, with minor changes in the description of some adverse events. The definitions of severity grades, however, did not change, with grade 3 being severe and grade 4 being life-threatening or disabling. Thus it is unlikely that there were any significant adverse events in our database that would have been changed in their SAE classifications based on the toxicity criteria version. Finally, although a wide range of MTAs were included in this study from 27 different phase II clinical trials, it is possible that a novel MTA may have an entirely different adverse event profile, for which our model does not apply. For instance, a new agent may have a mechanism of action that interacts with blood pressure, thus the likelihood of an SAE is highly dependent on the patient's baseline blood pressure. The baseline risk for any patient would need to be specifically adjusted based on this, or any factor, that is not accounted for directly in the model. In conclusion, a regression model was constructed that predicts the risk of SAE during cycle 1 for patients in a phase II clinical trial of an MTA-based regimen in solid tumors. Nomograms were constructed, which visually simplify the regression model and allow investigators to easily and quickly calculate an individual patient's risk. Internal validation of the regression model was performed using bootstrapping with moderate to good validity; however, external validation is still required.
The author(s) indicated no potential conflicts of interest.
Conception and design: Gregory R. Pond, Lillian L. Siu, Carol A. Townsley Financial support: Lillian L. Siu, Malcolm Moore Provision of study materials or patients: Lillian L. Siu, Malcolm Moore, Amit Oza, Hal W. Hirte, Eric Winquist, Glenwood Goss, Gary Hudes Collection and assembly of data: Gregory R. Pond Data analysis and interpretation: Gregory R. Pond, Carol A. Townsley Manuscript writing: Gregory R. Pond, Lillian L. Siu, Eric Winquist Final approval of manuscript: Gregory R. Pond, Lillian L. Siu, Malcolm Moore, Amit Oza, Hal W. Hirte, Eric Winquist, Glenwood Goss, Gary Hudes, Carol A. Townsley
Supported in part by National Cancer Institute Contract No. N01-CO-124001. Presented in part at the 43rd Annual Meeting of the American Society of Clinical Oncology, June 1-5, 2007, Chicago, IL. Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.
1. Agulnik M, Oza AM, Pond GR, et al: Impact and perceptions of mandatory tumor biopsies for correlative studies in clinical trials of novel anticancer agents. J Clin Oncol 24:4801-4807, 2006 2. Horstmann E, McCabe MS, Grochow L, et al: Risks and benefits of phase 1 oncology trials, 1991 through 2002. N Engl J Med 352:895-904, 2005 3. Parulekar WR, Eisenhauer EA: Phase I trial design for solid tumor studies of targeted, non-cytotoxic agents: Theory and practice. J Natl Cancer Inst 96:990-997, 2004 4. Korn EL, Arbuck SG, Pluda JM, et al: Clinical trial designs for cytostatic agents: Are new approaches needed? J Clin Oncol 19:265-272, 2001 5. Rogatko A, Babb JS, Wang H, et al: Patient characteristics compete with dose as predictors of acute treatment toxicity in early phase clinical trials. Clin Cancer Res 10:4645-4651, 2004 6. Charlson ME, Pompei P, Ales KL, et al: A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. J Chronic Dis 40:373-383, 1987[CrossRef][Medline] 7. Therasse P, Arbuck SG, Eisenhauer EA, et al: New guidelines to evaluate the response to treatment in solid tumors: European Organization for Research and Treatment of Cancer, National Cancer Institute of the United States, National Cancer Institute of Canada. J Natl Cancer Inst 92:205-216, 2000 8. Liang KY, Zeger SL: Regression analysis for correlated data. Annu Rev Public Health 14:43-68, 1993[CrossRef][Medline] 9. Kattan MW: Validating a prognostic model. Cancer 107:2523-2524, 2006[CrossRef][Medline] 10. Efron B, Tibshirani, RJ: An Introduction to the Bootstrap. London, United Kingdom, Chapman and Hall, 1994 Submitted August 27, 2007; accepted November 28, 2007.
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||
|
|||||||||||
|
|
Copyright © 2008 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|
TOP
ABSTRACT
INTRODUCTION
