|
|||||
|
|
||||||
Journal of Clinical Oncology, Vol 22, No 17 (September 1), 2004: pp. 3485-3490 © 2004 American Society of Clinical Oncology. DOI: 10.1200/JCO.2004.03.025 How Accurate Is Clinician Reporting of Chemotherapy Adverse Effects? A Comparison With Patient-Reported Symptoms From the Quality-of-Life Questionnaire C30From the Department of Medicine, Divisions of General Medicine and Geriatrics, and Hematology and Medical Oncology, Oregon Health & Science University; the Oregon Health & Science University Center for Ethics in Health Care; and Biostatistics & Bioinformatics Shared Resource, Oregon Health & Science University Cancer Institute, Portland, OR Address reprint requests to Tomasz M. Beer, MD, Department of Medicine, Oregon Health & Science University, Mail Code CR145, 3181 SW Sam Jackson Park Rd, Portland, OR 97239; e-mail: beert{at}ohsu.edu
PURPOSE: Adverse events in chemotherapy clinical trials are assessed and reported by clinicians, yet clinician accuracy in assessing symptoms has been questioned. We compared patient reporting of eight symptoms using a validated instrument, the European Organization for the Research and Treatment of Cancer Quality-of-Life Questionnaire C30 (QLQ-C30 or QLQ) with physicians' reporting of the same symptoms in the study's adverse events log. PATIENTS AND METHODS: Thirty-seven men with metastatic, androgen-independent prostate cancer enrolled onto a phase II trial of weekly calcitriol and docetaxel completed the QLQ every 4 weeks for up to 28 weeks. A patient-reported symptom was defined as an increase in a QLQ symptom score by at least 10 points (0 to 100 scale), sustained for at least 4 weeks. A physician-reported symptom was considered present if it was ever documented in the adverse event log.
RESULTS: Forty-nine (new or worsened) symptoms were detected by both physician and QLQ, 48 symptoms were detected by the physician alone, and 55 symptoms were detected by the QLQ alone. They agreed on the absence of a symptom in 102 instances of 254 possible opportunities. Their uncorrected agreement was 59.4%, but Cohen's CONCLUSION: Even in a tightly controlled clinical trial, physician reporting was neither sensitive nor specific in detecting common chemotherapy adverse effects. Tools for collecting patient-reported adverse event data in chemotherapy clinical trials should be developed.
Adverse events (AEs) are defined by the National Cancer Institute as "any unfavorable and unintended sign (including an abnormal laboratory finding), symptom, or disease temporally associated with the use of a medical treatment or procedure..."1 Physicians and study nurses routinely identify and report AEs in clinical trials of cancer chemotherapy. These AE data are then disseminated to physicians and used to estimate the risks of therapy that patients and their physicians weigh when choosing cancer treatment strategies. For many AEs, such as abnormal laboratory findings or specific disease states, clinician reporting makes sense. It is less clear if clinician reporting of symptoms is reliable when they are neither directly observable nor objective. In symptom research, the patient's report is the gold standard for assessing symptoms2-5 and studies consistently show that physicians and nurses underestimate (or, occasionally, overestimate) symptom frequency and severity in comparison with patient ratings.6-9 Physician reporting of adverse effects has been shown to be insufficiently reliable in studies of prostatectomy for localized adenocarcinoma of the prostate, for which investigators have found that patients report higher frequencies of erectile dysfunction and urinary incontinence after treatment than do physicians.10-12 Because these findings have been so consistent,13 patient report has replaced physician report as the accepted standard for assessing postprostatectomy complications. Common complications of prostatectomy, such as erectile dysfunction and incontinence, are often uncomfortable topics to discuss, particularly as complications of medical treatment. One might expect physician reporting of less intimate symptoms to be more accurate than reporting of these sensitive symptoms, but this is not necessarily the case. For patients with prostate cancer participating in the Cancer of the Prostate Strategic Urologic Research Endeavor cohort study, although overall concordance was low for all symptoms, urologists somewhat more closely matched patients in identifying erectile dysfunction (52% urologist-reported rate v 97% patient-reported rate) and incontinence (21% urologist-reported rate v 97% patient-reported rate) and less closely matched patients in identifying other symptoms such as bone pain (5% urologist-reported rate v 43% patient-reported rate), fatigue (10% urologist-reported rate v 75% patient-reported rate), and diarrhea or rectal urgency (2% urologist-reported rate v 33% patient-reported rate).7 The accuracy of physician reporting of adverse effects has not been scrutinized as carefully for chemotherapy as it has for prostate surgery. We hypothesized that physician reporting of chemotherapy adverse effects would lack sensitivity when compared with patient self-reporting. We then sought to test this hypothesis by comparing patient reporting of eight common cancer symptoms using a validated instrument, the European Organization for the Research and Treatment of Cancer (EORTC) Quality-of-Life Questionnaire C30, version 3.014 (QLQ) with a physician's reporting of the same symptoms in the study's AE log.
Patients Thirty-seven men with metastatic, androgen-independent prostate cancer enrolled in a phase II trial of weekly calcitriol and docetaxel administered for 6 consecutive weeks repeated every 8 weeks were observed for up to 28 weeks for this analysis. The details of the design of this clinical trial have been published previously.15 Informed consent was obtained from all patients and the study was approved by the institutional review boards of the Oregon Health & Science University and the Portland VA Medical Center (Portland, OR).
Measures
Procedures
Analysis Physician sensitivity and specificity for each symptom was calculated using the QLQ as the gold standard. Assuming that false-positives were much less likely than false-negatives, imputed incidence was calculated by counting a symptom as new or worsened if it was detected by either the QLQ or the physician. Multiple methodologic approaches have been described for analyzing longitudinal symptom and quality-of-life data.21,22 Although no single method is the best in every circumstance, contrasting the results obtained by different methods can aid data interpretation. If the results are consistent across methods, this supports the robustness of the findings.23 Accordingly, we also calculated the means of all eight QLQ scores for each symptom for each of four situations: QLQ-positive/physician (MD) -positive, the QLQ was positive and the physician did report that symptom as an AE; QLQ-positive/MD-negative, the QLQ was positive but the physician did not report that symptom as an AE; QLQ-negative/MD-positive, the QLQ was negative but the physician did report that symptom as an AE; or QLQ-negative/MD-negative, the QLQ was negative and the physician did not report that symptom as an AE. We hypothesized that patients who were both QLQ-positive and MD-positive should have the highest mean symptom scores, given that both methods identified the presence of a new or worsened symptom. Conversely, the patients who were both QLQ-negative and MD-negative should have the lowest mean symptom scores. To test this hypothesis, we performed analysis of variance analyses including contrast coefficients that reflected our hypothesis: QLQ-positive/MD-positive patients were assigned a coefficient of 1 and QLQ-negative/MD-negative patients were assigned a coefficient of 1. We did not have sufficient power to test the hypothesis that symptom scores would be intermediate in patients who were QLQ-positive/MD-negative or QLQ-negative/MD-positive.
Patient Characteristics Thirty-seven men were observed during treatment for up to 28 weeks (median, 28 weeks; mean, 24 weeks) of chemotherapy for this analysis. Their median age was 73 years (range, 46 to 83 years). Their baseline Eastern Cooperative Oncology Group performance scores were 0 or 1 for 75%, 2 for 22%, and 3 for 3%. Metastases were present in bone only (59%), bone and lymph nodes (32%), or lymph nodes only (8%). Eighteen patients had their docetaxel or calcitriol dose decreased by the investigator, and 12 patients discontinued chemotherapy early, after a median of 20 weeks. Compliance with QLQ questionnaires was 94%.
Symptoms Detected
Comparison of QLQ and Physician
When data for all eight symptoms were pooled,
Table 2 lists the mean QLQ symptom scores for the entire study period for patients whose symptoms were detected by the QLQ alone, physician alone, both, or neither. Our hypothesis (that QLQ-positive/MD-positive mean symptom scores would be higher than QLQ-negative/MD-negative symptom scores) was supported for all symptoms except fatigue and dyspnea.
Speculating that in some instances of MD-positive/QLQ-negative symptoms the physician might have labeled a symptom new when it had been present at baseline, we compared the baseline QLQ scores for the MD-positive/QLQ negative (proposed false-positives) with the baseline QLQ scores for the MD-negative/QLQ-negative (proposed true-negatives). The baseline means for MD-positive/QLQ-negative symptoms were 33.0 v 17.3 for the MD-negative/QLQ-negative symptoms (t = 3.48; df = 191; P < .001).
The physician did not report approximately one half of the symptoms identified by the QLQ as AEs, and the QLQ did not detect approximately one half of the symptom AEs reported by the physician. The failure of the physician to identify a substantial number of symptoms identified by the patient-reported instrument is not surprising considering the published experience with prostatectomy and other symptom research. To our knowledge, however, this is the first such analysis aimed specifically at examining the reliability of physician identification of AEs in a chemotherapy clinical trial.
On the basis of the excellent performance of the QLQ in validation studies, we were surprised to find that it failed to identify so many symptoms identified by the physician. We explored the possibility that the physician misclassified a baseline symptom as a new symptom. Our data support this possibility insofar as the baseline scores were significantly higher in the QLQ-negative/MD-positive symptoms than in the QLQ-negative/MD-negative symptoms. However, it is also possible that the QLQ was not sufficiently sensitive to detect a significant number of symptoms. Groenvold et al27 compared 46 female cancer patients' self-reported score for each QLQ question item with an observer's score that was based on the patient's response to the same question in an open-ended interview. They found the agreement for the 12 symptom items to be excellent (median, Other QLQ-negative/MD-positive cases may have been missed because the sustained 10-point increase was insufficiently sensitive. Like any instrument, the sensitivity and specificity of the QLQ depend on how a significant change is defined. We used a 10-point increase sustained over at least two measurements at least 4 weeks apart to increase the likelihood that the increased score represented an AE rather than simply a daily fluctuation in symptom severity. A less stringent criterion would be expected to improve the sensitivity, likely at the expense of specificity. It is reassuring that the mean symptom scores were highest when both the QLQ and MD were positive and lowest when neither the QLQ nor MD were positive, except in the patients with fatigue and dyspnea. Although fatigue is a well-recognized adverse effect of docetaxel,28 it is among the most difficult symptoms to define and measure.2 This ambiguity may have limited the sensitivity and specificity of either the physician or QLQ. In contrast, recognition that dyspnea is an adverse effect of docetaxel has been more recent, following reports of its association with pulmonary complications.29-31 This may explain why physician sensitivity was poorest for this symptom (23%) and why the physician-noted dyspnea was reported in only five patients compared with 13 detected by the QLQ. Finally, missed QLQs could explain some missed data points. Compliance with the QLQ was excellent, however, with 94% of questionnaires returned. This study has limitations that are important to note. First, although the QLQ has been used in more than 3,000 studies and has been extensively validated,22 it has not been used to identify AEs, per se. Second, the design of the QLQ allowed us to examine only eight adverse experiences rather than a comprehensive set of adverse effects of chemotherapy. Other obvious limitations of these data are that they represent a relatively small sample size and a data set from a single-institution clinical trial. However, this problem is shared by each clinician charged with recording AEs for clinical trialsthe sensitivity, specificity, reliability, and validity of their reports may vary among individuals and are in any case unknown. These findings should be confirmed in a larger, multi-institutional study. If verified, they call into question the reliability of chemotherapy AE reporting that relies solely on the physician and strongly suggest that a mechanism for patient self-reporting of adverse experience should be developed, validated, and incorporated into cancer chemotherapy trials. Because both methods identified potentially important symptoms that the other did not, our findings also suggest that patient self-reporting should complement rather than replace physician assessment of AEs. There are a number of methodologic challenges that need to be overcome before patient self-reporting of AEs in chemotherapy clinical trials can move forward. Lack of truly well-validated instruments, the problems presented by missing data,32 uncertainty about the optimal timing and frequency of administration of questionnaires,33 and uncertainty about the optimal threshold of positivity34 are among the issues that will need to be addressed. The need for improvement in AE reporting standards in oncology clinical trials was well articulated in a recent Journal of Clinical Oncology editorial.35 Our study suggests that validated methods for collecting and analyzing patient-reported AE data will be a necessary element of such an effort.
The authors indicated no potential conflicts of interest.
Supported in part by grant 3M01RR00334-33S2, and a Cancer Center Support grant (P30 CA 69533) from the National Institutes of Health. Authors' disclosures of potential conflicts of interest are found at the end of this article.
1. National Cancer Institute: Cancer Therapy Evaluation Program: NCI GuidelinesExpedited adverse event reporting requirements for NCI investigational agents. Bethesda, MD, 2001, pp 1-12 2. Ingham J, Portenoy RK: The measurement of pain and other symptoms, in Doyle D, Hanks G, MacDonald N (eds): Oxford Textbook of Palliative Medicine (ed 2). Oxford, Oxford University Press, 2001, pp 203-222 3. Grossman SA, Sheidler VR: Cancer pain, in Abeloff D, Armitage JO, Lichter SA, et al (eds): Clinical Oncology (ed 2). Philadelphia, PA, Churchill Livingstone, 2000, pp 539-554 4. Kroenke K, Casper G: National Institute of Dental and Craniofacial Research, National Institutes of Health: Somatization and symptoms evaluationSomatization and symptoms evaluation. http://symptomresearch.nih.gov/Chapter_16/index.htm
5. Patrick DL, Ferketich SL, Frame PS, et al: National Institutes of Health State-of-the-Science Conference Statement: Symptom management in cancerPain, depression, and fatigue, July 15-17, 2002. J Natl Cancer Inst 95:1110-1117, 2003 6. Sprangers MA, Aaronson NK: The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease: A review. J Clin Epidemiol 45:743-760, 1992[CrossRef][Medline] 7. Litwin MS, Lubeck DP, Henning JM, et al: Differences in urologist and patient assessments of health related quality of life in men with prostate cancer: Results of the CaPSURE database. J Urol 159:1988-1992, 1998[CrossRef][Medline] 8. Grossman SA, Sheidler VR, Swedeen K, et al: Correlation of patient and caregiver ratings of cancer pain. J Pain Symptom Manage 6:53-57, 1991[CrossRef][Medline] 9. Vogelzang NJ, Breitbart W, Cella D, et al: Patient, caregiver, and oncologist perceptions of cancer-related fatigue: Results of a tripart assessment surveyThe Fatigue Coalition. Semin Hematol 34:4-12, 1997[Medline] 10. McCammon KA, Kolm P, Main B, et al: Comparative quality-of-life analysis after radical prostatectomy of external beam radiation for localized prostate cancer. Urology 54:509-516, 1999[CrossRef][Medline] 11. Fowler FJ Jr, Barry MJ, Lu-Yao G, et al: Patient-reported complications and follow-up treatment after radical prostatectomy: The National Medicare Experience1988-1990 (updated June 1993). Urology 42:622-629, 1993[CrossRef][Medline]
12. Talcott JA, Rieker P, Propert KJ, et al: Patient-reported impotence and incontinence after nerve-sparing radical prostatectomy. J Natl Cancer Inst 89:1117-1123, 1997 13. Robinson JW, Moritz S, Fung T: Meta-analysis of rates of erectile function after treatment of localized prostate carcinoma. Int J Radiat Oncol Biol Phys 54:1063-1068, 2002[CrossRef][Medline]
14. Aaronson NK, Ahmedzai S, Bergman B, et al: The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 85:365-376, 1993
15. Beer TM, Eilers KM, Garzotto M, et al: Weekly high-dose calcitriol and docetaxel in metastatic androgen-independent prostate cancer. J Clin Oncol 21:123-128, 2003 16. Fayers PM, Aaronson NK, Bjordal K, et al: The EORTC QLQ-C30 Scoring Manual (ed 3). Brussels, Belgium, European Organisation for Research and Treatment of Cancer, 2001
17. Osoba D, Rodrigues G, Myles J, et al: Interpreting the significance of changes in health-related quality-of-life scores. J Clin Oncol 16:139-144, 1998
18. Osoba D, Tannock IF, Ernst DS, et al: Health-related quality of life in men with metastatic prostate cancer treated with prednisone alone or mitoxantrone and prednisone. J Clin Oncol 17:1654-1663, 1999 19. Cohen J: A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37-46, 1960[CrossRef] 20. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 33:159-174, 1977[CrossRef][Medline] 21. Matthews JN, Altman DG, Campbell MJ, et al: Analysis of serial measurements in medical research. BMJ 300:230-235, 1990 22. European Organization for the Research and Treatment of Cancer Study Group on Quality of Life: Guidelines for Assessing Quality of Life in EORTC Clinical Trials. Brussels, Belgium, 2002, pp 12 23. Qian W, Parmar MK, Sambrook RJ, et al: Analysis of messy longitudinal data from a randomized clinical trial: MRC Lung Cancer Working Party. Stat Med 19:2657-2674, 2000[CrossRef][Medline] 24. Feinstein AR, Cicchetti DV: High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 43:543-549, 1990[CrossRef][Medline] 25. Fayers PM, Weeden S, Curran D: QLQ-C30 Reference Values. Brussels, Belgium, European Organization for Research and Treatment of Cancer, 1998 26. Bruera E: Research into symptoms other than pain, in Doyle D, Hanks GWC, MacDonald N (eds): Oxford Textbook of Palliative Medicine (ed 2). Oxford, Oxford University Press, 1998, pp 180 27. Groenvold M, Klee MC, Sprangers MA, et al: Validation of the EORTC QLQ-C30 quality of life questionnaire through combined qualitative and quantitative assessment of patient-observer agreement. J Clin Epidemiol 50:441-450, 1997[CrossRef][Medline] 28. Taguchi T: An early phase II clinical study of RP56976 (docetaxel) in patients with cancer of the gastrointestinal tract. Gan To Kagaku Ryoho 21:2431-2437, 1994[Medline] 29. Behrens RJ, Gulley JL, Dahut WL: Pulmonary toxicity during prostate cancer treatment with docetaxel and thalidomide. Am J Ther 10:228-232, 2003[CrossRef][Medline] 30. Read WL, Mortimer JE, Picus J: Severe interstitial pneumonitis associated with docetaxel administration. Cancer 94:847-853, 2002[CrossRef][Medline]
31. Dunsford ML, Mead GM, Bateman AC, et al: Severe pulmonary toxicity in patients treated with a combination of docetaxel and gemcitabine for metastatic transitional cell carcinoma. Ann Oncol 10:943-947, 1999 32. Bernhard J, Cella DF, Coates AS, et al: Missing quality of life data in cancer clinical trials: Serious problems and challenges. Stat Med 17:517-532, 1998[CrossRef][Medline] 33. Pater J, Osoba D, Zee B, et al: Effects of altering the time of administration and the time frame of quality of life assessments in clinical trials: An example using the EORTC QLQ-C30 in a large anti-emetic trial. Qual Life Res 7:273-278, 1998[Medline] 34. Osoba D: A taxonomy of the uses of health-related quality-of-life instruments in cancer care and the clinical meaningfulness of the results. Med Care 40:III31-III38, 2002 (suppl 6)[Medline]
35. Trotti A, Bentzen SM: The need for adverse effects reporting standards in oncology clinical trials. J Clin Oncol 22:19-22, 2004 Submitted March 3, 2004; accepted June 3, 2004.
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||
|
Copyright © 2004 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|