|
|||||
|
|
||||||
Originally published as JCO Early Release 10.1200/JCO.2005.05.948 on August 8 2005 © 2005 American Society of Clinical Oncology.
A Process for Measuring the Quality of Cancer Care: The Quality Oncology Practice InitiativeFrom Oncology Hematology Care, Cincinnati, OH; Virginia Cancer Institute, Richmond; American Society of Clinical Oncology, Alexandria; American Society for Therapeutic Radiology and Oncology, Fairfax, VA; California Cancer Care, Greenbrae, CA; Oncology Associates, Cedar Rapids, IA; N Shore Cancer Center, Peabody, MA; Boston Baskin Cancer Group, Memphis, TN; Louisiana Oncology Associates, Lafayette, LA; University of Cincinnati College of Pharmacy; Simone Consulting Company, Dunwoody, GA Address reprint requests to Michael N. Neuss, Oncology Hematology Care, 4725 E Galbraith, Suite 320, Cincinnati, OH 45236; e-mail: mneuss{at}ohcmail.com
PURPOSE: The Quality Oncology Practice Initiative (QOPI) is a practice-based system of quality self-assessment sponsored by the participants and the American Society of Clinical Oncology (ASCO). The process of quality evaluation, development of the pilot questionnaire, and preliminary results are reported. METHODS: Physicians from seven oncology groups developed medical record abstraction measures based on practice guidelines and consensus-supported indicators of quality care. Each practice completed two rounds of records review and received practice and aggregate results. Mean frequencies of responses for each indicator were compared among practices. RESULTS: Participants universally, if informally, find QOPI helpful, and results show statistically significant variation among practices for several indicators, including assessing pain in patients close to death, documentation of informed consent for chemotherapy, and concordance with granulocytic and erythroid growth factor administration guidelines. Measures with universally high concordance include the use of serotonin antagonist antiemetics according to the ASCO guideline; the presence of a pathology report in the record; the use of chemotherapy flow sheets; and adherence to standard chemotherapy recommendations for patients with certain stages of breast, colon, and rectal cancer. Concordance with quality indicators significantly changed between survey rounds for several measures. CONCLUSION: Pilot results indicate that the QOPI process provides a rapid and objective measurement of practice quality that allows comparisons among practices and over time. It also provides a mechanism for measuring concordance with published guidelines. Most importantly, it provides a tool for practice self-examination that can promote excellence in cancer care.
Medical practice is learned through both study and the emulation of model practitioners.1 Once formal training is completed, it becomes more difficult to find "best practices" to follow or imitate. And although most physicians admit that the general level of care could be improved, they usually have a better opinion of their own abilities. This opinion, however, lacks an objective basis. This is despite many and varied efforts to evaluate and improve the quality of cancer care undertaken in the 5 years since the Institute of Medicine (IOM) publicized the problem in the landmark 1999 publication, Ensuring Quality Cancer Care.2 Many publications explore the nature of oncology care by "slicing and dicing" information derived from secondary sources that provide valuable information on outcomes and treatment patterns involving large groups of patients.3-5 For example, publications have reported on the frequency of procedures and complications by mining one of a number of large claims, diagnoses, or procedure databases.6 Others have described characteristics of care that relate to quality measures for a particular cancer diagnosis,7-9 but none of these have been able to investigate the care as prescribed by individuals.6 While important, these efforts provide limited insight to practicing oncologists about the quality of their own work. Several factors hinder attempts to measure quality at the practice or provider level. As few oncology-specific quality indicators have been developed or tested, comparison values or benchmarks are lacking for the individual practice that completes a measurement. The financial expense or time commitment to complete such a process may be difficult to support. Results indicating that a process is in need of improvement may be perceived as a medicolegal liability.10 Also, any one physician may see too few patients of a particular type to allow statistically meaningful measures.11 Nonetheless, we believed that a process for gathering data concerning the quality of oncology care in individual practices could be expeditiously designed and implemented, and that information derived from this process would help each practice bootstrap the level of its achievement to that of the best group. A small group of medical oncologists attempted to develop quality measures and a process for data collection, reporting, and sharing of innovations. Named the Quality Oncology Practice Initiative (QOPI), this group of oncologists devised medical record abstraction measures based on published practice guidelines and their own consensus-derived indicators of quality care. The QOPI system allows rapid feedback of performance data for each indicator, which can be compared with aggregate results. We here report the QOPI development process, present initial data from the pilot, and discuss how the preliminary results inform the ongoing evolution of this initiative.
The process of selecting experts to generate ideal attributes is called the "Delphi Method." Our process of selecting physicians and groups and designing measures is a modification of this method.12
Selection of Participating Groups
Selection of Quality Measures
Medical Record Selection and Review Process
Data Entry
Statistical Analysis
Patient Confidentiality, Health Insurance Portability and Accessibility Act, and Exemption From Consent For research involving the collection or study of existing data, documents, and records, the Department of Health and Human Services states that "if the information is recorded by the investigator in such a manner that the subjects cannot be identified directly, or through identifiers linked to the subjects," then the research is exempt from review by an institutional review board,19 and therefore, institutional review board approval was not required.
Table 1 presents each quality measure within a framework of quality elements as has been described by the IOM.20
Table 2 presents the characteristics of the practices that participated in the first two rounds of QOPI data collection. The practices varied in size and location, but all cared for medical oncology patients and administered chemotherapy on site. Most practices had active clinical research or quality improvement programs, and staff from those areas generally performed the medical record abstraction.
The primary analysis of QOPI data is presented in Table 3. The first columns show the frequency of concordance or "yes" responses. The middle columns of the table report the statistical comparison of the first and second rounds of data collection. These were completed only when the questions were identical in both rounds.
There was some change in agreement with the quality measures between survey rounds. These measures were addressing pain (though this did not achieve statistical significance with P = .10) and administering granulocyte colony-stimulating factors by the standards of the ASCO guideline14 (P = .01). The use of erythroid growth factors according to the ASCO/American Society of Hematology (ASH) recommendation17 showed decreasing guideline adherence from 72% to 60% between rounds, with statistical significance (P = .048). No differences were seen in measures with very high concordance/achievement in the first round (uses of antiemetics,16 availability of pathology reports, and use of flow sheets). Variation in concordance among the groups is presented in the last three columns of Table 3. The measures demonstrate statistically significant differences among the groups for eight of the 11 measures. The ranges are noteworthy. The use of granulocyte colony-stimulating factor according to ASCO guidelines14 ranged from 0% to 88%. Likewise, the documentation of consent for chemotherapy varied widely. Formal informed consent documents were found in a range from 2% of medical records in one practice to 100% in others, though some form of consent was documented in 62% of medical records at a minimum. Abstractors found a specific medical record notation of staging (including comments such as metastatic, incurable, or advanced as a surrogate for M1 or stage IV) approximately 75% of the time, though there was still a significant disparity among groups. On the other hand, those aspects of medical oncology documentation expected in a medical record, like a pathology report and the presence of a flow sheet, were highly utilized, and little variation was seen. There seemed to be good concordance regarding the use of serotonin antagonists to prevent nausea (95%), but the use of steroids with these medications range from 60% to 97%, despite a clear guideline recommendation.16 Concordance with National Initiative for Cancer Care Quality measures for chemotherapy in colorectal and breast cancer patients and hormonal therapy in breast cancer patients18 is presented in Table 4. In this small sample, data show 100% compliance for the initiation of chemotherapy treatment for patients with these cancers and 83% agreement for use of hormonal therapy for breast cancer.
The costs associated with QOPI medical record reviews and data submission were tracked. Practice expenses varied from approximately $4 to $14 per medical record reviewed; variability was due both to employee wages and time taken to complete the abstraction. The total practice costs are estimated to be less than $7,500, or just more than $1,000 per practice for the two rounds of medical record abstraction. Administrative costs supported by ASCO, totaling an estimated $40,000 included information technology support and legal fees, in addition to general administrative support.
The QOPI pilot provides a framework for practice-based oncology quality self-measurement, which could lead to improvement. Our experience demonstrates that medical oncology practices can measure processes of care indicative of quality, based on published practice guidelines and consensus, in a cost-effective and rapid fashion. The QOPI process permits an ongoing self-assessment tool to provide practice-level information that can be compared to aggregated practice results, prior observations, and/or best practices. Naturally, the method of restricting medical record selection concerning physician, practice location, or even type of patient (eg, race, payor, age), will expand the power of observations that can be made, though each will be specific to the criteria used to choose medical records for analysis. The IOM framework for types of quality issues establishes six domains of quality: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity of access.20 The IOM schema is only one method of categorizing quality, and it tends to minimize what many would consider the most important measures of true effectivenesssurvival and minimization of suffering. Unfortunately, as others have found, due to the very large numbers needed to demonstrate small but important differences in these areas, we have as yet been unable to design an indicator to look into these important measures.11 Of course, it is our hope to expand into this and other areas. Many of these pilot results strongly support ongoing assessment of QOPI quality indicators. For instance, although there is clear and widespread recommendation for oncologists to address the issue of pain with dying patients,13,20 our results show room for improvement. At best, as noted in the second round of data extraction, only 69% of patients had a medical record notation that pain was addressed in the last visit to the oncologist's office before death. Similarly, a basic precept of high-quality care is that the patient should consent to it,2,20 yet consent for the administration of chemotherapy is documented in only 92% (297 of 324) of patients overall. Because consent is such a basic element of quality, it is reasonable to expect 100% medical record documentation as a quality standard. The range seen here, from 62% to 100% (100% measured in five of the seven groups), demonstrates the potential for both improvement and achievement. While the range observed for completing staging was not as great, being from 78% to 93%, there is room for improvement here, too. The practices were similar in many regards; variation was not significant in the use of flow sheets when giving chemotherapy, use of serotonin antagonist antiemetics, or obtaining a copy of a pathology report. Although continued assessment of these indicators among these seven practices may not be necessary, a broader group of practices may show more variation. Adherence to clinical guidelines is voluntary, and there are few published studies regarding the rates of adherence to oncology guidelines. Often, these are single-institution studies,21 or ones that reflect results of surveys of highly selected practices or diagnoses.22,23 We anticipate that QOPI can offer important and ongoing insight to ASCO and the oncology community regarding guideline adoption and adherence. QOPI pilot data show a broad range of concordance with clinical guidelines, from 24% for the use of granulocytic growth factors to 97% for the use of a serotonin antiemetic when a patient was given highly emetogenic chemotherapy. QOPI did not provide follow-up to assess reasons for discordance with the guidelines; for instance, to determine whether any patients had contraindications to prophylactic steroid use. Since QOPI is implemented within the context of a busy practice, it is unlikely that data acquisition regarding antiemetic guideline compliance as detailed as that accomplished by DeMoor et al is achievable.23 The practice guidelines themselves are subject to change and the influence of confounding factors, as occurred with the expansion of inclusion criteria for Medicare guidelines for the use of erythroid factors during the course of the study (for example, in Ohio24 and New England25). The only measure to show less compliance in the second round concerned the use of erythroid growth factors. Coincidentally or not, payment guidelines became more inclusive during the period between the two surveys (a higher baseline hemoglobin could be used to support Medicare claims for payment), though the ASCO/ASH guideline did not change concurrently. Perhaps nothing an oncologist does is more charged with anxiety and importance than the recommendation for an apparently well patient to receive adjuvant therapy. It is therefore reassuring that preliminary data showed 100% concordance with the guidelines in the narrow subset of patients with breast and colorectal cancer for whom chemotherapy is unequivocally recommended. Measuring this indicator in a medical oncology practice may simply confirm the assertion that the problem of underutilization of chemotherapy seen in community surveys or Medicare audits is related to eligible patients not being referred to (or agreeing to visit) a medical oncologist. A broader survey of medical oncology practices will be required to know this with certainty. Therefore, this important measure will be continuously reassessed. There are limitations to the QOPI approach. QOPI indicators assess processes of care in selected clinical areas. While inclusion of stage-specific outcome measures would be ideal, cancer-related survival is extremely difficult to reliably measure.26 Further, cancer patient outcomes are influenced by factors beyond the behavior of the medical oncologist, including the practice of other providers, care provided in hospitals, patient preferences, and comorbidity. This tool permits oncologists to measure the processes they have control over and can work to improve. Medical record selection was not random, nor were attempts made to insure all offices of a practice or each doctor were equally represented in the data collection. (Upcoming rounds of QOPI data collection with broader scale implementation will request practices to perform medical record reviews at each office site and for individual practitioners when appropriate.) Further, these preliminary results were collated without audit verification of data extraction results. Of course, any quality review that relies on the written medical record may under- or overestimate what really happened to the patient. This is especially problematic when studying measures that are not associated directly with billing, such as addressing the issue of pain in a dying patient, where there is really no secondary reason for documentation. Finally, smaller groups and those without clinical research capabilities or employees comfortable with computer-based data entry may find it difficult to accomplish the tasks required here, though it is an over-riding goal that this process be kept simple. Overall, pilot results and participant feedback indicate that the QOPI survey instrument provides a relatively inexpensive, practical tool that can be used to help promote practice quality. This project is scalable; within the context and sponsorship of an international organization like ASCO, new practice participants can be identified, and infrastructure is sufficient to support required staff and technology. ASCO coordinates the effort by providing logistical support, accepting and analyzing data, and providing comparative information to practices that can be used for self-examination and improvement. ASCO is committed to maintaining the confidentiality of individual practice results and will release results only to the practice or in aggregate. With this support, QOPI measures are subject to rapid updates as new treatments or quality standards evolve, and it is our goal to update measures frequently, adding and subtracting measures as group consensus opinions change. The history of the project will allow new participants to compare their results with those of earlier participants, providing a reference standard that can itself be improved. This is a unique process developed within the context of the busy clinical practice. We hope to eventually offer this program to any medical oncology practice meeting basic criteria for inclusion such as computer familiarity, Internet access, and willingness to provide all physicians in the participating practice with feedback concerning performance. The process has considerable potential to rapidly determine practice patterns and inform the ASCO guideline development committees about what is happening in the field. It is possible that some of the survey questions could be adopted as performance measures on a national basis. Most importantly, the successful implementation of QOPI will help fill a national void by providing a practical means of quality self-assessment and improvement in oncology.
Although all authors completed the disclosure declaration, the following authors or their immediate family members indicated a financial interest. No conflict exists for drugs or devices used in a study if they are not being evaluated as part of the investigation. For a detailed description of the disclosure categories, or for more information about ASCOs conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.
Dollar Amount Codes (A) < $10,000 (B) $10,000-99,999 (C)
Authors' disclosures of potential conflicts of interest are found at the end of this article.
1. Rae A: Osler vindicated: The ghost of Flexner laid to rest. CMAJ 164:1860-1861, 2001 2. Hewitt M, Simone JV: Ensuring Quality Cancer Care. Washington, DC, National Academy Press, 1999 3. Desch CE, Penberthy LT: Using state and federal claims data to evaluate the patterns and costs of cancer care. Cancer Treat Res 97:53-69, 1998[Medline]
4. Earle CC, Burstein HJ, Winer EP, et al: Quality of non-breast cancer health maintenance among elderly breast cancer survivors. J Clin Oncol 21:1447-1451, 2003 5. Khuri SF, Daley J, Henderson W, et al: The Department of Veterans Affairs' NSQIP: The first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical careNational VA Surgical Quality Improvement Program. Ann Surg 228:491-507, 1998[CrossRef][Medline] 6. Hillner BE, Smith TJ: The quality of cancer care: Does the literature support the rhetoric? Report to the National Cancer Policy Board. http://www.iom.edu/Object.File/Master/12/799/0.pdf
7. Krzyzanowska MK, Weeks JC, Earle CC: Treatment of locally advanced pancreatic cancer in the real world: Population-based practices and effectiveness. J Clin Oncol 21:3409-3414, 2003 8. Hillner BE, McDonald MK, Desch CE, et al: A comparison of patterns of care of nonsmall cell lung carcinoma patients in a younger and Medigap commercially insured cohort. Cancer 83:1930, 1998[CrossRef][Medline] 9. Desch CE, Penberthy L, Newschaffer CJ, et al: Factors that determine the treatment for local and regional prostate cancer. Med Care 34:152-162, 1996[CrossRef][Medline] 10. Altman DE, C Clancy, RJ Blendon: Improving patient safety-five years after the IOM report. N Engl J Med 351:2041-2043, 2004
11. Hofer RP, Hayward RA, Greenfield S, et al: The unreliability of individual physician "report cards" for assessing the costs and quality of care of a chronic disease. JAMA 281:2098-2105, 1999 12. Okoli C, Pawlowski SD: The Delphi Method as a research tool: An example, design considerations and applications. Inf Manag 42:15-30, 2004
13. Jacox A, Carr DB, Payne R: New clinical-practice guidelines for the management of pain in patients with cancer. N Engl J Med 330:651-655, 1994
14. Ozer H, Armitage JO, Bennett CL, et al: 2000 update of recommendations for the use of hematopoietic colony-stimulating factors: Evidence-based clinical practice guidelines. J Clin Oncol 18:3558-3585, 2000
15. Bast RC, Ravdin P, Hayes DF, et al: 2000 update of recommendations for the use of tumor markers in breast and colorectal cancer: Clinical practice guidelines of the American Society of Clinical Oncology. J Clin Oncol 19:1865-1878, 2001
16. Gralla RJ, Osoba D, Kris MG, et al: Recommendations for the use of antiemetics: Evidence based, clinical practice guidelinesAmerican Society of Clinical Oncology. J Clin Oncol 17:2971, 1999
17. Rizzo JD, Lichtin AE, Woolf SH, et al: Use of epoetin in patients with cancer: Evidence-based clinical practice guidelines of the American Society of Clinical Oncology and the American Society of Hematology. J Clin Oncol 20:4083-4117, 2002 18. Malin JL, Schneider EC, Epstein AM, et al: Results of the National Initiative for Cancer Care Quality (NICCQ): How can we improve the quality of cancer care in the United States? J Clin Oncol (in press) 19. Code of Federal Regulations, Title 45: Public Welfare, Department of Health and Human Services, National Institutes of Health Office for Protection from Research Risks, Part 46: Protection of Human Subjects, 2001 20. Institute of Medicine Committee on Quality Health Care in America.Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC, National Academy Press, 2001 21. Chan A, Yau J, Huan S, et al: Adherence to clinical practice guidelines for adjuvant therapy of breast cancer. J Clin Oncol 22:55, 2004 (suppl, abstr 716) 22. Ottevanger PB, De Mulder PH, Grol RP, et al: Adherence to the guidelines of the CCCE in the treatment of node-positive breast cancer patients. Eur J Cancer 40:198-204, 2004 23. Demoor C, Cohen L, Eisenberg PD, et al: Oncologists' compliance with antiemetic guidelines (AEG) and outcomes of patients (pts) receiving emetogenic chemotherapy (CT). Proc Am Soc Clin Oncol 22:727, 2003 (abstr 2924) 24. Palmetto GBA: Erythropoiesis stimulating proteins for non endstage renal disease (non-ESRD) patients. http://www.palmettogba.com/palmetto/LMRPs_PartB_OHWV.nsf/fe068764f91f744085256a7200679fc1/720812412abd201585256e92005d7bd3?OpenDocument 25. Medicare NHIC: LCD for Epoetin Alpha (L3191). http://www.cms.hhs.gov/mcd/viewlcd.asp?lcd_id=18024&lcd_version=2&show=all
26. Begg CB, Shrag D: Attribution of deaths following cancer treatment. JNCI 94:1044-1045, 2002 Submitted November 12, 2004; accepted March 15, 2005.
Related Editorial
This article has been cited by other articles:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||
|
Copyright © 2005 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|