|
Advertisement |
|
|
||
 |
|
|||||
|
|||||
|
 |
||||||
|
||||||
|
||||||
|
Journal of Clinical Oncology, Vol 26, No 11 (April 10), 2008: pp. 1893-1898 © 2008 American Society of Clinical Oncology. DOI: 10.1200/JCO.2007.14.2992 Improvement in Oncology Practice Performance Through Voluntary Participation in the Quality Oncology Practice Initiative
From the North Shore Medical Center, Salem, MA; Oncology/Hematology Care, Cincinnati, OH; American Society of Clinical Oncology, Alexandria, VA; Virginia Health Quality Center, Glen Allen, VA; California Cancer Care, Greenbrae, CA; and Simone Consulting, Atlanta, GA Corresponding author: Joseph O. Jacobson, MD, Department of Medicine, North Shore Medical Center, 81 Highland Avenue, Salem, MA 01970; e-mail: jjacobson{at}partners.org
Purpose The Quality Oncology Practice Initiative (QOPI) became available to all American Society of Clinical Oncology member physicians in 2006 as a voluntary medical oncology practice-based quality measurement and improvement project. QOPI assesses practice performance for a series of evidence- and consensus-based process measures, relying on practices to complete structured chart reviews and submit data via a secure Web-based portal. Methods This analysis focused on the 71 practices that participated in both the March and September 2006 data collections (7,624 charts abstracted in March and 10,240 in September). Among 33 measures common to both collections, five measures were closely correlated, and 28 are included in the final analysis. Composite scores were created for six different domains of care. Statistical significance was tested on both absolute changes and relative changes (relative failure reduction) of quality measures from baseline to follow-up and between the lower quartile and all other quartiles. Results Practice performance on individual measures varied between 18.8% and 98.6%. Mean overall performance as measured by a composite score increased from 78.7% in March to 82.3% in September (P < .05). Improvement was most marked among practices originally performing in the bottom quartile. Using a composite score, the absolute and relative performance for the bottom quartile improved by 27% and 35%, respectively, statistically superior to that of all others. Conclusion Practices that participated in QOPI demonstrated improved performance in self-reported process measures, with the greatest improvement demonstrated in initially low-performing practices.
Health care institutions are under intense pressure to deliver high-quality care that can be measured and improved. Cancer care, both at the institutional and practitioner levels, is not immune from this pressure. In 1997, the Institute of Medicine created a National Cancer Policy Board to assess the state of cancer care within the United States. The final report, Ensuring Quality Cancer Care, identified a need to create systems to assess this care.1 The American Society of Clinical Oncology (ASCO), in response to the National Cancer Policy Board report, initiated a 5-year study, the National Initiative on Cancer Care Quality (NICCQ), to assess the care of adult patients with breast and colorectal cancer treated within five geographic areas.2 This cross-sectional view of cancer care demonstrated relatively high adherence to generally accepted standards of care, but also demonstrated broad variation in performance for some measures.3 The Quality Oncology Practice Initiative (QOPI) was conceived by community medical oncologists as a voluntary means to assess and improve process of care in oncology practices. Begun in 2002 by ASCO as a pilot project, QOPI became available to all ASCO member medical oncologists and their practices in March 2006. QOPI measures processes of care semiannually at the practice level, relying on office personnel to collect data by retrospective chart abstraction using a structured questionnaire. De-identified data are entered into an analytic database via a secure website, and results are made available to practices that allow comparison of their performance to all other participants. The costs of data collection and reporting are borne by the practices. The administration of the program is funded by ASCO, and member oncologists' volunteer time for content and program support. The results of pilot data from QOPI were published in 2005.4 The report demonstrated that these self-selected oncology practices were willing to participate in a voluntary program of practice assessment, bearing the costs and meeting reporting deadlines. Results from these seven pilot practices confirmed the findings of the NICCQ, identifying high overall rates of compliance with a series of evidence-based and consensus guidelines, but also confirming great variability among practices for many measures. This analysis focuses on the first two QOPI data collections conducted after the program was opened nationally. We sought to determine whether practices that had voluntarily participated in the first data collection would continue their participation. We also sought to determine how continued participation in QOPI would affect practice performance.
Overview of QOPI In January 2006, ASCO members were invited to enroll in QOPI as a voluntary means to measure the processes of care within their ambulatory practices. Physicians were solicited by paper-based and electronic mailings and by journal and ASCO Web-based advertisements. Medical oncology practices enroll electronically at the ASCO website. At the time of enrollment, practices agree to participate in at least two semiannual data collections, and share the survey findings with their staff. The chart abstraction methodology has been previously described and is available online for review (www.asco.org/qopi).4,5 Practices generate a list of patients with invasive cancer seen within the practice in the preceding 6 months, proceeding backward in time from the date of analysis until the required number of records has been obtained. The number of full-time physician practitioners (or equivalent positions when part-time physicians are within the practice) determines the chart sample size. Additional records are identified as needed for disease- and domain-specific measures, as described later herein. Chart abstraction is performed by nursing staff, physicians, or research staff, with the stipulation that a physician should not abstract his or her own patient records. Abstracted data are entered directly on to a secure ASCO website during the 4-week data entry phase. Only de-identified data are entered in compliance with Health Insurance Portability and Accountability Act (HIPAA) requirements. Abstractors are prompted to answer questions using a branched-chain logic pathway based on specific responses. If, for example, a patient is noted to have received emetogenic chemotherapy, the abstractor is directed to a field asking whether proper antiemetic therapy was prescribed. Data logic checks are built into the Web-based collection tool to ensure completeness of data. Most of the data collected are dichotomous or based on limited data values. For the few fields with open text entry, validation checks were not built into the tool, but have since been added for future collection rounds. Data directly populate a Microsoft SQL Server database (Microsoft Corp, Redmond, WA). Within 4 weeks of the close of the data entry phase, practices may review the results of their data collection for each measure, with results compared with aggregated data from all participating sites.
Performance Measures The Measures Workgroup classified measures into a core set and several disease- and domain-specific sets. The core measures are composed of many of the "common sense" consensus-derived measures. Disease-specific sets include breast cancer, colorectal cancer, and non-Hodgkin's lymphoma. The domain-specific sets include end-of-life care and symptom/toxicity management. Thirty-three measures were common to both data collections. Five measures were excluded from the analysis because they were highly correlated with other measures. An end-of-life measure that assessed only a subset of the population was also excluded. All of the measures used for the two data collections are included in Table 1.
Statistical Analysis The primary outcome of interest for analyses was the change in rates in the performance in measures between the two data collection periods, March and September 2006. The null hypotheses included the following: (1) practices participating in QOPI do not show improvement over sequential measurement periods; and (2) lower-performing practices are no more likely than all others to improve. Measures, collected at the practice level for all participating centers, were analyzed using weighted mean comparisons at several levels. Results were grouped at the following levels: individual measure, practice performance, and composite score. Composite scores were calculated by combining measure results from all charts submitted at the individual practices into seven categories, including an overall composite score. The change in composite scores was calculated in two different ways. Absolute improvement was defined as the change in performance between the two data collections. Relative improvement was calculated as the relative failure rate, which was defined as the absolute change in performance divided by the difference in baseline performance and perfect performance.6,7
For composite scores, weighted means were created to account for differences in sample size for the different domains of care. The means or weighted means between collection periods at the various levels were compared for significant change using paired t tests. The statistical significance of the difference between two proportions (of quality measure) between periods and between groups (quartiles) was assessed by means of the z score or critical value ratio statistic. All tests were performed at P
Eighty-seven practices successfully completed the March 2006 data collection. A total of 9,357 charts were successfully entered into the QOPI database. One hundred thirteen practices completed data collection successfully in September 2006, with 14,291 charts abstracted. A total of 71 practices participated successfully in both data collections, abstracting 7,624 charts in March and 10,240 charts in September. The self-reported characteristics of those 71 practices, which constitute the study group, are summarized in Table 2, and are compared with the 16 practices that discontinued participation. The only statistically significant difference between the two groups is that practices that discontinued participation in QOPI saw a fewer number of patients each year (mean, 593 v 1,751 patients; P < .05).
Reported adherence to the measures is summarized in Table 1. Measure adherence varied between 18.8% and 98.6%. The lowest compliance for both data collections was for the measure assessing the proper use of aprepitant with highly emetogenic chemotherapy (an ASCO Clinical Practice Guideline recommendation). The highest compliance was for the appropriate use of rituximab for CD-20 positive lymphoma patients receiving CHOP (cyclophosphamide, doxorubicin, vincristine, prednisone) chemotherapy. For all practices, concordance on four measures improved between March and September; only one measure worsened (P < .05). Composite scores were created to compare overall practice performance. Figure 1 demonstrates practice performance for the overall composite score, including all 28 measures. Wide variation is present. The mean composite score was 78.7% in March and 82.3% in September, demonstrating significant improvement (P < .05). Among the 71 practices, the overall composite score improved significantly for nine practices; none had significant worsening.
Composite scores also were created for the core QOPI measures and for each of the disease- and domain-specific measure sets. The findings are summarized in Figure 2. Adherence was highest for both data collections for breast cancer management and lowest for end-of-life care. The composite for core measures improved significantly (P < .05) with mean performance increasing from 51.2% to 54.6%. Individual practice performance varied; core measure scores improved significantly for 33 practices and declined for 11 practices.
We compared the performance of the lowest quartile of practices to all others. The overall composite score increased for 12 of the bottom-quartile practices between the first and second data collections, whereas improvement was significantly improved in only one practice from all other quartiles. Assessing absolute change, bottom-quartile practices improved significantly for five of the six composite measure sets; none of the composite scores for the remaining practices improved significantly between the two data collections; in fact, performance declined in three (Fig 3).
Because an assessment of absolute improvement in performance may favor practices that initially performed poorly, we undertook a second analysis looking for changes in relative performance (relative failure reduction [RFR]). The RFR analysis was undertaken for each of the seven composite scores. RFR improvement for the bottom quartile was similar or better than the performance assessed by absolute improvement. The RFR analysis for "all others" exaggerated minor declines in performance as measured by the absolute change. The differences between the bottom quartile and all others were statistically significant for all composite scores as assessed by either absolute or relative improvement.
Health care institutions are increasingly faced with the requirement to report data on clinical performance. These have largely focused on processes of care as originally defined by Donabedian.8 The Veterans Health Administration adopted performance measurement as a tool to increase the quality of care in the 1990s, and has been able to demonstrate higher rates of adherence to these measures than was achieved in a national sample of non–Veterans Health Administration facilities.9 The Hospital Quality Alliance collects process performance data for acute-care nonfederal hospitals, and makes the data available to the public, payers, and care providers. The level of adherence to measured processes of care for patients with community-acquired pneumonia, myocardial infarction, and heart failure has been positively correlated with risk-adjusted mortality, although the absolute benefits are small.10,11 Recent efforts to improve care have focused on the need to increase the accountability of individual physicians. There is a growing trend for specialty societies to define best practices, create performance measures, and collect and report the findings. Ferris et al11a recently surveyed physician specialty societies to determine the extent of commitment to the development of quality measures. Only 35% percent of societies responded affirmatively, and only a few organizations had created or planned to create data-collection tools. Resistance to involvement included perceived lack of interest by society members, limited society expertise, and limited financial and structural resources. Two important exceptions are the American College of Cardiology and the Society of Thoracic Surgeons.12,13 Both have devoted major effort to defining best practice, as well as creating the tools needed to disseminate, measure, collect, and report the results of practitioner adherence to best practices. We now report the results of the first two QOPI national data collections. We demonstrated that practices are willing to participate in a voluntary program for measuring processes of care within ambulatory oncology practices. Of the 87 practices that participated in the first data collection, 71 (82%), successfully completed a second data collection 6 months later. We demonstrated a wide range of adherence to performance measures in the different domains of care, varying from 13.1% for end-of-life care to 91.1% for management of early-stage colorectal cancer. Practice performance in QOPI improved between data collections. Improvement was demonstrable at the measure and the practice level. Using an overall composite score, adherence to the measures increased from a mean of 78.7% in the first data collection to 82.3% (P < .05) 6 months later. We found that those practices originally performing in the bottom quartile showed the most marked improvement (Fig 3). Twelve of the bottom-quartile practices experienced significant improvement compared with only one practice from all other quartiles. Compared with all others, the bottom-quartile practices experienced significantly better improvement in each of the domains of care. Because measuring changes in absolute improvement may exaggerate the performance of poorly performing practices, and underestimate improvement in high-performing ones, we reanalyzed the data to assess for RFR. The improvement differences between the bottom quartile and all others remained significantly better. The RFR analysis tended to exaggerate small decreases in performance in the "all others" group. Eighteen percent of practices that submitted data for the first data collection did not participate in the second data collection. We did not poll those practices to understand why they did not participate in round 2. Table 3 compares the characteristics of practices that continued or discontinued participation. The only significant difference is the self-reported number of new patients seen each year. Practices that discontinued participation see significantly fewer patients. This likely reflects overall practice size, given that the number of self-reported medical doctor full-time equivalents was also lower in this group. It can be speculated that smaller practices might face a proportionally greater financial or personnel challenge to participate in QOPI.
The early growth of QOPI largely mirrors how Rogers14 has described the pattern of diffusion of a new innovation. As recently reviewed by Berwick,15 Rogers defines five personality types among adopters of a new innovation: innovators, early adopters, early majority, late majority, and laggards. The initial geographically disparate practices that participated in the pilot testing of QOPI are best characterized as innovators, accounting for a tiny fraction of practicing medical oncologists.4 Those physicians who have joined QOPI to participate in the initial two national data collections are best characterized as early adopters, constituting a small, but meaningful, minority of oncologists. These oncologists are well-respected opinion leaders who have the resources and willingness to try something new. Further growth of QOPI will now depend on recruitment of the early and late majorities. Berwick15 describes the former group as being rooted locally, and susceptible to local efforts of diffusion. State specialty societies meetings may be a useful means to attract this large group of oncologists. Further diffusion of QOPI may depend on secondary benefits of participation. Physicians participating in QOPI are eligible for Continuing Medical Education (CME) credit and may use QOPI participation to meet the practice performance measurement and improvement component required by American Board of Medicine for Maintenance of Certification. The cost of QOPI participation is currently borne by the practices and, because data collection is currently performed manually, costs may be prohibitive for some practices. As electronic health records become more widespread in oncology practices, it should be possible to automate the collection of QOPI data, substantially lowering costs. This, too, should favorably influence the diffusion of QOPI. Other potential means to influence the diffusion of QOPI could come through collaborations with payers in which successful participation could be tied to recognition (eg, "preferred status"). There are several limitations to the findings of the current analysis. Most importantly, we report observational data on a group of volunteer practices. To be certain that our findings do not simply reflect a secular trend in care, a control group would have been necessary. We do not know whether our findings can be generalized to nonparticipating practices because the practices that have participated thus far may not be representative of ambulatory oncology care in the United States. QOPI data are self-reported, and verification audits were not conducted on the analyzed data. In addition, the stability of QOPI measures needs to be confirmed with subsequent data collections and in a variety of settings and practice locations. QOPI measures processes of care; we do not know if high-performing QOPI practices provide better patient outcomes or higher patient satisfaction. Finally, the willingness of practices to continue to participate in QOPI without incentives other than altruism needs to be demonstrated. In summary, medical oncologists are willing to participate in a voluntary process of practice measurement and improvement, assuming the financial burden of data collection and reporting. Without any other intervention, we demonstrate overall improvement in practice performance, with the lowest-performing practices showing the most improvement. Although conclusions from our findings need to be tempered by the limitations of the study, they are potentially important to practicing oncologists, cancer patients, payers, and regulatory agencies.
The author(s) indicated no potential conflicts of interest.
Conception and design: Joseph O. Jacobson, Michael N. Neuss, Kristen K. McNiff, Pamela Kadlubek, Peter D. Eisenberg, Joseph V. Simone Collection and assembly of data: Kristen K. McNiff, Pamela Kadlubek Data analysis and interpretation: Kristen K. McNiff, Pamela Kadlubek, Leroy R. Thacker II, Frank Song Manuscript writing: Joseph O. Jacobson, Michael N. Neuss, Kristen K. McNiff, Pamela Kadlubek Final approval of manuscript: Joseph O. Jacobson, Michael N. Neuss, Kristen K. McNiff, Pamela Kadlubek, Frank Song, Peter D. Eisenberg, Joseph V. Simone
We thank the late Christopher Desch, MD, for his vital participation in the preparation of this manuscript; the practices that have volunteered staff time to collect data for the QOPI project; and the physician and nursing staff who have provided the necessary leadership.
Presented in abstract format at the 43rd Annual Meeting of the American Society of Clinical Oncology, June 1-5, 2007, Chicago, IL. Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.
1. Hewitt ME, Simone JV: Ensuring Quality Cancer Care. Washington, DC, National Academies Press, 1999 2. Schneider EC, Malin JL, Kahn KL, et al: Developing a system to assess the quality of cancer care: ASCO's national initiative on cancer care quality. J Clin Oncol 22:2985-2991, 2004 [Erratum: J Clin Oncol 22:4656, 2004] 3. Malin JL, Schneider EC, Epstein AM, et al: Results of the National Initiative for Cancer Care Quality: How can we improve the quality of cancer care in the United States? J Clin Oncol 24:626-634, 2006 [Erratum: J Clin Oncol 24:1966, 2006] 4. Neuss MN, Desch CE, McNiff KK, et al: A process for measuring the quality of cancer care: The Quality Oncology Practice Initiative. J Clin Oncol 23:6233-6239, 2005 5. McNiff K: The Quality Oncology Practice Initiative: Assessing and improving care within the medical oncology practice. J Oncol Pract 2:26-30, 2006 6. Fleiss J, Levin B, Myunghee CP: Statistical Methods for Rates and Proportions (ed 3). Hoboken, NJ, John Wiley & Sons Inc, 2003 7. Jencks SF, Huff ED, Cuerdon T: Change in the quality of care delivered to Medicare beneficiaries, 1998-1999 to 2000-2001. JAMA 289:305-312, 2003 [Erratum: JAMA 289:2649, 2002] 8. Donabedian A: Evaluating the quality of medical care. Milbank Memorial Fund Quarterly 44:166-203, 1966[CrossRef][Medline] 9. Asch SM, McGlynn EA, Hogan MM, et al: Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med 141:938-945, 2004 10. Werner RM, Bradlow ET: Relationship between Medicare's hospital compare performance measures and mortality rates. JAMA 296:2694-2702, 2006 11. Jha AK, Orav EJ, Li Z, et al: The inverse relationship between mortality rates and performance in the Hospital Quality Alliance measures. Health Aff 26:1104-1110, 2007 11. Ferris T, Vogeli C, Marder J, et al: Physician speciality societies and the development of physician performance measures. Health Affairs 26:1712-1719, 2007 12. Eagle KA, Garson AJ Jr, Beller GA, et al: Closing the gap between science and practice: The need for professional leadership. Health Aff 22:196-201, 2003 13. Ferguson TB Jr, Dziuban SW Jr, Edwards FH, et al: The STS National Database: Current changes and challenges for the new millennium—Committee to Establish a National Database in Cardiothoracic Surgery, The Society of Thoracic Surgeons. Ann Thorac Surg 69:680-691, 2000 14. Rogers EM: Diffusion of Innovations (ed 4). New York, NY, Free Press, 1995 15. Berwick DM, Berwick DM: Disseminating innovations in health care. JAMA 289:1969-1975, 2003 Submitted September 10, 2007; accepted January 14, 2008.
This article has been cited by other articles:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||
|
|||||||||||
|
|
Copyright © 2008 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|
TOP
ABSTRACT
INTRODUCTION
.01 or P 
