|
Advertisement |
|
|
||
 |
|
|||||
|
|||||
|
 |
||||||
|
||||||
|
||||||
|
© 2003 American Society for Clinical Oncology
Perspectives on the Value of American Society of Clinical Oncology Clinical Guidelines as Reported by Oncologists and Health Maintenance OrganizationsFrom the VA Midwest Center for Health Services and Policy Research, Department of Veterans Affairs, Chicago Healthcare System/Lakeside Division, Chicago; the Division of Hematology/Oncology, the Department of Medicine, and the Robert H Lurie Cancer Center, Northwestern University, Chicago, IL; the Health Services Research Committee of the American Society of Clinical Oncology, Alexandria, VA; the Division of Solid Tumor Oncology, the Health Outcomes Research Group, the Department of Epidemiology and Biostatistics, and the Department of Medicine, Memorial Sloan-Kettering Cancer Center, New York, NY Address reprint requests to Charles L. Bennett, MD PhD, VA Medical Science Building, 400 East Ontario, Ste 204, Chicago, IL 60611; email: cbenne{at}northwestern.edu.
Purpose: Although the American Society of Clinical Onoclogy’s (ASCO) Health Services Research (HSR) committee activities have primarily focused on clinical guideline development, little is known about the value placed on these guidelines by the desired end users. ASCO members and Health Maintenance Organizations (HMOs) were surveyed on the value and implementation of ASCO guidelines. In this article, we summarize our findings. Methods: ASCO members (n = 1500) were queried about whether they had read ASCO’s first four clinical guidelines and technology assessment; whether they agreed with the recommendations; whether they used guidelines in clinical practice; and how guidelines had affected reimbursement. HMOs (n = 131) were queried on how they identify, implement, and value the first four ASCO clinical guidelines. Results: The membership survey indicated that ASCO guidelines were read more often by physicians in private healthcare settings compared with physicians in academic practices (P < .02). Disagreement rates were low for all guidelines (range, 1% to 7%). One quarter of respondents reported that the guidelines were difficult to find and difficult to apply to the practice setting, and approximately one tenth of respondents indicated that the guidelines were difficult to evaluate, interpret, or read. The HMO survey indicated that one third of HMOs reported use of ASCO guidelines, with higher rates of usage by larger HMOs and by those with higher National Committee on Quality Assurance (NCQA) ratings. Respondent HMOs valued guidelines for various purposes and used multiple methods of guideline identification and implementation. Conclusion: ASCO guidelines are generally highly supported by physicians and HMOs. Additional studies are needed to identify implementation barriers and to see whether guidelines have resulted in improvements in healthcare.
THE LAST 15 years of health services research have seen the emergence of two related endeavors: evidence-based medicine and practice guideline development. Parallel to these activities, the American Society of Clinical Oncology (ASCO), beginning in 1994 with the publication of a guideline on colony-stimulating factors, has assembled expert panels to develop evidence-based oncology practice guidelines on a number of topics.1 ASCO has published 12 such guidelines to date, as well as several updates of these guidelines and two technology assessments. These guidelines are designed to support quality-improvement initiatives in oncology. Care of cancer patients is both complex and expensive because treatment strategies vary according to effectiveness and cost. There is empirical evidence of large variations in both the intensity and quality of medical care for patients with cancers of the prostate, colon, lung, and breast.2–6 These treatment differences have been attributed to physician-related factors, whereas others have been attributed to health system-related factors.7,8 Thus, quality improvement efforts for cancer care are directed at both the physician level and the system level.9,10 One aspect of these efforts is incorporation of clinical guidelines into practice by physicians and/or health care systems.9,10 To date, the Health Services Research (HSR) committee of ASCO has devoted the majority of its financial and time resources to the development of clinical guidelines in an effort to ultimately improve the quality of cancer care. However, although one prior study has reported that these guidelines are of moderate to high quality,11 little is known about how these guidelines are viewed by potential users either at the level of the cancer specialist or at the broader level of health care organizations. To obtain insights into these questions, we conducted two surveys. The first survey targeted ASCO membership and included questions about familiarity and agreement with the clinical guidelines. A second survey targeted managed care organizations (HMOs), and sought information on whether ASCO guidelines had been incorporated into internal clinical guideline programs. In this article, we summarize the findings for the two surveys.
Membership Survey All U.S. physicians whose names appeared on ASCO’s 1999 membership list were eligible to participate in the survey. Surgical and radiation oncologists were included among eligible members. A sample of 1,500 ASCO members was randomly selected from the membership list. The survey instrument was sent to the selected members, along with a stamped business reply envelope and a brief, personalized cover letter, which was signed by ASCO’s president and the Chair of the Subcommittee on Guideline Evaluation. The cover letter that accompanied the survey explained the purpose and importance of the survey, assured the confidentiality of responses, and encouraged member participation. No personal identifying information appeared on the survey, and respondents were instructed not to put their names anywhere on the survey. Identification numbers, which were linked to respondents’ names and addresses in a secured electronic file, were placed on each survey to allow for response tracking. No financial or other incentives were provided in exchange for member participation. The initial mailing was conducted on February 7, 2000. Four weeks after the first mailing, a second package of survey materials (ie, a replacement survey, personalized cover letter, and stamped business reply envelope) was sent to all members who had not returned their surveys. Follow-up telephone calls were initiated 3 weeks after the second mailing. Completed surveys were accepted through June 9, 2000, which was the end of the 4-month survey period. Survey responses were entered into a Microsoft ACCESS database that was specifically designed for this study. A double data-entry process was used so that survey responses could be verified. Every effort was made to decode illegible responses. In addition, we responded to surveys that were returned with more than 10% of critical items left blank by sending a personalized letter, along with the survey items of interest and a stamped business reply envelope, appealing for the missing information. On completion of data entry, the electronic data set was thoroughly inspected for any errors. All data entry errors and other invalid responses were corrected before generating descriptive statistics for each survey question. The Public Health Research Institute conducted all survey mailings and response collection.
The survey solicited data on respondents’ familiarity and agreement with ASCO clinical practice guidelines. The survey evaluated four clinical guidelines, use of hematopoietic colony-stimulating factors (CSFs), treatment of unresectable non–small-cell lung cancer, breast cancer surveillance, and use of tumor markers in breast and colorectal cancer; and one technology assessment of raloxifene and tamoxifen in breast cancer risk reduction. The survey included questions about the preferred medium for receiving information included in the guidelines, self-described thoroughness in reviewing the guideline materials and agreement with the guideline conclusions, end-use of the guidelines in clinical practice either in full or as part of practice-generated clinical guidelines, and the effect of guidelines on reimbursement.
HMO Survey Twenty-two percent (137 of 630) of the HMOs from the Competitive Edge, The HMO Directory (Interstudy Inc, Minneapolis, MN) database, were selected for this study. Ten HMOs were selected for ranking highest in covered lives (> 1% range of covered lives), and 127 HMOs were randomly selected from the remaining database. Six HMOs were ultimately eliminated because of missing contact information. Surveys were mailed to the medical directors of 131 HMOs on April 1, 2000, and telephone calls were subsequently placed to each organization. Four weeks after the initial mailing, a second package of survey materials was sent to all HMOs that did not send back their surveys. Nonrespondents also received follow-up telephone calls, faxes, and additional copies of the survey. Completed surveys were accepted through June 30, 2000.
Respondent HMOs were compared with nonrespondents according to geographic distribution, median number of covered lives, and percentage accepting Medicare. Individual respondent HMOs were further evaluated according to the value they placed on practice guidelines, their modes of guideline identification and dissemination, their guideline application policies, and their enforcement of physician compliance with guidelines. ASCO guideline use was assessed in relation to HMO size (ie, number of covered lives), HMO model type, and NCQA quality rating. For the analyses, ASCO guidelines were considered to be in use if the respondent indicated that "the organization uses ASCO guidelines to set policy for oncology practice" and/or "the organization has implemented (specific guideline)."
Membership Survey A total of 691 members returned a completed survey, 83 members chose not to complete the survey, 150 members were found to be ineligible, and 93 of 576 members who did not return a survey were estimated to be ineligible. Of the 150 members that were found to be ineligible, 32 of them practice medicine outside of the United States, 59 are retired, four are deceased, and 55 are nonphysician or nonpracticing physician members. Based on these results, the overall usable response rate was determined to be 55%, or 691 respondents out of 1257 eligible physicians. Because the survey responses were anonymous, we could not compare the characteristics of respondents with those of nonrespondents. Approximately 75% of the respondents were medical oncologists or hematologists/oncologists, 10% were radiation oncologists, 5% were surgical oncologists, and 5% were pediatric oncologists. One third of the respondents were in private healthcare settings characterized by having 40% or more managed care patients, and an additional one third of the respondents were in academic medical practices. Physician practice management companies were affiliated with 9% of the respondents. The average respondent reported seeing 7.8 new patients per week. The survey addressed general issues related to guidelines and dissemination of educational materials. The preferred dissemination medium for educational materials was either through journals (92%) or at medical conferences (84%) compared with medical grand rounds (36%) and audiotapes (23%). With respect to guidelines or technology assessments that influence reimbursement, most respondents supported ASCO having either a leadership (51% and 46%, respectively) or advocacy role (25% and 33%, respectively). Characteristics of the guideline process that were most important to the respondents included review of the relevant literature (78%), basing guidelines on randomized clinical trials (69%), and providing sufficient detail about the guideline (69%), whereas less important features included input from health professionals (54%), comprehensiveness (52%), and having recognized national experts on the committee (47%).
The first ASCO guideline, on the use of CSFs, which is the most recently published document, was read least often (55%; Table 1
Rates of agreement with individual guidelines were high, with 76% to 83% of respondents stating an opinion agreeing with the guidelines (Table 1  ). Moreover, among oncologists who had read the ASCO guidelines, fee-for-service oncologists were least likely to agree with the tumor markers guideline (P < .02), whereas academic oncologists were least likely to agree with the tamoxifen/raloxifene technology assessment (P < .04). For the other three ASCO guidelines, there were no significant differences in frequency of disagreement by type of practice setting. Most respondents were supporters of the patient guidelines, with 68% of respondents who had an opinion stating that these guidelines were important, 74% agreeing with their content, and 64% indicating that they were useful.
Some problems were identified with guideline dissemination and implementation. In particular, approximately one quarter of the respondents reported that the guidelines were both difficult to find and to apply to the practice setting, and approximately one tenth of the respondents indicated that the guidelines were difficult to evaluate, interpret, or read. (Table 2
We also queried oncologists about the use of guidelines in their clinical practice. Among oncologists who had clinical guidelines in their practice, ASCO’s guidelines for CSFs, breast cancer, and tamoxifen/raloxifene were used more frequently than internally derived practice-specific guidelines. (Table 1 ) In contrast, for non–small-cell lung cancer, practice-generated guidelines were used more frequently than ASCO guidelines (31% v 18%; P < .001). Whereas 7% of respondents had experienced a reimbursement denial because of care provided that was inconsistent with some clinical guideline, less than half of these denials occurred because of ASCO’s guidelines; only 2% of respondents had ever been financially rewarded for guideline compliance.
HMO Survey
In total, 33% of respondent HMOs indicated using ASCO guidelines. HMOs that were larger (ie, mean covered lives, 427,158) appeared more likely to use ASCO guidelines than smaller HMOs (mean covered lives, 191,273). In addition, 56% (9 of 16) of HMOs that received "commendable" ratings from the NCQA stated that they used ASCO guidelines, whereas only 18% (7 of 40) of HMOs that received lower ratings or had not been evaluated by NCQA stated that they used ASCO guidelines (P = .004; Table 6  ).
ASCO’s HSR committee has conducted two surveys to evaluate perceptions of ASCO clinical guidelines by physicians and HMOs. The membership survey identified a high level of readership of the guidelines and good support for the content. The HMO survey found that, at the system level, respondents valued practice guidelines and generally preferred ASCO guidelines to internally derived guidelines. This support was a result of a range of reasons, spanning from reduction of practice variation to determination of clinical trial coverage. In interpreting the findings of these two surveys, several factors should be considered. First, there continues to be uncertainty about how guidelines can be incorporated into clinical practice. The membership survey provides reassurance that as often as one third of the time, the guidelines were adopted by a clinical practice, whereas they were rarely used in support of a reimbursement denial. However, guidelines represent only a small part of a quality improvement process.9,10 Organizations that use the guidelines well must have strategic support that emphasizes the importance of quality improvement, cultural support that promotes guideline adherence, technical infrastructure to disseminate and monitor guideline compliance, and structural features that allow for easy use of guidelines in the daily practice of oncology.9 Without support for all four of these dimensions, guidelines are unlikely to have much effect on clinical practice. Second, the HMO survey identified opportunities for adoption of ASCO guidelines at the system level. Two thirds of respondent HMOs reported that they depend on a general awareness of major journals, 41% expected notification from authoring agencies, and 23% reviewed journals systematically. Respondent HMOs that reported use of more than one source of information about guidelines form a relatively accessible target group. Wider dissemination of ASCO guidelines could be facilitated by dissemination of the guidelines in other peer-reviewed journals, publications from organizations, such as the National Cancer Institute, the National Comprehensive Cancer Network, and the American Cancer Society, and via electronic publication through guideline clearinghouse programs. ASCO guidelines are posted on ASCO Online (www.asco.org) to enhance access. Third, enhancement of guideline content and dissemination does not ensure proper clinical application of the guidelines. The greatest reported weakness of the ASCO guidelines has been their failure to address monitoring and clinical audit activities, which are important components of clinical practice facilitation.11 Ultimately, successful standardization of efficacious patient care through guideline use will require a gradual enhancement of the entire process—from guideline development and dissemination to implementation. Thus, guidelines must be viewed as one of a series of quality improvement interventions. Fourth, these findings have implications for planning the ASCO future health-services agenda. To date, the major activity of the HSR committee has been guideline development.1 Although the committee plans to continue producing clinical practice guidelines, attention has been focused on the large amount of volunteer time and effort needed to develop each guideline. The most recent guideline effort has been greatly aided by ASCO’s decision to pursue collaborative relationships. Collaboration with the American Society of Hematology (ASH) has resulted in a joint review of erythropoeitin use and a joint request for the Agency for Health Research and Quality to provide funds for an evidence-based practice center to review this subject area. This review, conducted by the Blue Cross/Blue Shield technology evaluation center is now complete12 and has provided much of the background material for the recently completed joint ASCO/ASH erythropoietin guideline effort. Of note, the Blue Cross/Blue Shield technology evaluation center report includes a systematic and comprehensive literature review of randomized clinical trials, a feature of the guideline process supported most often by respondents to the ASCO membership survey. There are several limitations of the surveys that are important to consider. First, data on actual practice patterns are needed to determine whether HSR committee guidelines are facilitating quality improvement. The recently initiated Harvard/RAND/ASCO study of quality of breast and colorectal cancer care will provide empirical data on the frequency with which clinical practice is consistent with some of ASCO’s clinical guideline recommendations. Second, the surveys addressed assessments for only four of the ASCO clinical practice guidelines published to date. Third, each survey is limited by nonresponse considerations. However, the response rates for both surveys were similar to those reported for prior cancer-related surveys that were targeted to ASCO members and HMOs.13–15 In conclusion, the two surveys described herein provide insights into how ASCO’s guidelines are viewed by two of the targeted end-users: ASCO members and HMOs. Taken together, the studies indicate that the guidelines have reached a broad audience and have been well received by many of the targeted users. Future studies are needed to see whether the guidelines have resulted in improvements in the quality of medical care provided to cancer patients.
This article is based on research supported by a grant from the American Society of Clinical Oncology. We gratefully acknowledge the assistance of members of the Working Group on Guideline Evaluation of the ASCO HSR Committee.
1. Somerfield M, Padberg J, Pfister D, et al: ASCO clinical practice guidelines: Process, progress, pitfalls, and prospects. Classic Papers Curr Comments 4:881–886, 2000
2. Begg CB, Riedel ER, Bach PB, et al: Variations in morbidity after radical prostatectomy. N Engl J Med 346:1138–1144, 2002
3. Bach PB, Cramer LD, Schrag D, et al: The influence of hospital volume on survival after resection for lung cancer. N Engl J Med 345:181–188, 2001 4. Bennett CL, Greenfield S, Aronow H, et al: Patterns of care related to age of men with prostate cancer. Cancer 67:2633–2641, 1991[CrossRef][Medline] 5. Mandelblatt J, Hadley J, Kerner JF, et al: Patterns of breast carcinoma treatment in older women: Patient preference and clinical and physical influences. Cancer 89:561–573, 2000[CrossRef][Medline] 6. Howard J, Lund P, Bell G: Hospital variations in metastatic breast cancer. Med Care 18:442–455, 1980[Medline] 7. Hewitt M, Simone JV (eds). Ensuring quality cancer care. National Cancer Policy Board, Institute of Medicine and Commission on Life Sciences, National Research Council. Washington, DC, National Academy Press, 1999
8. Begg CB, Cramer LD, Hoskins WJ, et al: Impact of hospital volume on operative mortality for major cancer surgery. J Am Med Assoc 280:1747–1751, 1998 9. Shortell SM, Bennett CL, Byck G: Assessing the impact of continuous quality improvement on clinical practice: What will it take to accelerate progress. Milbank Q 76:593–624, 1998[CrossRef][Medline] 10. Shortell SM, O’Brien JL, Carman JM, et al: Assessing the impact of continuous quality improvement/total quality management: Concept versus implementation. Health Serv Res 30:377–401, 1995[Medline] 11. Brouwers M, Browman G: Assessment of the American Society of Clinical Oncology practice guidelines. J Clin Oncol Classic Papers and Current Comments 4:1081–1088, 2000
12. Seidenfeld J, Piper M, Flamm C, et al: Epoetin treatment of anemia associated with cancer therapy: A systematic review and meta-analysis of controlled clinical trials. J Natl Cancer Inst 93:1204–1214, 2001 13. Bennett CL, Smith T, Weeks J, et al: Use of hematopoietic colony stimulating factors: The American Society of Clinical Oncology Survey. J Clin Oncol 14:2511–2520, 1997
14. Bennett CL, Weeks J, Somerfield M, et al: Use of hematopoietic colony stimulating factors: Comparison of the 1994 and 1997 American Society of Clinical Oncology survey regarding ASCO clinical practice guidelines. J Clin Oncol 17:3676–3681, 1999 15. Bennett CL, Buchner DA, Ullman M: Approaches to prostate cancer by managed care organizations. Urology 50:79–86, 1997[CrossRef][Medline] Submitted July 26, 2002; accepted November 11, 2002.
This article has been cited by other articles:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||
|
|||||||||||
|
|
Copyright © 2003 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|
TOP
ABSTRACT
INTRODUCTION
2 tests (two-sided) were used to assess association between categorical variables. Student’s t tests (two-sided) were used to test differences in continuous variables. 
