|
Advertisement |
|
|
||
 |
|
|||||
|
|||||
|
 |
||||||
|
||||||
|
||||||
|
Journal of Clinical Oncology, Vol 23, No 1 (January 1), 2005: pp. 113-119 © 2005 American Society of Clinical Oncology. DOI: 10.1200/JCO.2005.06.179 Practitioners As Experts: The Influence of Practicing Oncologists "in-the-Field" on Evidence-Based Guideline DevelopmentFrom the Margaret and Charles Juravinski Cancer Centre, Hamilton Health Sciences; Cancer Care Ontario, Program in Evidence-Based Care; and Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada Address reprint requests to George P. Browman, MD, Tom Baker Cancer Centre, 1331-29th St NW, Calgary, Alberta, Canada T2N 4N2; e-mail: georgebr{at}cancerboard.ab.ca
PURPOSE: Panels of experts are used to develop clinical practice guidelines (CPGs) intended to be used by practitioners "in-the-field." Therefore, oncologists’ participation in CPG development is an important strategy to promote CPG adoption. The purpose of this study was to evaluate the contributions of oncologists in-the-field to evidence-based CPG development using data from Ontario’s cancer system. METHODS: CPG development in Ontario includes surveys of oncologists’ opinions, using a structured questionnaire, about draft recommendations that were developed from rigorous systematic reviews of evidence prepared by expert panels. Two research assistants reviewed background documents to trace the changes in CPG recommendations from draft to final stage to determine the contribution of oncologists’ input to final recommendations. Changes to recommendations were categorized as either substantive (content or tone) or minor (ideas clarification or edits). RESULTS: From 2000 to 2003, 43 CPGs were developed. There were 87 changes to draft recommendations for 31 CPGs, of which 40 changes to 19 CPGs could be attributed to survey input from practicing oncologists. Of the 40 changes, 28 (70%) were judged to be substantive. CONCLUSION: Despite a rigorous evidence-based process for CPG development, practicing oncologists contribute substantially to the final recommendations approved by the expert panel. It is hypothesized that the responsiveness of expert panels to input from oncologists in-the-field will facilitate adoption of CPGs.
Clinical practice guidelines (CPGs) are intended to serve as a guide to appropriate practice to influence clinical practice patterns. Deciding on appropriate practice for specific clinical circumstances requires careful evaluation and interpretation of the research evidence with the participation of experienced practitioners. Certain practitioners may be acknowledged as experts in their field for a variety of reasons. Those who conduct and publish influential research are often recognized broadly as experts in their field. Often, such individuals are brought together because of their acknowledged expertise to provide advice on appropriate practice through educational venues, for consultation exercises, to develop a practice guideline, and for other purposes, such as consensus development conferences. The theoretical value of the expert consensus approach to influencing practice is that there is the assumption that such acknowledged experts have an implicit and comprehensive mastery of the scientific and practical information that would yield the most appropriate recommendations. The use of acknowledged experts to set practice standards is consistent with research studies suggesting that an important intervention for changing practice behaviors is the use of so-called opinion leaders, or educational influentials.1,2 There is also an important body of work demonstrating that clinical medical practice is a highly social undertaking (eg, Mittman2) and that effective opinion leaders reside within practicing communities, defined either geographically or functionally. Such opinion leaders are respected locally but are not necessarily widely known researchers or clinicians. The National Comprehensive Cancer Network (NCCN) Guideline Program3 relies heavily on the implicit knowledge and judgment of clinical experts for the development of guidelines. Three other well-known oncology guideline initiatives, the Cancer Care Ontario (CCO) Program in Evidence-Based Care (PEBC),4 the French Standards, Options, and Recommendations (SOR) Program,5 and the American Society of Clinical Oncology (ASCO) Guidelines Program6 also use clinical experts within their guideline development panels. However, in the latter three programs, the processes are more explicit in how evidence is handled and interpreted to satisfy the due diligence requirements of an evidence-based approach.7 All four programs, NCCN, PEBC, SOR, and ASCO, include an independent review step (internal or external) as part of the guideline development process to help validate the final product. For NCCN, this process again focuses on acknowledged experts as external reviewers, whereas for PEBC and SOR, this step uses ordinary cancer care practitioners "in-the-field."8,9 Furthermore, PEBC guidelines (available at http://www.cancercare.on.ca/access_PEBC.htm) include a section that describes the results of this consultation process, including how the input of practitioners, mainly oncologists, in-the-field has influenced the final guideline recommendations. Although not as extensive as the PEBC process, ASCO distributes its draft guidelines to community oncologists, national and international content experts, and members of its Health Services Committee for feedback. ASCO also uses its board members to review guidelines before they are finally approved. The purpose of this article is to present the experience of CCO’s PEBC in including ordinary oncologists as part of the process of guideline development and to review their contributions as independent reviewers of practice guidelines.
Study Context: PEBC Guideline Development Initiative CCO uses a formal process for the development of evidence-based guidelines that has been published.4 The process includes a step called practitioner feedback that consists of formal surveys of Ontario practitioners’ evaluations of draft guidelines before they become approved as official guidelines.8 The survey is a structured instrument that also allows for written open-ended comments and is sent with each mailing of a draft guideline. Guidelines are sent to practitioners for whom the topic is relevant to their clinical practice. Thus a practitioner will receive as many surveys to complete as the number of guidelines on which they are asked to comment. Over the first 7 years of the Initiative, the database of cancer care–related Ontario practitioners in the community in Ontario has grown to more than 1,200 entries. Early experience with this process has been published.10 The oncology practitioner can be involved in CCO practice guideline development at one or more of three different phases of the process: first, as a member of an expert panel assigned to develop the guideline from a formal scientific systematic review where the evidence is assembled, synthesized, and first interpreted. The panels are composed of clinicians (medical oncologists, radiation oncologists, surgeons, other specialists, nurses, and so on) and methodologists who represent the province geographically. This group is composed of both community and academic clinicians, although proportionally more of the latter. Second, the oncology practitioner can be involved as a practitioner in-the-field through participation in the practitioner feedback surveys that accompany each draft guideline. The program has a database of practicing clinicians (oncologist specialists, surgeons, other specialists, and so on) in the province with contact information and details about their professional discipline and disease site(s) treated. In contrast to the panels, there are proportionally more community physicians than academic physicians. The characteristics of the sample of reviewers are matched to the characteristics of the report sent out for feedback to create a unique sample for each document. We strive to ensure balanced geographic representation for the reviewers chosen. Nonclinician methodologists are not part of this review process. Further, panel members are not eligible to review their own documents for this phase. Third, the oncology practitioner can be involved as a member of the CCO Practice Guidelines Coordinating Committee charged with reviewing the final guideline product for approval on behalf of CCO. This is a small panel composed of clinicians representing the spectrum of care and policy makers. At this stage, the Coordinating Committee can send a draft guideline back to the expert panel for modifications before final approval. On the basis of the first 38 guidelines produced by the program involving 4,002 mailings, we have previously demonstrated that the participation rate of practitioners in-the-field over the first 4 years of the program was sustained at a 66% response rate, with 33% of surveys containing written comments.10 There was no evidence of response fatigue over time or with increasing workload. Over the first 4 years, approval ratings for the program’s guidelines were high (79%) and consistent across oncology specialty areas (medical oncologists, radiation oncologists, surgeons, and pharmacists). The guideline development methodology of the program includes the systematic review of the evidence and its interpretation by the expert panel to produce a draft guideline. Subsequently, the draft guideline is sent to practitioners in Ontario accompanied by a structured questionnaire (practitioner feedback questionnaire) using formal survey methods. For those not responding, reminders are sent at 2 weeks and 4 weeks. The questionnaire consists of 21 items representing statements about aspects of the draft guideline. For each item, respondents are provided a five-point response option ranging from strongly agree to strongly disagree. The questionnaire covers domains of guideline quality (eg, the literature review is relevant and complete), applicability in practice (eg, the draft recommendations will be technically difficult to apply), acceptability (eg, the draft recommendations are appropriate to the patients to whom they apply), and comparative value to practice (the draft recommendations reflect a more effective approach to improving patient outcomes than current usual practice). Respondents are also asked about their intention to use the guideline once it becomes approved. The sample of practitioner records eligible for survey is stored in an accumulating database using Microsoft Access software (Redmond, WA). At March 2004, the program’s database contained 1,236 eligible records with identifiers for main disease site of interest, geographical region, practice setting (community v academic), participation in an expert panel, and specialty. The expert panel is responsible for dealing with survey results, including relevant modifications to the draft that will be sent to the Coordinating Committee for final approval. The Coordinating Committee and expert panel then engage in an iterative process to finalize the guideline. The process includes explicit documentation of any changes to the guideline recommendations such that a paper trail of change is available.
Protocol Two research assistants collaborated using a consensus approach in a retrospective review of all draft guideline documents, correspondence, and final guideline documents over the study period. The guideline documents were categorized before the review into two categories: those guidelines related to the Ontario new drug funding approval process11 and those related to topics other than new drug funding. The term recommendation refers to formal statements contained in the guideline that are intended to directly influence practice; a single guideline may have more than one recommendation. Feedback from practitioner surveys or review by the Coordinating Committee can affect any section of a guideline; this article documents changes made only to recommendations. Changes were coded into four different hierarchical categories in the following order (examples are listed in Table 1): (1) content: addition or deletion of substantive information; (2) tone: change resulting in a softening or strengthening of a recommendation; (3) clarity: addition, deletion, or rewording of text to improve comprehension; (4) editorial: grammatical, punctuation, or minor word changes or sentence restructuring. When there was more than one category of change in a guideline, the change was classified only within the category that represents the highest order in the hierarchy. For example, for a particular guideline, a change in content and clarity would be coded only as a change in content. When a change within a document affected both a recommendation and another aspect of the guideline, the change was coded as a change in the recommendation as the more important outcome.
Table 2 lists the sampling frame of guidelines. There were 43 guidelines, of which 18 (42%) related to assessments for new drug funding approvals and 25 (58%) related to practice issues. Of the 43 guidelines, 16% were completed in 2000 and 26%, 28%, and 30% were completed in 2001, 2002 and 2003, respectively.
Table 3 lists the number of changes prompted by all sources (Coordinating Committee and practitioner surveys) to recommendation statements across all guidelines. For the 43 guidelines, there were 87 changes made to recommendations from all sources. For 12 guidelines (28%), feedback resulted in no changes, and for 31 guidelines (72%), changes were made to at least one of the recommendations. Among these, one or two changes were made to recommendations in 19 guidelines, three to five changes were made to recommendations in nine guidelines, and six or more changes to recommendations were made in three guidelines.
The influence of practicing oncologists in-the-field on the development of evidence-based guidelines is best reflected by the expert panels’ changes prompted by oncologist survey responses. Table 3 also lists the patterns of change to recommendations prompted by the survey of oncologists. Changes to recommendations occurred in 19 (44%) of the 43 guidelines reviewed. Among these, one or two changes were made to recommendations in 12 guidelines, and three to five changes were made to recommendations in seven guidelines. Thus 40 of the 87 changes to draft recommendations were due to input from surveyed oncologists. Although the number of guidelines approved per year was small, from 2000 to 2003 there was a change prompted in recommendations each year: six of seven, nine of 11, seven of 12, and nine of 13 guidelines in 2000, 2001, 2002, and 2003, respectively. When this analysis is restricted to oncologist survey data alone, there were changes to four of seven, three of 11, five of 12, and seven of 13 guidelines over these years, respectively. Table 4 lists the categories of change to recommendations. Of the 40 changes across 19 guidelines prompted by oncologist surveys, 18 changes related to content, 10 related to tone, eight related to changes to improve clarity, and four related to editorial changes. Thus 70% (28 of 40) of the changes prompted by oncologist surveys resulted in a substantive change in either content or tone in 19 guidelines. By contrast, 50% (24 of 47) of the changes prompted by the Coordinating Committee were considered substantive (Table 4).
When focusing on the 87 changes according to guideline type, 37 changes affected 72% (13 of 18) of the new drug funding guidelines, and 49 changes affected the identical proportion (72%; 18 of 25) of the practice pattern category of guidelines.
There are different approaches to the development of CPGs and recommendations for practice. In oncology, there is a sharp contrast, for instance, between the expert opinion–based approach of the NCCN that has produced highly influential guidelines and that of CCO, SOR, and ASCO, which have preferred the so-called evidenced-based approach. The main difference is in the implicit versus explicit consideration of the evidence and how it is reported for expert-based versus opinion-based guideline development methods. In both cases, however, experts must be involved to interpret the evidence, either implicitly or explicitly, and to judge the applicability of the evidence from well-controlled research studies to the real world of practice. The role of the acknowledged expert for mainly opinion–based guidelines seems quite clear, but the role of the expert in evidence-based guidelines has not been precisely documented. Any evidence-based approach must blend the complementary contributions of seasoned clinicians with the rigorous synthesis of research evidence. This is so that the evidence can be interpreted appropriately in the context of the clinical circumstances of patients and of the values and circumstances of the health system within which the evidence will be applied. There is no universally accepted definition of what constitutes a clinical expert. Any patient with cancer would want to believe that their own particular oncologist was an expert in the field relevant to their particular clinical condition. Thus there may be different perspectives about who can be labeled an expert. In this study, we have used data collected and rigorously documented over 4 years regarding an evidence-based guideline development program to determine the contribution of the ordinary oncologist in-the-field to the construction of recommendations in evidence-based guidelines. In this process, oncologists are provided the opportunity to comment on draft guidelines before they become officially approved. Our data demonstrate that despite the very rigorous processes used by CCO’s PEBC to comply with the standards of a rigorous evidence-based approach, the program’s final recommendations owe much to the perspectives of ordinary practicing oncologists. Their input occurs after the completion of the systematic review and the drafting of recommendations by panel experts based on the systematic reviews. Despite the rigorous synthesis of evidence and its careful consideration by panel experts, ordinary practitioners were able to convince such panels that substantive changes (content and tone) to recommendations, consistent with the evidence, were needed in 28 cases across 19 of 43 guidelines. This information confirms the value of consulting with the oncology community when developing CPGs. Furthermore, based on adoption theory, we hypothesize that involving practicing clinicians in such processes not only improves evidence-based guidelines, but also contributes to their ultimate adoption.12-14 As a next step, we are engaged in a project to develop models to predict clinicians’ adoption of guidelines in terms of clinical decisions that are in keeping with clinical recommendations. Clinician participation, as a member of a guidelines panel, the approval body, or external reviewer, is a factor in the model. We will be able to study the independent role of this factor in predicting behavior as well as how this factor interacts with other factors involved in the guideline enterprise. We hope that this information will assist others in designing effective guideline development programs that can better influence oncology practice. The next step in our research is to determine the relationship between actual practice patterns and the responses that oncologists provide to guideline surveys.
The authors indicated no potential conflicts of interest.
Supported by Cancer Care Ontario, Program in Evidence-Based Care, and Ontario Ministry of Health and Long Term Care. Presented in part at the 40th Annual Meeting of the American Society of Clinical Oncology, New Orleans, LA, June 5-8, 2004. Authors’ disclosures of potential conflicts of interest are found at the end of this article.
1. Soumerai SB, McLaughlin TJ, Gurwitz JH, et al: Effect of local medical opinion leaders on quality of care for acute myocardial infarction: A randomized controlled trial. JAMA 279:1358-1363, 1998 2. Mittman BS: Implementing clinical practice guidelines: Social influence strategies and practitioner behavior change. QRB Qual Rev Bull 18:413-422, 1992[Medline] 3. Winn RJ, Botnick W, Bitsura JM: Evaluation of preliminary NCCN guidelines by external review. Oncology 10:311-315, 1996 (suppl 11)[Medline]
4. Browman GP, Levine MN, Mohide EA, et al: The practice guidelines development cycle: A conceptual tool for practice guidelines development and implementation. J Clin Oncol 13:502-512, 1995 5. Philip T, Fervers B, Haugh M, et al: European cooperation for clinical practice guidelines in cancer. Br J Cancer 89:S1-S3, 2003 (suppl 1) 6. American Society of Clinical Oncology: Practice Guideline. http://www.asco.org/ac/1,1003_12-002562,00.asp
7. Mulrow CD, Cook DJ, Davidoff F: Systematic reviews: Critical links in the great chain of evidence. Ann Intern Med 126:389-391, 1997 8. Browman GP, Newman TE, Mohide EA, et al: Progress of clinical oncology guidelines development using the practice guidelines development cycle: The role of practitioner feedback. J Clin Oncol 16:1226-1231, 1998[Abstract] 9. Fervers B, Hardy J, Blanc-Vincent MP, et al: SOR: Project methodology. Br J Cancer 84:8-16, 2001 (suppl 2) 10. Browman GP, Brouwers M, DeVito C, et al: Participation patterns of oncologists in the development of clinical practice guidelines. Curr Oncol 7:252-257, 2002
11. Pater JL, Browman G, Brouwers M, et al: Funding new cancer drugs in Ontario: Closing the loop in the Practice Guidelines Development Cycle. J Clin Oncol 19:3392-3396, 2001 12. Gates PE: Think globally, act locally: An approach to implementation of clinical practice guidelines. Jt Comm J Qual Improv 21:71-84, 1995[Medline] 13. Cain M, Mittman R: Diffusion of Innovation in Healthcare. IHealthreports, May 2002. http://www.chcf.org/documents/ihealth/DiffusionofInnovation.pdf 14. Plsek P: Complexity and the adoption of innovation in health care. Presented at: Accelerating Quality Improvement in Health Care Strategies to Speed the Diffusion of Evidence-Based Innovations. January 27-28, 2003, Washington, DC. http:/www.aacn.nche.edu/Publications/WhitePapers/PLSEK.pdf Submitted June 23, 2004; accepted October 2, 2004. This article has been cited by other articles:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||
|
|||||||||||
|
|
Copyright © 2005 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|
TOP
ABSTRACT
INTRODUCTION
