|
Advertisement |
|
|
||
 |
|
|||||
|
|||||
|
 |
||||||
|
||||||
|
||||||
|
Journal of Clinical Oncology, Vol 24, No 28 (October 1), 2006: pp. 4553-4557 © 2006 American Society of Clinical Oncology. DOI: 10.1200/JCO.2006.06.7819 Processes to Activate Phase III Clinical Trials in a Cooperative Oncology Group: The Case of Cancer and Leukemia Group B
From the Center for Management Research in Healthcare; Management of Technology Program; Owen Graduate School of Management; Vanderbilt-Ingram Cancer Center, Vanderbilt University, Nashville, TN; and Cancer and Leukemia Group B, Chicago, IL Address reprint requests to David M. Dilts, PhD, Owen Graduate School of Management, 401 21st Ave S, Nashville, TN 37203; e-mail: david.dilts{at}vanderbilt.edu
PURPOSE: National Cancer Institute–sponsored cooperative oncology groups are major sponsors of phase III clinical trials, yet the time and steps required to design and activate such studies has not been well studied. We examine the processes and document the calendar time required to activate such studies opened by the Cancer and Leukemia Group B (CALGB). METHODS: Setup steps were documented by (1) interviewing CALGB headquarters and statistical center staff and committee chairs to discover the steps required to transit from concept development to final study activation, (2) reviewing procedure manuals, and (3) inspecting all study records, documents, and e-mails to identify any additional steps. Calendar time was collected for each major process. RESULTS: Thirteen phase III studies were activated by CALGB during the study period of May 2002 to May 2005. More than 370 distinct processes were required for study activation: 317 work steps, 42 decision points, and 29 processing loops. Sixty-three percent of the decision points were outside CALGB. The complete process map measures 243.5" x 41" in 8-point font. Median calendar days to activate a phase III study at CALGB was 580 days (range, 295 to 1,248 days) from concept approval and 784 days (range, 537 to 1,130 days) from initial conception of the study. CONCLUSION: Setup of a phase III study at a major cooperative oncology group is a complex and lengthy process, with the majority of decision points external to the cooperative group. To improve the activation process, research should to be directed toward both internal and external groups and processes.
There is scant research on the processes needed to design and activate a study in a cooperative oncology group such as the Cancer and Leukemia Group B (CALGB). The calendar time required to activate a study is of particular importance with the rapidly changing landscape in cancer treatment and the large number of agents available for study.1 Cooperative groups are often characterized as being slow and inefficient in study start-up, yet the reasons for the postulated inefficiency have not been well documented. Administrative, or setup, barriers faced by cooperative groups are classified into four types: procedural, structural, infrastructural, and synchronicity. Generically, the first three barriers are common with those found in comprehensive cancer centers and in community oncology practices2; however, the fourth barrier is unique. Procedural barriers are policies, either formal or informal, that arise from the processes or series of steps required to activate a study and that may inhibit problem-solving actions. For example, a procedural barrier occurs when, after a concept is approved by CALGB, the concept must then be reviewed by an outside agency before additional work can be done on the concept development. As we will show, such procedural issues occur throughout the activation process and they occur both internally within CALGB and at the external interface with other stakeholders, such as the National Cancer Institute (NCI; National Institutes of Health, Bethesda, MD) Cancer Therapy Evaluation Program (CTEP), the NCI Central Institutional Review Board (CIRB), the US Food and Drug Administration (FDA), and with pharmaceutical firms. Structural barriers are created when different participants in the process follow a different ordering of steps, which can lead to miscommunication. An example of this barrier is a circular mismatch loop that arises because of multiple participants in the process. A loop occurs when one participant sends information to another, who replies back to the originator. For example a pharmaceutical sponsor, which holds the investigational new drug (IND) application, may require information that can only be provided by CALGB, which will only supply the necessary information after the sponsor agrees to some condition. This can lead to a "Catch-22" situation: one group cannot collect the required information until they approve a condition, but they will not approve the condition without the information. Infrastructural barriers concern how the underlying system is designed and how it supports the interconnection of various system aspects. For example, protocol review, drug distribution, NCI Common Data Element, and CIRB review are each a basic part of the CTEP required infrastructure.3,4 Although individually each is productive, they can cause delays in activation as studies wait for final approvals from multiple parts of the system, each of which may have a different support system, timeline or oversight. Synchronicity is the need for the tight coordination of the various components of a protocol before it can be submitted to other participants in the process. In CALGB, protocol assembly, forms/database development, contract negotiations, and drug acquisition/distribution plans, each the responsibility of a different functional group, all must be completed and synchronized before a study can be activated. Interactive effects among the barriers can also exacerbate hurdles. For example, an interaction between administrative and synchronicity barriers occurs as CTEP, pharmaceutical company, FDA, and CIRB reviews are frequently sequential. External reviews may be fragmented and sometimes contradictory because they collect information at various times from various individuals. Thus, these represent both timing and synchronizing barriers.
Study Settings and Timeframe The setting is the CALGB, a national clinical research group sponsored by the NCI, with its central office headquartered at the University of Chicago (Chicago, IL) and its statistical center located at Duke University (Durham, NC).5 CALGB has the goal of bringing together clinical oncologists and laboratory investigators to develop better treatments for cancer. Since its formation in 1956, CALGB has grown into a national network of 29 academic medical centers, more than 225 community hospitals/physician practices and more than 3,000 oncology specialists who collaborate in clinical research studies aimed at reducing the morbidity and mortality from cancer, relating the biologic characteristics of cancer to clinical outcomes and developing new strategies for the early detection and prevention of cancer. All CALGB phase III studies activated in a 3-year period (May 1, 2002, through May 31, 2005) were investigated. Phase III studies were selected for two reasons: (1) being the most complex, they contain all the relevant steps in study activation needed for all types of treatment trials, and (2) other research has shown that there is no significant difference in calendar time to open by phase.2 Because some studies that opened in 2002 began the process much earlier, the scope of time investigated was a period of 5 years (March 6, 2000, through May 31, 2005) to acquire all relevant timing data. All trials were therapeutic studies; intergroup studies were excluded so as to concentrate on identifying where CALGB could reduce internal process steps and times. Only process flows within CALGB were inspected. Processes external to CALGB were considered "black-box" processes; that is, data collected were the initial date that the information was sent to the outside agency and the final approval date when CALGB was informed that the agency accepted all changes.
Part A: Process Mapping For Part A: Process Mapping, a team of experts from the Center for Management Research in Healthcare (cMHRc) was engaged. The cMHRc is composed of members from the Vanderbilt-Ingram Cancer Center (Nashville, TN), the Vanderbilt University (Nashville) School of Engineering (VUSE) and the Vanderbilt University Owen Graduate School of Management (OGSM). This team was composed of a medical oncologist, three modeling experts from VUSE and five experts in process management methodologies from OGSM. The team conducted multiple, repeated in-depth face-to-face, telephone, and electronic interviews with staff at CALGB headquarters in Chicago and at the CALGB statistical center in Durham, NC. These data were supplemented by review of the written CALGB procedures manuals. Using these sources of information, the team developed initial process flows needed for all major activities required to activate a study, including: concept development, protocol development, forms and database development, contract negotiations (if applicable) and regulatory affairs. The flows were verified by presenting them to relevant participants and by validating the steps detailed in individual study documentation, in both print and electronic formats. Additionally, committee chairs who proposed these studies were interviewed via telephone to verify dates and steps based on their paper and electronic records.
Part B: Timing Analysis
Part A: Process Mapping There are two strategies when creating process maps: (1) list all process steps on one large complete diagram, or (2) create a hierarchy of diagrams, each of which provides additional detail from a master, or Level 0, diagram.2,7 The complete process diagram, because of its size (243.5" x 41" in 8-point font), is impossible to reproduce in a journal. Therefore, we elected in Figure 1 to present only the Level 0 map, with both the entire and specific segments of the process map available online at http://cmrhc.org/processmaps.htm.
The CALGB protocol activation process can be divided into four phases: (1) initial concept development and approval, (2) protocol development, (3) CIRB review, and (4) study activation. Initial concept development and approval is composed of five sets of steps. First, in concept development, are the steps required from initial presentation of a potential study by a principal investigator (PI) to a CALGB committee chair, to the formal concept submission by the committee for CALGB executive committee concept review. The first of the 29 processing loops discovered to activate a study occur in this step, as the committee can send the concept back to the PI for revision before bringing it forward. Formal review by the CALGB executive committee, concept rating, and concept approval (or disapproval or return for revision) by the executive committee are the next process set. Third, concept approval is evaluated by the group chair on the basis of the priority score of the executive committee, competing studies currently under development, and resource availability. If approved, the concept is sent to CTEP for review and on approval, a study team teleconference is initiated and the initial development phase is complete. The teleconference launches the most resource- and time-intensive part of the protocol activation process, namely protocol development. This is composed of seven sets of process steps, four of which are under CALGB control. The first, and most resource intensive, is the formal protocol development. This process begins internally between CALGB and its investigators. Once a protocol has been developed, it is sent to CTEP for review. This initiates a series of process loops between CALGB and CTEP. Limited or extensive changes can be requested by CTEP, and CALGB responds to each. A limited change is one that can be handled within a local process loop, whereas an extensive change requires the study to return to a previous step in the process, possibly even returning to the initial concept development step. While the above processes are underway, forms and database development are being completed simultaneously. Again, the process begins with internal CALGB staff and, once draft forms have been completed, they are submitted for NCI common data element compliance review. As with protocol review, a series of process loops commences, with limited or extensive changes occurring. If additional funding is needed for the study, CALGB begins grant development or contract negotiations with a potential sponsor. This is the fastest of the processes in the protocol development phase. The primary organizations involved in this aspect are CALGB and potential sponsors, such as pharmaceutical firms. Finally, the fourth major process flow that occurs simultaneously is drug acquisition (if required) and regulatory affairs. This identifies the source of the drug to be studied, establishes a drug distribution plan, and determines IND requirements. Once these are completed, the IND application (if required) is submitted by CALGB to the FDA for review. One vital aspect to understanding the process and timing requirements of these steps is the need for synchronicity; that is, the need for various elements of the protocol to come together at the same time for the study to be approved by internal and external organizations. This is similar to the problem faced in the automotive industry when making cars: the correct engine must be matched with the correct body in the assembly line before any additional work can be done on the vehicle. If the correct engine is not ready the assembly process halts until the engine is ready and the line restarts. In study activation, such synchronicity is needed between protocol development and forms/database development before the study can be sent for CIRB review. Additionally, all steps (ie, CTEP, CIRB, FDA, and sponsor negotiations) must be synchronized before a study can be placed on the CALGB Web site and activated. The final two phases in the activation process, namely CIRB review and study activation, are where most synchronization difficulties take place because it is here where the different parts of a study must come together for approval. One interesting feature of the process that is not evident in the Level 0 process map is the number of loops in the process. A process loop is when one element in the process sends part or all of the study back to another process for additional work. For example, the CALGB executive committee can send a proposed study back to the principal investigator for revision and reapproval by a scientific committee. There are 29 processing loops to activate a phase III study at CALGB. Each loop constitutes a set of processes that require one party to respond to another party’s questions. For example, potentially the longest loop is final CIRB approval, one of the last steps in the process, which, if an extensive revision is required, can send a proposed study back into protocol development. While a process step may be required to reply by a specific time, there is no limitation on the number of times a study may loop before being accepted or stopped. Table 1 details the number of process steps (> 370), participants (> 30), signatories (> 70), decision points (42) and processing loops (29) necessary to activate a phase III study at CALGB. It is important to remember that these activities take place simply to activate a study. After activation, the study must undergo additional steps before it can be opened at a community practice site (CPS) or at an NCI-designated comprehensive cancer center (CCC). To better understand the total number of steps, statistics from CPS and CCC are listed in Table 1.2 Therefore, to open a CALGB study at a CPS requires more than 430 steps, and more than 480 steps at a CCC; and nearly 60 groups or individuals are involved in the process.
Part B: Timing Analysis Data were retrospectively collected on 13 phase III studies. The first step verified data completeness and ensured data consistency. By cross checking electronic records with paper records, data consistency was established. The date of initial concept submission to committee chairs was unavailable for five studies; however dates of executive committee approval and final activation were available for all studies. The median calendar time from initial concept proposal by a PI to a CALGB committee chair to study activation by CALGB was 784 days (n = 8; range, 537 to 1,130 days), with the median time from formal approval of the concept by the CALGB executive committee to study activation of 580 days (n =13; range, 295 to 1,248 days; Table 2). The longest single process was protocol development (median, 477 days; range, 266 to 1,200 days), with forms/database development requiring nearly as much calendar time (median, 434 days; range, 259 to 1,183 days). Regulatory affairs required 127 fewer days (median, 350 days; range, 113 to 496 days) than protocol development and forms/database development. Grant development required the fewest number of days at 222 median days (range, 169 to 302 days).
Our investigation assesses the impact of procedural, structural, and infrastructural barriers, as well as the process of synchronicity, in the activation of a phase III treatment trial at a major NCI-sponsored cooperative group, the CALGB. The calendar time evaluated was measured from the time of concept inception to study activation. Aspects such as concept development and review, evaluation by the NCI CTEP, protocol, forms and database development, the process to obtain supplemental funding and regulatory affairs for drug acquisition and distribution were studied (Fig 1; Table 1). Although documentation of the processes has been reported infrequently, numerous opinions on the weaknesses of the process have been described.8 Reports have implicated various process aspects as barriers to clinical trial activation, such as contracting.8 Our study reveals that the process of activating a clinical trial at a major cooperative group is daunting and complex, requiring multiple processes, loops, and reviews by different organizations resulting in more than 370 steps, 30 groups or individuals, 70 signatures, 40 decision points, and nearly 30 processing loops. The overall median time from conception of a study to protocol activation is 784 days (n = 8; range, 537 to 1,130 days). It should be noted that the initial period of concept development has a median time of 193 days. This time is not official CALGB time because the study has not yet been formally submitted to the CALGB executive committee for review. We have included it in the process map as it is an integral part of study activation. However, if we exclude this time, the net CALGB time is a median of 580 days, or approximately 19 months. It is these times and process steps that can be most immediately influenced by process improvements at CALGB and external agencies. The process map shows that cooperative groups must deal with multiple organizations and agencies, none of which are under the direct control of the cooperative group. Coordination of studies through multiple organizations will delay rapid study activation. The result of this research is to document the complexity of the process and timing issues required to activate a phase III cooperative group study. This study is limited to only one cooperative group and evaluation of only 13 phase III studies. Additionally, no external agency documents were collected. It is important to extend this study to other cooperative groups and to the application of a similar methodology to external agencies in order to more fully evaluate the entire process of activating a clinical trial. Additional research is needed to investigate the impact of phase (I or II) and intergroup studies on such times. With the process now documented, we intend to implement revisions in those areas controlled by CALGB that should improve efficiency and reduce the time to protocol activation. It is important to note that improving CALGB processes, without addressing the processes of the external organizations that interface with CALGB, may have minimal impact on the total time to activate a study.
The authors indicated no potential conflicts of interest.
Supported by Grant No. CA 31946 from the National Cancer Institute to R.L.S. and by subcontract to D.M.D. and A.B.S. Presented in part at the 42nd Annual Meeting of the American Society of Clinical Oncology, June 2-6, 2006, Atlanta, GA. Authors’ disclosures of potential conflicts of interest and author contributions are found at the end of this article.
1. Rothenberg ML, Carbone DP, Johnson DH: Improving the evaluation of new cancer treatments: Challenges and opportunities. Nat Rev Cancer 3:303-309, 2003[CrossRef][Medline] 2. Dilts DM, Sandler AB: The "invisible" barriers to clinical trials: The impact of structural, infrastructural, and procedural barriers to opening oncology clinical trials. J Clin Oncol 24:4545-4552, 2006 3. Cancer Therapy Evaluation Program: NCI Clinical Trials Cooperative Group Program Guidelines. Bethesda, MD, Division of Cancer Treatment and Diagnosis, National Cancer Institute, 2005, p 105 4. Fukuda H: Infrastructure of cancer clinical trial cooperative groups in western countries [Japanese]. Gan To Kagaku Ryoho 27:1144-1151, 2000[Medline] 5. CALGB: Home Page. http://www.calgb.org 6. Harrington HJ: Business Process Improvement. New York, NY, McGraw-Hill, 1991 7. Gane C, Sarson T: Structured Systems Analysis: Tools and Techniques. Englewood Cliffs, NJ, Prentice-Hall, 1979 8. Mello MM, Clarridge BR, Studdert DM: Academic medical centers’ standards for clinical-trial agreements with industry. N Engl J Med 352:2202-2210, 2005 Submitted March 29, 2006; accepted July 21, 2006. Related Correspondence
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||
|
|||||||||||
|
|
Copyright © 2006 by the American Society of Clinical Oncology, Online ISSN: 1527-7755. Print ISSN: 0732-183X
|
TOP
ABSTRACT
INTRODUCTION
