This Article Abstract Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Save to my personal folders Download to citation manager Rights & Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Scharf, O. Articles by Colevas, A. D. Search for Related Content PubMed PubMed Citation Articles by Scharf, O. Articles by Colevas, A. D. Related Articles Related Correspondence Social Bookmarking                            What's this?

Adverse Event Reporting in Publications Compared With Sponsor Database for Cancer Clinical Trials

Orit Scharf, A. Dimitrios Colevas

From PSI International Inc, Fairfax, VA; and the Cancer Therapy Evaluation Program, National Cancer Institute, Rockville, MD

Address reprint requests to A. Dimitrios Colevas, MD, Investigational Drug Branch, Cancer Therapy Evaluation Program, National Cancer Institute, Executive Plaza North Room 7131, 6130 Executive Blvd, Rockville, MD 20852; e-mail: colevasd{at}ctep.nci.nih.gov

	ABSTRACT

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION Authors' Disclosures of... Author Contributions REFERENCES

 
PURPOSE: Prospectively planned collection and analysis of adverse event (AE) data are essential parts of well-conducted clinical trials. The AE data in a trial sponsor's database should be comparable with what is stipulated in the protocol and with the AE data published. We examined whether the published AE data differ from those in the sponsor's database and from the data collection requirements stated in study protocols.

METHODS: We searched the National Cancer Institute (NCI) Clinical Data Update System (CDUS) for studies that used the Common Toxicity Criteria version 2.0 and for which a final study publication was available. We extracted from the protocols information pertaining to AE collection and reporting methods and compared it with the methods cited in the article. We also compared the AE data in the trial publication with the AE data submitted by the investigators to CDUS.

RESULTS: We identified 22 studies meeting the criteria for this review. There was considerable inconsistency between AE collection and reporting methods cited in the protocols versus final publications. AE data in the article and CDUS were not identical. Twenty-seven percent of article high-grade AEs could not be matched to agent-attributable AEs in the CDUS. Twenty-eight percent of CDUS high-grade AEs could not be matched to AEs in the corresponding article. In 14 of 22 articles, the number of high-grade AEs in CDUS differed from the number in the articles by 20% or more.

CONCLUSION: Lack of consistency in and reporting of AEs are associated with NCI database and trial publication AE data discrepancy.

	INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION Authors' Disclosures of... Author Contributions REFERENCES

 
The purpose of investigational agent clinical trials is to determine the safety and efficacy of the agents in the treatment of human disease. A trial may be performed for the purposes of licensing (US Food and Drug Administration approval) or for the purpose of generating information of use to patients and physicians when making treatment decisions. The requirements and processes by which data are generated to fulfill these two needs are not identical. All clinical trials, regardless of purpose, should be conducted according to good clinical practice.¹ However, for trials designed to support licensure of a drug product, the adverse event (AE) and efficacy data are collected in a way consistent with the requirements of the relevant regulatory oversight organization. In the United States, the US Food and Drug Administration determines these standards. For trials not specifically designed to support licensure of the drug product, one goal is to generate data that will be acceptable to the peer-review process typically sought when the results of a trial are submitted for publication. There are no rules or guidelines governing this latter process for early-stage trials, although the Consolidated Standards of Reporting Trials² group has begun to address this issue for randomized controlled clinical trials. We are unaware of any mechanism by which article data are routinely audited for accuracy and completeness with respect to AE reporting.

Although many different efficacy end points may be investigated, trials are usually designed to address one primary efficacy end point, such as overall survival, disease-free survival, or response rate. Traditionally, early-phase clinical development of investigational cancer agents has focused on response rate as the primary end point. The methodology for collection and reporting of response data has been extensively discussed.^3-5 The WHO standards for response assessment, initially proposed in 1977, have been virtually universally accepted as the standard for use in early clinical trials, and the more recently disseminated Response Evaluation Criteria for Solid Tumors are based on the WHO foundation.^3-5 The guidelines for response evaluation put forward by the Response Evaluation Criteria for Solid Tumors (RECIST) group have been particularly helpful in facilitating standardization of data acquisition, including specifics of imaging technique parameters, and reporting, including specific definitions of partial response, complete response, stable disease, and so on.

Clinical trial sponsors and investigators are aware of the importance of developing rules for AE reporting and have struggled with the conflict between the burden of complete AE reporting and the importance of safety monitoring.^2,6,7 However, although National Cancer Institute (NCI) –sponsored trials are audited to ensure compliance with the protocol-specified data collection and submission requirements,⁸ we are unaware of any specific standard rules or guidance concerning the reporting of AEs in the medical literature. Therefore, we decided to conduct a comparison between the AE data submitted to the NCI and the AE data in the corresponding clinical trial publications for single-agent phase II clinical trials studying investigational agents. We assumed that the AE data presented in published articles should closely match those available in the NCI database or that the articles should explicitly state the criteria and rationale used when presenting less than the full AE data set. We also examined how closely the AE data collection methodology, as stipulated in the protocols, was recapitulated in the corresponding published clinical trials reports. Our hypothesis was that cancer clinical trial publication and sponsor database AE data should be consistent.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION Authors' Disclosures of... Author Contributions REFERENCES

 
Common Toxicity Criteria
The NCI developed the Common Toxicity Criteria (CTC) AE lexicon to provide standard language for collecting, describing, and reporting AEs occurring in cancer clinical trials. CTC version 2.0 (v.2.0) was issued to improve accuracy, precision, and completeness of the CTC and to standardize reporting. All NCI database AEs analyzed for this study were in the CTC v.2.0 format, but in some cases, we translated non-CTC v.2.0 format AEs reported in articles into CTC v.2.0 terminology to facilitate the comparison.⁹

Clinical Data Update System
The Clinical Data Update System (CDUS) is the NCI's electronic database of clinical trials information. For each protocol, the lead group or institution is responsible for submitting CDUS data for all sites participating in the trial. AE reporting using the most up-to-date CTC version is mandatory for all studies assigned to complete CDUS reporting, and AEs must be reported by the course (cycle) in which they occurred. Reporting is required for grade 1 and 2 AEs assessed to be possibly, probably, or definitely related to the investigational agent and for all grade 3, 4, and 5 AEs regardless of attribution. All AEs must be attributed to at least one possible cause.¹⁰ The NCI maintains a site-based partial auditing program that verifies the accuracy of clinical trial AE data by comparison of the primary medical record with relevant research databases.⁸ CDUS data are accessible to study principal investigators.

Clinical Trial Selection Process
We searched the NCI database for single-agent, CDUS-monitored phase II studies that were active between March 1998 and October 2003, which was the time frame in which CTC v.2.0 was the mandated lexicon for AE reporting and grading. Three hundred fifty-five studies were identified, 213 of which were single-agent chemotherapeutic studies. A PubMed search using the agent names and principal investigators from these studies identified 24 published articles in peer-reviewed journals reporting these clinical trial results. Two of these 24 studies were eliminated from our analyses; one was eliminated because only a subset of patients was included in the article, and the other was eliminated because of AE reporting that was not in a format translatable to allow comparison with CDUS AE data. The number of patients reported in the articles was confirmed to be identical to the number of patients in CDUS for every trial.

Protocol Instructions for AE Collection and Reporting
We extracted information from the protocols pertaining to AE collection and reporting methods. We generally used directions from the Study Calendar or Table of Required Observations sections in the study protocol to determine required AE collections. These sections typically specify the evaluation times for each of the specified tests and assessments during protocol treatment. We also extracted relevant information from the protocol text describing the required AE evaluation of patients during study.

Extraction of AE Data From Articles and CDUS
AE information from articles was extracted from both the text and AE tables (when available) to tabulate the most severe grade of each AE reported per patient. All grade 3 AEs reported in the articles, regardless of attribution to the investigational drug, were analyzed in accordance with the reporting rules of CDUS. AE information from CDUS was retrieved electronically for all studies evaluated and analyzed both for all AEs reported and for the subset of AEs attributed specifically to the investigational agent.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION Authors' Disclosures of... Author Contributions REFERENCES

 
Comparison Between Protocol and Article Methodology
In our review of 22 protocols and associated publications, we observed that, although all protocols identified the CTC as their AE classification system (with two protocols using an additional set of neurotoxicity criteria), only 16 of the subsequent publications (73%) identified any AE criteria system (Tables 1 and 2); 13 articles (59%) cited the CTC as the method of AE classification, and three articles (14%) reported other standard toxicity criteria, such as the WHO criteria,¹¹ despite the fact that this lexicon was not referenced in these protocols. All NCI-sponsored phase II protocols are required to report AEs via CDUS. However, nine of the protocols we investigated (41%) did not refer to any reporting system for AEs, and the CDUS was not mentioned anywhere in these protocols. Five protocols (23%) named a responsible party for data reporting (ie, the principal investigator or a research study assistant), whereas the rest did not specify any such responsibility.

Eighteen of the 22 publications presented AEs in the form of a table in the results section, whereas four publications discussed the AEs in text form alone. In only three articles (14%) was attribution of AEs (ie, investigational agent related or not) clearly specified for all AEs reported. Attribution of the reported AEs to the investigational agent was not specified in any way in 12 articles (54%) and incompletely specified in an additional three articles (14%). In the remaining four publications, attribution was mentioned in a vague manner and only in reference to certain AEs or to AEs of grade 3. For the purpose of our comparisons, we assumed that AE tables and text discussions in the articles referred to AEs attributed to the investigational agent unless otherwise specified in the article. We made this assumption for several reasons. It was our assumption that AEs not attributable to the investigational agent would not be of interest to the investigator or article reader. No publication explicitly stated that all AEs, without regard to attribution, were reported. When comparing high-grade AEs without regard to attribution, there were many more grade 3 to 5 AEs in CDUS versus the publications (611 v 413 AEs, respectively) and an average of 28 and 19 high-grade AEs reported per trial, respectively, in the two data sets. Only three trials had more total grade 3 to 5 AEs cited in the publications than in CDUS; one trial had 17 in the publication versus 12 in CDUS, and two trials had a difference of only one AE between the data sets. A similar comparison between publication and CDUS agent-attributable events demonstrated much closer congruence (413 and 423 AEs, respectively). Therefore, we concluded that the analysis that could best evaluate differential reporting of like events was the publication versus the CDUS investigational agent–attributable AE data.

Fourteen of the publications (64%) reported AEs of all grades, either separately (grades 1, 2, 3, 4, and 5) or combining grades 1 and 2 (grades 1+2, 3, 4, and 5). Four studies (18%) showed only grade 3 AEs. The AEs were usually presented as the number or percentage of patients who experienced a specific AE and grade. One article included the cumulative AE burden as represented by the most severe AE per cycle (ie, multiple occurrences of same AE per patient). In all other articles, most severe AE per patient reporting was inferred by the way the results were presented but not explicitly stated.

The AE text section in the articles discussed primarily the most common AEs of any grade, most severe AEs defined as grade 3, and AEs that led to dose adjustments or treatment interruption. Patient deaths (grade 5) were usually discussed at length, and detailed information was included about the patients' baseline parameters and the AEs they experienced before death. In some studies, hematologic and nonhematologic AEs were presented separately in the text, as well as in the AE table. The protocol and article specifications are listed in Tables 1 and 2.

Comparison of AE Reporting Between Articles and CDUS
A comparison between grade 3 AEs reported in the articles versus CDUS revealed that 27% (111 of 413 AEs) of all AEs cited in the articles could not be matched directly with investigational agent–attributable AEs in CDUS (Fig 1A). Forty eight of these publication-cited AEs could be matched to AEs in CDUS not attributed to the investigational agent. Twenty nine of these publication-cited AEs could be matched to different AEs in CDUS within the same general category (eg, constitutional in article matched to fatigue in CDUS). The remaining 34 AEs did not appear at all in CDUS.

View larger version (32K): [in this window] [in a new window]   Fig 1. (A) Adverse events (AEs) reported in articles mapped to AEs reported in the Clinical Data Update System (CDUS). (B) AEs reported in CDUS mapped to AEs reported in articles. PB, publication.

The total number of grade 3 AEs reported in the 22 articles was comparable to the total number of investigational agent–attributable grade 3 AEs reported in the CDUS (413 and 423 AEs, respectively). However, for many trials, the discrepancy between the number of AEs in CDUS versus the number in the article was considerable (Figs 2A and 2B). In only eight (36%) of 22 trials were the numbers of grade 3 AEs in the articles and CDUS within ± 10% of each other. Publications 14 and 19 specified that only AEs occurring in 10% of patients were included (Fig 2A). Publication 15 cited only AEs of patients who withdrew from therapy for toxicity and "the most common toxicities of any grade." Therefore, in some cases, articles systematically excluded reporting a fraction of the high-grade AEs. When considering all investigational agent–attributed AEs (grades 1 to 5), the number of AEs reported in the articles was less than the number reported in CDUS (1,910 and. 2,995 AEs, respectively), and only six (27%) of 22 trials' total AE numbers in the CDUS and articles were within ± 20% of each other (Fig 2B). In addition, a comparison between article grade 3 AEs and the total number of grade 3 AEs in CDUS, regardless of attribution, was also warranted because as many as 15 articles (68%) possibly report nonattributable AEs to some extent, given that attribution was not specified or was incompletely specified in those articles. The total number of grade 3 AEs in these 15 articles (n = 260) represents 74% of all grade 3 AEs that are in CDUS for these trials (350 AEs), regardless of attribution. Because attribution was not specified in many of the articles, further resolution of this CDUS versus article discrepancy was not possible.

View larger version (21K): [in this window] [in a new window]   Fig 2. (A) Grade 3 adverse events (AEs) reported in articles compared with the Clinical Data Update System (CDUS). See Results for discussion of AEs excluded in articles 14, 15, and 19. (B) All grade AEs in articles compared with investigational agent–attributed AEs in CDUS. For both figures, the bar values are the number of AEs in CDUS subtracted from the number of AEs in the article.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION Authors' Disclosures of... Author Contributions REFERENCES

 
Our review illustrates that investigators drafting protocols are aware of the importance of specifying the AE lexicon and AE collection and reporting systems to be used in a clinical trial. All protocols we reviewed specified the AE lexicon to be used, and the majority specified how AE data were to be collected and reported to the NCI. Unfortunately, when drafting manuscripts for publication, an AE lexicon is often not specified, the attribution to the agent under study is not provided, and the AE burden is poorly characterized and usually underrepresented relative to what has been submitted to the NCI database. Therefore, the only AE data available to the public are inconsistent with the information available to the NCI.

Our analysis of AE reporting discrepancies identified three distinct areas in which publication reporting of AEs was problematic: (1) underreporting of low-grade AEs; (2) underreporting of recurrent AEs; and (3) inconsistent and incomplete characterization and reporting of high-grade AEs.

Low-grade AEs contribute to the overall AE burden and, therefore, should not be omitted when reporting clinical outcomes. More comprehensive reporting will enhance the understanding and interpretation of clinical trial results as well as contribute to future drug development and planning of drug-combination studies. For example, a pharmacogenetic syndrome could be detected by a high rate of low-grade AEs. Systematic exclusion of high-grade AEs in articles, even if specified in the article, can be misleading and, by definition, underrepresents the burden of toxicity. Similarly, reporting the burden of recurrent AEs could have a major influence on the view of the therapeutic index of an agent. For example, intractable nausea during one treatment course of an agent, which is subsequently controlled with antiemetic prophylaxis, is much less distressing than recurrent intractable nausea with every treatment course. Although this difference can be evaluated in CDUS, such information was not available in virtually all of the articles we reviewed. Lastly, when an article fails to clearly delineate investigational agent–attributable AEs from other AEs, it is not possible for a reader to assess how much of the AE burden is thought to be drug related. Therefore, the reader's ability to form an opinion concerning the therapeutic index of an agent is constrained.

To enhance our ability to garner useful data from publications of clinical trial results, we suggest that an appropriate AE lexicon be used by investigators and more uniform and comprehensive standards for publication reporting of AEs be implemented. In general, the methods and extent of AE data collection and submission to the sponsor's database should provide the basis for what methods and data are cited in articles. Investigators should review the AE data they have submitted to the sponsor's database when drafting the final trial manuscript so that inconsistencies can be minimized. There are presently no efficient auditing systems for data in manuscripts. Implementation of such standards would not impose an undue burden on clinical investigators because the tools needed to abide by such standards are already in place for most trials of investigational agents. These tools and methods merely need to be applied to clinical trial articles. It is unlikely that the ad hoc, peer review process will be an effective method for imposition of such standards. We propose that a model analogous to the Consolidated Standards of Reporting Trials group's efforts at the journal editorial level be developed to address this issue. Minimum standards would include explicit discussion of the methods of collection and reporting of AEs and should include AE description, grading, and attribution according to a standard AE lexicon. Journal editors could specify that protocols and AE data sets be part of the required supplemental materials to be submitted with clinical trial manuscripts. This would allow reviewers to judge the fidelity of the manuscript with respect to these primary documents. Investigators should be encouraged to include in the manuscript a summary of all investigational agent–attributable AEs, or a link should be provided to supplemental data via the Internet, as is often done now with many scientific publications. Investigators should use the sponsor's AE database as a starting point for AE reporting and describe explicitly any alterations or omissions in the publication relative to this database. Implementation of these publication AE reporting standards will enhance the quality and integrity of clinical trial publications.

	Authors' Disclosures of Potential Conflicts of Interest

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION Authors' Disclosures of... Author Contributions REFERENCES

 
The authors indicated no potential conflicts of interest.

	Author Contributions

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION Authors' Disclosures of... Author Contributions REFERENCES

 

Conception and design: A. Dimitrios Colevas Administrative support: Orit Scharf, A. Dimitrios Colevas Provision of study materials or patients: Orit Scharf, A. Dimitrios Colevas Collection and assembly of data: Orit Scharf, A. Dimitrios Colevas Data analysis and interpretation: Orit Scharf, A. Dimitrios Colevas Manuscript writing: Orit Scharf, A. Dimitrios Colevas Final approval of manuscript: Orit Scharf, A. Dimitrios Colevas

 

	NOTES

 
Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.

	REFERENCES

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION Authors' Disclosures of... Author Contributions REFERENCES

 
1. International Conference on Harmonisation: Guideline for good clinical practice. ICH harmonized tripartite guideline. 1-57, 1996 http://www.ich.org/cache/compo/276-254-1.html

2. Moher D, Schulz KF, Altman DG: The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357: 1191-1194, 2001[CrossRef][Medline]

3. WHO: WHO Handbook for Reporting Results of Cancer Treatment. Geneva, Switzerland, WHO, WHO offset publication 48, 1979

4. Therasse P, Arbuck SG, Eisenhauer EA, et al: New guidelines to evaluate the response to treatment in solid tumors: European Organization for Research and Treatment of Cancer, National Cancer Institute of the United States, National Cancer Institute of Canada. J Natl Cancer Inst 92: 205-216, 2000[Abstract/Free Full Text]

5. Hayward JL, Carbone PP, Heuson JC, et al: Assessment of response to therapy in advanced breast cancer: A project of the Programme on Clinical Oncology of the International Union Against Cancer, Geneva, Switzerland. Cancer 39: 1289-1294, 1977[CrossRef][Medline]

6. Martin PJ, Antin JH, Weisdorf DJ, et al: Reporting of adverse event data in hematopoietic stem cell transplantation clinical trials involving investigational new drugs or devices: A report from the William Guy Forbeck Foundation 2001 focus meeting on clinical trials in hematopoietic stem cell transplantation. Biol Blood Marrow Transplant 8: 295-302, 2002[Medline]

7. Goldberg RM, Sargent DJ, Morton RF, et al: Early detection of toxicity and adjustment of ongoing clinical trials: The history and performance of the North Central Cancer Treatment Group's real-time toxicity monitoring program. J Clin Oncol 20: 4591-4596, 2002[Abstract/Free Full Text]

8. National Cancer Institute: Investigator's handbook, 2002: Section 16: Monitoring and quality assurance. http://ctep.cancer.gov/forms/Hndbk.pdf

9. National Cancer Institute: Common Toxicity Criteria version 2.0 (CTC), 1999. http://ctep.cancer.gov/reporting/CTC-3.html

10. National Cancer Institute: Clinical Data Update System (CDUS), 2003. http://ctep.cancer.gov/reporting/cdus.html

11. Miller AB, Hoogstraten B, Staquet M, et al: Reporting results of cancer treatment. Cancer 47: 207-214, 1981[CrossRef][Medline]

Submitted December 18, 2005; accepted April 10, 2006.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Facebook    Reddit    Technorati    Twitter    What's this?

Related Correspondence

• More Thoughts on the Reporting of Adverse Events in Cancer Clinical Trials
  Martine Extermann
  JCO 2007 25: 918 [Full Text]

This article has been cited by other articles:

				J P Leombruno, T R Einarson, and E C Keystone The safety of anti-tumour necrosis factor treatments in rheumatoid arthritis: meta and exposure-adjusted pooled analyses of serious adverse events Ann Rheum Dis, July 1, 2009; 68(7): 1136 - 1145. [Abstract] [Full Text] [PDF]

				D. M. Brizel, B. A. Murphy, D. I. Rosenthal, K. J. Pandya, S. Gluck, H. E. Brizel, R. F. Meredith, D. Berger, M.-G. Chen, and W. Mendenhall Phase II Study of Palifermin and Concurrent Chemoradiation in Head and Neck Squamous Cell Carcinoma J. Clin. Oncol., May 20, 2008; 26(15): 2489 - 2496. [Abstract] [Full Text] [PDF]

				A. Trotti, A. D. Colevas, A. Setser, and E. Basch Patient-Reported Outcomes and the Evolution of Adverse Event Reporting in Oncology J. Clin. Oncol., November 10, 2007; 25(32): 5121 - 5127. [Abstract] [Full Text] [PDF]

				K. D. Crew, H. Greenlee, J. Capodice, G. Raptis, L. Brafman, D. Fuentes, A. Sierra, and D. L. Hershman Prevalence of Joint Symptoms in Postmenopausal Women Taking Aromatase Inhibitors for Early-Stage Breast Cancer J. Clin. Oncol., September 1, 2007; 25(25): 3877 - 3883. [Abstract] [Full Text] [PDF]

				A. Surbone, M. Rowe, and T. H. Gallagher Confronting Medical Errors in Oncology and Disclosing Them to Cancer Patients J. Clin. Oncol., April 20, 2007; 25(12): 1463 - 1467. [Full Text] [PDF]

				M. Extermann More Thoughts on the Reporting of Adverse Events in Cancer Clinical Trials J. Clin. Oncol., March 1, 2007; 25(7): 918 - 918. [Full Text] [PDF]

				O. Scharf and A. D. Colevas In Reply J. Clin. Oncol., March 1, 2007; 25(7): 918 - 919. [Full Text] [PDF]

				S. J. Anderson Some Thoughts on the Reporting of Adverse Events in Phase II Cancer Clinical Trials J. Clin. Oncol., August 20, 2006; 24(24): 3821 - 3822. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Save to my personal folders Download to citation manager Rights & Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Scharf, O. Articles by Colevas, A. D. Search for Related Content PubMed PubMed Citation Articles by Scharf, O. Articles by Colevas, A. D. Related Articles Related Correspondence Social Bookmarking                            What's this?