Journal of Clinical Oncology, Vol 21, Issue 13
(July), 2003: 2574-2582
© 2003 American Society for Clinical Oncology
Interobserver and Intraobserver Variability in Measurement of Non–Small-Cell Carcinoma Lung Lesions: Implications for Assessment of Tumor Response
Jeremy J. Erasmus,
Gregory W. Gladish,
Lyle Broemeling,
Bradley S. Sabloff,
Mylene T. Truong,
Roy S. Herbst,
Reginald F. Munden
From the Departments of Diagnostic Radiology, Biostatistics, and Thoracic Head and Neck Medical Oncology, University of Texas M.D. Anderson Cancer Center, Houston, TX.
Address reprint requests to Jeremy J. Erasmus, MD, M.D. Anderson Cancer Center, Department of Diagnostic Radiology, 1515 Holcombe Blvd., Box 57, Houston, TX 77030; email: jerasmus{at}di.mdacc.tmc.edu.
Purpose: Response of solid malignancies to therapy is usually determined by serial measurements of tumor size. The purpose of our study was to assess the consistency of measurements performed by readers evaluating lung tumors.
Materials and Methods: The study group was composed of 33 patients with lung tumors more than 1.5 cm. Bidimensional (BD) and unidimensional (UD) measurements were performed on computed tomography (CT) scans according to the World Health Organization (WHO) criteria and the Response Evaluation Criteria in Solid Tumors (RECIST), respectively. Measurements were performed independently by five thoracic radiologists using printed film and were repeated after 5 to 7 days. Inter- and intraobserver measurement variations were estimated through statistical modeling.
Results: There were 40 tumors with an average size of 1.8 to 8.0 cm (mean, 4.1 cm). Analysis of variance showed a significant difference (P < .05) among readers and among the measured nodules for UD and BD measurements. Interobserver misclassification rates were more than intraobserver misclassification rates using either progressive disease or response criteria. The probability of misclassifying a tumor with the WHO criteria or RECIST was greatest with interobserver measurements when criteria for progression (43% BD, 30% UD) were used and lowest with intraobserver measurements when criteria for response (2.5% BD, 3.0% UD) were used. In addition, interobserver misclassification rates were more than intraobserver misclassification rates for both regular and irregular tumors.
Conclusion: Measurements of lung tumor size on CT scans are often inconsistent and can lead to an incorrect interpretation of tumor response. Consistency can be improved if the same reader performs serial measurements for any one patient.
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Facebook  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
R. Thiam, L. S. Fournier, L. Trinquart, J. Medioni, G. Chatellier, D. Balvay, B. Escudier, C. Dromain, C. A. Cuenod, and S. Oudard
Optimizing the size variation threshold for the CT evaluation of response in metastatic renal cell carcinoma treated with sunitinib
Ann. Onc.,
November 4, 2009;
(2009)
mdp466v1.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
H. A. Jacene, S. Leboulleux, S. Baba, D. Chatzifotiadis, B. Goudarzi, O. Teytelbaum, K. M. Horton, I. Kamel, K. J. Macura, H.-L. Tsai, et al.
Assessment of Interobserver Reproducibility in Quantitative 18F-FDG PET and CT Measurements of Tumor Response to Therapy
J. Nucl. Med.,
November 1, 2009;
50(11):
1760 - 1769.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. A. Tang, G. R. Pond, and E. X. Chen
Influence of an independent review committee on assessment of response rate and progression-free survival in phase III clinical trials
Ann. Onc.,
October 29, 2009;
(2009)
mdp478v1.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. Wang, N. Wu, M. D. Cham, and Y. Song
Tumor Response in Patients With Advanced Non-Small Cell Lung Cancer: Perfusion CT Evaluation of Chemotherapy and Radiation Therapy
Am. J. Roentgenol.,
October 1, 2009;
193(4):
1090 - 1096.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
K MARTEN, C DULLIN, W MACHANN, J S SCHMID, M DAS, K-P HERMANN, and C ENGELKE
Comparison of flat-panel-detector-based CT and multidetector-row CT in automated volumetry of pulmonary nodules using an anthropomorphic chest phantom
Br. J. Radiol.,
September 1, 2009;
82(981):
716 - 723.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
B. Zhao, L. P. James, C. S. Moskowitz, P. Guo, M. S. Ginsberg, R. A. Lefkowitz, Y. Qin, G. J. Riely, M. G. Kris, and L. H. Schwartz
Evaluating Variability in Tumor Measurements from Same-day Repeat CT Scans of Patients with Non-Small Cell Lung Cancer
Radiology,
July 1, 2009;
252(1):
263 - 272.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. J. Nietert, J. G. Ravenel, W. M. Leue, J. V. Miller, K. K. Taylor, E. S. Garrett-Mayer, and G. A. Silvestri
Imprecision in Automated Volume Measurements of Pulmonary Nodules and Its Effect on the Level of Uncertainty in Volume Doubling Time Estimation
Chest,
June 1, 2009;
135(6):
1580 - 1587.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
R. L. Wahl, H. Jacene, Y. Kasamon, and M. A. Lodge
From RECIST to PERCIST: Evolving Considerations for PET Response Criteria in Solid Tumors
J. Nucl. Med.,
May 1, 2009;
50(Suppl_1):
122S - 150S.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. A. Gavrielides, L. M. Kinnard, K. J. Myers, and N. Petrick
Noncalcified Lung Nodules: Volumetric Assessment with Thoracic CT
Radiology,
April 1, 2009;
251(1):
26 - 37.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
W. Cai, A. Kassarjian, M. A. Bredella, G. J. Harris, H. Yoshida, V. F. Mautner, R. Wenzel, and S. R. Plotkin
Tumor Burden in Patients with Neurofibromatosis Types 1 and 2 and Schwannomatosis: Determination on Whole-Body MR Images
Radiology,
March 1, 2009;
250(3):
665 - 673.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
W. A. Weber
The Use of Positron Emission Tomography versus Computerized Tomography Scans in the Assessment of Response to Therapy
ASCO Educational Book,
January 1, 2009;
2009(1):
474 - 478.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
T. Tanvetyanon, E. A. Eikman, E. Sommers, L. Robinson, D. Boulware, and G. Bepler
Computed Tomography Response, But Not Positron Emission Tomography Scan Response, Predicts Survival After Neoadjuvant Chemotherapy for Resectable Non-Small-Cell Lung Cancer
J. Clin. Oncol.,
October 1, 2008;
26(28):
4610 - 4616.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
F. Girvin and J. P. Ko
Pulmonary Nodules: Detection, Assessment, and CAD
Am. J. Roentgenol.,
October 1, 2008;
191(4):
1057 - 1069.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
C. H. Takimoto
Commentary: Tumor Growth, Patient Survival, and the Search for the Optimal Phase II Efficacy Endpoint
Oncologist,
October 1, 2008;
13(10):
1043 - 1045.
[Full Text]
[PDF]
|
|
|
|
|
|
|
P S MURPHY, T J McCARTHY, and A S K DZIK-JURASZ
The role of clinical imaging in oncological drug development
Br. J. Radiol.,
September 1, 2008;
81(969):
685 - 692.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
L. E. Dodd, E. L. Korn, B. Freidlin, C. C. Jaffe, L. V. Rubinstein, J. Dancey, and M. M. Mooney
Blinded Independent Central Review of Progression-Free Survival in Phase III Clinical Trials: Important Design Element or Unnecessary Expense?
J. Clin. Oncol.,
August 1, 2008;
26(22):
3791 - 3796.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
Y. Wang, R. J. van Klaveren, H. J. van der Zaag-Loonen, G. H. de Bock, H. A. Gietema, D. M. Xu, A. L. M. Leusveld, H. J. de Koning, E. T. Scholten, J. Verschakelen, et al.
Effect of Nodule Characteristics on Variability of Semiautomated Volume Measurements in Pulmonary Nodules Detected in a Lung Cancer Screening Program
Radiology,
August 1, 2008;
248(2):
625 - 631.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. N. Lara Jr, M. W. Redman, K. Kelly, M. J. Edelman, S. K. Williamson, J. J. Crowley, and D. R. Gandara
Disease Control Rate at 8 Weeks Predicts Clinical Benefit in Advanced Non-Small-Cell Lung Cancer: Results From Southwest Oncology Group Randomized Trials
J. Clin. Oncol.,
January 20, 2008;
26(3):
463 - 467.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
W. M. Stadler
Tumor Burden Endpoints and Phase II Clinical Trial Design
ASCO Educational Book,
January 1, 2008;
2008(1):
89 - 93.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
D. S. Gierada, T. K. Pilgram, M. Ford, R. M. Fagerstrom, T. R. Church, H. Nath, K. Garg, and D. C. Strollo
Lung Cancer: Interobserver Agreement on Interpretation of Pulmonary Findings at Low-Dose CT Screening
Radiology,
December 1, 2007;
246(1):
265 - 272.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
T. G. Karrison, M. L. Maitland, W. M. Stadler, and M. J. Ratain
Design of Phase II Cancer Trials Using a Continuous Endpoint of Change in Tumor Size: Application to a Study of Sorafenib and Erlotinib in Non Small-Cell Lung Cancer
J Natl Cancer Inst,
October 3, 2007;
99(19):
1455 - 1461.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
C. Beigelman-Aubry, P. Raffy, W. Yang, R. A. Castellino, and P. A. Grenier
Computer-Aided Detection of Solid Lung Nodules on Follow-Up MDCT Screening: Evaluation of Detection, Tracking, and Reading Time
Am. J. Roentgenol.,
October 1, 2007;
189(4):
948 - 955.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
E. A. Hahn, D. Cella, O. Chassany, D. L. Fairclough, G. Y. Wong, R. D. Hays, and Clinical Significance Consensus Meeting Group
Precision of Health-Related Quality-of-Life Data Compared With Other Clinical Measures
Mayo Clin. Proc.,
October 1, 2007;
82(10):
1244 - 1254.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. D. Berry and G. J. R. Cook
Positron emission tomography in oncology
Br. Med. Bull.,
December 14, 2006;
(2006)
ldl013v1.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
M P Sampat, G J Whitman, T W Stephens, L D Broemeling, N A Heger, A C Bovik, and M K Markey
The reliability of measuring physical characteristics of spiculated masses on mammography
Br. J. Radiol.,
December 1, 2006;
79(Special_Issue_2):
S134 - S140.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
G. T. Budd, M. Cristofanilli, M. J. Ellis, A. Stopeck, E. Borden, M. C. Miller, J. Matera, M. Repollet, G. V. Doyle, L. W.M.M. Terstappen, et al.
Circulating Tumor Cells versus Imaging--Predicting Overall Survival in Metastatic Breast Cancer.
Clin. Cancer Res.,
November 1, 2006;
12(21):
6403 - 6409.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. G. Jennings, H. T. Winer-Muram, M. Tann, J. Ying, and I. Dowdeswell
Distribution of Stage I Lung Cancer Growth Rates Determined with Serial Volumetric CT Measurements
Radiology,
November 1, 2006;
241(2):
554 - 563.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
C. C. Jaffe
Measures of Response: RECIST, WHO, and New Alternatives
J. Clin. Oncol.,
July 10, 2006;
24(20):
3245 - 3251.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. P. Ko, R. Marcus, E. Bomsztyk, J. S. Babb, C. Stefanescu, M. Kaur, D. P. Naidich, and H. Rusinek
Effect of blood vessels on measurement of nodule volume in a chest phantom.
Radiology,
April 1, 2006;
239(1):
79 - 85.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
L. R. Goodman, M. Gulsun, L. Washington, P. G. Nagy, and K. L. Piacsek
Inherent Variability of CT Lung Nodule Measurements In Vivo Using Semiautomated Volumetric Measurements.
Am. J. Roentgenol.,
April 1, 2006;
186(4):
989 - 994.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. G. Armato III, J. L. Ogarek, A. Starkey, N. J. Vogelzang, H. L. Kindler, M. Kocherginsky, and H. MacMahon
Variability in mesothelioma tumor response classification.
Am. J. Roentgenol.,
April 1, 2006;
186(4):
1000 - 1006.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
G. D. Shah, S. Kesari, R. Xu, T. T. Batchelor, A. M. O'Neill, F. H. Hochberg, B. Levy, J. Bradshaw, and P. Y. Wen
Comparison of linear and volumetric criteria in assessing tumor response in adult high-grade gliomas
Neuro-oncol,
January 1, 2006;
8(1):
38 - 46.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
R. F. Munden, S. S. Swisher, C. W. Stevens, and D. J. Stewart
Imaging of the Patient with Non-Small Cell Lung Cancer
Radiology,
December 1, 2005;
237(3):
803 - 818.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
H. I. Scher, M. J. Morris, W. K. Kelly, L. H. Schwartz, and G. Heller
Prostate Cancer Clinical Trial End Points: "RECIST"ing a Step Backwards
Clin. Cancer Res.,
July 15, 2005;
11(14):
5223 - 5232.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
V. Goh, S. Halligan, J.-A. Hugill, P. Bassett, and C. I. Bartram
Quantitative Assessment of Colorectal Cancer Perfusion Using MDCT: Inter- and Intraobserver Agreement
Am. J. Roentgenol.,
July 1, 2005;
185(1):
225 - 231.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
D. R. Aberle, C. Chiles, C. Gatsonis, B. J. Hillman, C. D. Johnson, B. L. McClennan, D. G. Mitchell, E. D. Pisano, M. D. Schnall, and A. G. Sorensen
Imaging and Cancer: Research Strategy of the American College of Radiology Imaging Network
Radiology,
June 1, 2005;
235(3):
741 - 751.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. M. Goo, T. Tongdee, R. Tongdee, K. Yeo, C. F. Hildebolt, and K. T. Bae
Volumetric Measurement of Synthetic Lung Nodules with Multi-Detector Row CT: Effect of Various Image Reconstruction Parameters and Segmentation Thresholds on Measurement Accuracy
Radiology,
June 1, 2005;
235(3):
850 - 856.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. J. Byrne and A. K. Nowak
Modified RECIST criteria for assessment of response in malignant pleural mesothelioma
Ann. Onc.,
February 1, 2004;
15(2):
257 - 260.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. O. Park, S. I. Lee, S. Y. Song, K. Kim, W. S. Kim, C. W. Jung, Y. S. Park, Y.-H. Im, W. K. Kang, M. H. Lee, et al.
Measuring Response in Solid Tumors: Comparison of RECIST and WHO Response Criteria
Jpn. J. Clin. Oncol.,
October 1, 2003;
33(10):
533 - 537.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
