This Article Abstract Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Save to my personal folders Download to citation manager Rights & Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Timmerman, D. Articles by Valentin, L. Search for Related Content PubMed PubMed Citation Articles by Timmerman, D. Articles by Valentin, L. Social Bookmarking                            What's this?

Logistic Regression Model to Distinguish Between the Benign and Malignant Adnexal Mass Before Surgery: A Multicenter Study by the International Ovarian Tumor Analysis Group

From the Department of Obstetrics and Gynecology, University Hospitals Katholieke Universiteit Leuven; Department of Electrical Engineering, Katholieke Universiteit Leuven, Leuven, Belgium; Istituto di Clinica Ostetrica e Ginecologica, Università Cattolica del Sacro Cuore, Rome; Dipartimento di Scienze Cliniche, Sacco, Università di Milano, Milan, Italy; Department of Obstetrics and Gynaecology, St George's Hospital Medical School, University of London; King's College London, London, United Kingdom; and Department of Obstetrics and Gynecology, University Hospital Malmö, Malmö Sweden.

Address reprint requests to Dirk Timmerman, MD, PhD, Department of Obstetrics and Gynecology, University Hospitals, Katholieke Universiteit Leuven, Herestraat 49, B-3000 Leuven, Belgium; e-mail: dirk.timmerman{at}uz.kuleuven.ac.be

	ABSTRACT

TOP ABSTRACT INTRODUCTION PATIENTS AND METHODS RESULTS DISCUSSION Appendix Authors' Disclosures of... REFERENCES

 
PURPOSE: To collect data for the development of a more universally useful logistic regression model to distinguish between a malignant and benign adnexal tumor before surgery.

PATIENTS AND METHODS: Patients had at least one persistent mass. More than 50 clinical and sonographic end points were defined and recorded for analysis. The outcome measure was the histologic classification of excised tissues as malignant or benign.

RESULTS: Data from 1,066 patients recruited from nine European centers were included in the analysis; 800 patients (75%) had benign tumors and 266 (25%) had malignant tumors. The most useful independent prognostic variables for the logistic regression model were as follows: (1) personal history of ovarian cancer, (2) hormonal therapy, (3) age, (4) maximum diameter of lesion, (5) pain, (6) ascites, (7) blood flow within a solid papillary projection, (8) presence of an entirely solid tumor, (9) maximal diameter of solid component, (10) irregular internal cyst walls, (11) acoustic shadows, and (12) a color score of intratumoral blood flow. The model containing all 12 variables (M1) gave an area under the receiver operating characteristic curve of 0.95 for the development data set (n = 754 patients). The corresponding value for the test data set (n = 312 patients) was 0.94; and a probability cutoff value of .10 gave a sensitivity of 93% and a specificity of 76%.

CONCLUSION: Because the model was constructed from multicenter data, it is more likely to be generally applicable. The effectiveness of the model will be tested prospectively at different centers.

	INTRODUCTION

TOP ABSTRACT INTRODUCTION PATIENTS AND METHODS RESULTS DISCUSSION Appendix Authors' Disclosures of... REFERENCES

 
The preoperative assessment of adnexal tumors remains a major challenge for the gynecologist. Advances in surgery have provided more treatment options, but their potential usefulness depends on a prior assessment of the mass using noninvasive procedures. Appropriate surgical treatment is essential because the rupture of a stage 1 ovarian cancer during the operation may worsen the prognosis.¹

In 1989, Granberg et al² reported that transvaginal sonographic images of an ovarian mass could be used to predict the likelihood of malignancy. This approach was later incorporated into scoring systems to improve test performance.^3-5 In 1989, it was also shown that an ultrasound-derived index of tumoral blood flow might be an important risk factor.⁶ Other workers used menopausal status and the serum CA-125 level to produce a risk-of-malignancy index.⁷ The logical extension of this work was to use logistic regression models^8,9 or neural networks^10,11 to establish the most important variables and improve test performance. However, the results of prospective evaluations of these models at different centers were not encouraging.^12-14 One study reported values for sensitivity and specificity figures as low as 62% and 79%, respectively,¹⁴ whereas another study reported unbalanced figures of 9% and 91%, respectively.¹² A retrospective assessment of all reports suggests that the study populations were insufficiently large for model development or evaluation because of differences in the prevalence of different types of ovarian malignancies and rare benign tumors. There were also variations in the interpretation of ultrasound-derived end points.

The aim of the International Ovarian Tumor Analysis Group study was to minimize the limitations of previous reports by studying a total of 1,000 patients with a persistent adnexal mass at a minimum of six centers working with a common protocol. More than 50 defined variables were recorded on a database for the development of a logistic regression model that could be used to achieve high sensitivity (> 90%) for malignant ovarian tumors (and a specificity > 75%).

The multicenter approach was chosen as the most likely method to achieve a model that might be more effectively applicable to prospective studies in different clinic populations.

	PATIENTS AND METHODS

TOP ABSTRACT INTRODUCTION PATIENTS AND METHODS RESULTS DISCUSSION Appendix Authors' Disclosures of... REFERENCES

 
The study was prospective and multicenter. The protocol was ratified by the local ethics committee at each recruitment center.

Recruitment Centers
There were nine centers from five countries: University Hospital Malmö (Sweden), University Hospital Leuven (Belgium), Universita del Sacro Cuore Rome (Italy), Dipartimento di Scienze Cliniche, Sacco University of Milan (Italy), Hôpital Boucicaut Paris (France), Centre Medical des Pyramides, Maurepas (France), King's College Hospital London (United Kingdom), Istituto di Scienze Biomediche Ospedale, San Gerado Universita di Milano, Monza (Italy), and Universita degli Studi di Napoli, Naples (Italy).

Patients
Inclusion criteria were as follows: patients presenting with at least one overt persistent adnexal mass who were assessed by a principal investigator at one of the participating centers were eligible for inclusion in the study. Data from the apparent worse case mass were used for this study.

Patients were excluded from study for the following reasons: pregnancy or refusal of transvaginal sonography, surgery more than 120 days after sonographic assessment, disagreement in the classification (malignant or benign) between the original pathology report and the report of an expert reviewer, or incomplete submission of the data.

Data Collection
A dedicated, secure data collection system was developed for the study.¹⁵ A unique identifier was generated automatically for each patient's record based on the identifier for the center, the patient's birthday, the date of the ultrasound scan, and the follow-up number. Clinicians at each center could only view or update patient records from their own center. Data security was ensured by not recording the patient's name and by encrypting all data communication between the browser-based data entry and the central database using a 56-bit SSL (Secure Socket Layer) certificate. Data integrity was ensured by client-side JavaScript checks, server-side XML-encoded rules based on the study protocol, and manual checks by three experts.

Clinical Variables
A family history that included the number of first-degree relatives with ovarian or breast cancer was taken from each patient. Demographic data included the patient's age, menopausal status, day of menstrual cycle (if appropriate), previous hormonal therapy, and surgical history. Women 50 years of age who had undergone a hysterectomy were defined as postmenopausal.

Sonographic End Points
A transvaginal scan was performed in all cases. Transabdominal sonography was used to examine a large mass that could not be seen in its entirety using a transvaginal probe. A standardized sonographic procedure was used at all centers. Gray scale and color Doppler images were used to obtain over 40 morphologic and blood flow end points to characterize each adnexal mass. These criteria have been illustrated, described and defined.¹⁶ When intratumoral blood flow velocity waveforms were not detected, the peak systolic velocity, time averaged maximum velocity, the pulsatility index, and the resistance index were coded as 2.0 cm/sec, 1 cm/sec, 3.0 cm/sec, and 1.0 cm/sec, respectively, for use in mathematical modeling. The presence or absence of pain during the examination was recorded. Finally, the investigator gave a subjective assessment of whether the mass was likely to be malignant or benign.

Serum Tumor Marker
Centers were encouraged to measure the level of serum CA-125 in peripheral blood from all patients, but the availability of this biochemical end point was not an essential requirement for recruitment into the study. The immunoradiometric assay CA-125 II (Centocor, Malvern, PA; or Cis-Bio, Gif-sur-Yvette, France; or Abbott Axsym system, REF 3B41-22, Abbott Laboratories Diagnostic Division, Abbott Park, IL; or Immuno-l-analyser; Bayer Diagnostics, Tarrytown, NY; or Vidas; bioMétrieux, Marcy l’Etoile, France) was used and the results expressed in units per milliliter.

Outcome Measures
The final outcome measures of the study were the histologic diagnosis and, in the cases of malignancy, the surgical stage. Surgery was performed in the case of a mass classified as persistent (ie, still present 6 to 12 weeks after the initial scan). In cases of symptomatic masses, suspected malignancy, or at the patient's request, surgery was performed more quickly, either by laparoscopy or laparotomy according to the surgeon's judgment. All excised tissues were sampled for histologic examination at the local center. Tumors were classified according to the criteria recommended by the International Federation of Gynecology and Obstetrics.¹⁷ The degree of differentiation of malignant tumors was recorded. The pathologic samples from approximately 10% of the patients were randomly selected for peer review by Professor Ph. Moerman (Katholieke Universiteit, Leuven, Belgium).

Statistical Analysis
The data were thoroughly inspected using univariate analyses (contingency tables and basic descriptive statistics) and multivariate analyses (principal component analysis, bi-plots, scatter matrices, canonical correlation analysis, and logistic regression) to explore the data and to detect multicollinearity and outliers. Manual checks of any outliers were performed to eliminate mistakes that had occurred during submission of the data. To address the problem of characterizing preoperatively a mass as malignant or benign, we randomly stratified 70% of patient data to construct the logistic regression model and 30% for use as the test data. The data were stratified to ensure that the proportion of malignant and benign masses and the proportion of masses derived from each contributing center were the same for the development set and the test set. Then based on forward-backward selection methods (using the statistical significance level of the differences in ²) and after checking for interactions among the factors and determining whether the model was linear in the logit for continuous factors, an input selection was made for logistic regression models.^18-20 Receiver operating characteristic (ROC) curves were constructed, and the method proposed by DeLong et al²¹ was used to check for statistically significant differences in the area under the ROC curves.

	RESULTS

TOP ABSTRACT INTRODUCTION PATIENTS AND METHODS RESULTS DISCUSSION Appendix Authors' Disclosures of... REFERENCES

 
A total of 1,149 patients were recruited. Data from 83 patients (7%) were excluded (eight because of pregnancy, 31 because surgery was undertaken more than 120 days from the sonographic assessment, 42 because of incomplete submission of data, and two because of disagreement over the histologic diagnosis). Most missing data were the result of missing pathology owing to no operation being performed. Data from 1,066 patients (93%) were available for statistical analysis and model development.

The overall mean age of the patients was 47 years (range, 17 to 94 years), the overall proportion of patients who were postmenopausal was 40.5%, the overall proportion who were nulliparous was 37.7%, and the overall proportion receiving hormonal therapy was 22.1% (Table 1). There were a total of 52 stage I primary ovarian cancers.

Logistic Regression Analysis
Stepwise multivariate regression analysis of the development data set (754 patients) resulted in the delineation of the optimal set of variables that could be included in the equation: (1) personal history of ovarian cancer (yes = 1, no = 0), (2) current hormonal therapy (yes = 1, no = 0), (3) age of the patient (in years), (4), maximum diameter of the lesion (in millimeters), (5) the presence of pain during the examination (yes = 1, no = 0), (6) the presence of ascites (yes = 1, no = 0), (7) the presence of blood flow within a solid papillary projection (yes = 1, no = 0), (8) the presence of a purely solid tumor (yes = 1, no = 0), (9) maximal diameter of the solid component (expressed in millimeters, but with no increase > 50 mm), (10) irregular internal cyst walls (yes = 1, no = 0), (11) the presence of acoustic shadows (yes = 1, no = 0), and (12) the color score (1, 2, 3, or 4).

The logistic regression model (M1) provided the estimated probability of malignancy for a particular patient with an adnexal tumor. This probability was equal to y = 1/(1 + e^–z), where z = –6.7468 + 1.5985 (1) –0.9983 (2) + 0.0326 (3) + 0.00841 (4) –0.8577 (5) + 1.5513 (6) + 1.1737 (7) + 0.9281 (8) + 0.0496 (9) + 1.1421 (10) –2.3550 (11) + 0.4916 (12), and e is the mathematical constant and base value of natural logarithms.

Independent risk factors for the presence of malignancy included the patient's age (a 3.3% increase of odds for malignancy with each additional year of age), a personal history of ovarian cancer (odds ratio, 4.95), the maximum diameter of the lesion (0.8% for every millimeter increment), the maximum diameter of the solid component (5.1% for every millimeter increase with an upper limit of 50 mm; no increase > 50 mm), the presence of ascites (odds ratio, 4.72), the presence of blood flow within a solid papillary projection (odds ratio, 3.23), the presence of a purely solid lesion (odds ratio, 2.53), the presence of irregular internal cyst walls (odds ratio, 3.13), increase in color score (odds ratio, 1.64 for every one unit increase). Factors that reduced the risk of malignancy included the current use of hormonal therapy (odds ratio, 0.369), the presence of pain during the ultrasound examination (odds ratio, 0.424), and the presence of acoustic shadows (odds ratio, 0.095).

A simpler version (M2) using six selected variables was also developed. These were the six variables that were first entered into the model M1 when using stepwise selection of variables. The variables included in the equation M2 were (1) age of the patient (in years), (2) the presence of ascites (yes = 1, no = 0), (3) the presence of blood flow within a solid papillary projection (yes = 1, no = 0), (4) maximal diameter of the solid component (expressed in millimeters, but with no increase > 50 mm), (5) irregular internal cyst walls (yes = 1, no = 0), and (6) the presence of acoustic shadows (yes = 1, no = 0). For the reduced model the probability was equal to y = 1/(1 + e^–z), where z = –5.3718 + 0.0354 (1) + 1.6159 (2) + 1.1768 (3) + 0.0697 (4) + 0.9586 (5) –2.9486 (6).

Model Evaluation
The final model (M1) applied to the development data set gave the ROC shown in Figure 1. The area under the curve (AUC) was 0.95 (SE = 0.009), and the corresponding value for model (M2) was 0.93 (SE = 0.012). The numbers of malignant and benign tumors that were correctly or incorrectly classified using M1 is shown against the use of different probability levels in Table 5. A probability value of .10 gave a sensitivity of 93% and a specificity of 77%. The corresponding values for M2 at a probability level of .10 were 92% and 75%, respectively.

View larger version (11K): [in this window] [in a new window]   Fig 1. The receiver operating characteristic curves of the logistic regression model constructed on the development data set (n = 754) and applied to the test set (n = 312). The areas under the curve are 0.946 and 0.942, respectively.

View larger version (14K): [in this window] [in a new window]   Fig 2. The receiver operating characteristic (ROC) curves of the logistic regression model (M1) and ROC of the Risk of Malignancy Index (RMI)7 and ROC of an old logistic regression model (LR) by Timmerman et al9 applied to the test set cases with serum CA-125 results available (n = 236). The areas under the curve (AUC) are 0.94, 0.87, and 0.90, respectively.

	DISCUSSION

TOP ABSTRACT INTRODUCTION PATIENTS AND METHODS RESULTS DISCUSSION Appendix Authors' Disclosures of... REFERENCES

 
This is the first report of a prospective, multicenter study to collect data for the development of a mathematical model to distinguish between malignant and benign overt adnexal masses before surgery. The study identified several defined indices that can be used to discriminate between malignant and benign masses. The developed models are based on a large number and variety of malignant and benign ovarian masses. We are optimistic that this development should provide a more robust model than previous versions. Furthermore, the multicenter aspect of this study implies that any equation derived from the study is more likely to be generally applicable to other populations. In this context, it is reassuring that when the model (M1) was tested on the data from each individual center, the overall test performance did not deteriorate.

The results of the internal validation from this study also compare favorably with the results of previous attempts on smaller datasets to validate mathematical models prospectively.^12,13,14,22 Moreover, our model (M1) had significantly better test results on the test set cases than previously published models that had been developed on smaller datasets.^7,9 Our simpler model (M2) might be easier to use than the full model (M1). However, our full model included all significant variables, and the simpler model performed significantly less well than this full model on the test data set.

Although the area under the ROC curve reflects the overall test performance, the optimal probability level must be chosen to suit each individual clinical setting. There is always a compromise between the sensitivity (true-positive rate) and the number of false-positive test results. For example, by selecting a probability level of .10, the sensitivity for malignancy (model M1) was 93% and the specificity was 76%. In previous studies, we showed that highly experienced operators using subjective impression as the basis to define malignancy gave a sensitivity and specificity of 96% and 90%, respectively,²³ or 88% and 96%, respectively.²⁴ For less experienced operators, the corresponding values were 86% and 80%.²³ Therefore, our model could be helpful to less experienced operators.

Our study confirms the results of previous studies that simple unilocular cysts have only a small risk of being malignant.^25-30 We know that some benign pathology is relatively easy to characterize,^24,31 whereas late-stage cancers are usually quite obvious. Previous reports relating to subjective impression of ovarian masses suggest that some are difficult to classify even for the most experienced operators.^23,24 We have shown in our study that with sufficient numbers of patients and variables, a multimodal model can be built that will classify most tumors accurately before surgery. New models may be developed that concentrate on the characterization of more complex or difficult pathology.

The best model (M1) was retrospectively developed and then evaluated on a separate test set of patients. Although the internal validation of the model with our test data gave encouraging results, a true assessment can only be made by prospective evaluation on different study populations. In the near future, we will start the second phase of the IOTA collaboration, where models M1 and M2 will be prospectively evaluated in new centers.

	Appendix

TOP ABSTRACT INTRODUCTION PATIENTS AND METHODS RESULTS DISCUSSION Appendix Authors' Disclosures of... REFERENCES

 
Members of the IOTA Steering Committee: Dirk Timmerman, Lil Valentin, Thomas H. Bourne, William P. Collins, Sabine Van Huffel, and Ignace Vergote.

IOTA principal investigators (in alphabetical order): Jean-Pierre Bernard, Maurepas, France; Thomas H. Bourne, London, United Kingdom; Enrico Ferrazzi, Milan, Italy; Davor Jurkovic, London, United Kingdom; Fabrice Lécuru, Paris, France; Andrea Lissoni, Monza, Italy; Ulrike Metzger, Paris, France; Dario Paladini, Napels, Italy; Antonia Testa, Roma, Italy; Dirk Timmerman, Leuven, Belgium; Lil Valentin, Malmö, Sweden; Caroline Van Holsbeke, Leuven, Belgium; Sabine Van Huffel, Leuven, Belgium; Ignace Vergote, Leuven, Belgium; Gerardo Zanetta [deceased], Monza, Italy.

	Authors' Disclosures of Potential Conflicts of Interest

TOP ABSTRACT INTRODUCTION PATIENTS AND METHODS RESULTS DISCUSSION Appendix Authors' Disclosures of... REFERENCES

 
The authors indicated no potential conflicts of interest.

	NOTES

 
Supported by interdisciplinary research grants of the research council of the Katholieke Universiteit Leuven, Belgium (IDO/99/03 and IDO/02/09 projects), by the Belgian Programme on Interuniversity Poles of Attraction (phase V-22), by the Concerted Action Project AMBioRICS of the Flemish Community, by the FWO project G.0407.02, by the European Union Network of Excellence BIOPATTERN (contract No. FP6-2002-IST 508803), by research grants from the Swedish Medical Research Council (Grants No. K98-17X - 11,605 - 03A, K2001 - 72X - 11,605 - 06A and K2002- 72X- 11,605- 07B), by funds administered by Malmö University Hospital, Allmänna Sjukhusets i Malmö Stiftelse för bekämpande av cancer (the Malmö General Hospital Foundation for fighting against cancer), and ALF-medel (a Swedish governmental grant from the region of Scania).

Presented in part at an invited lecture at the 14th World Congress of Ultrasound in Obstetrics and Gynecology, organized by the International Society of Ultrasound in Obstetrics and Gynecology, August 31- September 4, 2004, Stockholm, Sweden.

Authors' disclosures of potential conflicts of interest are found at the end of this article.

	REFERENCES

TOP ABSTRACT INTRODUCTION PATIENTS AND METHODS RESULTS DISCUSSION Appendix Authors' Disclosures of... REFERENCES

 
1. Vergote I, De Brabanter J, Fyles A, et al: Prognostic importance of degree of differentiation and cyst rupture in stage I invasive epithelial ovarian carcinoma. Lancet 357:176-182, 2001[CrossRef][Medline]

2. Granberg S, Wikland M, Jansson I: Macroscopic characterization of ovarian tumors and the relation to the histological diagnosis: Criteria to be used for ultrasound evaluation. Gynecol Oncol 35:139-144, 1989[CrossRef][Medline]

3. Sassone AM, Timor-Tritsch IE, Artner A, et al: Transvaginal sonographic characterisation of ovarian disease: Evaluation of a new scoring system to predict ovarian malignancy. Obstet Gynecol 78:70-76, 1991[Medline]

4. Timor-Tritsch LE, Lerner JP, Monteagudo A, et al: Transvaginal ultrasonographic characterization of ovarian masses by means of color flow-directed Doppler measurements and a morphologic scoring system. Am J Obstet Gynecol 168:909-913, 1993[Medline]

5. Lerner JP, Timor-Tritsch IE, Federman A, et al: Transvaginal ultrasonographic characterization of ovarian masses with an improved, weighted scoring system. Am J Obstet Gynecol 170:81-85, 1994[Medline]

6. Bourne T, Campbell S, Steer CV, et al: Transvaginal colour flow imaging: A possible new screening technique for ovarian cancer. BMJ 299:1367-1370, 1989

7. Jacobs I, Oram D, Fairbanks J, et al: A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynaecol 97:922-929, 1990[Medline]

8. Tailor A, Jurkovic D, Bourne TH, et al: Sonographic prediction of malignancy in adnexal masses using multivariate logistic regression analysis. Ultrasound Obstet Gynecol 10:41-47, 1997[CrossRef][Medline]

9. Timmerman D, Bourne T, Tailor A, et al: A comparison of methods for the pre-operative discrimination between benign and malignant adnexal masses: The development of a new logistic regression model. Am J Obstet Gynecol 181:57-65, 1999[CrossRef][Medline]

10. Timmerman D, Verrelst H, Bourne TH, et al: Artificial neural network models for the pre-operative discrimination between malignant and benign adnexal masses. Ultrasound Obstet Gynecol 13:17-25, 1999[CrossRef][Medline]

11. Tailor A, Jurkovic D, Bourne TH, et al: Sonographic prediction of malignancy in adnexal masses using an artificial neural network. Br J Obstet Gynaecol 106:21-30, 1999[Medline]

12. Aslam N, Banerjee S, Carr JV, et al: Prospective evaluation of logistic regression models for the diagnosis of ovarian cancer. Obstet Gynecol 96:75-80, 2000[CrossRef][Medline]

13. Mol BWJ, Boll D, De Kanter M, et al.: Distinguishing the benign and malignant adnexal mass: An external validation of prognostic models. Gynecol Oncol 80:162-167, 2001[CrossRef][Medline]

14. Valentin L, Hagen B, Tingulstad S, et al: Comparison of ‘pattern recognition’ and logistic regression models for discrimination between benign and malignant pelvic masses: A prospective cross validation. Ultrasound Obstet Gynecol 18:357-365, 2001[CrossRef][Medline]

15. Aerts S, Antal P, Timmerman D, et al: Web based data collection for ovarian cancer: A case study. Proceedings of the 15th IEEE Symposium on Computer Based Medical Systems (CBMS), Maribor, Slovenia, 2002 (abstr)

16. Timmerman D, Valentin L, Bourne TH, et al: Terms, definitions and measurements to describe the ultrasonographic features of adnexal tumors: A consensus opinion from the international ovarian tumor analysis (IOTA) group. Ultrasound Obstet Gynecol 16:500-505, 2000[CrossRef][Medline]

17. Heintz APM, Odicino F, Maisonneuve P, et al: Carcinoma of the Ovary: 25th Annual Report on the Results of Treatment in Gynecological Cancer. Int J Gynaecol Obstet 83(suppl 1):S135-S137, 2003

18. Harrell FE Jr: Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY, Springer Series in Statistics, 2001

19. Hastie T, Tibshirani R: Generalized Additive Models: Monographs on Statistics and Applied Probability. Boca Ratan, FL, Chapman and Hall, 1999

20. Hosmer DW, Lemeshow S: Applied Logistic Regression: Wiley Series in Probability and Statistics. New York, NY, John Wiley & Sons, 2000

21. De Long ER, De Long DM, Clarke-Pearson DL: Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 44:837-845, 1988[CrossRef][Medline]

22. Timmerman D, Verrelst H, Collins WP, et al: Distinguishing the benign and malignant adnexal mass: An external validation of prognostic models. Gynecol Oncol 83:166-168, 2001[CrossRef][Medline]

23. Timmerman D, Schwärzler P, Collins WP, et al: Subjective assessment of adnexal masses using ultrasonography: An analysis of interobserver variability and experience. Ultrasound Obstet Gynecol 13:11-16, 1999[CrossRef][Medline]

24. Valentin L: Pattern recognition of pelvic masses by gray-scale ultrasound imaging: The contribution of Doppler ultrasound. Ultrasound Obstet Gynecol 14:338-347, 1999[CrossRef][Medline]

25. Goldstein SR, Subramanyam B, Snyder J, et al: The postmenopausal cystic adnexal mass: The potential role of ultrasound in conservative management. Obstet Gynecol 73:8-10, 1988

26. Obwegeser R, Deutinger J, Bernascheck G: The risk of malignancy with an apparently simple adnexal cyst on ultrasound. Arch Gynecol Obstet 253:117-120, 1993[CrossRef][Medline]

27. Kroon E, Andolf E: Diagnosis and follow-up of simple ovarian cysts detected by ultrasound in postmenopausal women. Obstet Gynecol 85:211-214, 1995[CrossRef][Medline]

28. Gerber B, Muller H, Kulz T, et al: Simple ovarian cysts in premenopausal patients. Int J Gynaecol Obstet 57:49-55, 1997[CrossRef][Medline]

29. Bailey C, Ueland F, Land GL, et al: The malignant potential of small cystic ovarian tumors in women over 50 years of age. Gynecol Oncol 69:3-7, 1998[CrossRef][Medline]

30. Ekerhovd E, Wienerroith H, Staudach A, et al: Preoperative assessment of unilocular adnexal cysts by transvaginal sonography: A comparison between sonographic morphological imaging and histopathologic diagnosis. Am J Obstet Gynecol 184:48-54, 2001[CrossRef][Medline]

31. Jermy K, Luise C, Bourne T: The characterization of common ovarian cysts in premenopausal women. Ultrasound Obstet Gynecol 17:140-144, 2001[CrossRef][Medline]

Submitted February 25, 2005; accepted September 7, 2005.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Facebook    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				A. Bokor, C.M. Kyama, L. Vercruysse, A. Fassbender, O. Gevaert, A. Vodolazkaia, B. De Moor, V. Fulop, and T. D'Hooghe Density of small diameter sensory nerve fibres in endometrium: a semi-invasive diagnostic test for minimal to mild endometriosis Hum. Reprod., August 18, 2009; (2009) dep283v1. [Abstract] [Full Text] [PDF]

				F. Amor, H. Vaccaro, J. L. Alcazar, M. Leon, J. M. Craig, and J. Martinez Gynecologic Imaging Reporting and Data System: A New Proposal for Classifying Adnexal Masses on the Basis of Sonographic Findings J. Ultrasound Med., March 1, 2009; 28(3): 285 - 291. [Abstract] [Full Text] [PDF]

				C. Van Holsbeke, B. Van Calster, A. C. Testa, E. Domali, C. Lu, S. Van Huffel, L. Valentin, and D. Timmerman Prospective Internal Validation of Mathematical Models to Predict Malignancy in Adnexal Masses: Results from the International Ovarian Tumor Analysis Study Clin. Cancer Res., January 15, 2009; 15(2): 684 - 691. [Abstract] [Full Text] [PDF]

				P. Yoruk, O. Dundar, B. Yildizhan, L. Tutuncu, and T. Pekin Comparison of the Risk of Malignancy Index and Self-Constructed Logistic Regression Models in Preoperative Evaluation of Adnexal Masses J. Ultrasound Med., October 1, 2008; 27(10): 1469 - 1477. [Abstract] [Full Text] [PDF]

				B. Van Calster, D. Timmerman, T. Bourne, A. C. Testa, C. Van Holsbeke, E. Domali, D. Jurkovic, P. Neven, S. Van Huffel, and L. Valentin Discrimination Between Benign and Malignant Adnexal Masses by Specialist Ultrasound Examination Versus Serum CA-125 J Natl Cancer Inst, November 21, 2007; 99(22): 1706 - 1714. [Abstract] [Full Text] [PDF]

				D. Timmerman, B. Van Calster, D. Jurkovic, L. Valentin, A. C. Testa, J.-P. Bernard, C. Van Holsbeke, S. Van Huffel, I. Vergote, and T. Bourne Inclusion of CA-125 Does Not Improve Mathematical Models Developed to Distinguish Between Benign and Malignant Adnexal Tumors J. Clin. Oncol., September 20, 2007; 25(27): 4194 - 4200. [Abstract] [Full Text] [PDF]

				R. G. Moore and R. C. Bast Jr How Do You Distinguish a Malignant Pelvic Mass From a Benign Pelvic Mass? Imaging, Biomarkers, or None of the Above J. Clin. Oncol., September 20, 2007; 25(27): 4159 - 4161. [Full Text] [PDF]

				C. Van Holsbeke, B. Van Calster, L. Valentin, A. C. Testa, E. Ferrazzi, I. Dimou, C. Lu, P. Moerman, S. Van Huffel, I. Vergote, et al. External Validation of Mathematical Models to Distinguish Between Benign and Malignant Adnexal Tumors: A Multicenter Study by the International Ovarian Tumor Analysis Group Clin. Cancer Res., August 1, 2007; 13(15): 4440 - 4447. [Abstract] [Full Text] [PDF]

				S. Guerriero, S. Ajossa, M. Gerada, G. B. Melis, J. L. Alcazar, E. Ferrazzi, A. A Lissoni, D. Dordoni, and D. Trio Pattern Recognition and Descriptive Sonographic Scoring in the Diagnosis of Ovarian Cancer * Reply J. Ultrasound Med., April 1, 2006; 25(4): 558 - 560. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Save to my personal folders Download to citation manager Rights & Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Timmerman, D. Articles by Valentin, L. Search for Related Content PubMed PubMed Citation Articles by Timmerman, D. Articles by Valentin, L. Social Bookmarking                            What's this?