- Research article
- Open Access
Identifying factors that may influence the classification performance of radiomics models using contrast-enhanced mammography (CEM) images
Cancer Imaging volume 22, Article number: 22 (2022)
Radiomics plays an important role in the field of oncology. Few studies have focused on the identification of factors that may influence the classification performance of radiomics models. The goal of this study was to use contrast-enhanced mammography (CEM) images to identify factors that may potentially influence the performance of radiomics models in diagnosing breast lesions.
A total of 157 women with 161 breast lesions were included. Least absolute shrinkage and selection operator (LASSO) regression and the random forest (RF) algorithm were employed to construct radiomics models. The classification result for each lesion was obtained by using 100 rounds of five-fold cross-validation. The image features interpreted by the radiologists were used in the exploratory factor analyses. Univariate and multivariate analyses were performed to determine the association between the image features and misclassification. Additional exploratory analyses were performed to examine the findings.
Among the lesions misclassified by both LASSO and RF ≥ 20% of the iterations in the cross-validation and those misclassified by both algorithms ≤5% of the iterations, univariate analysis showed that larger lesion size and the presence of rim artifacts and/or ripple artifacts were associated with more misclassifications among benign lesions, and smaller lesion size was associated with more misclassifications among malignant lesions (all p < 0.050). Multivariate analysis showed that smaller lesion size (odds ratio [OR] = 0.699, p = 0.002) and the presence of air trapping artifacts (OR = 35.568, p = 0.025) were factors that may lead to misclassification among malignant lesions. Additional exploratory analyses showed that benign lesions with rim artifacts and small malignant lesions (< 20 mm) with air trapping artifacts were misclassified by approximately 50% more in rate compared with benign and malignant lesions without these factors.
Lesion size and artifacts in CEM images may affect the diagnostic performance of radiomics models. The classification results for lesions presenting with certain factors may be less reliable.
It is important to find an accurate and efficient way to detect and diagnose breast cancer. In recent years, radiomics has played an increasingly important role in the field of oncology [1,2,3,4]. In radiomics, a high-throughput computer algorithm extracts large amounts of image features and converts medical images into quantitative data, showing decent results [5,6,7]. For breast cancer, radiomics has been extensively studied in research settings for diagnosis, treatment evaluation, and prognosis prediction [1,2,3,4, 8].
Contrast-enhanced mammography (CEM) is a technique that can simultaneously show the morphological and angiogenic characteristics of breast lesions [9, 10] and has a high spatial resolution comparable to that of conventional mammography [11, 12]. Several studies have developed and validated radiomics models in an attempt to achieve high diagnostic accuracy for breast lesions [13,14,15,16,17,18]. Although the diagnostic performance of radiomics models is promising, concerns still persist, as radiomics approaches are often regarded as black boxes and are less acceptable for clinical application [1, 2, 19, 20]. In other words, improvement in the overall diagnostic performance of radiomics models is still difficult to convert into practical clinical benefits, such as a reduction in unnecessary biopsies. Radiomics models are still not sufficiently reliable and interpretable to be used in the real-world diagnostic setting. In addition, few studies have examined imaging factors that may influence the diagnostic performance of the models.
The purpose of this study was to examine the performance of radiomics analysis in breast cancer diagnosis and preliminarily disentangle the black box of radiomics by identifying factors that may influence the classification results of radiomics models. Our study focused on breast lesions that were more likely to be misclassified by radiomics analysis and attempted to identify the potential image features that may influence the classification results from an interpretable perspective.
Materials and methods
This retrospective study was approved by the Institutional Review Board and Ethics Committee of the research center. The requirement for patient informed consent was waived. We collected consecutive CEM images between November 2018 and February 2020. The indications for CEM in this study included (1) problem solving for inconclusive findings on mammography or ultrasound screening; and (2) evaluation of symptomatic patients. The inclusion criteria were as follows: (1) patients with suspected breast lesions after physical examination or screening; (2) patients with referral for CEM by breast surgeons as part of diagnostic imaging; and (3) patients with final diagnoses that were confirmed by histopathological results. We excluded patients (1) with missing data and (2) with a history of breast surgery, breast radiotherapy, chemotherapy, or hormone treatment within 6 months prior to CEM examination. The patient inclusion and exclusion workflows are shown in Fig. 1. A total of 157 women with 161 breast lesions (47 benign, 29.2%; 114 malignant, 70.8%) were included in the study. The median age of the patients was 49 years (range, 21–70 years).
All CEM examinations were performed using Senographe Essential mammography units (GE Healthcare). Before the examination, a dose of 1.5 mL/kg body weight iodinated contrast material (Iohexol, 300–350 mg I/mL) was injected intravenously using an automated power injector at a flow rate of 3.0 mL/s, followed by a 10-mL bolus of saline. Two minutes after the injection, bilateral craniocaudal (CC) views were obtained first, beginning with the suspicious breast. Then, bilateral mediolateral oblique (MLO) views were acquired in the same order. In a single projection, a pair of low-energy (LE) and high-energy (HE) exposures was performed within 1.5 seconds. The HE and LE images were recombined to generate dual-energy subtraction (DES) images. All of the HE, LE, and DES images were used to construct the radiomics models.
CEM image evaluation
Two radiologists with 5–10 years of experience in breast imaging reviewed and interpreted all of the CEM images to obtain the image features. The radiologists were blinded to the histopathology results. When a discrepancy occurred in image evaluation, the final decision was made by consensus. The image features could be divided into two main groups: (1) basic image features and (2) artifact features. The basic image features included breast density, degree of background parenchymal enhancement (BPE), and lesion size. Breast density (a, b, c, or d) was evaluated using the LE images according to the Breast Imaging Reporting and Data System (BI-RADS) mammography lexicon . The degree of BPE (minimal, mild, moderate, or marked) was assessed using the DES images referring to the BI-RADS MRI lexicon . Lesion size was obtained by calculating the mean value of the largest lesion diameters on DES images measured by two independent radiologists. The artifact features included the presence of rim artifacts, ripple artifacts, vascular artifacts, and air trapping artifacts in DES images, as these artifacts occurred more often and might interfere with image quality [23, 24]. We defined artifacts located outside the lesion area as being absent since all the radiomics features were extracted from inside the lesion area and therefore might not interfere with artifacts outside the lesion area.
In addition, we extracted three objective quantitative features that might reflect the enhancement degree of the lesions. These features include the signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and background contrast ratio (BCR). Since these features are obtained through calculation and can also be affected by the abovementioned image features, such as artifacts, they were excluded in the factor analysis. We only examined the distribution pattern of these features among the lesions with different classification results. The detailed processes and calculation methods of these features are provided in the Supplemental Materials (Appendix E1).
Lesion delineation and feature extraction
The lesion contours were manually delineated with ITK-SNAP (version 3.6; www.itksnap.org) (25) by two radiologists together. For each lesion, a total of 6 regions of interest (ROIs) were delineated on the HE, LE, and DES images in the CC and MLO views. For multiple lesions within one breast, only the largest lesion was delineated.
Because the voxel was isotropic in-plane, we omitted the image resampling step. Gray-level discretization was performed to discretize all the images to 256 Gy levels. Spectral Mammography Kit (SMK) software (version 1.2.0, GE Healthcare) was used to extract the radiomics features. For each ROI, a total of 680 features, including 14 shape features, 18 first-order features, 24 Gy-level cooccurrence matrix (GLCM) features, 16 Gy-level run length matrix (GLRLM) features, 16 Gy-level size zone matrix (GLSZM) features, and 592 wavelet features, were extracted (Supplemental Table 1).
Feature selection and Radiomics model building
We employed two algorithms, L1-based least absolute shrinkage and selection operator (LASSO) regression  and the random forest (RF) algorithm , with all the radiomics features (680 features for each ROI), to construct the classification models. The “one-standard-error” rule  was used to select the best model when implementing LASSO regression. The reference standard of the classification results was the histopathological results. To obtain robust results regarding how the radiomics models classified each lesion, we conducted 100 rounds of five-fold cross-validation. During each round of cross-validation, to account for imbalanced class numbers between malignant and benign lesions, adjusted weights inversely proportional to the frequencies of each class in the training data were calculated and incorporated in building RF and LASSO regression [28,29,30]. Before analysis, all the extracted radiomics features were normalized. We performed the feature normalization using the training data and calculated the mean and standard deviation for each feature. Subsequently, the values of mean and standard deviation were used to normalize the features in the testing data. Besides, the dimensions of radiomics features were reduced using the training data (80% of the whole data). We removed highly correlated redundant radiomics features if the pairwise correlations were greater than 0.8. Specifically, if two radiomics features had a correlation greater than 0.8, the radiomics feature with the largest mean absolute correlation was removed. Then, the models were built on the remaining features in the training data and the classification results for the testing data determined by using the best cutoff value based on the Youden index  for both the LASSO and RF models were summarized. The area under the curve (AUC), accuracy, sensitivity, and specificity values in the testing dataset were calculated. The misclassification probability for each lesion was obtained. The details of this statistical procedure are provided in the Supplemental Materials (Appendix E2).
Definition of lesion with high/low misclassification probability
For both the LASSO and RF method, we defined a lesion as having a high misclassification probability if it was incorrectly classified for no less than 20.0% of 100 iterations and as having a low misclassification probability if it was incorrectly classified for no more than 5.0% of the iterations. To combine the results of the LASSO regression and RF models, we defined a lesion as having a high misclassification probability for both algorithms if the lesion was defined with a high misclassification probability by each algorithm at the same time; the equivalent definition was used to identify lesions with a low misclassification probability for both algorithms. Unless otherwise specified, lesions described below as having a high/low misclassification probability are those with a high/low misclassification probability as determined by both algorithms simultaneously.
Identification of factors influencing the classification performance of Radiomics models
Multivariate logistic regression was conducted using the type of lesion (high misclassification probability vs. low misclassification probability) as a dependent variable and the image features as independent variables. A factor that showed a statistically significant high or low odds ratio (OR) was determined as an influential factor.
Additional exploratory analyses
To directly evaluate how the factors identified in the previous analysis influence the performances of the radiomics models, we compared the correct classification rates between lesions with certain factors and the lesions without these factors based on the results of cross-validation.
In addition, to evaluate the performance of radiomics models on the data with/without influential factors, we performed two more sets of 100 rounds of five-fold cross-validation with both radiomics algorithms built on the data, including the lesions with/without the factors identified by the factor analysis. The AUC, accuracy, sensitivity, and specificity values in the testing dataset were calculated for comparison.
General statistical analysis
Continuous variables were described as the means ± standard deviations, and categorical variables were summarized as proportions (%). Independent t tests, Wilcoxon rank-sum tests, and Fisher’s exact tests were used as appropriate for the univariate analyses and additional exploratory analyses. A p value less than 0.050 was considered statistically significant. All analyses were implemented in R software (version 3.6.3) .
Summary of the study cohort and image features
A summary of the study cohort and image features is shown in Table 1. The mean age and lesion size in the malignant lesion group were significantly greater than those in the benign lesion group (p < 0.050). The distributions of different types of breast densities and degrees of BPE were significantly different between the two groups (both p < 0.050). For the different kinds of artifacts, significant differences were observed in the presence of ripple artifacts (p = 0.005) and vascular artifacts (p = 0.042), but no differences were found in the presence of rim artifacts (p = 1.000) and air trapping artifacts (p = 0.104) between the two groups. For the objective quantitative features, the benign lesion group showed lower SNR (p < 0.001), CNR (p < 0.001), and BCR values (p < 0.001) than the malignant lesion group.
Performance of Radiomics models based on cross-validation results
For the LASSO regression models, the average AUC, accuracy, sensitivity, and specificity values were 0.926 ± 0.047, 0.895 ± 0.061, 0.891 ± 0.085, and 0.908 ± 0.096, respectively. For the RF models, the average AUC, accuracy, sensitivity, and specificity values were 0.915 ± 0.055, 0.880 ± 0.068, 0.878 ± 0.097, and 0.886 ± 0.108, respectively. The statistics of the features used ≥20% of the times by LASSO and features with the largest permutation importance scores generated by RF in the cross-validation are given in the Supplemental Tables 2 and 3.
Summary of classification results for the lesions
The lesion classification results are shown in Fig. 2 (LASSO regression) and Fig. 3 (RF). For the LASSO regression models, 20 (12.4%) of the 161 lesions (5 benign; 15 malignant) were incorrectly classified for no less than 20.0% of the 100 iterations, and 116 (72.0%) of the 161 lesions (37 benign; 79 malignant) were incorrectly classified for no more than 5.0% of the iterations; for the RF models, 33 (20.5%) lesions (8 benign; 25 malignant) were misclassified for no less than 20.0% of 100 iterations, and 116 (72.0%) lesions (35 benign; 81 malignant) were incorrectly classified for no more than 5.0% of the iterations. Based on our definition, a total of 16 (9.9%) lesions (5 benign; 11 malignant) were defined as having a high misclassification probability, and 101 (62.7%) lesions (32 benign; 69 malignant) were defined as having a low misclassification probability.
Factors identified that may influence the classification performance of Radiomics models
A summary of the image features and the objective quantitative features in the subgroups of interest is shown in Table 2. The univariate analysis showed that larger lesion size (p = 0.003), the presence of rim artifacts (p < 0.001), and ripple artifacts (p = 0.042) may increase the misclassification rate for benign lesions. Among the malignant lesions, a smaller lesion size (p < 0.001) was found to be a factor that may be associated with misclassification. The distributions of the objective quantitative features are shown in Fig. 4. Among the benign lesions, compared with lesions with a low misclassification probability, lesions with a high misclassification probability showed higher values for the SNR, CNR, and BCR. Among the malignant lesions, compared with the lesions with low misclassification probability, the lesions with high misclassification probability showed lower values for the SNR, CNR, and BCR. All of the differences between the lesions with a high misclassification probability and lesions with a low misclassification probability were statistically significant (p < 0.050).
Multivariate analysis was only performed in the malignant lesion group since the small number of lesions in the benign lesion group prevented the logistic regression model from converging. In Table 3, the results show that a smaller lesion size (odds ratio [OR] = 0.699, p = 0.002) and the presence of air trapping artifacts (OR = 36.568, p = 0.025) may be factors that may result in the misclassification of malignant lesions.
In addition, both the univariate and multivariate analyses based on the LASSO regression models and RF models showed similar results (Supplemental Table 4-Table 7).
Results of additional exploratory analyses
Correct classification rates for lesions with/without influential factors
A summary of correct classification rates between lesions with and without certain influential factor is given in Table 4. A smaller lesion size (< 20 mm) increased the correct classification rate among benign lesions by 0.223 ± 0.098 (mean ± standard deviation) and 0.231 ± 0.095, and decreased the correct classification rate among malignant lesions by − 0.140 ± 0.049 and − 0.256 ± 0.069 for LASSO and RF, respectively. The present of rim artifacts decreased the correct classification rate among benign lesions by − 0.613 ± 0.193 and − 0.624 ± 0.140 for LASSO and RF, respectively. The present of ripple artifacts decreased the correct classification rate among benign lesions by − 0.126 ± 0.075 and − 0.165 ± 0.106 for LASSO and RF, respectively. The present of air trapping artifacts decreased the correct classification rate among malignant lesions by − 0.148 ± 0.056 and − 0.088 ± 0.054 for LASSO and RF, respectively. However, the presence of both smaller lesion size and air trapping artifacts decreased the correct classification rate among malignant lesions by − 0.458 ± 0.168 and − 0.559 ± 0.145 for LASSO and RF, respectively.
Performance of Radiomics models in the data with/without influential factors
We performed two more sets of 100 rounds of cross-validations among the data on the lesions with or without rim artifacts, ripple artifacts, and/or air trapping artifacts (with: 87 in total, 16 benign and 71 malignant; without: 74 in total, 31 benign and 43 malignant). We only considered valid classification results without prediction issues due to the small number of prediction categories. For the LASSO regression models in lesions with/without the abovementioned artifacts, the average AUC, accuracy, sensitivity, and specificity values were 0.875 ± 0.078 vs. 0.970 ± 0.071, 0.858 ± 0.097 vs. 0.965 ± 0.066, 0.851 ± 0.099 vs. 0.967 ± 0.088, and 0.898 ± 0.123 vs. 0.967 ± 0.092, respectively. For the RF models in lesions with/without the abovementioned artifacts, the average AUC, accuracy, sensitivity, and specificity values were 0.852 ± 0.085 vs. 0.961 ± 0.094, 0.830 ± 0.100 vs. 0.952 ± 0.079, 0.822 ± 0.123 vs. 0.953 ± 0.121, and 0.907 ± 0.124 vs. 0.968 ± 0.090, respectively.
Overall, the performance of the two algorithms (LASSO and RF) used in this study was comparable to that of the models in the published literature using radiomics features of CEM to classify breast lesions (AUC = 0.848–0.950, accuracy = 78.4–90.0%) [13,14,15,16].
The results of factor analyses showed that small lesion size and the presence of rim artifacts, ripple artifacts, and air trapping artifacts might influence classification performances in the LASSO regression models and RF radiomics models. To illustrate the findings, we provided a set of CEM images as examples in Fig. 5. As shown in Fig. 5A-C, benign lesions with larger lesion size and presenting with rim artifacts or ripple artifacts were more likely to be misclassified. Benign lesions that were less likely to be misclassified (Fig. 5D-F) were smaller in size and generally did not contain rim or ripple artifacts. In Fig. 5G-H, malignant lesions with smaller lesion size and presenting with air trapping artifacts were more likely to be misclassified. Malignant lesions that were less likely to be misclassified were generally larger and did not present with air trapping artifacts (Fig. 5J-L). The presence of artifacts seemed to be an influential factor that resulted in misclassification, and the influence could be bidirectional: some artifacts, such as rim artifacts and ripple artifacts, tended to influence the classification of a lesion as malignant, probably because these artifacts increase the signal intensity and/or heterogeneity of the lesions, while other artifacts, such as air trapping artifacts and negative enhancement artifacts, decrease the signal intensity of the lesions. Thus, lesions with such artifacts might be more likely to be classified as benign.
Our findings were further examined by the results of additional exploratory analyses. Based on the cross-validation results, correct classification rates could obviously decrease (approximately 50% on average) for benign lesions with rim artifacts and smaller malignant lesions (< 20 mm) with air trapping artifacts. Furthermore, model accuracy could obviously decrease by an average of 10–12% when the analyses were only performed for lesions with rim artifacts, ripple artifacts, and/or air trapping artifacts versus lesions without the artifacts.
Our findings could also be potentially explained by objective quantitative image features in an interpretable way. The SNR, CNR, and BCR values showed significantly different distributions between lesions with high misclassification probability and lesions with low misclassification probability in both the benign and malignant lesion groups. These results were also in line with the abovementioned findings and inferences. It is worth mentioning that the quantitative features may be associated with the presence of artifacts as well, so we did not include these features in our exploratory analyses. Benign lesions with high misclassification probability showed higher signal intensity after enhancement (Fig. 5A-C), while malignant lesions with high misclassification probability showed lower signal intensity (Fig. 5G-I). Several aspects could contribute to high lesion signal intensity, including the inherent characteristics of the lesion itself and external influential factors, which may further cause lesion misclassification by the radiomics models. Several quantitative studies of CEM have demonstrated that malignant lesions tend to show more obvious enhancement than benign lesions [33,34,35]. Some studies [36, 37] have noted that the enhancement intensity depends on the size of the tumor and is more obvious for larger lesions than for smaller lesions. In other words, larger benign lesions can also display strong enhancement, and smaller malignant lesions can also display slight enhancement. Furthermore, as reported by Yagil et al. , rim and ripple artifacts were the main artifacts commonly seen on CEM. Researchers [23, 39, 40] have stated that DES images are prone to rim artifacts of increased density as a result of radiation scattering. Additionally, BPE, which refers to the uptake of contrast medium by normal fibroglandular breast tissue [41, 42], may also add the signal intensity of the lesions. In contrast, air trapping artifacts, which represent the presence of air and create a dark area due to incomplete contact between the skin and the detector or compression paddle [23, 24], may result in more neutral signal intensity.
Although some scholars have considered that some artifacts in CEM images might not compromise image quality [24, 38], we found that some artifacts in CEM images might affect the diagnostic performance of radiomics models, and other scholars [23, 43, 44] have proposed that some artifacts may present challenges to image interpretation. Therefore, it is still necessary to stress the importance of high-quality images. Neppalli et al. reported  that the type, incidence, and severity of CEM-specific artifacts differ between image device vendors. To date, several image-processing algorithms have been developed to reduce artifacts and improve the image quality [46,47,48]. For example, scatter correction techniques are becoming commercially available , and the rim artifacts are not present in the newer systems . Furthermore, except for equipment- or technique-related factors, CEM-specific artifacts can also be alleviated by patient- or technologist-related factors. Therefore, it is also important to use standard and appropriate protocols during image acquisition and perform regular quality control tests  to prevent or minimize these artifacts.
There are some limitations in our study. First, the relatively small sample size is the main limitation. A larger sample may help provide more information with the same accuracy. Second, radiomics features derived from CEM, in general, could have inherent limitations caused by the two-dimensional nature of the images and compression. Third, more homogeneous baseline characteristics between benign and malignant lesions may potentially help better interpret the results. To avoid bias, we used 100 rounds of cross-validation instead of a single round to obtain “averaged” classification results. Performing factor analysis separately for benign and malignant lesions could further limit the impact of unbalanced characteristics.
Our study found that large lesion size and the presence of rim and/or ripple artifacts were associated with misclassification of benign lesions, and small lesion size and presence of air trapping artifacts were associated with misclassification of malignant lesions. The results imply that we should be aware that the results of radiomics models could be less reliable when these influential factors are present. Based on these findings, some methods, such as alleviating artifacts by using specific postprocessing algorithms , applying adequate compression of the breast , referring to the image information around the lesion , and employing an adjusted algorithm that considers these influential factors, can potentially help to build more accurate and interpretable radiomics classification models.
Availability of data and materials
The datasets generated and/or analysed during the current study are not publicly available due confidential information but are available from the corresponding author on reasonable request.
area under the curve
background parenchymal enhancement
gray-level cooccurrence matrix
gray-level run length matrix
gray-level size zone matrix
least absolute shrinkage and selection operator
Liu Z, Wang S, Dong D, Wei J, Fang C, Zhou X, et al. The applications of radiomics in precision diagnosis and treatment of oncology: opportunities and challenges. Theranostics. 2019;9:1303–22.
Rogers W, Thulasi Seetha S, Refaee TAG, Lieverse RIY, Granzier RWY, Ibrahim A, et al. Radiomics: from qualitative to quantitative imaging. Br J Radiol. 2020;93:20190948.
Conti A, Duggento A, Indovina I, Guerrisi M, Toschi N. Radiomics in breast cancer classification and prediction. Semin Cancer Biol. 2020;72:238–50.
Lee SH, Park H, Ko ES. Radiomics in breast imaging from techniques to clinical applications: a review. Korean J Radiol. 2020;21:779–92.
Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RGPM, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48:441–6.
Kumar V, Gu Y, Basu S, Berglund A, Eschrich SA, Schabath MB, et al. Radiomics: the process and the challenges. Magn Reson Imaging. 2012;30:1234–48.
Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology. 2015;278:563–77.
Valdora F, Houssami N, Rossi F, Calabrese M, Tagliafico AS. Rapid review: radiomics and breast cancer. Breast Cancer Res Tr. 2018;169:217–29.
Dromain C, Thibault F, Muller S, Rimareix F, Delaloge S, Tardivon A, et al. Dual-energy contrast-enhanced digital mammography: initial clinical results. Eur Radiol. 2011;21:565–74.
Ghaderi KF, Phillips J, Perry H, Lotfi P, Mehta TS. Contrast-enhanced mammography: current applications and future directions. Radiographics. 2019;39:1907–20.
Lalji UC, Jeukens CRLPN, Houben I, Nelemans PJ, van Engen RE, van Wylick E, et al. Evaluation of low-energy contrast-enhanced spectral mammography images by comparing them to full-field digital mammography using EUREF image quality criteria. Eur Radiol. 2015;25:2813–20.
Francescone MA, Jochelson MS, Dershaw DD, Sung JS, Hughes MC, Zheng J, et al. Low energy mammogram obtained in contrast-enhanced digital mammography (CEDM) is comparable to routine full-field digital mammography (FFDM). Eur J Radiol. 2014;83:1350–5.
Fanizzi A, Losurdo L, Basile TMA, Bellotti R, Bottigli U, Delogu P, et al. Fully automated support system for diagnosis of breast Cancer in contrast-enhanced spectral mammography images. J Clin Med. 2019;8:891.
Danala G, Patel B, Aghaei F, Heidari M, Li J, Wu T, et al. Classification of breast masses using a computer-aided diagnosis scheme of contrast enhanced digital mammograms. Ann Biomed Eng. 2018;46:1419–31.
Patel BK, Ranjbar S, Wu T, Pockaj BA, Li J, Zhang N, et al. Computer-aided diagnosis of contrast-enhanced spectral mammography: a feasibility study. Eur J Radiol. 2018;98:207–13.
Fusco R, Vallone P, Filice S, Granata V, Petrosino T, Rubulotta MR, et al. Radiomic features analysis by digital breast tomosynthesis and contrast-enhanced dual-energy mammography to detect malignant breast lesions. Biomed Signal Process Control. 2019;53:101568.
Losurdo L, Fanizzi A, Basile TMA, Bellotti R, Bottigli U, Dentamaro R, et al. Radiomics analysis on contrast-enhanced spectral mammography images for breast Cancer diagnosis: a pilot study. Entropy. 2019;21:1110.
Lin F, Wang Z, Zhang K, Yang P, Ma H, Shi Y, et al. Contrast-enhanced spectral mammography-based Radiomics Nomogram for identifying benign and malignant breast lesions of Sub-1 cm. Front Oncol. 2020;10:573630.
Limkin EJ, Sun R, Dercle L, Zacharaki EI, Robert C, Reuzé S, et al. Promises and challenges for the implementation of computational medical imaging (radiomics) in oncology. Ann Oncol. 2017;28:1191–206.
Verma V, Simone CB 2nd, Krishnan S, Lin SH, Yang J, Hahn SM. The rise of Radiomics and implications for oncologic management. J Natl Cancer Inst. 2017;109.
D’Orsi CJSE, Mendelson EB, Morris EA. ACR BI-RADS® atlas: breast imaging reporting and data system. Reston: American College of Radiology; 2013.
D’Orsi CJ, Sickles EA, Mendelson EB, Morris EA. ACR BI-RADS Atlas. Breast Imaging Reporting and Data System 2013.
Bhimani C, Li L, Liao L, Roth RG, Tinney E, Germaine P. Contrast-enhanced spectral mammography: modality-specific artifacts and other factors which may interfere with image quality. Acad Radiol. 2017;24:89–94.
Nori J, Gill MK, Vignoli C, Bicchierai G, De Benedetto D, Di Naro F, et al. Artefacts in contrast enhanced digital mammography: how can they affect diagnostic image quality and confuse clinical diagnosis? Insights Imaging. 2020;11:16.
Yuan G-X, Ho C-H, Lin C-J. An improved glmnet for l1-regularized logistic regression. J Machine Learn Res. 2012;13:1999–2030.
Breiman L. Random forests. Mach Learn. 2001;45:5–32.
Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33:1.
Maalouf M. Logistic regression in data analysis: an overview. Int J Data Analysis Techniques Strategies. 2011;3:281–99.
Chen C, Liaw A, Breiman L. Using random forest to learn imbalanced data. Berkeley: University of California; 2004;110:24.
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine learning in Python. J Machine Learn Res. 2011;12:2825–30.
Fluss R, Faraggi D, Reiser B. Estimation of the Youden index and its associated cutoff point. Biometrical J. 2005;47:458–72.
Chambers J. Software for data analysis: programming with R: Springer Science & Business Media; 2008.
Rudnicki W, Heinze S, Niemiec J, Kojs Z, Sas-Korczynska B, Hendrick E, et al. Correlation between quantitative assessment of contrast enhancement in contrast-enhanced spectral mammography (CESM) and histopathology-preliminary results. Eur Radiol. 2019;29:6220–6.
Deng CY, Juan YH, Cheung YC, Lin YC, Lo YF, Lin G, et al. Quantitative analysis of enhanced malignant and benign lesions on contrast-enhanced spectral mammography. Br J Radiol. 2018;91:20170605.
Lv Y, Chi X, Sun B, Lin S, Xing D. Diagnostic value of quantitative gray-scale analysis of contrast-enhanced spectral mammography for benign and malignant breast lesions. J Comput Assist Tomogr. 2020;44:405–12.
Dromain C, Vietti-Violi N, Meuwly JY. Angiomammography: a review of current evidences. Diagn. Interv. Imaging. 2019;100:593–605.
Luczynska E, Niemiec J, Heinze S, Adamczyk A, Ambicka A, Marcyniuk P, et al. Intensity and pattern of enhancement on CESM: prognostic significance and its relation to expression of Podoplanin in tumor Stroma - a preliminary report. Anticancer Res. 2018;38:1085–95.
Yagil Y, Shalmon A, Rundstein A, Servadio Y, Halshtok O, Gotlieb M, et al. Challenges in contrast-enhanced spectral mammography interpretation: artefacts lexicon. Clin Radiol. 2016;71:450–7.
Lancaster RB, Gulla S, De Los SJ, Umphrey HR. Contrast-enhanced spectral mammography in breast imaging. Semin Roentgenol. 2018;53:294–300.
James JJ, Tennant SL. Contrast-enhanced spectral mammography (CESM). Clin Radiol. 2018;73:715–23.
Savaridas SL, Taylor DB, Gunawardana D, Phillips M. Could parenchymal enhancement on contrast-enhanced spectral mammography (CESM) represent a new breast cancer risk factor? Correlation with known radiology risk factors. Clin Radiol. 2017;72:1085.e1081–1085.e1089.
Sogani J, Morris EA, Kaplan JB, D’Alessio D, Goldman D, Moskowitz CS, et al. Comparison of background parenchymal enhancement at contrast-enhanced spectral mammography and breast MR imaging. Radiology. 2016;282:63–73.
Zamora K, Allen E, Hermecz B. Contrast mammography in clinical practice: current uses and potential diagnostic dilemmas. Clin Imaging. 2021;71:126–35.
Gluskin J, Click M, Fleischman R, Dromain C, Morris EA, Jochelson MS. Contamination artifact that mimics in-situ carcinoma on contrast-enhanced digital mammography. Eur J Radiol. 2017;95:147–54.
Neppalli S, Kessell MA, Madeley CR, Hill ML, Vlaskovsky PS, Taylor DB. Artifacts in contrast-enhanced mammography: are there differences between vendors? Clin Imaging. 2021;80:123–30.
Knogler T, Homolka P, Hörnig M, Leithner R, Langs G, Waitzbauer M, et al. Contrast-enhanced dual energy mammography with a novel anode/filter combination and artifact reduction: a feasibility study. Eur Radiol. 2016;26:1575–81.
Sistermanns M, Kowall B, Hörnig M, Beiderwellen K, Uhlenbrock D. Motion artifact reduction in contrast-enhanced dual-energy mammography - a multireader study about the effect of nonrigid registration as motion correction on image quality. Rofo. 2021;193:1183–8.
Lu Y, Peng B, Lau BA, Hu Y-H, Scaduto DA, Zhao W, et al. A scatter correction method for contrast-enhanced dual-energy digital breast tomosynthesis. Phys Med Biol. 2015;60:6323–54.
Sensakovic WF, Carnahan MB, Czaplicki CD, Fahrenholtz S, Panda A, Zhou Y, et al. Contrast-enhanced mammography: how does it work? Radiographics. 2021;41:829–39.
Wang S, Sun Y, Li R, Mao N, Li Q, Jiang T, et al. Diagnostic performance of perilesional radiomics analysis of contrast-enhanced mammography for the differentiation of benign and malignant breast lesions. Eur Radiol. 2022;32:639–49.
The authors are thankful to Boran Pang, MD, for providing technical support and inspiration in experimental design. Permission was obtained from him. We also acknowledge funding and support from the Clinical Research Plan of SHDC (SHDC2020CR2008A), the National Natural Science Foundation of China (NSFC82071878), Shanghai Science and Technology Foundation (19DZ1930502), Shanghai Anticancer Association EYAS PROJECT (SACA-CY20B01), Shanghai Anticancer Association FLIGHT PROJECT (SACA-AX-201903), and Shanghai Science and Technology Innovation Action Plan Medical Innovation Research Project (21Y11910200).
This project was supported by the grant from the Clinical Research Plan of SHDC (SHDC2020CR2008A), the National Natural Science Foundation of China (NSFC 82071878), Shanghai Science and Technology Foundation (19DZ1930502), Shanghai Anticancer Association EYAS PROJECT (SACA-CY20B01), Shanghai Anticancer Association FLIGHT PROJECT (SACA-AX-201903), and Shanghai Science and Technology Innovation Action Plan Medical Innovation Research Project (21Y11910200).
Ethics approval and consent to participate
Institutional Review Board approval was obtained.
Consent for publication
Consent forms were obtained.
The datasets generated during and/or analyzed during the current study are not publicly available due confidentiality but are available from the corresponding author on reasonable request.
One of the coauthors (S.D.) is an employee of General Electric (GE) Healthcare (Shanghai, China). No other potential conflict of interest relevant to this article has been reported.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1.
Additional file 2.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Sun, Y., Wang, S., Liu, Z. et al. Identifying factors that may influence the classification performance of radiomics models using contrast-enhanced mammography (CEM) images. Cancer Imaging 22, 22 (2022). https://doi.org/10.1186/s40644-022-00460-8
- Breast Cancer