Deep learning models to predict mammographic density jointly on standard dose and low dose images.

Introduction

High breast density, usually defined as the relative proportion of radio-opaque fibroglandular to radiolucent fatty tissue in the breast, is known to be a risk factor for developing cancer1and is also associated with a decreased ability to detect breast cancers mammographically due to masking.2Whilst screening usually starts at around age 50 in the United Kingdom, around one in five breast cancers are detected at an earlier age. Producing accurate mammographic density predictions for women below screening age could enable personalization of screening with better targeting of alternative or supplementary imaging modalities or changes to the frequency of screening. However, full field digital mammography, in which radiologists’ estimates of density from mammograms show a particularly strong relationship with cancer risk,3is not recommended for younger women because of the risk of radiation induction of tumors and the increased density resulting in more frequent recalls. An alternative is to utilize mammography with substantially reduced radiation dose with automated density analysis, previous work has shown that breast density measures can be made when using low dose mammography.4An ongoing project aims to assess ways of predicting breast cancer risk in women aged 30-39 including the use of low dose mammography.5The aim of this paper is to investigate how to produce automated mammographic density predictions on low dose images that are comparable in quality to predictions on standard dose images.

Previous work on density estimation in low dose mammography has primarily utilized three strategies.6One is to use models trained on standard dose mammograms7for direct inference on low dose images with the addition of small alterations to the output to correct for systematic differences. The second approach is to train models on simulated low dose mammograms and use those models for low dose prediction.8,9The third is to use the low dose mammograms to fine tune a model pre-trained on standard dose images.6

The first approach utilizes models trained on standard dose mammograms and makes direct predictions on low dose images. This assumes a high enough degree of similarity between the low and standard dose images that any differences in density predictions are small or can be corrected post hoc, which has been partially shown in previous work.6This may be due to the image features relating to mammographic density being clear enough such that they are not fully obscured in the low dose images where the noise levels are higher, i.e. the density signal in the mammograms is strong, something which has been demonstrated in previous work on density estimation from mammograms.10

The second approach is to train models on standard dose and simulated low dose images separately. However, the way to assess the quality of low dose predictions is to compare to standard dose predictions; because there are two trained models it is hard to disentangle the effects of different training models and of different image doses. Another problem is that there is likely less information available in the low dose images than standard dose images due to increased noise. Ideally, we would train a model that is able to compensate for that information loss by implicitly filling in the missing information.

The third approach is to fine tune a model on real (rather than simulated) low dose images; this requires a substantial amount of training data. In medical imaging access to such data, especially labelled data, is often a challenge, for example in this study we have access to 147 sets of low dose images. Another problem is “model forgetting”11where training on a new task causes the model to perform worse on the previous task, as it can become too well tuned to the new domain. When fine-tuning the model on the small number of low dose images the model may then “forget” about images in the broader image-space and perform worse on low dose images it has not seen than if it had not been fine-tuned on the low dose images.

The approach we set out in this paper is to build a joint model trained on both standard and simulated low dose images together. We believe this approach solves all the previously mentioned issues and we demonstrate that the results produced by our models show both state-of-the-art performance on the standard dose and simulated low dose data as well as showing similar predictions on the real low dose images compared to their standard dose equivalents. The model has been trained on (simulated) low dose images so should be able to correct for any variations between the two dose levels. As there is only one model trained which can make both standard and low dose image predictions the final comparison is not confounded by between model variation. Further, as the model sees both the standard and simulated low dose matched images it should be able to learn, if it is possible to do so, how to compensate for any lost information in the simulated low dose images.

In addition to the jointly trained models, we also train models individually on the standard and simulated low dose images to ensure the jointly trained models are not showing a reduction in performance. We train three separate deep learning model architectures to demonstrate that the results we have are not specific to one model type. Finally, we investigate whether the similarity in model predictions between real low dose and standard dose is the same as between the simulated low dose and standard dose images.

Methods and materials

Data

We utilize two separate datasets, the first is constructed with mammograms from the Predicting Risk Of Cancer at Screening (PROCAS) study.12We use this dataset to produce simulated low dose images at 10% of standard dose from physics-based software.13,14These simulated low dose images are equivalent to the standard dose images except for the simulated alteration in dose. We utilized mammograms from 32 441 women between ages of 47 and 73, all produced using GE Senographe Essential machines. The mammograms consisted of right/left mediolateral oblique (RMLO/LMLO) and right/left craniocaudal (RCC/LCC) views; the mammogram images used in this study were of pixel size 2294 × 1914 or 3062 × 2394. In total we have 107 619 images which we partition, at random, into training (70 171 images), validation (17 605 images) and test sets (19 843 images) always keeping images from one woman within the same partition. During the PROCAS study these images were viewed by two expert readers drawn from a pool who each provided a density score on a visual analogue scale (VAS) between 0 and 100%, a histogram showing the distribution of these scores is shown inFigure 1. The final VAS density score for each mammographic image is the average of the two readers’ scores, which is used as the label for training our models. We use this VAS measure of mammographic density as it has been shown to have a stronger relationship to breast cancer risk than other measures tested.3

(Left) Distribution of VAS scores for the PROCAS data. The distribution is skewed towards lower VAS scores with little data at high scores. (Right) Distribution of predicted scores for the ALDRAM full dose images from the DenseNet-161 model.

The second dataset is from the Automated Low Dose Risk Assessment Mammography (ALDRAM) study6; this consists of images from 147 women between ages of 30 and 45 who had had cancer in one breast and were attending routine screening. The ALDRAM images were taken using the same type of mammogram machine as PROCAS (GE Senographe Essential) and with the same sizes (2294 × 1914 or 3062 × 2394). There are standard dose images taken at the four standard mammographic views (right and left breast with craniocaudal and mediolateral oblique views) and whilst the right breast was in compression the dose was reduced to 10% of the standard dose (or the machine minimum) and a second image was taken. The low dose images are thus available for right craniocaudal (RCC) and right mediolateral oblique (RMLO) views and should be directly comparable with the standard dose equivalents, examples are shown of ALDRAM full and low dose images inFigure 2. There are no associated VAS density scores with the ALDRAM data; we show the distribution of the model predicted scores on the standard dose ALDRAM images inFigure 1.

Two examples of ALDRAM full and low dose images. (Left pair) RCC full and low dose example images. (Right pair) RMLO full and low dose example images.

All images in this study are in the “for-processing” (“raw”) state and are reduced in size down to 640 by 512 pixels whilst maintaining their aspect ratio. The choice of size of 640 × 512 is pragmatic as larger images require more processing power/time and there is little evidence of substantially improved predictive power when using larger images9. Histogram equalization is applied, and the images are normalized. The same procedure is applied for standard, simulated low dose and low dose images. This is the same procedure applied previously for mammographic density predictions.7,9

Our aim is to produce a model that can accurately predict mammographic density on low dose images. We have no VAS scores for the ALDRAM dataset (the images have not been scored by expert readers) so accurate predictions on the low dose images are considered with reference to the predictions on the corresponding standard dose images, that is, the predictions on the standard dose images effectively become the labels. We use both root mean squared error (RMSE) and the Spearman rank correlation coefficient between predictions on standard and low dose images to measure the quality of the model predictions on the low dose images. This also means our models need to accurately estimate mammographic density in standard dose images and simulated low dose images in the PROCAS dataset which can be compared to the VAS density scores.

Joint and independent training

Our approach is to use the training and validation sets from the PROCAS data and train models on a joint dataset of standard and simulated low dose images. The trained model would then be used for inference on standard and low (simulated and real) dose images.

As a direct comparison we also train models separately on the standard and simulated low dose image sets. The standard dose image trained models are then tested on just the standard dose images from the test sets. The simulated low dose image trained models are tested on the low dose (simulated and real) images from the test sets.

Models

There are many deep learning model architectures and they are likely to show modest differences in performance. We therefore take a pragmatic approach trading off time and computational requirements with final model performance. We train three separate types of models: ResNet-18,15ResNet-5015and DenseNet-161.16ResNet-18 has been used before to make mammographic density predictions and has shown good performance.17The logic of using ResNet-50 is to test the hypothesis that more layers within the same model architecture would improve model performance. We include DenseNet-161 as a separate model from a different family to establish whether the ResNet family has weaknesses on this dataset and problem domain.

All results shown are on the held-out test set from the PROCAS dataset or the independent (not trained on) ALDRAM dataset. Spearman’s rank correlation coefficients and RMSE are shown. The uncertainties are generated by bootstrapping and reported at the 95% level.

Model training

We train the three different model types (ResNet-18, ResNet-50 and DenseNet-161) and the joint/independent approaches with the same procedure where possible to ensure a fair comparison can be made. The models are initialized with weights pre-trained from ImageNet.18We train with the Adam optimizer19with a mean-squared error (MSE) objective function altering only the learning rates. Both ResNet models are trained with a batch-size of 20 while the DenseNet models utilize a batch-size of 5 due to limitations of computer memory. The model parameters are saved at the end of each training epoch if the validation MSE is lower than any previous epoch. The final models are selected using the lowest value of the mean squared error of the validation set–for the joint models this is the combined standard and simulated low dose predictions.

Results

The metrics of density predictions compared to VAS density scores of the joint models on the PROCAS standard and simulated low dose images are shown inTable 1. Comparable scores for previous work7have RMSE of 9.18% and Spearman rank correlation coefficient of 0.798 on the standard dose images. While the results are shown separately for the standard dose and simulated low dose images the predictions are produced by the same model.

Table: The performance of the three joint models against labels for standard dose and simulated low dose from the PROCAS test set.

The equivalent results of predictions compared to VAS density scores for the separate models are shown inTable 2. Unlike for the joint models, here the predictions on the standard dose and simulated low dose images are from the two separately trained models, so the table shows results from a total of six models.

Table: The performance of the six independent models against labels for standard dose and simulated low dose from the PROCAS test set.

InTable 3we show metrics to compare predictions on the standard dose images and predictions on the simulated low dose images for the PROCAS test set. Effectively we are considering the predictions on the standard dose models to be the label to test the quality of the predictions on the simulated low dose images. Results for both the joint and individual models are shown in the same table. The joint predictions are generated by the same model while the individual predictions are from the two separately trained models. The joint models produce more similar predictions on standard and simulated low dose images than the individual models.

Table: The metrics between predictions made on the standard dose images compared to predictions made on the simulated low dose images from the PROCAS dataset.

To ensure the metrics we show do not disguise differences in prediction, inFigure 3we show plots of the predictions of our final joint models on the PROCAS standard and simulated low dose data for a direct comparison of the models. We show only 2,000, randomly selected, data-points in the plots to improve readability.

Plots of simulated low dose predictions versus standard dose predictions on the PROCAS data from the three jointly trained models. The number of data-points shown is reduced to 2000, by random selection, to improve readability.

InTable 4we present results comparing predictions on the ALDRAM standard and low dose images for both jointly trained models and the individual trained models. The similarity in predictions is reduced compared to the PROCAS results (Table 3). The gap in performance between the joint and individual models has also reduced.

Table: The metrics between predictions made on the standard dose images compared to low dose images from the ALDRAM dataset.

InFigure 4we show plots of the predictions of the joint models on the ALDRAM standard and low dose data. These are equivalent plots, with predictions from the same models, as inFigure 3.

Plots of low dose versus standard dose predictions on the ALDRAM data for the joint models.

InFigure 5we show how the RMSE between predictions and labels changes for different values of the mammographic density; where the mammographic density is defined as the specific label used. We show label-prediction pairs of: PROCAS expert reader VAS scores with standard dose predictions; PROCAS expert reader VAS scores with low dose predictions; PROCAS standard dose predictions with PROCAS low dose predictions; ALDRAM standard dose predictions with ALDRAM low dose predictions. We used the labels to bin the images into bins of width 30 with a sliding window with stride length of one. Therefore, each value on the y-axis is the RMSE between predictions and labels for those images with a label within 15 points either side of the value on the x-axis. Results are shown for both the PROCAS and ALDRAM datasets. The discontinuities in the ALDRAM results are caused by the small number of data-points in some bins.

We show how the RMSE between predictions and labels changes with the mammographic density, defined as the specific label. We show label-prediction pairs of: PROCAS expert reader VAS scores with standard dose predictions (P_Lab_Std); PROCAS expert reader VAS scores with low dose predictions (P_Lab_Low); PROCAS standard dose predictions with PROCAS low dose predictions (P_Std_Low); ALDRAM standard dose predictions with ALDRAM low dose predictions (ALDRAM).

Discussion

We have shown that our joint models can make accurate predictions on both standard and simulated low dose images; there is no statistically significant reduction in similarity to radiologist scores compared to models trained separately on the two different sets of images. We do not see a significant improvement for the simulated low dose images when training jointly, which implies that including standard dose images alongside simulated low dose images does not improve performance on the simulated low dose images. However, the difference between performance on the standard and simulated low dose images is also statistically insignificant which suggests that the performance gap is small.

The separately trained models perform worse than the jointly trained models when making a direct comparison between standard dose and simulated low dose predictions. This is likely to be due to the removal of model variability with the joint models. This effect is an important advantage of the joint approach as we can now directly compare the performance of the models on the different doses without having to consider how the differences depend on the training of the models.

There is a greater difference in model performance between predictions on the standard and low dose images from the ALDRAM dataset than from the PROCAS dataset (with simulated low dose images). The jointly trained models show this difference clearly with differences in performance between the PROCAS and ALDRAM data. The individual models show these differences too but some of the differences will be due to the different training of the two models making the result less clear.

Automated VAS density prediction models have shown poorer performance at higher densities.7,17Therefore, part of the reason for the larger model prediction differences on the standard and low dose images in ALDRAM compared to PROCAS is that the ALDRAM images have higher distributions of densities. However, this variation in distribution of densities does not account for all the differences in model similarity as shown inFigure 5. The models perform worse (larger differences between low and standard dose predictions) for all density levels. It is possible that there are differences between the simulated low dose images from PROCAS and the low dose images from ALDRAM which cause the differences in model performance. It is notable that there are outlier points in the ALDRAM plots (Figure 4) which we do not see in the PROCAS plots, even though we show significantly more data for PROCAS. Further research would be valuable to ascertain what is causing these differences and what approaches could improve the results. However, the results (seeFigure 4) are very similar with most differences being small between predictions on the standard and low dose images.

In the introduction we discussed three previous methods of estimating VAS scores on low dose mammograms: (1) models trained on standard dose images and used on low dose images, (2) models trained on simulated low dose mammograms, (3) models fine-tuned on low dose mammograms. As we do not have VAS scores on the ALDRAM data we also need to consider the quality of predictions of that approach on standard dose data as well as the similarity between low dose and standard dose predictions. Comparisons are not straightforward as different test sets and metrics are used. For the first method the Pearson correlation coefficient between predictions and VAS scores of standard dose images was 0.817; between low and standard dose predictions it was 0.98.6For the second method Spearman rank correlations for equivalents were 0.87 and 0.979respectively. For the third method equivalents were 0.817(used predictions for training) and 0.92.6The Spearman rank correlations for equivalents for our method were 0.84 and 0.98. The results on PROCAS have different testing sets so are not directly comparable–the PROCAS data from the second method is a subset of the PROCAS data designed to reduce label variability resulting in the higher correlation. However, overall our method shows high correlation between low and standard dose images as well as good correlation to the standard dose VAS scores.

A limitation of this study is the lack of expert reader VAS scores for the ALDRAM data; instead we relied upon the model predictions on the standard dose images to act as the labels. While this is a limitation the model predictions on standard dose images correlate well with VAS scores and therefore should act as an effective proxy.

Another limitation is that all the images were from the GE Senographe Essential system and the models have not been tested on other systems. Also the VAS measure is not commonly used by radiologists around the world so validation elsewhere may be challenging. Furthermore, we used raw (“for processing”) images which may not always be available, and our models have not been test on processed images. Our study has demonstrated that mammographic density estimates can be made using low dose mammograms; this has the potential to be used for women below screening ages to aid with assessing risk of breast cancer. In combination with other factors such as genetics and family history this could lead to the use of earlier screening or other strategies.5

It would be valuable to study how diagnostic quality of breast cancer varies with radiation dose; it may be that substantially reduced radiation dose mammograms could be used for assessment of breast cancer. With the relationship between radiation dose and quality of breast cancer prediction known there could be informed choice for the level of reduction in radiation dose; it is possible that a reduced dose would still provide substantial diagnostic capacity even if it was somewhat higher (e.g. at 20% of full dose rather than 10%) radiation dose. If that was the case, then there would be the potential for the low dose mammograms to be used for diagnosis as well as risk assessment.

Conclusions

The three deep learning models investigated (ResNet-18, ResNet-50 and DenseNet-161) provide similar density prediction performance on simulated low dose image as on standard dose images for both joint and independently trained models. The jointly trained models show greater similarity in prediction between simulated low dose images and standard dose equivalents compared to separately trained models.

There is a reduction in similarity of predictions between ALDRAM standard and low dose images when compared to PROCAS standard and simulated low dose images. It would be valuable to understand the cause of these differences and, if possible, to improve the models such that they perform as well on the ALDRAM low dose data as they do on the PROCAS simulated low dose data. However, for most images, the predictions between standard and low dose images in the ALDRAM data have only small differences which are unlikely to be problematic when the methods are used clinically. Our work shows that with low dose mammography, automated density assessments can be provided for younger women. This conclusion holds for both jointly trained models and models trained on just simulated low dose images as well as the three model types (ResNet-18, ResNe50 and DenseNet-161) all of which show high performance.

Contributor Information

Steven Squires, Department of Computer Science, University of Exeter, The Queen's Drive, Exeter, EX4 4QJ, United Kingdom.

Alistair Mackenzie, NCCPM, Royal Surrey NHS Foundation Trust, 18 Frederick Sanger Road, Guildford, GU2 7YD, United Kingdom.

Dafydd Gareth Evans, Division of Evolution, Infection and Genomics, School of Biological Sciences, University of Manchester, Oxford Road, Manchester, M13 9PL, United Kingdom.

Sacha J Howell, Division of Cancer Sciences, University of Manchester, Wilmslow Road, Manchester, M20 4BX, United Kingdom.

Susan M Astley, Division of Informatics, Imaging and Data Science, University of Manchester, Oxford Road, Manchester, M13 9PT, United Kingdom.

Conflicts of interest

None declared.

Funding

D. Gareth Evans and Susan M. Astley are supported by the National Institute for Health Research (NIHR) Manchester Biomedical Research Centre (Grant No. IS-BRC-1215-20007). Steven Squires was supported by the Medical Research Council through the ALDRAM grant..

References

1. Boyd NF, Martin LJ, Bronskill M, Yaffe MJ, Duric N, Minkin S. Breast tissue composition and susceptibility to breast cancer. J Natl Cancer Inst. 2010;102:1224-1237. 10.1093/jnci/djq239 doi.org/10.1093/jnci/djq239
2. Kolb TM, Lichy J, Newhouse JH. Comparison of the performance of screening mammography, physical examination, and breast US and evaluation of factors that influence them: an analysis of 27,825 patient evaluations. Radiology. 2002;225:165-175. 10.1148/radiol.2251011667 doi.org/10.1148/radiol.2251011667
3. Astley SM, Harkness EF, Sergeant JC, et al. A comparison of five methods of measuring mammographic density: a case-control study. Breast Cancer Res. 2018;20:10. 10.1186/s13058-018-0932-z doi.org/10.1186/s13058-018-0932-z
4. Chen L, Ray S, Keller BM, et al. The impact of acquisition dose on quantitative breast density estimation with digital mammography: results from ACRIN PA 4006. Radiology. 2016;280:693-700. 10.1148/radiol.2016151749 doi.org/10.1148/radiol.2016151749
5. Hindmarch S, Howell SJ, Usher-Smith JA, Gorman L, Evans DG, French DP. Feasibility and acceptability of offering breast cancer risk assessment to general population women aged 30–39 years: a mixed-methods study protocol. BMJ Open. 2024;14:e078555. 10.1136/bmjopen-2023-078555 doi.org/10.1136/bmjopen-2023-078555
6. Squires S, Ionescu G, Harkness EF, et al. Automatic density prediction in low dose mammography. In: 15th International Workshop on Breast Imaging (IWBI2020). 2020:115131D.
7. Ionescu GV, Fergie M, Berks M, et al. Prediction of reader estimates of mammographic density using convolutional neural networks. J Med Imag. 2019;6:1. 10.1117/1.JMI.6.3.031405 doi.org/10.1117/1.JMI.6.3.031405
8. Squires S, Mackenzie A, Evans DG, Howell SJ, Astley SM. Capability and reliability of deep learning models to make density predictions on low-dose mammograms. J Med Imag. 2024;11:44506. 10.1117/1.JMI.11.4.044506 doi.org/10.1117/1.JMI.11.4.044506
9. Squires S, Harkness EF, Mackenzie A, Evans DG, Howell SJ, Astley SM. Breast density prediction from low and standard dose mammograms using deep learning: effect of image resolution and model training approach on prediction quality. Biomed Phys Eng Express. 2024;10:45021. 10.1088/2057-1976/ad470b doi.org/10.1088/2057-1976/ad470b
10. Squires S, Harkness EF, Evans DG, Astley SM. The effect of variable labels on deep learning models trained to predict breast density. Biomed Phys Eng Express. 2023;9:35030. 10.1088/2057-1976/accaea doi.org/10.1088/2057-1976/accaea
11. Mccloskey M, Cohen NJ. Catastrophic interference in connectionist networks: the sequential learning problem. In: Bower GH eds. Psychology of Learning and Motivation. Elsevier; 1989:109-165.
12. Evans DGR, Warwick J, Astley SM, et al. Assessing individual breast cancer risk within the UK national health service breast screening program: a new paradigm for cancer prevention. Cancer Prev Res (Phila). 2012;5:943-951. 10.1158/1940-6207.CAPR-11-0458 doi.org/10.1158/1940-6207.CAPR-11-0458
13. Mackenzie A, Dance DR, Diaz O, Young KC. Image simulation and a model of noise power spectra across a range of mammographic beam qualities. Med Phys. 2014;41:121901. 10.1118/1.4900819 doi.org/10.1118/1.4900819
14. Boita J, Mackenzie A, Van Engen RE, Broeders M, Sechopoulos I. Validation of a mammographic image quality modification algorithm using 3D-printed breast phantoms. J Med Imag. 2021;8:33502. 10.1117/1.JMI.8.3.033502 doi.org/10.1117/1.JMI.8.3.033502
15. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016:770-778.
16. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017:4700–4708.
17. Squires S, Harkness E, Gareth Evans D, Astley SM. Automatic assessment of mammographic density using a deep transfer learning method. J Med Imag. 2023;10:24502. 10.1117/1.JMI.10.2.024502 doi.org/10.1117/1.JMI.10.2.024502
18. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. ImageNet: A Large-Scale Hierarchical Image Database. CVPR09; 2009.
19. Kingma DP, Ba J. Adam: a method for stochastic optimization. InInternational Conference on Learning Representations (ICLR); 2015.

---

Republished from the open web under CC-BY. Authors: Squires S, Mackenzie A, Evans DG, Howell SJ, Astley SM. Read the original.