Skip to main content
SpringerLink
Log in

Super-resolution application of generative adversarial network on brain time-of-flight MR angiography: image quality and diagnostic utility evaluation

  • Imaging Informatics and Artificial Intelligence
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To develop a generative adversarial network (GAN) model to improve image resolution of brain time-of-flight MR angiography (TOF-MRA) and to evaluate the image quality and diagnostic utility of the reconstructed images.

Methods

We included 180 patients who underwent 1-min low-resolution (LR) and 4-min high-resolution (routine) brain TOF-MRA scans. We used 50 patients’ datasets for training, 12 for quantitative image quality evaluation, and the rest for diagnostic validation. We modified a pix2pix GAN to suit TOF-MRA datasets and fine-tuned GAN-related parameters, including loss functions. Maximum intensity projection images were generated and compared using multi-scale structural similarity (MS-SSIM) and information theoretic-based statistic similarity measure (ISSM) index. Two radiologists scored vessels’ visibilities using a 5-point Likert scale. Finally, we evaluated sensitivities and specificities of GAN-MRA in depicting aneurysms, stenoses, and occlusions.

Results

The optimal model was achieved with a lambda of 1e5 and L1 + MS-SSIM loss. Image quality metrics for GAN-MRA were higher than those for LR-MRA (MS-SSIM, 0.87 vs. 0.73; ISSM, 0.60 vs. 0.35; p.adjusted < 0.001). Vessels’ visibility of GAN-MRA was superior to LR-MRA (rater A, 4.18 vs. 2.53; rater B, 4.61 vs. 2.65; p.adjusted < 0.001). In depicting vascular abnormalities, GAN-MRA showed comparable sensitivities and specificities, with greater sensitivity for aneurysm detection by one rater (93% vs. 84%, p < 0.05).

Conclusions

An optimized GAN could significantly improve the image quality and vessel visibility of low-resolution brain TOF-MRA with equivalent sensitivity and specificity in detecting aneurysms, stenoses, and occlusions.

Key Points

• GAN could significantly improve the image quality and vessel visualization of low-resolution brain MR angiography (MRA).

• With optimally adjusted training parameters, the GAN model did not degrade diagnostic performance by generating substantial false positives or false negatives.

• GAN could be a promising approach for obtaining higher resolution TOF-MRA from images scanned in a fraction of time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

3D TOF-MRA:

Three-dimensional time-of-flight magnetic resonance angiography

FN:

False negative

FP:

False positive

GAN:

Generative adversarial network

ISSM:

Information theoretic-based statistic similarity measure

LR:

Low-resolution

MS-SSIM:

Multi-scale structural similarity index measure

References

  1. Nishimura DG (1990) Time-of-flight MR angiography. Magn Reson Med 14:194–201

    Article  CAS  Google Scholar 

  2. Yan R, Zhang B, Wang L et al (2018) A comparison of contrast-free MRA at 3.0T in cases of intracranial aneurysms with or without subarachnoid hemorrhage. Clin Imaging 49:131–135

    Article  Google Scholar 

  3. Zhang X, Cao YZ, Mu XH et al (2020) Highly accelerated compressed sensing time-of-flight magnetic resonance angiography may be reliable for diagnosing head and neck arterial steno-occlusive disease: a comparative study with digital subtraction angiography. Eur Radiol 30:3059–3065

    Article  CAS  Google Scholar 

  4. Dong C, Loy CC, He K, Tang X (2014) Learning a Deep Convolutional Network for Image Super-Resolution. In: Computer Vision – ECCV 2014. Springer International Publishing, pp 184–199

  5. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 1637–1645

  6. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 5835–5843

  7. Lim B, Son S, Kim H, et al (2017) Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, pp 1132–1140

  8. Goodfellow IJ, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial networks. arXiv:1406.2661 [stat.ML]. https://doi.org/10.48550/arXiv.1406.2661

  9. Ledig C, Theis L, Huszár F et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 105–114

  10. Gu Y, Zeng Z, Chen H et al (2020) MedSRGAN: medical images super-resolution using generative adversarial networks. Multimed Tools Appl 79:21815–21840

    Article  Google Scholar 

  11. Kaji S, Kida S (2019) Overview of image-to-image translation by use of deep neural networks: denoising, super-resolution, modality conversion, and reconstruction in medical imaging. Radiol Phys Technol 12:235–248

    Article  Google Scholar 

  12. Tan C, Zhu J, Lio’ P (2020) Arbitrary scale super-resolution for brain MRI images. Artificial Intelligence Applications and Innovations 583:165

  13. Zhang H, Shinomiya Y, Yoshida S (2021) 3D MRI reconstruction based on 2D generative adversarial network super-resolution. Sensors 21. https://doi.org/10.3390/s21092978

  14. Sanchez I, Vilaplana V (2018) Brain MRI super-resolution using 3D generative adversarial networks. arXiv:1812.11440 [cs.CV]. https://doi.org/10.48550/arXiv.1812.11440

  15. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1125–1134

  16. Wang Z, Simoncelli EP, Bovik A (2003) Multi-scale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers. IEEE, pp 1398–1402

  17. Loizides F, Schmidt B (2016) Positioning and power in academic publishing: players, agents and agendas: Proceedings of the 20th International Conference on Electronic Publishing. IOS Press

  18. Chollet F (2015) Keras. In: GitHub. https://github.com/fchollet/keras

  19. Aljanabi MA, Hussain ZM, Shnain NAA, Lu SF (2019) Design of a hybrid measure for image similarity: a statistical, algebraic, and information-theoretic approach. Eur J Remote Sens 52:2–15

    Article  Google Scholar 

  20. Mason A, Rioux J, Clarke SE et al (2020) Comparison of objective image quality metrics to expert radiologists’ scoring of diagnostic quality of MR images. IEEE Trans Med Imaging 39:1064–1072

    Article  Google Scholar 

  21. Zhai G, Min X (2020) Perceptual image quality assessment: a survey. Sci China Inf Sci 63:211301

    Article  Google Scholar 

  22. Lucas A, Lopez-Tapia S, Molina R, Katsaggelos AK (2019) Generative adversarial networks and perceptual losses for video super-resolution. IEEE Trans Image Process 28:3312–3327

    Article  Google Scholar 

  23. Ahn N, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 252–268

  24. Wang Z, Simoncelli EP, Bovik A (2003) Multi-scale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers. IEEE, pp 1398–1402

  25. Snell J, Ridgeway K, Liao R, et al (2017) Learning to generate images with perceptual similarity metrics. In: 2017 IEEE International Conference on Image Processing (ICIP), pp 4277–4281

  26. Zhao H, Gallo O, Frosio I, Kautz J (2015) Loss Functions for Neural Networks for Image Processing. arXiv:1511.08861 [cs.CV]. https://doi.org/10.48550/arXiv.1511.08861

  27. Wang J, Chen Y, Wu Y, et al (2020) Enhanced generative adversarial network for 3D brain MRI super-resolution. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp 3616–3625

  28. Do H, Bourdon P, Helbert D et al (2021) 7T MRI super-resolution with generative adversarial network. IS&T Int Symp Electron Imaging 2021:106–106

    Google Scholar 

  29. Hagiwara A, Otsuka Y, Hori M et al (2019) Improving the quality of synthetic FLAIR images with deep learning using a conditional generative adversarial network for pixel-by-pixel image translation. AJNR Am J Neuroradiol 40:224–230

    Article  CAS  Google Scholar 

  30. Küstner T, Munoz C, Psenicny A et al (2021) Deep-learning based super-resolution for 3D isotropic coronary MR angiography in less than a minute. Magn Reson Med 86:2837–2852

    Article  Google Scholar 

Download references

Acknowledgements

Krishna Pandu Wicaksono, MD, likes to thank the Indonesia Endowment Fund for Education (LPDP) for providing the scholarship for his doctoral study.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Diagnostic Imaging and Nuclear Medicine, Graduate School of Medicine, Kyoto University, Kyoto, 606-8507, Japan

    Krishna Pandu Wicaksono, Yasutaka Fushimi, Akihiko Sakata, Sachi Okuchi, Takuya Hinoda, Satoshi Nakajima & Yuji Nakamoto

  2. Department of Real World Data Research and Development, Graduate School of Medicine, Kyoto University, 54 Shogoin Kawahara-cho, Sakyo-ku, Kyoto, 606-8507, Japan

    Koji Fujimoto

  3. Department of Neurosurgery, Graduate School of Medicine, Kyoto University, Kyoto, 606-8507, Japan

    Yukihiro Yamao & Kazumichi Yoshida

  4. Department of Advanced Medical Imaging Research, Graduate School of Medicine, Kyoto University, Kyoto, 606-8507, Japan

    Kanae Kawai Miyake, Hitomi Numamoto & Tsuneo Saga

Authors
  1. Krishna Pandu Wicaksono
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Koji Fujimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yasutaka Fushimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Akihiko Sakata
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sachi Okuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Takuya Hinoda
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Satoshi Nakajima
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yukihiro Yamao
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kazumichi Yoshida
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Kanae Kawai Miyake
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Hitomi Numamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Tsuneo Saga
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Yuji Nakamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Koji Fujimoto.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Koji Fujimoto.

Conflict of interest

Four authors (Koji Fujimoto, Kanae Kawai Miyake, Hitomi Numamoto, and Tsuneo Saga) belong to an Industry-Academia collaboration department. Conflicts of interest for each author will be submitted to the editorial office.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Since this was a retrospective study, written informed consent was waived from all subjects (patients) in this study.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• retrospective

• cross-sectional study

• performed at one institution

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

ESM 1

(DOCX 8013 kb)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wicaksono, K.P., Fujimoto, K., Fushimi, Y. et al. Super-resolution application of generative adversarial network on brain time-of-flight MR angiography: image quality and diagnostic utility evaluation. Eur Radiol 33, 936–946 (2023). https://doi.org/10.1007/s00330-022-09103-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-022-09103-9

Keywords

Search

Navigation