A Review of Deep Learning-Based Image Super-Resolution Reconstruction Methods
DOI:
https://doi.org/10.54097/phfrck02Keywords:
Image Super-Resolution, Deep Learning, Convolutional Neural Networks, Generative Adversarial Networks, TransformerAbstract
Image Super-Resolution (SR) technology aims to reconstruct High-Resolution (HR) images from Low-Resolution (LR) images, holding significant application value in fields such as medical imaging analysis, satellite remote sensing, video enhancement, and security surveillance. In recent years, deep learning methods have significantly advanced the development of image super-resolution technology due to their powerful feature extraction capabilities. This paper systematically reviews the current research status of Single Image Super-Resolution (SISR) technology, focusing on three mainstream deep learning frameworks: Convolutional Neural Networks (CNN), Generative Adversarial Networks (GAN), and Transformers, and summarizes their latest research progress. Firstly, the paper introduces the fundamental principles of traditional super-resolution methods and their limitations in complex scenarios. Secondly, it provides a detailed analysis of the network architectures, optimization strategies, and performance advantages of various deep learning-based super-resolution models. Finally, the paper discusses the challenges currently faced by deep learning-based super-resolution technology and outlines potential future research directions.
Downloads
References
[1] X. Yu et al., “Towards efficient and scale-robust ultra-high-definition image demoiréing,” in European Conference on Computer Vision, Springer, 2022, pp. 646–662.
[2] K. I. Kim and Y. Kwon, “Single-image super-resolution using sparse regression and natural image prior,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 6, pp. 1127–1133, 2010.
[3] W. Jenkins, B. Mather, and D. Munson, “Nearest neighbor and generalized inverse distance interpolation for Fourier domain image reconstruction,” in ICASSP’85. IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE, 1985, pp. 1069–1072.
[4] T. Blu, P. Thévenaz, and M. Unser, “Linear interpolation revitalized,” IEEE Trans. Image Process., vol. 13, no. 5, pp. 710–719, 2004.
[5] C. Lin, M. Sheu, H. Chiang, C. Liaw, and Z. Wu, “The efficient VLSI design of BI-CUBIC convolution interpolation for digital image processing,” in 2008 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2008, pp. 480–483.
[6] H. Prashanth, H. Shashidhara, and K. B. Murthy, “Image scaling comparison using universal image quality index,” in 2009 international conference on advances in computing, control, and telecommunication technologies, IEEE, 2009, pp. 859–863.
[7] K. T. Gribbon and D. G. Bailey, “A novel approach to real-time bilinear interpolation,” in Proceedings. DELTA 2004. Second IEEE international workshop on electronic design, test and applications, IEEE, 2004, pp. 126–131.
[8] R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Trans. Acoust. Speech Signal Process., vol. 29, no. 6, pp. 1153–1160, 1981.
[9] H. Xie, K. Xie, and H. Yang, “Research progress of image super-resolution methods,” Comput. Eng. Appl., vol. 56, no. 19, pp. 34–41, 2020.
[10] R. Y. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Multiframe Image Restor. Regist., vol. 1, pp. 317–339, 1984.
[11] L. Ye and C. Zou, “A Survey of Image Super-Resolution Reconstruction Based on Deep Learning,” in INTERNATIONAL CONFERENCE ON WIRELESS COMMUNICATIONS, NETWORKING AND APPLICATIONS, Springer, 2022, pp. 584–594.
[12] S. Dai, M. Han, Y. Wu, and Y. Gong, “Bilateral back-projection for single image super resolution,” in 2007 IEEE International Conference on Multimedia and Expo, IEEE, 2007, pp. 1039–1042.
[13] H. Chang, D.-Y. Yeung, and Y. Xiong, “Super-resolution through neighbor embedding,” in Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., IEEE, 2004, p. I–I.
[14] Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 12, pp. 6999–7019, Dec. 2022, doi: 10.1109/TNNLS.2021.3084827.
[15] C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp. 295–307, 2015.
[16] C. Dong, C. C. Loy, and X. Tang, “Accelerating the super-resolution convolutional neural network,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, Springer, 2016, pp. 391–407.
[17] W. Shi et al., “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1874–1883.
[18] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
[19] J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1646–1654.
[20] X.-J. Mao, “Image restoration using convolutional auto-encoders with symmetric skip connections,” ArXiv Prepr. ArXiv160608921, 2016.
[21] B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 136–144.
[22] C. Ledig et al., “Photo-realistic single image super-resolution using a generative adversarial network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4681–4690.
[23] J. Li, F. Fang, K. Mei, and G. Zhang, “Multi-scale residual network for image super-resolution,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 517–532.
[24] R. Lan et al., “Cascading and enhanced residual networks for accurate single-image super-resolution,” IEEE Trans. Cybern., vol. 51, no. 1, pp. 115–125, 2020.
[25] J. Kim, J. K. Lee, and K. M. Lee, “Deeply-recursive convolutional network for image super-resolution,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1637–1645.
[26] Y. Tai, J. Yang, and X. Liu, “Image super-resolution via deep recursive residual network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3147–3155.
[27] W. Han, S. Chang, D. Liu, M. Yu, M. Witbrock, and T. S. Huang, “Image Super-Resolution via Dual-State Recurrent Networks,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT: IEEE, Jun. 2018, pp. 1654–1663. doi: 10.1109/CVPR.2018.00178.
[28] Z. Li, J. Yang, Z. Liu, X. Yang, G. Jeon, and W. Wu, “Feedback Network for Image Super-Resolution,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA: IEEE, Jun. 2019, pp. 3862–3871. doi: 10.1109/CVPR.2019.00399.
[29] N. Ahn, B. Kang, and K.-A. Sohn, “Fast, accurate, and lightweight super-resolution with cascading residual network,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 252–268.
[30] Z. Hui, X. Wang, and X. Gao, “Fast and Accurate Single Image Super-Resolution via Information Distillation Network,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT: IEEE, Jun. 2018, pp. 723–731. doi: 10.1109/CVPR.2018.00082.
[31] Z. Hui, X. Gao, Y. Yang, and X. Wang, “Lightweight Image Super-Resolution with Information Multi-distillation Network,” in Proceedings of the 27th ACM International Conference on Multimedia, Nice France: ACM, Oct. 2019, pp. 2024–2032. doi: 10.1145/3343031.3351084.
[32] J. Liu, J. Tang, and G. Wu, “Residual feature distillation network for lightweight image super-resolution,” in Computer vision–ECCV 2020 workshops: Glasgow, UK, August 23–28, 2020, proceedings, part III 16, Springer, 2020, pp. 41–55.
[33] L. Wang et al., “Exploring Sparsity in Image Super-Resolution for Efficient Inference,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA: IEEE, Jun. 2021, pp. 4915–4924. doi: 10.1109/CVPR46437.2021.00488.
[34] I. Goodfellow et al., “Generative adversarial nets,” Adv. Neural Inf. Process. Syst., vol. 27, 2014.
[35] M. S. M. Sajjadi, B. Scholkopf, and M. Hirsch, “EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis,” in 2017 IEEE International Conference on Computer Vision (ICCV), Venice: IEEE, Oct. 2017, pp. 4501–4510. doi: 10.1109/ICCV.2017.481.
[36] X. Wang et al., “Esrgan: Enhanced super-resolution generative adversarial networks,” in Proceedings of the European conference on computer vision (ECCV) workshops, 2018, pp. 0–0.
[37] W. Zhang, Y. Liu, C. Dong, and Y. Qiao, “RankSRGAN: Generative Adversarial Networks With Ranker for Image Super-Resolution,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South): IEEE, Oct. 2019, pp. 3096–3105. doi: 10.1109/ICCV.2019.00319.
[38] A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 30, 2017.
[39] A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” ArXiv Prepr. ArXiv201011929, 2020.
[40] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-End Object Detection with Transformers,” May 28, 2020, arXiv: arXiv:2005.12872. Accessed: Jul. 26, 2023. [Online]. Available: http://arxiv.org/abs/2005.12872
[41] A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lucic, and C. Schmid, “ViViT: A Video Vision Transformer,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada: IEEE, Oct. 2021, pp. 6816–6826. doi: 10.1109/ICCV48922.2021.00676.
[42] J. Bi, Z. Zhu, and Q. Meng, “Transformer in computer vision,” in 2021 IEEE International conference on computer science, electronic information engineering and intelligent control technology (CEI), IEEE, 2021, pp. 178–188.
[43] H. Chen et al., “Pre-Trained Image Processing Transformer,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA: IEEE, Jun. 2021, pp. 12294–12305. doi: 10.1109/CVPR46437.2021.01212.
[44] J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool, and R. Timofte, “Swinir: Image restoration using swin transformer,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 1833–1844.
[45] Z. Liu et al., “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10012–10022.
[46] Z. Lu, H. Liu, J. Li, and L. Zhang, “Efficient transformer for single image super-resolution,” ArXiv Prepr. ArXiv210811084, vol. 2, 2021.
[47] N. Baghel, S. R. Dubey, and S. K. Singh, “SRTransGAN: Image Super-Resolution using Transformer based Generative Adversarial Network,” ArXiv Prepr. ArXiv231201999, 2023.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Journal of Computing and Electronic Information Management
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
