Depth-aware Image-Space Fogging Algorithm for Background Privacy Protection using ZoeDepth Framework

Xiyin Liang; Meng Wang; Peirong Pan

doi:10.54097/t37gnr69

Authors

Xiyin Liang
Meng Wang
Peirong Pan

DOI:

https://doi.org/10.54097/t37gnr69

Keywords:

ZoeDepth model, Depth Estimation, Fogging Algorithm, Background Privacy Protection, Depth-aware Image-Space

Abstract

Fogging algorithms are typically employed in fog dataset construction, natural scene generation for games, and fog scene rendering. This paper proposes a novel depth-aware image-space fogging algorithm for background privacy protection, leveraging the ZoeDepth depth estimation framework and atmospheric scattering model. The algorithm first estimates depth maps from single images using ZoeDepth, enabling segmentation into foreground and background regions. Subsequently, by integrating the atmospheric scattering model and an improved dark channel prior algorithm, the algorithm applies fogging processing to the background regions, transitioning naturally from near to far without retaining identifiable information. Evaluation metrics, including Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Information Entropy, demonstrate that the proposed approach preserves details and structural information in the foreground while effectively blurring background information. This method offers a practical solution for image privacy protection in diverse domains such as security, social media, and military applications, where background privacy is paramount.

Downloads

Download data is not yet available.

References

[1] Jizhe Zhou, Chi-Man Pun. Personal Privacy Protection via Irrelevant Faces Tracking and Pixelation in Video Live Streaming. IEEE Transactions on Information Forensics and Security 2021;16:1088-1103.

[2] Ruaa Sadoon Salman, Farah khiled Al-Jibory, Mauj Haider AbdAlkreem.Detect People's Faces and Protect Them by Providing High Privacy Based on Deep Learning. Tehnički glasnik2024;2024;18(1):92-99.

[3] Yifang Li, Nishant Vishwamitra, Hongxin Hu, Bart Knijnenburg, Kelly Caine. Effectiveness and Users’ Experience of Face Blurring as a Privacy Protection for Sharing Photos via Online Social Networks. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2017;61(1):803-807.

[4] Mohammad Khojaste, Nastaran Moradzadeh, Ahmad Nickabadi. GMFIM: A generative mask-guided facial image manipulation model for privacy preservation. Computers & Graphics 2023;112(1):81-91.

[5] Sara Newman, Kannappan Palaniappan, Jianping Fan, Dan Lin. “Do You Know You Are Tracked by Photos That You Didn’t Take”: Large-Scale Location-Aware Multi-Party Image Privacy Protection. IEEE Transactions on Dependable and Secure Computing 2023;20(1):301-312.

[6] Jianping He, Bin Liu, Deguang Kong, Xuan Bao;, Na Wang, Hongxia Jin, George Kesidis. PUPPIES: Transformation-Supported Personalized Privacy Preserving Partial Image Sharing. IEEE/IFIP International Conference on Dependable Systems and Networks 2016;359-370.

[7] Nishant Vishwamitra, Yifang Li, Hongxin Hu, Kelly Caine, Long Cheng, Ziming Zhao, Gail-Joon Ahn. Towards Automated Content-based Photo Privacy Control in User-Centered Social Networks. Association for Computing Machinery 2022;65-76.

[8] Rakibul Hasan. Reducing Privacy Risks in the Context of Sharing Photos Online. Association for Computing Machinery 2020;1-11.

[9] Chau Yi Li, Ali Shahin Shamsabadi, Ricardo Sanchez-Matilla, Riccardo Mazzon, Andrea Cavallaro. Scene Privacy Protection. IEEE International Conference on Acoustics, Speech and Signal Processing 2019;2502-2506.

[10] Muhammad Bilal Sakha. Image Enhancement and Adversarial Attack Pipeline for Scene Privacy Protection. MediaEval Benchmarking Initiative for Multimedia Evaluation 2019.

[11] Chen Li, Weiqi Yan, Hongwei Zhao, Shihua Zhou, Yueping Wang. TFFD-Net: an effective two-stage mixed feature fusion and detail recovery dehazing network. Visual Computer 2024. https://doi.org/10.1007/s00371-024-03642-6

[12] Javed Aymat Husen Shaikh, Shailendrakumar Mahadev Mukane, Santosh Nagnath Randive. Lightweight progressive recurrent network for video de-hazing in adverse weather conditions. Visual Computer 2024. https://doi.org/10.1007/s00371-024-03683-x

[13] Bin Sheng, Ping Li, Yuxi Jin, Ping Tan, Tong-Yee Lee.Intrinsic image decomposition with step and drift shading separation. IEEE Transactions on Visualization and Computer Graphics 2018;26(2):1332-1346.

[14] Bin Sheng, Ping Li, Xiaoxin Fang, Ping Tan, Enhua Wu. Depth-aware motion deblurring using loopy belief propagation. IEEE Transactions on Circuits and Systems for Video Technology 2019;30(4):955-969.

[15] Yu Zhou, Zhihua Chen, Ping Li, Haitao Song, C. L. Philip Chen, Bin Sheng. FSAD-Net: feedback spatial attention dehazing network. IEEE transactions on neural networks and learning systems 2022;34(10):7719-7733.

[16] Codruta O. Ancuti; Cosmin Ancuti; Radu Timofte, "NH-HAZE: An Image Dehazing Benchmark with Non-Homogeneous Hazy and Haze-Free Images. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2020;1798-1805.

[17] Codruta O.Ancuti, Cosmin Ancuti, Mateu Sbert, Radu Timofte. Dense-Haze: A Benchmark for Image Dehazing with Dense-Haze and Haze-Free Images. IEEE International Conference on Image Processing 2019;1014-1018.

[18] [18]Cosmin Ancuti, Codruta O.Ancuti, Christophe De Vleeschouwer. D-HAZY: A dataset to evaluate quantitatively dehazing algorithms. IEEE International Conference on Image Processing 2016;2226-2230.

[19] Yanfu Zhang, Li Ding, Gaurav Sharma. HazeRD: An outdoor scene dataset and benchmark for single image dehazing. IEEE International Conference on Image Processing 2017;3205-3209.

[20] Codruta Orniana Ancuti, Cosmin Ancuti, Radu Timofte, Christophe De Vleeschouwer. I-HAZE: A Dehazing Benchmark with Real Hazy and Haze-Free Indoor Images. Advanced Concepts for Intelligent Vision Systems 2018;1804.05091.

[21] Md Nasim Khan, Mohamed M. Ahmed. Machine and Deep Learning Techniques for Daytime Fog Detection in Real Time with In-Vehicle Vision Systems Using the SHRP 2 Naturalistic Driving Study Data. Transportation Research Record 2023;2677(1):995-1011.

[22] Md. Imtiyaz Anwar, Arun Khosla. Classification of foggy images for vision enhancement. International Conference on Signal Processing and Communication 2015;233-237.

[23] Chi Zhang, Zihang Lin, Liheng Xu, Zongliang Li, Wei Tang, Yuehu Liu, Gaofeng Meng, Le Wang, Li Li. Density-Aware Haze Image Synthesis by Self-Supervised Content-Style Disentanglement. IEEE Transactions on Circuits and Systems for Video Technology 2022;32(7):4552-4572.

[24] Christos Sakaridis, Dengxin Dai, Luc Van Gool. Semantic Foggy Scene Understanding with Synthetic Data.International Journal of Computer Vision 2018;126(9):973-992.

[25] Yading Zheng, Aizhong Mi, Yingxu Qiao, Yijiang Wang. Realistic Nighttime Haze Image Generation with Glow Effect. Association for Computing Machinery 2022;96–101.

[26] M.Colomb, K.Hirech, Philippe André, J.Boreux, P.Lacote, J.Dufour. An innovative artificial fog production device improved in the European project “FOG”. Atmospheric Research 2008;87:242-251.

[27] Chuxuan Li, Ran Yi, Ali S G,et al. RADepthNet: reflectance-aware monocular depth estimation.. Virtual Reality & Intelligent Hardware 2022;4(5):418-431.

[28] Zizhuo Wang, et al. Robust blind image watermarking based on interest points. Virtual Reality & Intelligent Hardware 2024;6(4): 308-322.

[29] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, Rob Fergus. Indoor Segmentation and Support Inference from RGBD Images. European Conference on Computer Vision 2012;7576:746-760.

[30] Fayao Liu, Chunhua Shen, Guosheng Lin, Ian Reid. Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 2016;38(10):2024-39.

[31] Rostam Affendi Hamzah, Haidi Ibrahim, Literature Survey on Stereo Vision Disparity Map Algorithms. Journal of Sensors 2016;2016(2):1-23.

[32] Wei Zhuo, Mathieu Salzmann, Xuming He, Miaomiao Liu. Indoor Scene Parsing with Instance Segmentation, Semantic Labeling and Support Relationship Inference. IEEE Conference on Computer Vision and Pattern Recognition 2017;6269-6277.

[33] Liang Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L.Yuille. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 2018;40(4)834-848.

[34] S.Bhat, R.Birkl, Diana Wofk, Peter Wonka, Matthias Muller. ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth. Advanced Concepts for Intelligent Vision Systems 2023;2302.12288.

[35] Kaiming He, Jian Sun, Xiaoou Tang. Single image haze removal using dark channel prior. IEEE Conference on Computer Vision and Pattern Recognition 2009;1956-1963.

[36] E.J.McCartney, Freeman F.Hall. Optics of the Atmosphere: Scattering by Molecules and Particles. Physics Today 1977;30(5):76–77.

[37] S.K.Nayar, S.G.Narasimhan. Vision in bad weather. Proceedings of the Seventh IEEE International Conference on Computer Vision 1999;2:820-827.

[38] Srinivasa G.Narasimhan, Shree K.Nayar. Vision and the Atmosphere. International Journal of Computer Vision 2002;48:233–254.

[39] Fan Guo, Jin Tang, Xiaoming Xiao. Foggy Scene Rendering Based on Transmission Map Estimation. International Journal of Computer Games Technology 2014;308629.

[40] Haoying Sun, Yutong Zheng, Qing Lang. Domain Adaptation for Synthesis of Hazy Images. Journal of Computer and Communications 2021;9:142-151.

[41] Zhou Wang, A.C.Bovik, H.R.Sheikh, E.P.Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004;13(4):600-612.