Dual Residual Dense Network Based on Fast Guided Filter for Image Rain Removal

Jianjun  Zhu; Jian’E  Zhao

doi:10.6180/jase.202411_27(11).0011

Dual Residual Dense Network Based on Fast Guided Filter for Image Rain Removal

Jianjun Zhu and Jian’E ZhaoThis email address is being protected from spambots. You need JavaScript enabled to view it.

School of Electronics and Electrical Engineering, Zhengzhou University of Science and Technology, No. 1 Xueyuan Road, hai Industrial Park, Erqi District, Zhengzhou, 450064 China

Received: October 23, 2023
Accepted: December 16, 2023
Publication Date: February 7, 2024

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202411_27(11).0011

In rainy days, rain streaks will cover the high-frequency information of images and reduce the quality of images or videos, which has a great impact on the measurement accuracy of outdoor visual systems. Therefore, how to effectively remove the rain lines, retain the background details of the image, improve the image quality has important research significance and application value. Rainfall weather often leads to the quality deterioration of outdoor surveillance video, which will cause image distortion and blur phenomenon. The traditional image rain-removal algorithms can produce multi-scale rain stripes and the detailed information loss. Therefore, this paper proposes a new image rain removal algorithm based on dual residual dense network via fast guided filter and information distillation. The image is decomposed into base layer and detail layer by using the fast guided filter. The network is trained by directly learning the residual of the detail layer in the rain image and the detail layer of rain-free image, so the mapping range is reduced. Three expansion convolutions with different expansion factors and information distillation are used to extract multi-scale features from the detail layer to obtain more context information and extract complex multi-directional raindrop features. The expansion dual residual dense block is used as the parameter layer of the network to enhance the feature transmission capacity and expand the receptive field. Experimental results on synthetic images and real images show that the values of SSIM and PSNR with the proposed method are higher than 0.95 and 35.0. The proposed method achieves the best values. The proposed algorithm can effectively remove rain streaks with different densities and recover the image details well compared with other state-of-the-art rain removal algorithms.

Keywords: image rain removal; dual residual dense network; fast guided filter; expansion convolution; information distillation

[1] M. R. Islam and M. Paul, (2021) “Video deraining using the visual properties of rain streaks" IEEE Access 10: 202–212. DOI: 10.1109/ACCESS.2021.3136551.
[2] S. Yin and H. Li, (2020) “Hot region selection based on selective search and modified fuzzy C-means in remote sensing images" IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 13: 5862–5871. DOI: 10.1109/JSTARS.2020.3025582.
[3] S.-C. Huang, Q.-V. Hoang, and T.-H. Le, (2022) “SFANet: A selective features absorption network for object detection in rainy weather conditions" IEEE Transactions on Neural Networks and Learning Systems: DOI: 10.1109/TNNLS.2021.3125679.
[4] S. Yin, (2023) “Object Detection Based on Deep Learning: A Brief Review" IJLAI Transactions on Science and Engineering 1(02): 1–6.
[5] H. Tian, T. Deng, and H. Yan, (2022) “Driving as well as on a sunny Day? Predicting driver’s fixation in rainy weather conditions via a dual-branch visual model" IEEE/CAA Journal of Automatica Sinica 9(7): 1335– 1338. DOI: 10.1109/JAS.2022.105716.
[6] Y. Jiang and S. Yin, (2023) “Heterogenous-view occluded expression data recognition based on cycle-consistent adversarial network and K-SVD dictionary learning under intelligent cooperative robot environment" Computer Science and Information Systems (00): 34–34. DOI: 10.2298/CSIS221228034J.
[7] F. Reuschling, A. Papenfuss, J. Jakobi, T. Rambau, E. Michaelsen, and N. Scherer-Negenborn. “Designing and Evaluating a Fusion of Visible and Infrared Spectrum Video Streams for Remote Tower Operations”. In: Virtual and Remote Control Tower: Research, Design, Development, Validation, and Implementation. Springer, 2022, 445–487. DOI: 10.1007/978-3-030-93650-1_19.
[8] L. Teng, Y. Qiao, M. Shafiq, G. Srivastava, A. R. Javed, T. R. Gadekallu, and S. Yin, (2023) “FLPK-BiSeNet: Federated Learning Based on Priori Knowledge and Bilateral Segmentation Network for Image Edge Extraction" IEEE Transactions on Network and Service Management: DOI: 10.1109/TNSM.2023.3273991.
[9] H. Wang, Y. Wu, Q. Xie, Q. Zhao, Y. Liang, S. Zhang, and D. Meng, (2021) “Structural residual learning for single image rain removal" Knowledge-Based Systems 213: 106595. DOI: 10.1016/j.knosys.2020.106595.
[10] L.-W. Kang, C.-W. Lin, and Y.-H. Fu, (2011) “Automatic single-image-based rain streaks removal via image decomposition" IEEE transactions on image processing 21(4): 1742–1755. DOI: 10.1109/TIP.2011.2179057.
[11] J.-H. Kim, C. Lee, J.-Y. Sim, and C.-S. Kim. “Singleimage deraining using an adaptive nonlocal means filter”. In: 2013 IEEE international conference on image processing. IEEE. 2013, 914–917. DOI: 10.1109/ICIP.2013.6738189.
[12] Y.-L. Chen and C.-T. Hsu. “A generalized low-rank appearance model for spatio-temporally correlated rain streaks”. In: Proceedings of the IEEE international conference on computer vision. 2013, 1968–1975.
[13] Y. Luo, Y. Xu, and H. Ji. “Removing rain from a single image via discriminative sparse coding”. In: Proceedings of the IEEE international conference on computer vision. 2015, 3397–3405.
[14] Y. Li, R. T. Tan, X. Guo, J. Lu, and M. S. Brown. “Rain streak removal using layer priors”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, 2736–2744.
[15] L. Zhu, C.-W. Fu, D. Lischinski, and P.-A. Heng. “Joint bi-layer optimization for single-image rain streak removal”. In: Proceedings of the IEEE international conference on computer vision. 2017, 2526–2534.
[16] C. Xu, J. Gao, Q. Wen, and B. Wang, (2021) “Generative adversarial network for image raindrop removal of transmission line based on unmanned aerial vehicle inspection" Wireless Communications and Mobile Computing 2021: 1–8. DOI: 10.1155/2021/6668771.
[17] X. Fu, J. Huang, D. Zeng, Y. Huang, X. Ding, and J. Paisley. “Removing rain from single images via a deep detail network”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, 3855–3863.
[18] W. Yang, R. T. Tan, J. Feng, J. Liu, Z. Guo, and S. Yan. “Deep joint rain detection and removal from a single image”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, 1357–1366.
[19] H. Zhang, V. Sindagi, and V. M. Patel, (2019) “Image de-raining using a conditional generative adversarial network" IEEE transactions on circuits and systems for video technology 30(11): 3943–3956. DOI: 10.1109/TCSVT.2019.2920407.
[20] H. Zhang and V. M. Patel. “Density-aware single image de-raining using a multi-stream dense network”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, 695–704.
[21] X. Fu, B. Liang, Y. Huang, X. Ding, and J. Paisley, (2019) “Lightweight pyramid networks for image deraining" IEEE transactions on neural networks and learning systems 31(6): 1794–1807. DOI: 10.1109/TNNLS.2019.2926481.
[22] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. “Densely connected convolutional networks”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, 4700–4708.
[23] M. Li, X. Cao, Q. Zhao, L. Zhang, and D. Meng, (2021) “Online rain/snow removal from surveillance videos" IEEE Transactions on Image Processing 30: 2029–2044. DOI: 10.1109/TIP.2021.3050313.
[24] K. Zhang, D. Li, W. Luo, and W. Ren, (2021) “Dual attention-in-attention model for joint rain streak and raindrop removal" IEEE Transactions on Image Processing 30: 7608–7619. DOI: 10.1109/TIP.2021.3108019.
[25] W. Wang, (2022) “Multi-scale attention generative adversarial network for single image rain removal" Pattern Recognition and Image Analysis 32(2): 436–447. DOI: 10.1134/S1054661822020201.
[26] C.-H. Son and X.-P. Zhang, (2020) “Rain detection and removal via shrinkage-based sparse coding and learned rain dictionary" Journal of Imaging Science and Technology 64(3): 30501–1.
[27] M.-W. Shao, L. Li, D.-Y. Meng, and W.-M. Zuo, (2021) “Uncertainty guided multi-scale attention network for raindrop removal from a single image" IEEE Transactions on Image Processing 30: 4828–4839. DOI: 10.1109/TIP.2021.3076283.
[28] Y. Mi, S. Yuan, X. Li, and J. Zhou, (2021) “Dense residual generative adversarial network for rapid rain removal" IEEE Access 9: 24848–24858. DOI: 10.1109/ACCESS.2021.3055527.
[29] Q. Chen and T. Wu, (2021) “Single image deraining based on the concatenation residual network" Chinese Journal of Liquid Crystal & Displays 36(2):
[30] S. T. Ahmed and S. Sankar, (2020) “Investigative protocol design of layer optimized image compression in telemedicine environment" Procedia Computer Science 167: 2617–2622. DOI: 10.1016/j.procs.2020.03.323.
[31] M. Gunashree, S. T. Ahmed, M. Sindhuja, P. Bhumika, B. Anusha, B. Ishwarya, et al., (2020) “A New Approach of Multilevel Unsupervised Clustering for Detecting Replication Level in Large Image Set" Procedia Computer Science 171: 1624–1633. DOI: 10.1016/j.procs.2020.04.174.
[32] Y. Yang, J. Guan, S. Huang, W. Wan, Y. Xu, and J. Liu, (2021) “End-to-end rain removal network based on progressive residual detail supplement" IEEE Transactions on Multimedia 24: 1622–1636. DOI: 10.1109/TMM.2021.3068833.