A Novel U-net Model for Low-light Image Enhancement and Its Application in Art Design

Jing  Yu; Lu  Zhao

doi:10.6180/jase.202412_27(12).0002

A Novel U-net Model for Low-light Image Enhancement and Its Application in Art Design

Computer Science and Information Engineering

Jing Yu and Lu Zhao

Lu Xun Academy of Fine Arts, Shenyang 110004, China

Received: December 10, 2023
Accepted: January 11, 2024
Publication Date: February 20, 2024

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202412_27(12).0002

In the field of environmental art design, computer image processing technology is widely used. First, the application of computer image processing technology in space landscape design. Whether it is a city or a residential area, spatial landscape design will be carried out during the plane planning. In order to complete the design work, designers usually need to make drawings, and drawing design drawings manually will take a lot of time, so designers usually use computer image processing software to draw design drawings. The residential garden made by computer image processing software Landscape design map, not only the design effect is more real and intuitive, the operation is also very simple, greatly improving the design efficiency and quality. Low-light image has some demerits such as low contrast and strong noise. Traditional image enhancement methods cannot fully solve the above problems. In this paper, we propose a novel low-light image enhancement method based on a new U-net model. Firstly, the improved U-net model is used to extract the features of different levels in the raw images. A standardization layer is added after each convolutional layer. The original convolution module is perfected into an optimization module with the form of jump connection. Meanwhile, pyramid pooling module is added at the bottom of the encoder to better extract global information, forming an improved U-net model. Then the enhanced image with the modified U-net is fused, and finally the enhanced low-light image is obtained. Through rich experiments, the new U-net can better extract the features. The low-light image enhancement effect is better. The proposed method not only improves the brightness and contrast, makes the color more real, and it is more consistent with the characteristics of human vision system, but also the objective image quality indexes such as PSNR (exceeding 11.4) and SSIM are the optimal compared to other state-of-the-art enhancement algorithms.

Keywords: Low-light Image Enhancement, U-net, pyramid pooling, standardization layer, jump connection

[1] J. Cai, S. Gu, and L. Zhang, (2018) “Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images" IEEE Transactions on Image Processing 27(4): 2049–2062. DOI: 10.1109/TIP.2018.2794218.
[2] A. Zaman, S. H. Park, H. Bang, C.-w. Park, I. Park, and S. Joung, (2020) “Generative approach for data augmentation for deep learning-based bone surface segmentation from ultrasound images" International Journal of Computer Assisted Radiology and Surgery 15(6): 931–941. DOI: 10.1007/s11548-020-02192-18.
[3] S. Yin and Y. Zhang, (2019) “Singular value decomposition-based anisotropic diffusion for fusion of infrared and visible images" International journal of image and data fusion 10(1/4): 146–163. DOI: 10.1080/19479832.2018.1487886.
[4] K. Kim, S. Park, S. Yu, and J. Paik. “Bright region preserving back-light image enhancement using clipped histogram equalization”. In: 2018, 1–3. DOI: 10.23919/ ELINFOCOM.2018.8330592.
[5] W. Wang, X. Wu, X. Yuan, and Z. Gao, (2020) “An Experiment-Based Review of Low-Light Image Enhancement Methods" IEEE Access 8: 87884–87917. DOI: 10.1109/ACCESS.2020.2992749.
[6] L. Ma, J. Lin, J. Shang, W. Zhong, X. Fan, Z. Luo, and R. Liu, (2020) “Learning Multi-scale Retinex with Residual Network for Low-Light Image Enhancement" Springer, Cham: DOI: 10.1007/978- 3-030-60633-6_24.
[7] J. Ma, X. Fan, J. Ni, X. Zhu, and C. Xiong, (2017) “Multi-scale retinex with color restoration image enhancement based on Gaussian filtering and guided filtering" International Journal of Modern Physics, B. Condensed Matter Physics, Statistical Physics, Applied Physics: DOI: 10.1142/s0217979217440775.
[8] X. Liu, J. Zhang, Y. Yin, W. Yang, C. Zhang, and X. Wu, (2020) “Underwater polarization image restoration based on logarithmic transformation and dark channel" Optoelectronics Letters v.16;No.89(02): 73–77. DOI: 10.1007/s11801-020-9135-9.
[9] Y.-F. Pu, P. Siarry, A. Chatterjee, Z.-N. Wang, Z. Yi, Y.-G. Liu, J.-L. Zhou, and Y. Wang, (2020) “Low-light image joint enhancement optimization algorithm based on frame accumulation and multi-scale Retinex" Ad Hoc Networks 113(4): 102398. DOI: 10.1016/j.adhoc.2020.102398.
[10] Y. F. Pu, P. Siarry, A. Chatterjee, Z. N. Wang, Z. Yi, Y. G. Liu, J. L. Zhou, and Y. Wang., (2017) “A Fractional-Order Variational Framework for Retinex: Fractional-Order Partial Differential Equation-Based Formulation for Multi-Scale Nonlocal Contrast Enhancement with Texture Preserving" IEEE Transactions on Image Processing: 1–1. DOI: 10.1109/TIP.2017.2779601.
[11] D. Zosso, G. Tran, and S. Osher, (2013) “A unifying retinex model based on non-local differential operators": DOI: 10.1117/12.2008839.
[12] Z. Shi, M. mei Zhu, B. Guo, M. Zhao, and C. Zhang. “Nighttime low illumination image enhancement with single image using bright/dark channel prior”. In: 2018. 2018. DOI: 10.1186/s13640-018-0251-4.
[13] X. Dong, Y. A. Pang, and J. G. Wen. “Fast efficient algorithm for enhancement of low lighting video”. In: 2011 IEEE International Conference on Multimedia and Expo. 2010. DOI: 10.1145/1836845.1836920.
[14] D. Park, M. Kim, B. Ku, S. Yoon, and D. K. Han. “Image enhancement for extremely low light conditions”. In: 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). 2014, 307–312. DOI: 10.1109/AVSS.2014.6918686.
[15] S. Ma, H. Ma, Y. Xu, S. Li, C. Lv, and M. Zhu, (2018) “A Low-Light Sensor Image Enhancement Algorithm Based on HSI Color Model" Sensors 18(10): DOI: 10.3390/s18103583.
[16] Y. F. Wang, H. M. Liu, and Z. W. Fu, (2019) “Low-Light Image Enhancement via the Absorption Light Scattering Model" IEEE Transactions on Image Processing 28(11): 5679–5690. DOI: 10.1109/TIP.2019.2922106.
[17] S.-L. Y. Jing Yu Hang Li and S. Karim, (2020) “Dynamic Gesture Recognition Based on Deep Learning in Human-to-Computer Interfaces" Journal of Applied Science and Engineering 23(1): 31–38. DOI: 10.6180/jase.202003_23(1).0004.
[18] Y. Sun, S.-L. Yin, and L. Teng, (2020) “Research on multi-robot intelligent fusion technology based on multimode deep learning" International Journal of Electronics and Information Engineering 12(3): 119–127. DOI: 10.6636/IJEIE.202009_12(3).03.
[19] L. Tao, C. Zhu, G. Xiang, Y. Li, and X. Xie. “LLCNN: A convolutional neural network for low-light image enhancement”. In: Visual Communications & Image Processing. 2018. DOI: 10.1109/VCIP.2017.8305143.
[20] Y. P. Loh, X. Liang, and C. S. Chan, (2019) “Low-light image enhancement using Gaussian Process for features retrieval" Signal Processing: Image Communication 74: 175–190. DOI: 10.1016/j.image.2019.02.001.
[21] N. Zhao, X. Wang, and S.-L. Yin, (2021) “Research of fire smoke detection algorithm based on video" International Journal of Electronics and Information Engineering 13(1): 1–9. DOI: 10.6636/IJEIE.202103_13(1).01.
[22] K. G. Lore, A. Akintayo, and S. Sarkar, (2017) “LLNet: A Deep Autoencoder Approach to Natural Low-light Image Enhancement" Pattern Recognition 61: 650–662. DOI: 10.1016/j.patcog.2016.06.008.
[23] L. Tao, C. Zhu, G. Xiang, Y. Li, and X. Xie. “LLCNN: A convolutional neural network for low-light image enhancement”. In: Visual Communications & Image Processing. 2018. DOI: 10.1109/VCIP.2017.8305143.
[24] X. Guo, Y. Li, and H. Ling, (2017) “LIME: Low-Light Image Enhancement via Illumination Map Estimation" IEEE Transactions on Image Processing 26: 982–993. DOI: 10.1109/TIP.2016.2639450.
[25] L. Tao, C. Zhu, J. Song, T. Lu, and X. Xie. “Low-light image enhancement using CNN and bright channel prior”. In: 2017 IEEE International Conference on Image Processing (ICIP). 2018. DOI: 10.1109/ICIP.2017. 8296876.
[26] C. Wang, Y. Huang, Y. Zou, and Y. Xu, (2021) “FWBNet:Front White Balance Network for Color Shift Correction in Single Image Dehazing via Atmospheric Light Estimation": DOI: 10.1109/ICASSP39728.2021.9414200.
[27] Y. Jiang, X. Gong, D. Liu, Y. Cheng, and Z. Wang, (2021) “EnlightenGAN: Deep Light Enhancement Without Paired Supervision" IEEE Transactions on Image Processing 30: 2340–2349. DOI: 10.1109/TIP.2021.3051462.
[28] S. W. Cho, N. R. Baek, J. H. Koo, and K. R. Park, (2020) “Modified Perceptual Cycle Generative Adversarial Network-Based Image Enhancement for Improving Accuracy of Low Light Image Segmentation" IEEE Access 9: 6296–6324. DOI: 10.1109/ACCESS.2020.3048366.
[29] J. Causey, J. Stubblefield, J. Qualls, J. Fowler, and X. Huang, (2022) “An Ensemble of U-Net Models for Kidney Tumor Segmentation with CT images" IEEE/ACM Transactions on Computational Biology and Bioinformatics 19(3): 1387–1392. DOI: 10.1109/TCBB.2021.3085608.
[30] N. Martinel, G. L. Foresti, and C. Micheloni, (2020) “Deep Pyramidal Pooling With Attention for Person ReIdentification" IEEE Transactions on Image Processing 29: 7306–7316. DOI: 10.1109/TIP.2020.3000904.
[31] Z. Qu, J. Mei, L. Liu, and D. Y. Zhou, (2020) “Crack Detection of Concrete Pavement With Cross-Entropy Loss Function and Improved VGG16 Network Model" IEEE Access 8: 54564–54573. DOI: 10.1109/ACCESS.2020.2981561.
[32] C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, and M. J. Cardoso, (2017) “Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations" 10553: 240–248. DOI: 10.1007/978-3-319-67558-9_28.
[33] Q. Yang, Y. Wu, D. Cao, M. Luo, and T. Wei, (2020) “A Lowlight Image Enhancement Method Learning from Both Paired and Unpaired Data by Adversarial Training" Neurocomputing 433: 83–95. DOI: 10.1016/j.neucom.2020.12.057.
[34] Y. Zhang, X. Di, B. Zhang, R. Ji, and C. Wang, (2020) “Better Than Reference In Low Light Image Enhancement: Conditional Re-Enhancement Networks": DOI: 10.1109/TIP.2021.3135473.
[35] W. Yang, S. Wang, Y. Fang, Y. Wanga, and J. Liu, (2021) “Band Representation-Based Semi-Supervised Low-Light Image Enhancement: Bridging the Gap Between Signal Fidelity and Perceptual Quality" IEEE Transactions on Image Processing 30: 461–3473. DOI: 10.1109/TIP.2021.3062184.
[36] C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong, and R. Cong, (2020) “Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement" 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR): 1777–1786.