A 2D Hidden Markov Model for Patch-based Super Resolution

Chen-Chiung  Hsieh; Po-Han Chuan

doi:10.6180/jase.2016.19.1.11

A 2D Hidden Markov Model for Patch-based Super Resolution

Computer Science and Information Engineering

Chen-Chiung Hsieh ¹ and Po-Han Chuan¹

¹Department of Computer Science and Engineering, Tatung University, Taipei, Taiwan 104, R.O.C.

Received: March 23, 2015
Accepted: November 9, 2015
Publication Date: March 1, 2016

Download Citation: ||https://doi.org/10.6180/jase.2016.19.1.11

ABSTRACT

Super resolution is developed to enhance the resolution of images and various kinds of learning based methods were proposed to magnify a single image. This paper presents a 2D hidden Markov model which could do super resolution by using learned image patch pair database. The image patch pairs store the correspondence relation of high-frequency information between low resolution (LR) patches and high resolution (HR) patches. For each input LR patch, the top five similar LR candidate patches in database are searched to construct a 3D cube which can then be modeled by the proposed 2D hidden Markov model (HMM). A novel 2D Viterbi algorithm is developed to find the optimal LR candidate patches that are the most compatible with each other. The resulting super resolution image could be formed by pasting back the corresponding HR patches from patch pair database according to the positions of found optimal LR patches. By objective comparisons of PSNRs/SSIMs and subjective judgment of the generated super resolution images, the proposed 2D HMM method is superior to the traditional interpolation methods and some existing state-of-the-art methods.

Keywords: Super Resolution, Image Patch, Hidden Markov Model, 2D HMM, Viterbi Algorithm

REFERENCES

[1] Van Ouwerkerk, J. D., “Image Super-resolution Survey,” Image and Vision Computing, Vol. 24, No. 10, pp. 10391052 (2006). doi: 10.1016/j.imavis.2006.02.026
[2] Park, S. C., Park, M. K. and Kang, M. G., “Super-resolution Image Reconstruction: a Technical Overview,” IEEE Signal Processing Magazine, Vol. 20, No. 3, pp. 2136 (2003). doi: 10.1109/MSP.2003.1203207
[3] Kim, S. P. and Bose, N. K., “Reconstruction of 2-D Band Limited Discrete Signals from Nonuniform Samples,” IEEE Proc. Radar Signal Processing, Vol. 137, pp. 197204 (1990). doi: 10.1049/ip-f-2.1990.0030
[4] Hsieh, C. C., Huang, Y. P., Chen, Y. Y. and Fuh, C. S., “Video Super-resolution by Motion Compensated Iterative Back-projection Approach,” Journal of Information Science and Engineering, Vol. 27, No. 3, pp. 1107 1122 (2011).
[5] Atkins, C. B., Bouman, C. A. and Allebach, J. P., “Optimal Image Scaling Using Pixel Classification,” Proc. of Intl. Conf. on Image Processing, pp. 864867 (2001). doi: 10.1109/ICIP.2001.958257
[6] Staelin, C., Greig, D., Fischer, M. and Maurer, R., “Neural Network Image Scaling Using Spatial Errors,” HP Tech. Rep. HPL-2003-26R1, pp. 111 (2003).
[7] Candocia, F. M. and Principe, J. C., “Super Resolution of Images Based on Local Correlations,” IEEE Transactions on Neural Networks, Vol. 10, No. 2, pp. 372 380 (1999). doi: 10.1109/72.750566
[8] Tappen, M. F., Russell, B. C. and Freeman, W. T., “Exploiting the Sparse Derivative Prior for Super-resolution and Image Demosaicing,” Proc. of the 3rd Intl. Workshop on Statistical and Computational Theories of Vision, Nice, France, October (2003).
[9] Freeman, W. T. and Pasztor, E. C., “Learning to Estimate Scenes from Images,” Advances in Neural Information Processing Systems, Vol. 11, pp. 775781 (1999).
[10] Freeman, W. T. and Pasztor, E. C., “Markov Networks for Super-resolution,” Proc. of the 34th Annual Conference on Information Sciences and Systems, New Jersey, USA, March (2000).
[11] Freeman, W. T., Jones, T. R. and Pasztor, E. C., “Example-based Super-resolution,” IEEE Computer Graphics and Applications, Vol. 22, No. 2, pp. 5665 (2002). doi: 10.1109/38.988747
[12] Wang, Q., Tang, X. and Shum, H., “Patch Based Blind Image Super Resolution,” Proceedings of the Tenth IEEE International Conference on Computer Vision, pp. 709716 (2005). doi: 10.1109/ICCV.2005.186
[13] He, R. and Zhang, Z., “Locally Affine Patch Mapping and Global Refinement for Image Super-resolution,” Pattern Recognition, Vol. 44, No. 9, pp. 22102219 (2011). doi: 10.1016/j.patcog.2011.03.004
[14] Li, X., Lam, K. M., Qiu, G., Shen, L. and Wang, S., “An Efficient Example-Based Approach for Image Super-resolution,” Proc. IEEE International Conf. on Neural Networks and Signal Processing, pp. 575580 (2008). doi: 10.1109/ICNNSP.2008.4590416
[15] Chang, H., Yeung, D. Y. and Xiong, Y., “Super-resolution through Neighbour Embedding,” Proc. IEEE Conference on Computer Vision and Pattern Classification (CVPR), Vol. 1, pp. 275282 (2004). doi: 10. 1109/CVPR.2004.1315043
[16] Roweis, S. T. and Saul, L. K., “Nonlinear Dimensionality Reduction by Locally Linear Embedding,” Science, Vol. 290, No. 5500, pp. 23232326 (2000). doi: 10.1126/science.290.5500.2323
[17] Yang, J., Wright, J., Huang, T. and Ma, Y., “Image Super-resolution via Sparse Representation,” IEEE Transaction on Image Processing, Vol. 19, No. 11, pp. 28612873 (2010). doi: 10.1109/TIP.2010.2050625
[18] Kim, K. I., Kim, D. H. and Kim, J. H., “Example-based Learning for Image Super-resolution,” Proc. the Third Tsinghua-KAIST Joint Workshop on Pattern Recognition, pp. 140148 (2004).
[19] Kim, K. I. and Kwon, Y., “Single-image Super-resolution Using Sparse Regression and Natural Image Prior,” IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 32, No. 6, pp. 11271133 (2010). doi: 10.1109/TPAMI.2010.25
[20] Nagai, T., Kurematsu, A. and Nguyen, T., “Image Interpolation Using Subband Pseudo 2-D HMM,” Proc. of the 15th IASTED Signal and Image Processing, pp. 276281 (2003).
[21] Jiazheng, S. and Reichenbach, S. E., “Image Interpolation by Two-dimensional Parametric Cubic Convolution,” IEEE Transaction on Image Processing, Vol. 15, No. 7, pp. 18571870 (2006). doi: 10.1109/TIP.2006. 873429
[22] MDSP Super-resolution and Demosaicing Datasets, Multi-Dimensional Signal Processing Research Group (MDSP), http://www.soe.ucsc.edu/~milanfar/software/ sr-datasets.html
[23] Wang, Z., Bovik, A. C., Sheikh, H. R. and Simoncelli, E. P., “Image Quality Assessment: from Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600612 (2004). doi: 10.1109/TIP.2003.819861