Development of 3D Feature Detection and on Board Mapping Algorithm from Video Camera for Navigation

Eng Zi Hao and Sutthiphong  Srigrarom

doi:10.6180/jase.2016.19.1.04

Development of 3D Feature Detection and on Board Mapping Algorithm from Video Camera for Navigation

Computer Science and Information Engineering

Eng Zi Hao and Sutthiphong Srigrarom This email address is being protected from spambots. You need JavaScript enabled to view it.¹

¹University of Glasgow in Partnership with Singapore Institute of Technology, 139660, Singapore

Received: September 3, 2015
Accepted: December 20, 2015
Publication Date: March 1, 2016

Download Citation: ||https://doi.org/10.6180/jase.2016.19.1.04

ABSTRACT

In this paper, a different approach is introduced to produce comparable 3D reconstruction outcomes similar to that of working geometry method but not as computationally extensive as well as mathematically complex. An image pair, capturing the left and right view of the object or surrounding, is used as inputs. The analogy is very similar to how the human eye perceives the world. The 3D reconstruction program is broken down into two sections, with 3 MATLAB codes been written in total. First, to generate the image frames, followed by the second section, generating the 3D point cloud. In the first part of the program, 2 MATLAB codes have been written with the end result of estimated image frames between the two views which are not captured by the camera will be generated. In the second half of the program, the image pair is now processed to generate 3D point clouds containing 3D co-ordinates of the features. This techniques allows the partial reconstruction of a 3D environment by stitching together these image frames, thus creating a video of the environment as if the camera is moving from the left camera point to the right, giving the user the depth perception one would get when viewing it in real life. After which a 3D point cloud is generated, however to achieve this, the camera must first be calibrated to obtain the camera parameter with the aid of a checkerboard. The camera positions are also estimated and this is combined with the 3D co-ordinates of the features, producing the 3D point cloud. This will give the 3D co-ordinates of the features in an interactive 3D plot within MATLAB extracted from just a pair of input images.

Keywords: 3D Point Cloud, Features Detection, Vision-based Navigation, MATLAB

REFERENCES

[1] Zhao, S., Lin, F., Peng, K., Chen, B. M. and Lee, T. H., “Homography-based Vision-aided Inertial Navigation of UAVs in Unknown Environments,” AIAA Guidance, Navigation and Control Conference 1316 August 2012, Minneapolis, Minnesota, National University of Singapore (2012). doi: 10.2514/6.2012-5033
[2] Sanfourche, M., Delaune, J., Le Besnerais, G., de Plinval, H., Israel, J., Cornic, P., Treil, A., Watanabe, Y. and Plyer, A., “Perception for UAV: Vision-Based Navigation and Environment, Modeling,” Aerospace Lab Journal, pp. 12 (2012).
[3] Scharstein, D. and Szeliski, R., “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” International Journal of Computer Vision, Vol. 47, No. 1, pp. 742 (2002).
[4] Seitz, S., Curless, B., Diebel, J., Scharstein, D. and Szeliski, R., “A Comparison and Evaluation of Multiview Stereo Reconstruction Algorithms,” Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, IEEE, Vol. 1, pp. 519528 (2006). doi: 10.1109/CVPR.2006.19
[5] Snavely, N., Seitz, S. and Szeliski, R., “Photo Tourism: Exploring Photo Collections in 3D,” ACM Transactions on Graphics (TOG), ACM, Vol. 25, pp. 835846 (2006).
[6] Snavely, N., Seitz, S. and Szeliski, R., “Modeling the World from Internet Photo Collections,” International Journal of Computer Vision, Vol. 80, No. 2, pp. 189 210 (2008). doi: 10.1007/s11263-007-0107-3
[7] Tan, P., Fang, T., Xiao, J., Zhao, P. and Quan, L., “Single Image Tree Modeling,” ACM Trans. Graph., Vol. 27, No. 5, pp. 108:1108:7 (2008).
[8] Xiao, J., Chen, J., Yeung, D.-Y. and Quan, L., “Learning Two-view Stereo Matching,” Proceedings of the 10th European Conference on Computer Vision: Part III, ECCV ’08, Berlin, Heidelberg, pp. 1527 (2008).
[9] Xiao, J., Fang, T., Tan, P., Zhao, P., Ofek, E. and Quan, L., “Image-based Facade Modeling,” ACM Trans. Graph., Vol. 27, No. 5, pp. 161:1161:10 (2008).
[10] Xiao, J., Fang, T., Zhao, P., Lhuillier, M. and Quan, L., “Image Based Street-side City Modeling,” ACM SIGGRAPH Asia 2009 Papers, SIGGRAPH Asia ’09, New York, NY, USA, pp. 114:1114:12 (2009).
[11] Kanade, T., Yoshida, A., Oda, K., Kano, H. and Tanaka, M., “A Stereo Machine for Video-rate Dense Depth Mapping and its New Applications,” Proc. of CVPR’96 (1996). doi: 10.1109/CVPR.1996.517074
[12] Konolige, K., “Small Vision Systems: Hardware and Implementation,” Proc. of 8th International Symposium on Robotics Research (1997).
[13] Marques, O., Practical Image and Video Processing, Hoboken, NJ: John Wiley & Sons (2011).
[14] Georgantzoglou, A., da Silva, J. and Jena, R., Image Processing with MATLAB and GPU, MATLAB Applications for the Practical Engineer, Mr Kelly Bennett (Ed.), ISBN: 978-953-51-1719-3, InTech (2014). doi: 10.5772/58300
[15] Digital Image Processing Using MATLAB’ 2015. Presentation, Haris Papasaika-Hanusch, Institute of Geodesy and Photogrammetry, ETH Zurich.
[16] Otsu, N., “A Threshold Selection Method from GrayLevel Histograms,” IEEE Transactions on Systems, Man and Cybernetics, Vol. SMC-9, No. 1, pp. 6266 (1979). doi: 10.1109/TSMC.1979.4310076
[17] Sridhar, S., Digital Image Processing, New Delhi: Oxford University Press (2011).
[18] Sun, W., Yang, X., Xiao, S. and Hu, W., “Robust Recognition of Checkerboard Pattern for Deformable Surface Matching In Multiple Views,” High Performance Computing & Simulation Conference, Shanghai, China: Print. ISBN: 978-0-9553018-7-2 (2008).
[19] The MathWorks Inc. MATLAB 7.0 (R14SP2). The MathWorks Inc. (2005).
[20] The MathWorks, Inc., “Image Processing Toolbox Documentation,” The MathWorks, Inc. (2015). [Online]. Available: http://www.mathworks.co.uk/help/images/ [Accessed 25 Jan 2015].
[21] The MathWorks, Inc., “Computer Vision System Toolbox - MATLAB & Simulink,” The MathWorks, Inc. (2015). [Online] http://www.mathworks.com/products/ computer-vision/?refresh=true [Accessed 21 Dec 2014].
[22] The MathWorks, Inc., “Single Camera Calibration App - MATLAB & Simulink,” The MathWorks, Inc. (2015). [Online] http://www.mathworks.com/help/vision/ug/ single-camera-calibrator-app.html [Accessed 1 Feb 2014].
[23] The MathWorks, Inc., “Calibrate A Single or Stereo Camera - MATLAB - Estimatecameraparameters,” The MathWorks, Inc. (2015). [Online]. http://www.math works.com/help/vision/ref/estimatecameraparameters. html [Accessed 22 Jan 2014].
[24] The MathWorks, Inc., “Object for Storing Camera Parameters - MATLAB,” The MathWorks, Inc. (2015). [Online] http://www.mathworks.com/help/vision/ref/ cameraparameters-class.html [Accessed 1 Feb 2014].
[25] The MathWorks, Inc., “Compute Location of Calibrated Camera - MATLAB Extrinsics,” The MathWorks, Inc. (2015). [Online] http://www.mathworks.com/help/ 2014].
[26] The MathWorks, Inc., “Camera Workion Matrix - MATLAB Cameramatrix,” The MathWorks, Inc. (2015). [Online] http://www.mathworks.com/help/vision/ref/ cameramatrix.html [Accessed 5 Feb 2014].
[27] Hartley, R. and Zisserman, A., Multiple View Geometry in Computer Vision., Second Edition. Cambridge (2000).
[28] Bardley, D. and Li, B., 3D Reconstruction Using the Direct Linear Transform with a Gabor Wavelet Based Correspondence Measure (2015).
[29] Stückler, J. and Cremers, D., Visual SLAM, Computer Vision Group, Technische Universität München [Online] https://vision.in.tum.de/research/vslam [Accessed 3 December 2015].
[30] Shultz, T. and Rodriguez, L., “3D Reconstruction from Two 2D Images,” ECE533 Digital Image Processing, University of Wisconsin-Madison [Online] http://home pages.cae.wisc.edu/~ece533/work/f03/shultzrodriguez. pdf [Accessed 3 December 2015].