Home  |  About Journal  |  Editorial Board  |  For Authors  |  For Referees  |  For Readers  |  Subscription  |  Contract Us
International Journal of Automation and Computing 2018, Vol. 15 Issue (4) :443-453    DOI: 10.1007/s11633-018-1114-2
Research Article Current Issue | Next Issue | Archive | Adv Search << Previous Articles | Next Articles >>
Elaborate Scene Reconstruction with a Consumer Depth Camera
Jian-Wei Li1,2, Wei Gao1,2, Yi-Hong Wu1,2
1 National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100080, China;
2 University of Chinese Academy of Sciences, Beijing 100049, China
Download: [PDF 3348KB] HTML()   Export: BibTeX or EndNote (RIS)      Supporting Info
Abstract A robust approach to elaborately reconstruct the indoor scene with a consumer depth camera is proposed in this paper. In order to ensure the accuracy and completeness of 3D scene model reconstructed from a freely moving camera, this paper proposes new 3D reconstruction methods, as follows: 1) Depth images are processed with a depth adaptive bilateral filter to effectively improve the image quality; 2) A local-to-global registration with the content-based segmentation is performed, which is more reliable and robust to reduce the visual odometry drifts and registration errors; 3) An adaptive weighted volumetric method is used to fuse the registered data into a global model with sufficient geometrical details. Experimental results demonstrate that our approach increases the robustness and accuracy of the geometric models which were reconstructed from a consumer-grade depth camera.
Service
Email this article
Add to my bookshelf
Add to citation manager
Email Alert
RSS
Articles by authors
Keywords3D reconstruction   image processing   geometry registration   simultaneous localization and mapping (SLAM)   volumetric integration     
Received: 2017-09-06; published: 2017-12-28
Fund:

This work was supported by the National Key Technologies R & D Program (No. 2016YFB0502002) and in part by National Natural Science Foundation of China (Nos. 61472419, 61421004 and 61572499).

Corresponding Authors: Wei Gao     Email: wgao@nlpr.ia.ac.cn
About author: Jian-Wei Li received the B.Sc.degree in measurement,control technology and instrument,the M.Sc.degree in detection technology and automatic equipment from Beijing Jiaotong University,China in 2008 and 2011,respectively.E-mail:jianwei.li@nlpr.ia.ac.cn;Wei Gao received the B.Sc.degree in computational mathematics,the M.Sc.degree in pattern recognition and intelligent system from Shanxi University and the Ph.D.degree in pattern recognition and intelligent system from Institute of Automation,Chinese Academy of Sciences,China in 2002,2005 and 2008,respectively.E-mail:wgao@nlpr.ia.ac.cn;Yi-Hong Wu received the Ph.D.degree in geometric invariants and applications from the Institute of Systems Science,Chinese Academy of Sciences,China in 2001. E-mail:yhwu@nlpr.ia.ac.cn
Cite this article:   
Jian-Wei Li, Wei Gao, Yi-Hong Wu. Elaborate Scene Reconstruction with a Consumer Depth Camera[J]. International Journal of Automation and Computing , vol. 15, no. 4, pp. 443-453, 2018.
URL:  
http://www.ijac.net/EN/10.1007/s11633-018-1114-2      或     http://www.ijac.net/EN/Y2018/V15/I4/443
 
[1] R Kmmerle and G. Grisetti. Strasdat h, konolige k and burgard w. g2o:A general framework for graph optimization. In IEEE International Conference on Robotics and Automation, 2010.
[2] A. Chatterjee and V. M. Govindu. Noise in structured-light stereo depth cameras:Modeling and its applications. Computer Science, 2015.
[3] Q. Chen and V. Koltun. Fast mrf optimization with application to depth reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3914-3921, 2014.
[4] S. Choi, Q.-Y. Zhou, and V. Koltun. Robust reconstruction of indoor scenes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5556-5565, 2015.
[5] B. Curless and M. Levoy. A volumetric method for building complex models from range images. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 303-312, 1996.
[6] M. Halber and T. Funkhouser. Structured global registration of rgb-d scans in indoor environments. arXiv preprint arXiv:1607.08539, 2016.
[7] A. Handa, T. Whelan, J. Mcdonald, and A. J. Davison. A benchmark for rgb-d visual odometry, 3d reconstruction and slam. In IEEE International Conference on Robotics and Automation, pp. 1524-1531, 2014.
[8] J. Huai, Y. Zhang, and A. Yilmaz. Real-time large scale 3d reconstruction by fusing kinect and imu data. ISPRS Annals ofPhotogrammetry, RemoteSensingandSpatialInformation Sciences, vol. Ⅱ-3/W5, pp. 491-496, 2015.
[9] S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, and A. Davison. Kinectfusion:real-time 3d reconstruction and interaction us-ing a moving depth camera. Proc Uist, pp. 559-568, 2011.
[10] C. Kerl, J. Sturm, and D. Cremers. Robust odometry estima-tion for rgb-d cameras. In IEEE International Conference on Robotics and Automation, pp. 3748-3754, 2013.
[11] K. Khoshelham. Accuracy analysis of kinect depth data.ISPRS-International Archives of the Photogrammetry, Re-mote Sensing and Spatial Information Sciences, XXXVⅢ-5/W12(5), pp. 133-138, 2012.
[12] K. Khoshelham and S. O. Elberink. Accuracy and resolution of kinect depth data for indoor mapping applications. Sen-sors, vol. 12, no. 2, pp. 1437-54, 2012.
[13] J. Kopf, M. F. Cohen, D. Lischinski, and M. Uyttendaele. Joint bilateral upsampling. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007), vol. 26, no. 3, 2007.
[14] W. E. Lorensen. Marching cubes:A high resolution 3d surface construction algorithm. Acm Siggraph Computer Graphics, vol. 21, no. 4, pp. 163-169, 1987.
[15] R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohi, J. Shotton, S. Hodges, and A. Fitzgibbon. Kinectfusion:Real-time dense surface mapping and tracking. In IEEE International Symposium on Mixed and Augmented Reality, pp. 127-136, 2011.
[16] C. V. Nguyen, S. Izadi, and D. Lovell. Modeling kinect sensor noise for improved 3d reconstruction and tracking. In International Conference on 3d Imaging, pp. 524-530, 2012.
[17] M. Nie?ner, A. Dai, and M. Fisher. Combining inertial navigation and icp for real-time 3d surface reconstruction. In Eurographics (Short Papers), pp. 13-16. Citeseer, 2014.
[18] S. Parker, P. Shirley, Y. Livnat, and C. Hansen. Interactive ray tracing for isosurface rendering. In Visualization 98. Proceedings, pp. 233-238, 1998.
[19] S. Rusinkiewicz and M. Levoy. Efficient variants of the icp algorithm. In 3dim, pp. 145, 2001.
[20] J. Smisek, M. Jancosek, and T. Pajdla. 3d with kinect. In IEEE International Conference on Computer Vision Workshops, ICCV 2011 Workshops, Barcelona, Spain, November, pp. 1154-1160, 2011.
[21] F. Steinbrcker, J. Sturm, and D. Cremers. Real-time visual odometry from dense rgb-d images. In IEEE International Conference on Computer Vision Workshops, pp. 719-722, 2011.
[22] C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In International Conference on Computer Vision, pp. 839-846, 1998.
[23] T. Whelan, M. Kaess, M. Fallon, H. Johannsson, J. Leonard, and J. Mcdonald. Kintinuous:Spatially extended kinectfusion. RSS Workshop on RGB-D:Advanced Reasoning with Depth Cameras, 2012.
[24] T. Whelan, M. Kaess, H. Johannsson, M. Fallon, J. J. Leonard, and J. McDonald. Real-time large-scale dense rgbd slam with volumetric fusion. The International Journal of Robotics Research, vol. 34, no. 4-5, pp. 598-626, 2015.
[25] T. Whelan, M. Kaess, J. J. Leonard, and J. McDonald. Deformation-based loop closure for large scale dense rgb-d slam. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 548-555, 2013.
[26] T. Whelan, S. Leutenegger, R. F. Salas-Moreno, B. Glocker, and A. J. Davison. ElasticFusion:Dense SLAM without a pose graph. In Robotics:Science and Systems (RSS), Rome, Italy, July 2015.
[27] J. Xiao, A. Owens, and A. Torralba. Sun3d:A database of big spaces reconstructed using sfm and object labels. In IEEE International Conference on Computer Vision, pp. 1625-1632, 2013.
[28] A. Zeng, S. Song, M. Niesner, M. Fisher, and J. Xiao. 3DMatch:Learning the matching of local 3d geometry in range scans. 2016.
[29] Q. Y. Zhou and V. Koltun. Dense scene reconstruction with points of interest. Acm Transactions on Graphics, vol. 32, no. 4, pp. 96-96, 2013.
[30] Q. Y. Zhou, S. Miller, and V. Koltun. Elastic fragments for densescenereconstruction. In 2013 IEEE International Conference on Computer Vision, pp. 473-480, 2013.
[31] Q. Y. Zhou, J. Park, and V. Koltun. Fast Global Registration. Springer International Publishing, 2016.
[32] Zollh, M. Fer, A. Dai, M. Innmann, C. Wu, M. Stamminger, C. Theobalt, Nie, and M. Ner. Shading-based refinement on volumetric signed distance functions. Acm Transactions on Graphics, vol. 34, no. 4, pp. 1-14, 2015.
[33] C. Kerl, J. Sturm, and D. Cremers. Dense visual slam for rgb-d cameras. In Ieee/rsj International Conference on Intelligent Robots and Systems, pp. 2100-2106, 2014.
[1] Tai-Zhi Lv, Chun-Xia Zhao, Hao-Feng Zhang.An Improved FastSLAM Algorithm Based on Revised Genetic Resampling and SR-UPF[J]. International Journal of Automation and Computing , 2018,15(3): 325-334
[2] Yi Yang, Fan Qiu, Hao Li, Lu Zhang, Mei-Ling Wang, Meng-Yin Fu.Large-scale 3D Semantic Mapping Using Stereo Vision[J]. International Journal of Automation and Computing , 2018,15(2): 194-206
[3] Mostafa Merras, Soulaiman El Hazzat, Abderrahim Saaidi, Khalid Satori, Abderrazak Gadhi Nazih.3D Face Reconstruction Using Images from Cameras with Varying Parameters[J]. International Journal of Automation and Computing , 2017,14(6): 661-671
[4] Sun-Chun Zhou, Rui Yan, Jia-Xin Li, Ying-Ke Chen, Huajin Tang.A Brain-inspired SLAM System Based on ORB Features[J]. International Journal of Automation and Computing , 2017,14(5): 564-575
[5] Ju-Ping Gu, Liang Hua, Xiao Wu, Hui Yang, Zhen-Tao Zhou.Color Medical Image Enhancement Based on Adaptive Equalization of Intensity Numbers Matrix Histogram[J]. International Journal of Automation and Computing , 2015,12(5): 551-558
[6] Huan Liu, Ying Xiao, Wei-Dong Tang, Yan-Hui Zhou.Illumination-robust and Anti-blur Feature Descriptors for Image Matching in Abdomen Reconstruction[J]. International Journal of Automation and Computing , 2014,11(5): 469-479
[7] Elder M. Hemerly.Automatic Georeferencing of Images Acquired by UAV’s[J]. International Journal of Automation and Computing , 2014,11(4): 347-352
[8] Junji Satake, Masaya Chiba, Jun Miura.Visual Person Identification Using a Distance-dependent Appearance Model for a Person Following Robot[J]. International Journal of Automation and Computing , 2013,10(5): 438-446
[9] Chao-Lei Wang, Tian-Miao Wang, Jian-Hong Liang, Yi-Cheng Zhang, Yi Zhou.Bearing-only Visual SLAM for Small Unmanned Aerial Vehicles in GPS-denied Environments[J]. International Journal of Automation and Computing , 2013,10(5): 387-396
[10] Han Wang, Wei Mou, Gerald Seet, Mao-Hai Li, M. W. S. Lau, Dan-Wei Wang.Real-time Visual Odometry Estimation Based on Principal Direction Detection on Ceiling Vision[J]. International Journal of Automation and Computing , 2013,10(5): 397-404
[11] Fei Liu, Xin Liu, Bin Zhang, Jing Bai.Extraction of Target Fluorescence Signal from In Vivo Background Signal Using Image Subtraction Algorithm[J]. International Journal of Automation and Computing , 2012,9(3): 232-236
[12] Li-Wei Han, De Xu.Statistic Learning-based Defect Detection for Twill Fabrics[J]. International Journal of Automation and Computing , 2010,7(1): 86-94
[13] Ning-Ning Zhou, Yu-Long Deng.Virtual Reality:A State-of-the-Art Survey[J]. International Journal of Automation and Computing , 2009,6(4): 319-325
[14] Kazuki Kozuka, Cheng Wan, Jun Sato.Rectification of 3D Data Obtained from Moving Range Sensor by Using Extended Projective Multiple View Geometry[J]. International Journal of Automation and Computing , 2008,5(3): 268-275
[15] Edmée Amstutz, Tomoaki Teshima, Makoto Kimura, Masaaki Mochimaru, Hideo Saito.PCA-based 3D Shape Reconstruction of Human Foot Using Multiple Viewpoint Cameras[J]. International Journal of Automation and Computing , 2008,5(3): 217-225
Copyright 2010 by International Journal of Automation and Computing