A Fast Compression Framework Based on 3D Point Cloud Data for Telepresence

Zun-Ran Wang, Chen-Guang Yang, Shi-Lu Dai. A Fast Compression Framework Based on 3D Point Cloud Data for Telepresence. International Journal of Automation and Computing. doi: 10.1007/s11633-020-1240-5
 Citation: Zun-Ran Wang, Chen-Guang Yang, Shi-Lu Dai. A Fast Compression Framework Based on 3D Point Cloud Data for Telepresence. International Journal of Automation and Computing.

A Fast Compression Framework Based on 3D Point Cloud Data for Telepresence

Author Bio: Zun-Ran Wang received the B. Eng. degree in automation from the South China University of Technology, China in 2017. He is currently a M.Sc. degree candidate in the South China University of Technology, China. His research interests include human-robot interaction, intelligent control and image processing. E-mail: zran.wang@qq.com ORCID iD: 0000-0002-0826-9376 Chen-Guang Yang received the B. Eng. degree in measurement and control from Northwestern Polytechnical University, China in 2005, the Ph.D. degree in control engineering from the National University of Singapore, Singapore in 2010. He received Best Paper Awards from IEEE Transactions on Robotics and over 10 international conferences. His research interests include robotics and automation. E-mail: cyang@ieee.org (Corresponding author)ORCID iD: 0000-0001-5255-5559 Shi-Lu Dai received his B. Eng. degree in thermal engineering, the M. Eng. and Ph. D. degrees in control science and engineering, Northeastern University, China in 2002, 2006, and 2010, respectively. He was a visiting student in Department of Electrical and Computer Engineering, National University of Singapore, Singapore from November 2007 to November 2009, and a visiting scholar at Department of Electrical Engineering, University of Notre Dame, USA from October 2015 to October 2016. Since 2010, he has been with the School of Automation Science and Engineering, South China University of Technology, China, where he is currently a professor. His research interests include adaptive and learning control, distributed cooperative systems. E-mail: audaisl@scut.edu.cn
• Figure  1.  The proposed framework. We send three P frames between two I frames.

Figure  2.  Diagram of judging the likelihood

Figure  3.  Diagram of resampling particles

Figure  4.  Principle of the compression algorithm

Figure  5.  Octree data structure

Figure  6.  Spherical grid for the local feature in SHOT descriptor[32]

Figure  7.  Color-based particle filter was applied to acquire the remote operator body. (a)–(f) show the results of acquiring the remote operator body through the 2D image (i.e., the red bounding box), (g)–(l) show the results of acquiring the point cloud data of the remote operator body through the proposed method. Color versions of the figures are available online.

Figure  8.  Results of the voxelization. (a)–(l): Before voxelizing 3D point cloud data of the remote operator body. (m)–(x): After voxelizing 3D point cloud data of the remote operator body, obviously, distribution of point cloud was uniform and the main information of point cloud data was retained.

Figure  9.  Results of the matching point and the proposed compression. (a)–(i): The results of the matching point, the content on the left of each of the images represented the frame on the time $t+1$, $t+2$, $t+3$ respectively. The content on the right of each of the images represented the frame on the time $t$. If two points were matched successfully, they will be connected by a line. (j)–(r): The $\Gamma_{t+1}$, $\Gamma_{t+2}$, and $\Gamma_{t+3}$ frames were decoded through using some information including the motion vectors, the reference frame $\Gamma_t$ and a small amount of the cluster subsets in the $\Gamma_{t+1}$, $\Gamma_{t+2}$ or $\Gamma_{t+3}$ frame, each image has been decoded successfully.

Figure  10.  Results of the octree-based compression algorithm

•  [1] C. G. Yang, Y. H. Ye, X. Y. Li, R. W. Wang. Development of a neuro-feedback game based on motor imagery EEG. Multimedia Tools and Applications, vol. 77, no. 12, pp. 15929–15949, 2018. DOI:  10.1007/s11042-017-5168-x. [2] F. Nagata, K. Watanabe, M. K. Habib. Machining robot with vibrational motion and 3D printer-like data interface. International Journal of Automation and Computing, vol. 15, no. 1, pp. 1–12, 2018. DOI:  10.1007/s11633-017-1101-z. [3] C. G. Yang, H. W. Wu, Z. J. Li, W. He, N. Wang, C. Y. Su. Mind control of a robotic arm with visual fusion technology. IEEE Transactions on Industrial Informatics, vol. 14, no. 9, pp. 3822–3830, 2018. DOI:  10.1109/TII.2017.2785415. [4] X. Y. Wang, C. G. Yang, Z. J. Ju, H. B. Ma, M. Y. Fu. Robot manipulator self-identification for surrounding obstacle detection. Multimedia Tools and Applications, vol. 76, no. 5, pp. 6495–6520, 2017. DOI:  10.1007/s11042-016-3275-8. [5] J. H. Zhang, M. Li, Y. Feng, C. G. Yang. Robotic grasp detection based on image processing and random forest. Multimedia Tools and Applications, vol. 79, no. 3–4, pp. 2427–2446, 2020. DOI:  10.1007/s11042-019-08302-9. [6] H. Y. Chen, H. L. Huang, Y. Qin, Y. J. Li, Y. H. Liu. Vision and laser fused slam in indoor environments with multi-robot system. Assembly Automation, vol. 39, no. 2, pp. 297–307, 2019. DOI:  10.1108/AA-04-2018-065. [7] Y. Yang, F. Qiu, H. Li, L. Zhang, M. L. Wang, M. Y. Fu. Large-scale 3D semantic mapping using stereo vision. International Journal of Automation and Computing, vol. 15, no. 2, pp. 194–206, 2018. DOI:  10.1007/s11633-018-1118-y. [8] J. Oyekan, A. Fischer, W. Hutabarat, C. Turner, A. Tiwari. Utilising low cost RGB-D cameras to track the real time progress of a manual assembly sequence. Assembly Automation, to be published. DOI: 10.1108/AA-06-2018-078. [9] G. L. Wang, X. T. Hua, J. Xu, L. B. Song, K. Chen. A deep learning based automatic surface segmentation algorithm for painting large-size aircraft with 6-DOF robot. Assembly Automation, vol. 40, no. 2, pp. 199–210, 2019. DOI:  10.1108/AA-03-2019-0037. [10] J. W. Li, W. Gao, Y. H. Wu. Elaborate scene reconstruction with a consumer depth camera. International Journal of Automation and Computing, vol. 15, no. 4, pp. 443–453, 2018. DOI:  10.1007/s11633-018-1114-2. [11] C. G. Yang, Z. R. Wang, W. He, Z. J. Li. Development of a fast transmission method for 3D point cloud. Multimedia Tools and Applications, vol. 77, no. 19, pp. 25369–25387, 2018. DOI:  10.1007/s11042-018-5789-8. [12] S. M. Prakhya, B. B. Liu, W. S. Lin, V. Jakhetiya, S. C. Guntuku. B-shot: A binary 3D feature descriptor for fast Keypoint matching on 3D point clouds. Autonomous Robots, vol. 41, no. 7, pp. 1501–1520, 2017. DOI:  10.1007/s10514-016-9612-y. [13] J. H. Hou, L. P. Chau, N. Magnenat-Thalmann, Y. He. Compressing 3-D human motions via Keyframe-based geometry videos. IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 1, pp. 51–62, 2015. DOI:  10.1109/TCSVT.2014.2329376. [14] J. Wingbermuehle. Towards automatic creation of realistic anthropomorphic models for realtime 3D telecommunication. Journal of VLSI Signal Processing Systems for Signal,Image and Video Technology, vol. 20, no. 1-2, pp. 81–96, 1998. DOI:  10.1023/A:1008018307114. [15] J. H. Zhang, C. B. Owen. Octree-based animated geometry compression. Computers &Graphics, vol. 31, no. 3, pp. 463–479, 2007. DOI:  10.1016/j.cag.2006.12.002. [16] Q. H. Yu, W. Yu, J. H. Zheng, X. Z. Zheng, Y. He, Y. C. Rong. A high-throughput and low-complexity decoding scheme based on logarithmic domain. Journal of Signal Processing Systems, vol. 88, no. 3, pp. 245–257, 2017. DOI:  10.1007/s11265-016-1143-4. [17] C. Loop, C. Zhang, Z. Y. Zhang. Real-time high-resolution sparse voxelization with application to image-based modeling. In Proceedings of the 5th High-performance Graphics Conference, ACM, New York, USA, pp. 73-79, 2013. DOI: 10.1145/2492045.2492053. [18] R. Schnabel, R. Klein. Octree-based point-cloud compression. In Proceedings of the 3rd Symposium on Point-based Graphics, ACM, Boston, USA, pp. 111–121, 2006. [19] J. Kammerl, N. Blodow, R. B. Rusu, S. Gedikli, M. Beetz, E. Steinbach. Real-time compression of point cloud streams. In Proceedings of IEEE International Conference on Robotics and Automation, IEEE, Saint Paul, USA, pp. 778–785, 2012. [20] C. Zhang, D. Florêncio, C. Loop. Point cloud attribute compression with graph transform. In Proceedings of IEEE International Conference on Image Processing, IEEE, Paris, France, pp. 2066–2070, 2014. DOI: 10.1109/ICIP.2014.7025414. [21] D. Sedlacek, J. Zara. Graph cut based point-cloud segmentation for polygonal reconstruction. In Proceedings of the 5th International Symposium on Visual Computing, Springer, Las Vegas, USA, pp. 218–227, 2009. DOI: 10.1007/978-3-642-10520-3_20. [22] L. Landrieu, C. Mallet, M. Weinmann. Comparison of belief propagation and graph-cut approaches for contextual classification of 3D lidar point cloud data. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, IEEE, Fort Worth, USA, pp. 2768–2771, 2017. DOI: 10.1109/IGARSS.2017.8127571. [23] X. M. Zhang, W. G. Wan, X. D. An. Clustering and DCT based color point cloud compression. Journal of Signal Processing Systems, vol. 86, no. 1, pp. 41–49, 2017. DOI:  10.1007/s11265-015-1095-0. [24] J. Euh, J. Chittamuru, W. Burleson. Power-aware 3D computer graphics rendering. Journal of VLSI Signal Processing Systems for Signal,Image and Video Technology, vol. 39, no. 1-2, pp. 15–33, 2005. DOI:  10.1023/B:VLSI.0000047269.03965.e9. [25] D. Thanou, P. A. Chou, P. Frossard. Graph-based compression of dynamic 3D point cloud sequences. IEEE Transactions on Image Processing, vol. 25, no. 4, pp. 1765–1778, 2016. DOI:  10.1109/TIP.2016.2529506. [26] Y. T. Shao, Q. Zhang, G. Li, Z. Li, L. Li. Hybrid point cloud attribute compression using slice-based layered structure and block-based intra prediction. In Proceedings of the 26th ACM Multimedia Conference on Multimedia Conference, ACM, Istanbul, Turkey, pp. 1199–1207, 2018. [27] P. M. Djuric, J. H. Kotecha, J. Zhang, Y. F. Huang, T. Ghirmai, M. F. Bugallo, J. Miguez. Particle filtering. IEEE Signal Processing Magazine, vol. 20, no. 5, pp. 19–38, 2003. DOI:  10.1109/MSP.2003.1236770. [28] Z. Chen. Bayesian filtering: From Kalman filters to particle filters, and beyond. Statistics:A Journal of Theoretical and Applied Statistics, vol. 182, no. 1, pp. 1–69, 2003. [29] J. S. Liu, R. Chen, T. Logvinenko. A theoretical framework for sequential importance sampling with resampling. Sequential Monte Carlo Methods in Practice, A. Doucet, N, de Freitas, N. Gordon, Eds., New York, USA: Springer, pp. 225–246, 2001. DOI: 10.1007/978-1-4757-3437-9_11. [30] X. H. Liu, S. Payandeh. Implementation of levels-of-detail in bayesian tracking framework using single RGB-D sensor. In Proceedings of the 7th IEEE Annual Information Technology, Electronics and Mobile Communication Conference, IEEE, Vancouver, Canada, 2016. DOI: 10.1109/IEMCON.2016.7746290. [31] S. Salti, F. Tombari, L. Di Stefano. SHOT: Unique signatures of histograms for surface and texture description. Computer Vision and Image Understanding, vol. 125, pp. 251–264, 2014. DOI:  10.1016/j.cviu.2014.04.011. [32] A. Frome, D. Huber, R. Kolluri, T. Bülow, J. Malik. Recognizing objects in range data using regional point descriptors. In Proceedings of the 8th European Conference on Computer Vision, Springer, Prague, Czech Republic, pp. 224–237, 2004. DOI: 10.1007/978-3-540-24672-5_18. [33] F. Tombari, S. Salti, L. Di Stefano. Unique signatures of histograms for local surface description. In Proceedings of the 11th European Conference on Computer Vision, Springer, Heraklion, Greece, pp. 356–369, 2010. DOI: 10.1007/978-3-642-15558-1_26. [34] R. Mekuria, K. Blom, P. Cesar. Design, implementation, and evaluation of a point cloud codec for tele-immersive video. IEEE Transactions on Circuits and Systems for Video Technology, vol. 27, no. 4, pp. 828–842, 2017. DOI:  10.1109/TCSVT.2016.2543039.
•  [1] Viet-Anh Le, Hai-Xuan Le, Linh Nguyen, Minh-Xuan Phan.  An Efficient Adaptive Hierarchical Sliding Mode Control Strategy Using Neural Networks for 3D Overhead Cranes . International Journal of Automation and Computing, doi: 10.1007/s11633-019-1174-y [2] Qiang Fu, Xiang-Yang Chen, Wei He.  A Survey on 3D Visual Tracking of Multicopters . International Journal of Automation and Computing, doi: 10.1007/s11633-019-1199-2 [3] Bing-Xing Wu, Suat Utku Ay, Ahmed Abdel-Rahim.  Pedestrian Height Estimation and 3D Reconstruction Using Pixel-resolution Mapping Method Without Special Patterns . International Journal of Automation and Computing, doi: 10.1007/s11633-019-1170-2 [4] Xian-Xia Zhang, Zhi-Qiang Fu, Shao-Yuan Li, Tao Zou, Bing Wang.  A Time/Space Separation Based 3D Fuzzy Modeling Approach for Nonlinear Spatially Distributed Systems . International Journal of Automation and Computing, doi: 10.1007/s11633-017-1080-0 [5] Mostafa El Mallahi, Jaouad El Mekkaoui, Amal Zouhri, Hicham Amakdouf, Hassan Qjidaa.  Rotation Scaling and Translation Invariants of 3D Radial Shifted Legendre Moments . International Journal of Automation and Computing, doi: 10.1007/s11633-017-1105-8 [6] Yi Yang, Fan Qiu, Hao Li, Lu Zhang, Mei-Ling Wang, Meng-Yin Fu.  Large-scale 3D Semantic Mapping Using Stereo Vision . International Journal of Automation and Computing, doi: 10.1007/s11633-018-1118-y [7] Mostafa El Mallahi, Amal Zouhri, Anass El Affar, Ahmed Tahiri, Hassan Qjidaa.  Radial Hahn Moment Invariants for 2D and 3D Image Recognition . International Journal of Automation and Computing, doi: 10.1007/s11633-017-1071-1 [8] Fusaomi Nagata, Keigo Watanabe, Maki K. Habib.  Machining Robot with Vibrational Motion and 3D Printer-like Data Interface . International Journal of Automation and Computing, doi: 10.1007/s11633-017-1101-z [9] Ling-Yi Xu, Zhi-Qiang Cao, Peng Zhao, Chao Zhou.  A New Monocular Vision Measurement Method to Estimate 3D Positions of Objects on Floor . International Journal of Automation and Computing, doi: 10.1007/s11633-016-1047-6 [10] Merras Mostafa, El Hazzat Soulaiman, Saaidi Abderrahim, Satori Khalid, Gadhi Nazih Abderrazak.  3D Face Reconstruction Using Images from Cameras with Varying Parameters . International Journal of Automation and Computing, doi: 10.1007/s11633-016-0999-x [11] Wei-Hua Chen, Yuan-Yuan Liu, Fu-Hua Zhang, Yong-Ze Yu, Hai-Ping Chen, Qing-Xi Hu.  Osteochondral Integrated Scaffolds with Gradient Structure by 3D Printing Forming . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0853-y [12] Fan Zhou, Wei Zheng, Zeng-Fu Wang.  Adaptive Noise Identification in Vision-assisted Motion Estimation for Unmanned Aerial Vehicles . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0857-7 [13] Fei Yan, Yi-Sha Liu, Ji-Zhong Xiao.  Path Planning in Complex 3D Environments Using a Probabilistic Roadmap Method . International Journal of Automation and Computing, doi: 10.1007/s11633-013-0750-9 [14] De Xu, Hua-Wei Wang, You-Fu Li, Min Tan.  A New Calibration Method for an Inertial andVisual Sensing System . International Journal of Automation and Computing, doi: 10.1007/s11633-012-0648-y [15] Xiao-Jing Zhou, Zheng-Xu Zhao.  The Skin Deformation of a 3D Virtual Human . International Journal of Automation and Computing, doi: 10.1007/s11633-009-0344-8 [16] Kazuki Kozuka,  Cheng Wan,  Jun Sato.  Rectification of 3D Data Obtained from Moving Range Sensor by Using Extended Projective Multiple View Geometry . International Journal of Automation and Computing, doi: 10.1007/s11633-008-0268-8 [17] Edmée Amstutz, Tomoaki Teshima, Makoto Kimura, Masaaki Mochimaru, Hideo Saito.  PCA-based 3D Shape Reconstruction of Human Foot Using Multiple Viewpoint Cameras . International Journal of Automation and Computing, doi: 10.1007/s11633-008-0217-6 [18] Yukiko Kenmochi, Lilian Buzer, Akihiro Sugimoto, Ikuko Shimizu.  Discrete Plane Segmentation and Estimation from a Point Cloud Using Local Geometric Patterns . International Journal of Automation and Computing, doi: 10.1007/s11633-008-0246-1 [19] Ming-Min Zhang,  Zhi-Geng Pan,  Li-Feng Ren,  Peng Wang.  Image-based Virtual Exhibit and Its Extension to 3D . International Journal of Automation and Computing, doi: 10.1007/s11633-007-0018-3 [20] Jindong Liu,  Huosheng Hu.  A 3D Simulator for Autonomous Robotic Fish . International Journal of Automation and Computing, doi: 10.1007/s11633-004-0042-5

计量
• 文章访问数:  14
• HTML全文浏览量:  29
• PDF下载量:  8
• 被引次数: 0
出版历程
• 收稿日期:  2020-05-11
• 录用日期:  2020-06-05
• 网络出版日期:  2020-07-31

A Fast Compression Framework Based on 3D Point Cloud Data for Telepresence

English Abstract

Zun-Ran Wang, Chen-Guang Yang, Shi-Lu Dai. A Fast Compression Framework Based on 3D Point Cloud Data for Telepresence. International Journal of Automation and Computing. doi: 10.1007/s11633-020-1240-5
 Citation: Zun-Ran Wang, Chen-Guang Yang, Shi-Lu Dai. A Fast Compression Framework Based on 3D Point Cloud Data for Telepresence. International Journal of Automation and Computing.

/

• 分享
• 用微信扫码二维码

分享至好友和朋友圈