Image Inpainting Based on Structural Tensor Edge Intensity Model

Jing Wang, Yan-Hong Zhou, Hai-Feng Sima, Zhan-Qiang Huo, Ai-Zhong Mi. Image Inpainting Based on Structural Tensor Edge Intensity Model. International Journal of Automation and Computing. doi: 10.1007/s11633-020-1256-x
 Citation: Jing Wang, Yan-Hong Zhou, Hai-Feng Sima, Zhan-Qiang Huo, Ai-Zhong Mi. Image Inpainting Based on Structural Tensor Edge Intensity Model. International Journal of Automation and Computing.

## Image Inpainting Based on Structural Tensor Edge Intensity Model

###### Author Bio: Jing Wang received the B. Sc. degree in computer science and technology from Henan University of Science and Technology, China in 2006, and the Ph. D. degree in computer application technology from College of Computing and Communication Engineering, Graduate University of Chinese Academy of Science, China in 2012. Currently, she is an associate professor in College of Computer Science and Technology, Henan Polytechnic University, China.Her research interests include image processing computer vision and machine learning.E-mail: wjasmine@hpu.edu.cnORCID iD: 0000-0002-3288-2111 Yan-Hong Zhou received the B. Eng. degree in computer science and technology from Fuyang Teachers College, China in 2018. Currently, she is a master student in software engineering at College of Computer Science and Technology, Henan Polytechnic University, China.Her research interests include image processing and computer vision.E-mail: zhouyanhong_zyh123@163.com Hai-Feng Sima received the B. Eng. and M. Eng. degrees in computer science from Zhengzhou University, China in 2004 and 2007, respectively, and the Ph. D. degree in software and theory from Beijing Institute of Technology, China in 2015. Since 2007, he has been with Faculty of Henan Polytechnic University, China, and is currently a lecturer with College of Computer Science and Technology, Henan Polytechnic University, China.His current research interests include pattern recognition, image processing, image segmentation and image classification.E-mail: smhf@hpu.edu.cn (Corresponding author)ORCID iD: 0000-0002-2049-3637 Zhan-Qiang Huo received the B. Sc. degree in mathematics and applied mathematics from the Hebei Normal University of Science and Technology, China in 2003. He received the M. Sc. degree in computer software and theory and the Ph. D. degreD. degree in circuit and system from Yanshan University, China in 2006 and 2009. Currently, he is an associate professor in the College of Computer Science and Technology, Henan Polytechnic University, China.His research interests include computer vision and machine learning.E-mail: hzq@hpu.edu.cn Ai-Zhong Mi received the M. Sc. degree in computer and application from Guangxi University, China in 2005, and the Ph. D. degree in computer application technology from University of Science and Technology Beijing, China in 2009. He is currently an associate professor in College of Computer Science and Technology, Henan Polytechnic University, China.His research interests include pattern recognition and network security.E-mail: miaizhong@163.com
• Figure  1.  Framework of the proposed algorithm. Colored figures are available in the online version.

Figure  2.  Symbol definition: (a) Source area $\Omega$, target area $\varPhi$ and scan patch $\varPsi_x$; (b) Patch to be matched $\varPsi_x$ and the best matching patch $\varPsi_y$; (c) Schematic diagram of the degree of image to be inpainted.

Figure  3.  Incorrect reconstruction of patch $\varPsi_x$ local structure

Figure  4.  Process of searching for matching patch in the image inpainting of Sill: (a) State 1; (b) State 2; (c) State 3.

Figure  5.  Inpainting results of fence, window and stair: (a) Target image; (b) Method in [20]; (c) Method in [11]; (d) Method in [10]; (e) Our method.

Figure  8.  Object removal for images people, tourist and boat: (a) Target image; (b) Method in [20]; (c) Method in [11]; (d) Method in[10]; (e) Our method.

Figure  6.  Inpainting results of island, mountain and highway: (a) Target image; (b) Method in [20]; (c) Method in [11]; (d) Method in [10]; (e) Our method.

Figure  7.  Object removal for images girl, woman and fox: (a) Target image; (b) Method in [20]; (c) Method in [11]; (d) Method in [10]; (e) Our method.

•  [1] V. K. Ha, J. C. Ren, X. Y. Xu, S. Zhao, G. Xie, V. Masero, A. Hussain. Deep learning based single image super-resolution: A survey. International Journal of Automation and Computing, vol. 16, no. 4, pp. 413–426, 2019. DOI:  10.1007/s11633-019-1183-x. [2] M. Wang, R. C. Hong, X. T. Yuan, S. C. Yan, T. S. Chua. Movie2Comics: Towards a lively video content presentation. IEEE Transactions on Multimedia, vol. 14, no. 3, pp. 858–870, 2012. DOI:  10.1109/TMM.2012.2187181. [3] J. B. Huang, S. B. Kang, N. Ahuja, J. Kopf. Temporally coherent completion of dynamic video. ACM Transactions on Graphics, vol. 35, no. 6, Article number 196, 2016. DOI:  10.1145/2980179.2982398. [4] M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester. Image inpainting. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, USA, pp. 417−424, 2000. DOI: 10.1145/344779.344972. [5] T. F. Chan and J. H. Shen. Nontexture inpainting by curvature-driven diffusions. Journal of Visual Communication and Image Representation, vol. 12, no. 4, pp. 436–449, 2001. DOI:  10.1006/jvci.2001.0487. [6] J. H. Shen and T. F. Chan. Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, vol. 62, no. 3, pp. 1019–1043, 2002. DOI:  10.1137/s0036139900368844. [7] K. S. Li, Y. S. Wei, Z. Yang, W. H. Wei. Image inpainting algorithm based on TV model and evolutionary algorithm. Soft Computing, vol. 20, no. 3, pp. 885–893, 2016. DOI:  10.1007/s00500-014-1547-7. [8] X. H. Yang, B. L. Guo, Z. L. Xiao, W. Liang. Improved structure tensor for fine-grained texture inpainting. Signal Processing:Image Communication, vol. 73, pp. 84–95, 2019. DOI:  10.1016/j.image.2018.02.006. [9] P. Buyssens, M. Daisy, D. Tschumperlé, O. Lézoray. Exemplar-based inpainting: Technical review and new heuristics for better geometric reconstructions. IEEE Transactions on Image Processing, vol. 24, no. 6, pp. 1809–1824, 2015. DOI:  10.1109/TIP.2015.2411437. [10] X. F. Li, J. Wang, H. M. Liu, Z. H. Wang. Image inpainting using feature precedence and patch matching. Journal of Computer-Aided Design &Computer Graphics, vol. 28, no. 7, pp. 1131–1137, 2016. DOI:  10.3969/j.issn.1003-9775.2016.07.012. (in Chinese [11] J. Wang, K. Lu, D. R. Pan, N. He, B. K. Bao. Robust object removal with an exemplar-based image inpainting approach. Neurocomputing, vol. 123, pp. 150–155, 2014. DOI:  10.1016/j.neucom.2013.06.022. [12] Y. Liu, C. J. Liu, H. L. Zou, S. S. Zhou, Q. Shen, T. T. Chen. A novel exemplar-based image inpainting algorithm. In Proceedings of International Conference on Intelligent Networking and Collaborative Systems, IEEE, Taipei, China, pp. 86−90, 2015. [13] X. H. Bi, H. M. Liu, G. M. Lu, J. Wei, Y. Chao. Exemplar-based image inpainting using automatic patch optimization. In Proceedings of the 2nd International Conference on Video and Image Processing, ACM, Hong Kong, China, pp. 128−133, 2018. DOI: 10.1145/3301506.3301549. [14] L. F. Zhang, L. C. Song, B. Du, Y. P. Zhang. Nonlocal low-rank tensor completion for visual data. IEEE Transactions on Cybernetics, published online. DOI: 10.1109/TCYB.2019.2910151. [15] M. El Mallahi, A. Zouhri, A. El Affar, A. Tahiri, H. Qjidaa. Radial Hahn moment invariants for 2D and 3D image recognition. International Journal of Automation and Computing, vol. 15, no. 3, pp. 277–289, 2018. DOI:  10.1007/s11633-017-1071-1. [16] T. Liu, Y. H. Wang, D. Z. Cheng. Dynamics and stability of potential hyper-networked evolutionary games. International Journal of Automation and Computing, vol. 14, no. 2, pp. 229–238, 2017. DOI:  10.1007/s11633-017-1056-0. [17] A. Newson, A. Almansa, Y. Gousseau, P. Pérez. Non-local patch-based image inpainting. Image Processing on Line, vol. 7, pp. 373–385, 2017. DOI:  10.5201/ipol.2017.189. [18] C. Barnes, E. Shechtman, A. Finkelstein, D. B. Goldman. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics, vol. 28, no. 3, Article number 24, 2009. DOI:  10.1145/1531326.1531330. [19] H. M. Liu, X. H. Bi, G. M. Lu, W. L. Wang, Z. Y. Zhang. Screen window propagating for image inpainting. IEEE Access, vol. 6, pp. 61761–61772, 2018. DOI:  10.1109/ACCESS.2018.2876161. [20] Criminisi A, Pérez P, Toyama K. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing, vol. 13, no. 9, pp. 1200–1212, 2004. DOI:  10.1109/TIP.2004.833105. [21] Y. Xu, L. C. Yu, H. T. Xu, H. Zhang, T. Nguyen. Vector sparse representation of color image using quaternion matrix analysis. IEEE Transactions on Image Processing, vol. 24, no. 4, pp. 1315–1329, 2015. DOI:  10.1109/TIP.2015.2397314. [22] V. Naumova, K. Schnass. Dictionary learning from incomplete data for efficient image restoration. In Proceedings of the 25th European Signal Processing Conference, IEEE, Kos, Greece, pp. 1425−1429, 2017. DOI: 10.23919/EUSIPCO.2017.8081444. [23] Z. J. Yao, J. Bi, Y. X. Chen. Applying deep learning to individual and community health monitoring data: A survey. International Journal of Automation and Computing, vol. 15, no. 6, pp. 643–655, 2018. DOI:  10.1007/s11633-018-1136-9. [24] N. Wang, S. H. Ma, J. Y. Li, Y. P. Zhang, L. F. Zhang. Multistage attention network for image inpainting. Pattern Recognition, vol. 106, Article number 107448, 2020. DOI:  10.1016/j.patcog.2020.107448.
•  [1] Huan Liu, Gen-Fu Xiao.  Remote Sensing Image Registration Based on Improved KAZE and BRIEF Descriptor . International Journal of Automation and Computing, doi: 10.1007/s11633-019-1218-3 [2] Kai Li, Tong Xu, Shuai Feng, Li-Sheng Qiao, Hua-Wei Shen, Tian-Yang Lv, Xue-Qi Cheng, En-Hong Chen.  The Propagation Background in Social Networks: Simulating and Modeling . International Journal of Automation and Computing, doi: 10.1007/s11633-020-1227-2 [3] Jian-Guo Wang, Qian-Ping Xiao, Tiao Shen, Shi-Wei Ma, Wen-Tao Rao, Yong-Jie Zhang.  Simultaneous Identification of Process Structure, Parameters and Time-delay Based on Non-negative Garrote . International Journal of Automation and Computing, doi: 10.1007/s11633-015-0948-0 [4] Hiroki Matsumori, Ming-Cong Deng, Yuichi Noge.  An Operator-based Nonlinear Vibration Control System Using a Flexible Arm with Shape Memory Alloy . International Journal of Automation and Computing, doi: 10.1007/s11633-018-1149-4 [5] Viet Khanh Ha, Jin-Chang Ren, Xin-Ying Xu, Sophia Zhao, Gang Xie, Valentin Masero, Amir Hussain.  Deep Learning Based Single Image Super-resolution: A Survey . International Journal of Automation and Computing, doi: 10.1007/s11633-019-1183-x [6] G. Dharanibai, Anupama Chandrasekharan, Zachariah C. Alex.  Automated Segmentation of Left Ventricle Using Local and Global Intensity Based Active Contour and Dynamic Programming . International Journal of Automation and Computing, doi: 10.1007/s11633-018-1112-4 [7] Tian-Rui Zong, Elbadry Suzan, Xiang Yong, Nahavandi Saeid.  Modified Moment-based Image Watermarking Method Robust to Cropping Attack . International Journal of Automation and Computing, doi: 10.1007/s11633-015-0926-6 [8] Shotaro Kawahata, Ming-Cong Deng.  Operator-based Nonlinear Temperature Control Experiment for Microreactor Group Actuated by Peltier Devices . International Journal of Automation and Computing, doi: 10.1007/s11633-016-0994-2 [9] Ju-Ping Gu, Liang Hua, Xiao Wu, Hui Yang, Zhen-Tao Zhou.  Color Medical Image Enhancement Based on Adaptive Equalization of Intensity Numbers Matrix Histogram . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0871-9 [10] Oche Alexander Egaji, Alison Griffiths, Mohammad S. Hasan, Hong-Nian Yu.  A Comparison of Mamdani and Sugeno Fuzzy Based Packet Scheduler for MANET with a Realistic Wireless Propagation Model . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0861-y [11] R. I. Minu, K. K. Thyagharajan.  Semantic Rule Based Image Visual Feature Ontology Creation . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0832-3 [12] Chun-Ling Cheng, Chun-Ju Sun, Xiao-Long Xu, Deng-Yin Zhang.  A Multi-dimensional Index Structure Based on Improved VA-file and CAN in the Cloud . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0772-y [13] Fan Guo, Jin Tang, Zi-Xing Cai.  Image Dehazing Based on Haziness Analysis . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0768-7 [14] Mei-Sen Pan, Jing-Tian Tang, Xiao-Li Yang.  Movement Invariants-based Algorithm for Medical Image Tilt Correction . International Journal of Automation and Computing, doi: 10.1007/s11633-010-0538-0 [15] S. Geetha, Siva S. Sivatha Sindhu, N. Kamaraj.  Passive Steganalysis Based on Higher Order Image Statistics of Curvelet Transform . International Journal of Automation and Computing, doi: 10.1007/s11633-010-0537-1 [16] Xiang-Hong Lin, Tian-Wen Zhang.  Structure and Dynamics of Artificial Regulatory Networks Evolved by Segmental Duplication and Divergence Model . International Journal of Automation and Computing, doi: 10.1007/s11633-010-0105-8 [17] Houda Mathlouthi, Kamel Abederrahim, Faouzi Msahli, Gerard Favier.  Crosscumulants Based Approaches for the Structure Identification of Volterra Models . International Journal of Automation and Computing, doi: 10.1007/s11633-009-0420-0 [18] Ming-Min Zhang,  Zhi-Geng Pan,  Li-Feng Ren,  Peng Wang.  Image-based Virtual Exhibit and Its Extension to 3D . International Journal of Automation and Computing, doi: 10.1007/s11633-007-0018-3 [19] Xiang-Yang Wang, Jun Wu.  A Feature-based Robust Digital Image Watermarking Against Desynchronization Attacks . International Journal of Automation and Computing, doi: 10.1007/s11633-007-0428-2 [20] Yue-Gang Tao, Wen-De Chen, Yi-Xin Yin.  Balance Properties and Stabilization of Min-Max Systems . International Journal of Automation and Computing, doi: 10.1007/s11633-006-0076-y

##### 计量
• 文章访问数:  9
• HTML全文浏览量:  26
• PDF下载量:  7
• 被引次数: 0
##### 出版历程
• 收稿日期:  2020-06-13
• 录用日期:  2020-09-16
• 网络出版日期:  2020-11-28

## Image Inpainting Based on Structural Tensor Edge Intensity Model

### English Abstract

Jing Wang, Yan-Hong Zhou, Hai-Feng Sima, Zhan-Qiang Huo, Ai-Zhong Mi. Image Inpainting Based on Structural Tensor Edge Intensity Model. International Journal of Automation and Computing. doi: 10.1007/s11633-020-1256-x
 Citation: Jing Wang, Yan-Hong Zhou, Hai-Feng Sima, Zhan-Qiang Huo, Ai-Zhong Mi. Image Inpainting Based on Structural Tensor Edge Intensity Model. International Journal of Automation and Computing.

/

• 分享
• 用微信扫码二维码

分享至好友和朋友圈