Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy

Wen-JuZhou Zi-XiangFei Huo-ShengHu LiLiu Jing-NaLi PeterJames Smith Wen-JuZhou Zi-XiangFei Huo-ShengHu LiLiu Jing-NaLi PeterJames Smith

Wen-JuZhou, Zi-XiangFei, Huo-ShengHu, LiLiu, Jing-NaLi, PeterJames Smith, Wen-JuZhou, Zi-XiangFei, Huo-ShengHu, LiLiu, Jing-NaLi, PeterJames Smith. Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy[J]. 国际自动化与计算杂志(英)/International Journal of Automation and Computing, 2016, 13(2): 99-107. doi: 10.1007/s11633-015-0946-2
引用本文: Wen-JuZhou, Zi-XiangFei, Huo-ShengHu, LiLiu, Jing-NaLi, PeterJames Smith, Wen-JuZhou, Zi-XiangFei, Huo-ShengHu, LiLiu, Jing-NaLi, PeterJames Smith. Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy[J]. 国际自动化与计算杂志(英)/International Journal of Automation and Computing, 2016, 13(2): 99-107. doi: 10.1007/s11633-015-0946-2
Wen-Ju Zhou, Zi-Xiang Fei, Huo-Sheng Hu, Liu Li, Jing-Na Li and Peter James Smith. Erratum to: Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy. International Journal of Automation and Computing, vol. 13, no. 6, pp. 665-665, 2016 doi:  10.1007/s11633-015-0946-2
Citation: Wen-Ju Zhou, Zi-Xiang Fei, Huo-Sheng Hu, Liu Li, Jing-Na Li and Peter James Smith. Erratum to: Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy. International Journal of Automation and Computing, vol. 13, no. 6, pp. 665-665, 2016 doi:  10.1007/s11633-015-0946-2

Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy

doi: 10.1007/s11633-015-0946-2
详细信息
    通讯作者: Huo-Sheng Hu received the M. Sc. degree in industrial automation from Central South University, China in 1982 and the Ph. D. degree in robotics from the University of Oxford, UK in 1993. He is currently a professor with the School of Computer Science and Electronic Engineering, University of Essex, UK, leading the Human Centered Robotics Group. He has published more than 370 research articles in journals, books and conference proceedings. He is a fellow of Institute of Engineering & Technology and Institution of Measurement & Control in the UK, a senior member of IEEE and ACM, and a chartered engineer. He is currently the editor-in-chief for International Journal of Automation and Computing, founding editor-in-chief for Robotics Journal and an executive editor for International Journal of Mechatronics and Automation. His research interests include autonomous robots, human-robot interaction, multi-robot collaboration, embedded systems, pervasive computing, sensor integration, intelligent control, cognitive robotics, and networked robots. E-mail: hhu@essex.ac.ukLi Jing-Na received the B. Sc. degree in physics from Ocean University of China, China in 1985, and the M. Sc. degree in condensed matter physics from Jinan University, China in 1991. She is now a lecturer with School of Information and Electronic Engineering, Ludong University, China. Her research interests include image processing, image registration and patter recognition. E-mail:ljn6502@126.comLi Liu graduated from Qufu Normal University, China in 2004. She received the M. Sc. degree from Dalian Maritime University, China in 2007. She is currently a Ph. D. candidate in Shanghai University, China, and she also is a lecturer at Ludong University, in the School of Information Science and Electrical Engineering. Her research interests include machine vision, image processing and pattern recognition. E-mail: liulildu@163.comPeter James Smith graduated from the University of Essex, England in 2001 with a B. Eng. (Hons.) in telecommunications systems engineering. He is currently working as a senior research engineer for Vitec Videocom in the area of robot navigation. E-mail: Peter.Smith@VitecGroup.comHuo-Sheng Hu received the M. Sc. degree in industrial automation from Central South University, China in 1982 and the Ph. D. degree in robotics from the University of Oxford, UK in 1993. He is currently a professor with the School of Computer Science and Electronic Engineering, University of Essex, UK, leading the Human Centered Robotics Group. He has published more than 370 research articles in journals, books and conference proceedings. He is a fellow of Institute of Engineering & Technology and Institution of Measurement & Control in the UK, a senior member of IEEE and ACM, and a chartered engineer. He is currently the editor-in-chief for International Journal of Automation and Computing, founding editor-in-chief for Robotics Journal and an executive editor for International Journal of Mechatronics and Automation. His research interests include autonomous robots, human-robot interaction, multi-robot collaboration, embedded systems, pervasive computing, sensor integration, intelligent control, cognitive robotics, and networked robots. E-mail: hhu@essex.ac.ukLi Jing-Na received the B. Sc. degree in physics from Ocean University of China, China in 1985, and the M. Sc. degree in condensed matter physics from Jinan University, China in 1991. She is now a lecturer with School of Information and Electronic Engineering, Ludong University, China. Her research interests include image processing, image registration and patter recognition. E-mail:ljn6502@126.comLi Liu graduated from Qufu Normal University, China in 2004. She received the M. Sc. degree from Dalian Maritime University, China in 2007. She is currently a Ph. D. candidate in Shanghai University, China, and she also is a lecturer at Ludong University, in the School of Information Science and Electrical Engineering. Her research interests include machine vision, image processing and pattern recognition. E-mail: liulildu@163.comPeter James Smith graduated from the University of Essex, England in 2001 with a B. Eng. (Hons.) in telecommunications systems engineering. He is currently working as a senior research engineer for Vitec Videocom in the area of robot navigation. E-mail: Peter.Smith@VitecGroup.com

Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy

计量
  • 文章访问数:  4254
  • HTML全文浏览量:  312
  • PDF下载量:  1772
  • 被引次数: 0
出版历程
  • 收稿日期:  2014-08-21
  • 录用日期:  2015-03-30
  • 刊出日期:  2016-01-25

Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy

doi: 10.1007/s11633-015-0946-2
    通讯作者: Huo-Sheng Hu received the M. Sc. degree in industrial automation from Central South University, China in 1982 and the Ph. D. degree in robotics from the University of Oxford, UK in 1993. He is currently a professor with the School of Computer Science and Electronic Engineering, University of Essex, UK, leading the Human Centered Robotics Group. He has published more than 370 research articles in journals, books and conference proceedings. He is a fellow of Institute of Engineering & Technology and Institution of Measurement & Control in the UK, a senior member of IEEE and ACM, and a chartered engineer. He is currently the editor-in-chief for International Journal of Automation and Computing, founding editor-in-chief for Robotics Journal and an executive editor for International Journal of Mechatronics and Automation. His research interests include autonomous robots, human-robot interaction, multi-robot collaboration, embedded systems, pervasive computing, sensor integration, intelligent control, cognitive robotics, and networked robots. E-mail: hhu@essex.ac.ukLi Jing-Na received the B. Sc. degree in physics from Ocean University of China, China in 1985, and the M. Sc. degree in condensed matter physics from Jinan University, China in 1991. She is now a lecturer with School of Information and Electronic Engineering, Ludong University, China. Her research interests include image processing, image registration and patter recognition. E-mail:ljn6502@126.comLi Liu graduated from Qufu Normal University, China in 2004. She received the M. Sc. degree from Dalian Maritime University, China in 2007. She is currently a Ph. D. candidate in Shanghai University, China, and she also is a lecturer at Ludong University, in the School of Information Science and Electrical Engineering. Her research interests include machine vision, image processing and pattern recognition. E-mail: liulildu@163.comPeter James Smith graduated from the University of Essex, England in 2001 with a B. Eng. (Hons.) in telecommunications systems engineering. He is currently working as a senior research engineer for Vitec Videocom in the area of robot navigation. E-mail: Peter.Smith@VitecGroup.comHuo-Sheng Hu received the M. Sc. degree in industrial automation from Central South University, China in 1982 and the Ph. D. degree in robotics from the University of Oxford, UK in 1993. He is currently a professor with the School of Computer Science and Electronic Engineering, University of Essex, UK, leading the Human Centered Robotics Group. He has published more than 370 research articles in journals, books and conference proceedings. He is a fellow of Institute of Engineering & Technology and Institution of Measurement & Control in the UK, a senior member of IEEE and ACM, and a chartered engineer. He is currently the editor-in-chief for International Journal of Automation and Computing, founding editor-in-chief for Robotics Journal and an executive editor for International Journal of Mechatronics and Automation. His research interests include autonomous robots, human-robot interaction, multi-robot collaboration, embedded systems, pervasive computing, sensor integration, intelligent control, cognitive robotics, and networked robots. E-mail: hhu@essex.ac.ukLi Jing-Na received the B. Sc. degree in physics from Ocean University of China, China in 1985, and the M. Sc. degree in condensed matter physics from Jinan University, China in 1991. She is now a lecturer with School of Information and Electronic Engineering, Ludong University, China. Her research interests include image processing, image registration and patter recognition. E-mail:ljn6502@126.comLi Liu graduated from Qufu Normal University, China in 2004. She received the M. Sc. degree from Dalian Maritime University, China in 2007. She is currently a Ph. D. candidate in Shanghai University, China, and she also is a lecturer at Ludong University, in the School of Information Science and Electrical Engineering. Her research interests include machine vision, image processing and pattern recognition. E-mail: liulildu@163.comPeter James Smith graduated from the University of Essex, England in 2001 with a B. Eng. (Hons.) in telecommunications systems engineering. He is currently working as a senior research engineer for Vitec Videocom in the area of robot navigation. E-mail: Peter.Smith@VitecGroup.com

摘要: In automatic visual inspection, the object image subspace should be segmented and matched quickly so that the affine relationship can be built between the template image and the sample image. When the interference is strong and the illumination is uneven, for example in an industrial application, this can make it difficult to obtain an objects subspace quickly and accurately in real-time. In this paper, a novel strategy is proposed to adopt discrete radial search paths instead of searching all points in an image. Therefore, the searching time can be substantially reduced. In order to reduce the influence coming from the industrial environment, the paper proposes another method that is local energy level set segmentation, which can locate the object subspace more efficiently and accurately. The detection of "crown caps" is presented as an example in this paper. Detection effects and computing time are compared between several detection methods, and the mechanisms of inspection have also been analyzed.

English Abstract

Wen-JuZhou, Zi-XiangFei, Huo-ShengHu, LiLiu, Jing-NaLi, PeterJames Smith, Wen-JuZhou, Zi-XiangFei, Huo-ShengHu, LiLiu, Jing-NaLi, PeterJames Smith. Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy[J]. 国际自动化与计算杂志(英)/International Journal of Automation and Computing, 2016, 13(2): 99-107. doi: 10.1007/s11633-015-0946-2
引用本文: Wen-JuZhou, Zi-XiangFei, Huo-ShengHu, LiLiu, Jing-NaLi, PeterJames Smith, Wen-JuZhou, Zi-XiangFei, Huo-ShengHu, LiLiu, Jing-NaLi, PeterJames Smith. Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy[J]. 国际自动化与计算杂志(英)/International Journal of Automation and Computing, 2016, 13(2): 99-107. doi: 10.1007/s11633-015-0946-2
Wen-Ju Zhou, Zi-Xiang Fei, Huo-Sheng Hu, Liu Li, Jing-Na Li and Peter James Smith. Erratum to: Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy. International Journal of Automation and Computing, vol. 13, no. 6, pp. 665-665, 2016 doi:  10.1007/s11633-015-0946-2
Citation: Wen-Ju Zhou, Zi-Xiang Fei, Huo-Sheng Hu, Liu Li, Jing-Na Li and Peter James Smith. Erratum to: Real-time Object Subspace Searching Based on Discrete Searching Paths and Local Energy. International Journal of Automation and Computing, vol. 13, no. 6, pp. 665-665, 2016 doi:  10.1007/s11633-015-0946-2
    • In automatic visual inspection, the target subspace should be located accurately so that the subspace can be segmented and detected in detail. In the production process, the spatial position of the target images are always changing because of mechanical vibration, the expansion and contraction changes of the supporting structures, and the delays of signals and so on. As a result, the segmentation of the target subspace cannot be completed effectively and efficiently. Thus, the mapping relationship between the target subspace and the standard subspace generates deviations. This results in increasing difficulties for detecting defects in the target images[1]. Consequently, the mapping relationship should be built between the subspace of the standard image and the subspace of the target image by locating and segmenting the target subspace quickly and accurately.

      Among many methods of image segmentation and image matching, the energy function method for segmenting images has become a very popular method amongst researchers. The basic idea of the energy function method is to use curves to express the target contour. The process of segmentation will be turned into the process of working out the minimum energy value. It can be realized by solving the corresponding Euler (Eider-Lagrange) equations. Then, the method of image segmentation can be divided into two categories based on the edge information[2] and also on region information[3]. The model based on the edge information uses a part of edge information to look for the boundary of the target, whose representatives are the Active Contour Model and Snake Model[4]. These models mainly depend on the levels of the edge information, which are irrelevant when considered with the distribution of the global brightness.

      Although, the model method has the ability to segment unevenly illuminated images, the segmentation model based on edge information is very sensitive to noise, and boundary leakage problems will easily occur in weak boundary points of the image. So, the models based on region information are proposed; for example the piecewise constant (PC)[5] model, which drives the evolution from active contour toward the target boundary through defining the region description. The whole region characteristics are used for realizing image segmentation. Therefore, this method has greater robustness than the edge-based segmentation methods. It is usually difficult to define the description of the region within the image since the description of the region is based on uniform illumination hypothesis.

      Effective segmentation cannot be carried out when the brightness of the image is uneven. Tsai et al.[6] proposed the piecewise smooth (PS)[7] model for segmenting in the case of unevenly illuminated images, which actually turns the image segmentation model into the optimization and approximation of the block smooth function. Zhou[8] proposed an improved energy function together with Bayesian analysis to tackle the deficiency in the presence of noise and adjacent other significant edges near the real boundary. Al-Shaikhli et al.[9] proposed an approach for multi-region segmentation based on a topological graph within a multi-level set (MLS) formulation, which is capable of segmenting adjacent objects with very close gray level that would be difficult to segment correctly using standard methods. Ji et al.[10] proposed the method for detecting interest points by combining local spatio-temporal feature and global positional distribution information (PDI) of interest points. So, these models can overcome the influence of the inconsistencies in brightness in some cases. However, it needs to search point by point over the whole image which generally results in a very substantial computation load. Therefore its usage in a real-time detection system on an industrial production line becomes unfeasible.

      To overcome the disadvantages that lie within the segmentation algorithm in the target subspace, this paper proposes novel methods which use local energy and discrete paths based on level set methods to match the target subspace with the template subspace. Our method can locate the object subspace quickly and accurately. The "crown caps" detection is used as an example in this paper, then the detection effects and computing time are compared between several detection methods, and then the mechanism of inspecting has been analyzed. The results show that our method can satisfy the real-time requirement in an industrial application and environment.

      This paper is organized as follows. We firstly describe our proposed methods in Section 2 including discrete paths level set method, and local energy level set method. The experimental results and their comparisons are presented in Section 3. Section 4 shows several applications in an industrial environment. Finally, in Section 5 we describe and draw our conclusion and present our future works.

    • In order to match the target subspace to the template subspace, firstly, the edge of the target subspace should be found. The level set method can be used to find the target subspace, which can reduce disturbances when searching for the template subspace edges under uneven lighting conditions. The level set method was proposed in 1988 by Osher and Sethian[11], they used this method based on the thermodynamics equations for solving the problem of flame shape changes. It is very difficult for the normal equations to express and describe the change of the flame shape since the flame shape is dynamic and the uncertainty of topological structure is unknown.

      Therefore, Osher and Sethian[11] suggested the level set method to describe the interface whose movement is time-dependent. Their main idea is to move the curve (surface) as the zero level set embedded in the higher dimensional function. Then, the evolution equation of the zero level set can be obtained from the hyper surface.

      Level set method is a numerical technique for modelling tracking the shape interface. The two main advantages are; the evolution of the curve (surface) only needs to be numerically calculated on a Cartesian coordinate system and does not need the parametric of the curve (surface). The calculation process is carried out on the fixed grid which is Euler's framework. Another advantage is that it is convenient to deal with the topology of the evolutionary curve (surface) changes and effectively avoid the hard problems in the parametric curve process. The zero level set is the middle part between the negative area and the positive area within the images. Assuming the curved surface function is $\phi $, the evolution curve at the time $t$ is $C(t)$, then the zero level set is $\phi (C(t), t)=0$. According to the composite function derivation rules, derivation of $\phi (C(t), t)=0$ can obtain \begin{align} \label{eq1} \frac{\partial \phi }{\partial t}+\nabla \phi \cdot \frac{\partial C}{\partial t}=0 \tag{1} \end{align} where $\nabla \phi $ is gradient. According to the curve evolution theory, the normal unit vector $\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}}\over {N}}$ on the curve is \begin{align} \label{eq2} \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}}\over {N}} =-\frac{\nabla \phi }{\left| {\nabla \phi } \right|}. \tag{2} \end{align}

      And the curve evolution equation is \begin{align} \frac{\partial C}{\partial t}=F\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}}\over {N}} \tag{3} \end{align} where $F$ is evolution velocity. By substituting (2) and (3) into (1), we get the level set evolution equation: \begin{align} \frac{{\partial \phi }}{{\partial t}} = F\left| {\nabla \phi } \right|. \tag{4} \end{align} In order to solve (4), a partial time-dependent differential equation is needed, which is the Hamilton-Jacobi expression[11]. The final result can be obtained by separating variables, that is, the variables should be processed in time domain and space domain. From formula (4) it can be seen that it is difficult and complex to solve the equation. So, the level set can be initialized as the signed distance function (SDF) approximation in a real world application. The SDF calculation complexity is ${\rm O}(nm)$[12], where $n$ denotes the number of points in a grid and $m$ is the number of grids.

      In order to accelerate the speed to generate the SDF, Sethian[12] proposed the fast matching method (FMM), which sorts all the neighborhood points using the complete binary tree.

      So, the computation complexity of the distance function is reduced from ${\rm O} (nm)$ to ${\rm O} (n\log m)$. In order to further reduce the computational complexity, Tsai[13] proposed the Voronoi source scanning method (VSSM) and also made further developments.

      The level set $\phi (x, y, t)$ can be represented using the discrete grid forms. Assuming the space length of a discrete grid is $h$, the time step is $\Delta t$, then the level set of grid point $(i, j)$ at the time $n$ is $\phi (ih, jh, n\Delta t)$ this can be abbreviated as $\phi _{ij}^n $. Therefore, evolution (4) can be discretized as the following.

      \begin{align} \label{eq5} \frac{\phi _{ij}^{n+1} -\phi _{ij}^n }{\Delta t}=F_{ij}^n \left| {\nabla _{ij} \phi _{ij}^n } \right| \tag{5} \end{align} where $F_{ij}^n $ denotes the value of speed function at time $n$. Equation (5) above can be solved using upwind finite differential method (UFDM). Firstly, six operators are defined as first-order difference, first-order forward difference and first-order back difference.

      \begin{align} \label{eq6} \begin{array}{l} \phi _x^0 =\dfrac{1}{2h}(\phi _{i+1, j} -\phi _{i-1, j} ), \quad \phi _y^0 =\dfrac{1}{2h}(\phi _{i, j+1} -\phi _{i, j-1} ), \\\ \phi _x^+ =\dfrac{1}{h}(\phi _{i+1, j} -\phi _{i, j} ), \quad \quad \phi _y^+ =\dfrac{1}{h}(\phi _{i, j+1} -\phi _{i, j} ), \\\ \phi _x^- =\dfrac{1}{h}(\phi _{i, j} -\phi _{i-1, j} ), \quad \quad \phi _y^0 =\dfrac{1}{h}(\phi _{i, j} -\phi _{i, j-1} ). \end{array} \tag{6} \end{align} By substituting (6) into (5), we can get \begin{align} \label{eq7} \phi _{ij}^{n+1} =\phi ^n+\Delta t(\max (F_{ij}^n , 0)\nabla ^++\min (F_{ij}^n , 0)\nabla ^-) \tag{7} \end{align} where $\nabla ^+$and $\nabla ^-$ are defined as follows: \begin{align} \label{eq8} \begin{array}{l} \nabla ^+=[\max (\phi _x^- , 0)^2+\min (\phi _x^+ , 0)^2 +\\ \quad \quad\;\; \max (\phi _y^- , 0)^2+\min (\phi _y^+ , 0)^2]^{\frac{1}{2}} \\ \nabla ^-=[\max (\phi _x^+ , 0)^2+\min (\phi _x^- , 0)^2+ \\ \quad \quad\;\; \max (\phi _y^+ , 0)^2+\min (\phi _y^- , 0)^2]^{\frac{1}{2}}. \end{array} \tag{8} \end{align} Equation (7) is used to segment image, the segment speed function is \begin{align} \label{eq9} F=F_{prop} +F_{curv} +F_{adv} \tag{9} \end{align} where $F_{prop} =V_0 $ denotes the evolution speed on the length, $F_{curv} =-\varepsilon k $ is the evolution speed on the curvature, $F_{adv} =\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}}\over {U}} \times \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}}\over {V}} $ is convection velocity on the level where $\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}}\over {U}} =(u(x, y, t), v(x, y, t))$. $u$ is the gradient in the $x$ orientation and $v$ is the gradient in the $y$ orientation. Thus, (7) can be rewritten as \begin{align} \label{eq10} \begin{array}{l} \phi _{ij}^{n+1} =\phi ^n+\Delta t[(\max (V_{0ij} , 0)\nabla ^++\min (V_{0ij} , 0)\nabla ^-)+ \\ \quad \quad \quad \quad (\max (u_{ij}^n , 0)\phi _x^- +\min (u_{ij}^n , 0)\phi _x^+ ) +\\ \quad \quad \quad \quad (\max (v_{ij}^n , 0)\phi _y^- +\min (v_{ij}^n , 0)\phi _y^+ )- \\ \quad \quad \quad \quad \varepsilon \kappa _{ij}^n ((\phi _x^0 )^2+(\phi _y^0 )^2)^{\frac{1}{2}}] . \end{array} \tag{10} \end{align}

      The level set can be updated continuously by (10). The step space $\Delta t$ should be fitted by Courant-Friedrichs-Levy (CFL) as follows. In the case that the grid space $h$ has been given, we have \begin{align} F\times \Delta t\le h. \tag{11} \end{align}

      Since (10) is calculated for the whole area in the image, the computation is heavy when the image size is large. In order to reduce the computation, Chopp[14] proposed the idea of Narrow Band in 1993, and Adalsteinsson et al.[15] gave its specific realization in 1995. The method can rapidly evolve to obtain the value of a level set. The main principle is only to update the narrow band around the shape of the level set region. Due to lower number of grid points, the computation is greatly reduced.

      Although narrow band method can reduce the calculation, it still does not satisfy the real-time application in the industrial field. Therefore, we propose a discrete path searching method, whose search is along several fixed lines but not the full image. The computational complexity of the search becomes 1D from 2D. The search space is shrunk greatly, and the search efficiency is improved. Thus, the real-time search can then be satisfied.

      In this paper, the crown cap image is used as an application example for exploring the level set method. Since the cap shape is a circular image, the search paths can be along the direction of the radius, the search method is shown in Fig.1.

      As shown in Fig.1, the level set search is performed from the outer circle to inner circle. The search path is along the radius and the search direction is the same as the arrow direction. Because the arrow direction is almost orthogonally aligned with the edge of the cap image, the search paths may be the shortest paths and the gradient may be the maximum along those paths. One search path can get one zero level set point. Thus, the number of level set points has been greatly reduced and the computing efficiency has been greatly enhanced.

      Because the crown cap image is captured as it is being transferred on the production line, the transferring equipment is also captured forming part of the image, which is shown in Fig.2. In Fig.2, the cap left edge is together with the holding ratchet, which will cause inaccuracies when locating the edge of the cap image. Therefore, this paper proposes the local energy level set method based on the discrete path level set search, which can accurately search caps edges according to the energy difference between the caps surface and the caps skirt.

    • Mumford and Shah[16] proposed image segmentation model based on energy minimization in 1989. The original idea was to find a contour curve $(I_0 , C)$ to approximate a given image $I$, where $I_0 $ is the piecewise smooth approximation of the original image, and $C$ denotes the smooth closed contour curvatures. The energy functional expression of the model is written as follows.

      \begin{align} \begin{array}{l} {E}(I, C)=\int\limits_\Omega {\left| {I(x, y)-I_0 (x, y)} \right|} ^2{\rm d}x{\rm d}y +\\ \qquad \mu \int\limits_{\Omega \backslash C} {\left| {\nabla I(x, y)} \right|^2{\rm d}x{\rm d}y+} v\times length(C) \\ \end{array} \tag{12} \end{align} where $\mu $ and $v$are positive constants, $\Omega $ denotes the image region, and $C\subset \Omega $ is the contour curve.

      The first term on the right side of (12) is named as fidelity, which is used to express the similarity between the segmented image and the original image. The second term denotes smoothness, which is used to ensure the smoothness of the segmented regions. The third term is the constraint, which is used to constrain the length to reach the minimum. When (12) reaches the minimum value on the left, $I$ and $C$ on the right can get the desired results. However, the solution process for (12) is very complex. So, the Mumford-Shah model should be simplified to a piecewise smooth function or a piecewise constant function, i.e., the model is approximated as a constant in each target area, the detail can be found in Chan-Vese (CV) model [17]. That is, CV is the simplified approximation of the Mumford-Shah model, which can be obtained by minimizing the energy function as follows: \begin{align} \begin{array}{l} {E}(u_1 , u_2 , C)=\mu \cdot Length(C) +\\ \quad \lambda _1 \times \int\limits_{inside(C)} {\left| {I(x, y)-u_1 } \right|^2{\rm d}x{\rm d}y}+ \\ \quad \lambda _2 \times \int\limits_{outside(C)} {\left| {I(x, y)-u_2 } \right|^2{\rm d}x{\rm d}y} \\ \end{array} \tag{13} \end{align} where $\mu $,$\lambda _1 $ and $\lambda _2 $ are positive constants, in this case, usually being set to $\mu =\lambda _1 =\lambda _2 =1$. $u_1 $ and $u_2 $ give the grayscale average at the outside and inside of the curve $C$. The first term of the energy function (13) is used to normalize the curve. The second and third together is giving the fidelity, its role is to attract the curve to the target contour.

      In order to obtain the minimum energy of the ${\rm E}(u_1 , u_2 , C)$, the level set idea is used, that is, the level set method may be used over the unknown evolution curve. When the points are inside of the curve, the level set is defined as $\phi (x, y)>0$. If the points are outside of the curve, the level set is defined as $\phi (x, y)< 0$. If the points are on the curve, then the level set is defined as $\phi (x, y)=0$. Thus, (13) can be rewritten as follows: \begin{align} \label{eq14} \begin{array}{l} {E}(u_1 , u_2 , C)=\mu \int\limits_\Omega {\delta _\varepsilon (\phi (x, y))\left| {\nabla \phi (x, y)} \right|^2{\rm d}x{\rm d}y}+ \\ \quad \quad \lambda _1 \times \int\limits_\Omega {\left| {I(x, y)-u_1 )} \right|} ^2H_\varepsilon (\phi (x, y)){\rm d}x{\rm d}y +\\ \quad \quad \lambda _2 \times \int\limits_\Omega {\left| {I(x, y)-u_2 )} \right|} ^2(1-H_\varepsilon (\phi (x, y))){\rm d}x{\rm d}y \\ \end{array} \tag{14} \end{align} where $\delta _\varepsilon (z)$ and $H_\varepsilon (z)$ are Dirac and Heaviside. $\delta _\varepsilon (z)$ and $H_\varepsilon (z)$ can be expanded as the following equations \begin{align} \label{eq15} &\delta _\varepsilon (z)=\left\{ {{\begin{array}{*{20}c} {\frac{1}{2\varepsilon }[1+\cos (\frac{\pi z}{\varepsilon })], \quad \left| z \right|\le \varepsilon } \hfill \\ {0, \quad {\kern 1pt}\quad \quad \quad \quad \quad \;\;{\kern 1pt}{\kern 1pt}{\kern 1pt}\left| z \right|>\varepsilon } \hfill \\ \end{array} }} \right.\\ \tag{15} \end{align} \begin{align} & H_\varepsilon (z)=\left\{ {{\begin{array}{*{20}c} {\frac{1}{2}(1+\frac{z}{\varepsilon }+\frac{1}{\pi }\sin (\frac{\pi z}{\varepsilon })), \quad \left| z \right|\le \varepsilon } \hfill \\ {1, \quad \quad \quad \quad \quad \quad \;\;\;\quad \quad z>\varepsilon } \hfill \\ {0, \quad \quad \quad \quad \;\quad \;\;\;\;\quad \quad z<-\varepsilon. } \hfill \\ \end{array} }} \right. \tag{16} \end{align} The minimized result of (14) can be solved by the energy functional form of Euler-Lagrange. The following level set evolution equations can be obtained.

      \begin{align} &\label{eq17} \frac{\partial \phi }{\partial t}=\delta _\varepsilon (\phi )[\mu {\rm div}(\frac{\nabla \phi }{\left| {\nabla \phi } \right|})-\lambda _1 (I-u_1 )^2+\lambda _2 (I-u_2 )^2]\\ \tag{17} \end{align} \begin{align} &\label{eq18} \phi (0, x, y)=\phi _0 (x, y)\quad \quad \subset\quad \Omega\\ \tag{18} \end{align} \begin{align} &\label{eq19} \frac{\delta _\varepsilon (\phi )}{\left| {\nabla \phi } \right|}\frac{\partial (\phi )}{\partial \vec {n}}=0\quad \quad \subset\quad \partial \Omega \tag{19} \end{align} where (18) is the initial condition, and (19) is the boundary. The grayscale $u_1 $ and $u_2 $ are updated by the following, \begin{align} & u_1 (\phi )=\frac{\int_\Omega {I(x, y)H_\varepsilon (\phi (x, y)){\rm d}x{\rm d}y} }{\int_\Omega {H_\varepsilon (\phi (x, y)){\rm d}x{\rm d}y} }\notag\\ & u_2 (\phi )=\frac{\int_\Omega {I(x, y)(1-H_\varepsilon (\phi (x, y))){\rm d}x{\rm d}y} }{\int_\Omega {(1-H_\varepsilon (\phi (x, y))){\rm d}x{\rm d}y} }. \tag{20} \end{align} In the practice, (14) and (20) need be converted to differential form to implement.

    • In this article, the results of the subspace matching experiments are compared using the level set method, discrete path level set and the local energy path level set methods. In the above matching experiments, the crown cap image is obtained from an actual industrial production line. According to the circular features of the caps, the search paths are pre-set to be along the radius of the caps' circle, which is shown in Fig.1. The experiments were carried out on an Intel i5 PC with 4G RAM. All the computations were performed with C#.

    • In order to compare the effectiveness and the efficiency of searching for cap edges using different methods, the experiments with different paths were carried out. Figs. 3 and 4 are the crown cap images captured from the production line. Fig.3 shows the search of the crown caps seal side using the discrete path level set method. Fig.4 shows the search of the crown caps surface side using the discrete path level set method. The numbers of searching paths in two figures are 15, 30 and 60. As shown in figures, the edges of the seal side and surface side can be successfully grasped using the discrete path level set method.

      Then, the seal circle region and surface circle region can be fitted. Fig.5 shows the circle fitted from Fig.3, and Fig.6 shows the circle fitted from Fig.4.

      As shown in Figs. 5 and 6, due to the skirts of the edge and the gripping device interference with the searching for the edge of the cap, the fitted circles tend to deviate away from the actual cap seal circle and the surface circle. Although the error can be reduced by increasing the number of search paths, the error still persists.

      From (17), Fig.7 shows the results using the local energy discrete path level set method to search for the edge of the bottle caps. The search path in Fig.7 is the same as that of Figs. 5 and 6.

      As shown in Fig.7, the upper images are the inner seals fitted and the bottom images are the surface circles fitted. The edge search results have been satisfactory as the number of search paths is 30. So Fig.7 is only showing the experiments results when the number of search paths are 15 and 30 respectively. The search results and fitting with 60 paths are the same as that of the 30 paths. Comparing the fitted circles in Figs. 5 and 7, the local energy discrete path level set method is found to be better than that of the discrete path level set method. In Fig.7, it shows a perfect match on the subspaces of bottle caps seals and surfaces by the circle spaces.

    • In order to verify the working efficiency of all of our proposed methods and several other methods, we run three tests, the narrow band level set method, the discrete paths level set method and the local energy discrete paths level set methods, each test is run 10 times to obtain the average time for correct detection using the same computer. The experimental results are presented in Tables 1 and 2.

      Table 1 gives the time consuming results of matching circles in the inner image of the crown cap, and Table 2 is for the surface image of the crown cap. As shown in Tables 1 and 2, the discrete paths level set method spends lesser time than that of the narrowband method. Especially, when the number of paths is 15, the average matching time of the discrete paths level set method is only 0.32 milliseconds in both tables. The average times of the local energy discrete paths level set method are slightly larger than that of discrete paths level set method. From Figs. 5-7, the matching accuracy of the local energy method is better than the discrete paths method. Therefore, there is a trade-off between the sensitive detection and the rough detection methods. It is worth noting that for a small cost in time, one can achieve a large improvement in accuracy. The time consumed in the local energy discrete paths level set method is less than 1 millisecond in both cases. It is apparent that our methods can meet real-time constraints and ensure an acceptable level of accuracy.

      Table .  Time consuming result of matching circles in inner image of crown cap (ms)

      Table .  Time consuming result of matching circles in surface image of crown cap (ms)

    • In order to verify the practical application of our method, we used our method in the crown caps production line to detect a variety of branded bottle caps. We saved the photos of the caps rejected by the detection program. Then, the photos with the failing matching surface circles were selected from the saved photos. Based on above data, the relevant results can be obtained. The definitions of $FPR$, false positive rate, is defined as follows: \begin{align} FPR=\frac{N}{P}\times 100 % \tag{21} \end{align} where $P$ is the total number of caps, $N$ is the total number of defective caps that are mistakenly matched circles, using our method. $FPR$ stands for the rate of erroneously matched circles when searching the subspace. Four kinds of bottle caps with different surface patterns were tested as shown in Fig.8. Results of tests performed for ten times are shown in Table 3.

      Table .  $FPR$ of matching circles in surface image of crown cap

      As shown in Table 3, the total $FPR$ of PEARL RIVER 1, PEARL RIVER 2, PEARL RIVER 3 and MEIWEIXIAN are 0.20 %, 0.24 %, 0.20 % and 0.25 %, respectively, even though the $FPR$ is changing for different color backgrounds of the caps. All $FPR$ levels of the four different kinds of caps are less than 0.3 %, hence it can meet the requirements of industrial application. Therefore, our method can eliminate the interference of the skirts, gripping device, and can accurately search the edge of the caps as shown in Fig.8. As can be seen the caps circle region subspace can be matched precisely.

    • In this paper, the subspace matching method based on the level set is studied, and the discrete paths level set method is proposed for raising the evolution speed of the level set. On this basis, the local energy discrete paths level set method is developed, which overcomes the disturbance produced by the uneven illumination and gripping device image. Experiments on practical images of bottle caps on the production lines show that the proposed method is the most effective in searching for the defect of bottle caps. It is also shown that method is robust for various types of crown caps. This precise subspace matching is a good foundation for the further detecting of defects in caps quickly, accurately and in real-time. All of the above work was carried out based on monochrome images, because it is difficult to detect tiny color deviations. Future work should focus on using color images in order to get more accurate detection.

目录

    /

    返回文章
    返回