Volume 13 Number 6
December 2016
Article Contents
Yuan Ge and Yaoyiran Li. SCHMM-based Compensation for the Random Delays in Networked Control Systems. International Journal of Automation and Computing, vol. 13, no. 6, pp. 643-652, 2016. doi: 10.1007/s11633-016-1001-7
Cite as: Yuan Ge and Yaoyiran Li. SCHMM-based Compensation for the Random Delays in Networked Control Systems. International Journal of Automation and Computing, vol. 13, no. 6, pp. 643-652, 2016. doi: 10.1007/s11633-016-1001-7

SCHMM-based Compensation for the Random Delays in Networked Control Systems

Author Biography:
  • Yaoyiran Li,is a senior student majoring in electrical engineering at University of Electronic Science and Technology of China, China. He is an IEEE student member (No. 93258350).
    His research interests include robotics, artificial intelligence and automatic control.
    E-mail:1622455452@qq.com;
    ORCID iD:0000-0001-9529-3878

  • Corresponding author: Yuan Ge received the B. Sc., M. Sc., and Ph.D. degrees from University of Science and Technology of China in 2002, 2005 and 2011, respectively. Now, he is a visiting scholar in the Department of Electrical Engineering at Tsinghua University, China, and he is also a professor in the College of Electrical Engineering at Anhui Polytechnic University, China.
    His research interests include networked control systems, telerobotic systems and optimal control.
    E-mail: ygetoby@mail.ustc.edu.cn ;
    ORCID iD: 0000-0003-2037-319X
  • Received: 2015-05-21
  • Accepted: 2015-10-16
  • Published Online: 2016-06-20
Fund Project:

This work was supported by National Natural Science Foundation of China 61203034 and 61572032

通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures (4)

Metrics

Abstract Views (424) PDF downloads (5) Citations (0)

SCHMM-based Compensation for the Random Delays in Networked Control Systems

  • Corresponding author: Yuan Ge received the B. Sc., M. Sc., and Ph.D. degrees from University of Science and Technology of China in 2002, 2005 and 2011, respectively. Now, he is a visiting scholar in the Department of Electrical Engineering at Tsinghua University, China, and he is also a professor in the College of Electrical Engineering at Anhui Polytechnic University, China.
    His research interests include networked control systems, telerobotic systems and optimal control.
    E-mail: ygetoby@mail.ustc.edu.cn ;
    ORCID iD: 0000-0003-2037-319X
Fund Project:

This work was supported by National Natural Science Foundation of China 61203034 and 61572032

Abstract: In order to compensate the network-induced random delays in networked control systems (NCSs),the semi-continuous hidden Markov model (SCHMM) is introduced in this paper to model the controller-to-actuator (CA) delay in the forward network channel.The expectation maximization algorithm is used to obtain the optimal estimation of the model s parameters,and the Viterbi algorithm is used to predict the CA delay in the current sampling period.Thus,the predicted CA delay and the measured sensor-tocontroller (SC) delay in the current sampling period are used to design an optimal controller.Under this controller,the exponentially mean square stability of the NCS is guaranteed,and the SC and CA delays are compensated.Finally,the effectiveness of the method proposed in this paper is demonstrated by a simulation example.Moreover,a comparative example is also given to illustrate the superiority of the SCHMM-based optimal controller over the discrete hidden Markov model (DHMM)-based optimal controller.

Yuan Ge and Yaoyiran Li. SCHMM-based Compensation for the Random Delays in Networked Control Systems. International Journal of Automation and Computing, vol. 13, no. 6, pp. 643-652, 2016. doi: 10.1007/s11633-016-1001-7
Citation: Yuan Ge and Yaoyiran Li. SCHMM-based Compensation for the Random Delays in Networked Control Systems. International Journal of Automation and Computing, vol. 13, no. 6, pp. 643-652, 2016. doi: 10.1007/s11633-016-1001-7
  • The rapid development of information technologies has boosted the emergence of networked control systems (NCSs) in which communication networks are used to connect sensors,controllers,and actuators[1, 2]. The introduction of networks brings a wealth of new advantages including reduced system wiring,increased system flexibility,and high system reliability. Owing to these distinctive benefits,typical application of NCSs ranges over various fields,such as mobile robot,advanced aircraft,manufacturing plant,and remote surgery[3].

    While NCSs have received increasing attention,they have also given rise to new challenges due to the inherent limitation of network bandwidth. Among all the challenges,the random network-induced delay and the intermittent packet dropout are known to be two of the main causes for the performance degradation or even the instability of NCSs. As pointed out,in [4],that the packet dropout can be seen as that the packet experiences an infinite delay,the degradation caused by the packet dropout can be treated as that caused by the infinite delay. So,random delays are the major problem and challenge in NCSs. The effect of random delays on the performance of NCSs can be compensated by designing proper controllers. Such issues have been investigated by many researchers in recent years,see [5],and the references therein.

    Generally,it is necessary to establish the mathematical model of random delays before compensation. Four types of modeling methods for the random delays have been surveyed in [4]. They are constant delay model,mutually independent stochastic delay model,Markov chain delay model,and hidden Markov model (HMM). Among them,the HMM describes the probabilistic relations between the random delays and the network states,or more precisely,the HMM models the random delays from the view of revealing the essential generation mechanism of random delays. It is worthy to notice that both the Markov chain delay model and the HMM delay model consider the random delay as a discrete stochastic variable. That is to say,the HMM given in [4] is strictly a discrete HMM (DHMM). But actually the random delay can arbitrarily take values from its acceptable interval,which means that the random delay itself is not strictly discrete. So,the semi-continuous HMM (SCHMM) was proposed to model the random delays in [6]. As demonstrated in [6],the SCHMM is superior to the DHMM in accuracy and to the continuous HMM (CHMM) in efficiency.

    Based on these delay models,various controllers can be designed to compensate the effect of the random delays on NCSs. For the constant delay model,a deterministic predictor-based delay compensation methodology was given in [7]. For the mutually independent stochastic delay model,the delay compensations were achieved by optimal control[8],robust control[9],fuzzy control[10],and predictive control[11]. For the Markov chain delay model,the NCS is usually modeled as a Markovian jump linear system (MJLS) and some controllers can be designed to compensate the random delays[12-15]. For the discrete HMM,a state feedback controller[16] and an optimal controller[17] were designed to compensate the random controller-actuator delays. But,for the SCHMM,how to design a controller for delay compensation has not been discussed in the literature of NCSs. In this paper,we will investigate the design method of an optimal controller based on the SCHMM delay model. Moreover,the compensation effect of this optimal controller will be compared with that of the optimal controller designed in [17]. The comparison will demonstrate the superiority of the SCHMM over the DHMM from the aspect of delay compensation,which is exactly the contribution of this paper.

    The rest of this paper is organized as follows. The SCHMM-based delay modeling and prediction methods are repeated in Section 2 for the sake of paper integrity. In Section 3,the optimal controller is designed based on the SCHMM delay model. Some examples are given in Section 4 to verify the effectiveness and superiority of the optimal controller designed in Section 3. Finally,Section 5 concludes the paper and suggests future work.

  • For the sake of clarity and integrity,the SCHMM delay model derived in [6] is briefly presented in this section. The typical NCS considered in this paper has the block diagram as shown in Fig. 1[6]. Similarly,the sensor is time-driven and its sampling period is h. The sensor measurement is represented by xk. Both the controller and the actuator are event-driven. There are mainly two kinds of random delays in NCSs. One is the sensor-to-controller delay (SC delay) in the backward network channel,and the other is the controller-to-actuator delay (CA delay) in the forward network channel. They are denoted by $\tau _k.{sc}$ and $\tau _k.{ca}$ respectively in the current (i.e.,the k-th) sampling period. The network nodes (i.e.,sensor,controller and actuator) can be clock-synchronous by using the method in [18],and then the SC and CA delays can be calculated by using the timestamp technology. The SC delay can be calculated by comparing the timestamp of the sensor measurement with the local time of the controller as soon as the controller receives the measurement. Similarly,the CA delay can be calculated by comparing the timestamp of the control law with the local time of the actuator as soon as the actuator receives the control law. For simplicity,the sum of the SC and CA delays is assumed to be not more than one sampling period (i.e.,$\tau _k.{sc} + \tau _k.{ca} \le h$).

    Figure 1.  Diagram of a typical NCS

    In Fig. 1,when designing the current controller ${u_k}$,the current SC delay is visible to the controller since it has occurred,but the current CA delay is not visible because it has not occurred. In order to compensate the current CA delay as well as the current SC delay by the controller ${u_k}$,a feasible way is to predict the current CA delay before designing the controller. Then,both the predicted CA delay (denoted as $\tilde \tau _k.{ca}$) and the calculated SC delay $\tau _k.{sc}$ are visible to the controller and can be compensated when they are considered into the design of the controller. In [6],the SCHMM was proposed to model the CA delay and obtain its current predicted value $\tilde \tau _k.{ca}$. Since the training of SCHMM needs plenty of historical CA delay data,a buffer is set at the controller node to collect all the past CA delays (i.e.,$\tau _{k - 1}.{ca},\cdots ,\tau _1.{ca}$). As is known,the adjacent previous CA delay $\tau _{k - 1}.{ca}$ can be calculated at the actuator node by using the timestamp technology. After that,it is packaged into the sensor measurement ${x_k}$ to generate a single packet ($\{ {x_k},\tau _{k - 1}.{ca}{\rm{\} }}$) at the sensor node and then is extracted from the packet to be put into the delay buffer at the controller node. In this way,all the past CA delays (denoted by ${\tau .{ca}}$,${\tau .{ca}} = \{ \tau _r.{ca}\} _{r = 1}.{k - 1}$) are collected in the delay buffer. Finally,${\tau .{ca}}$ can be used to train the SCHMM delay model and predict the current CA delay.

    The SCHMM delay model derived in [6] is rewritten as follows.

    $\lambda =(N,M,\pi ,A,B).$

    (1)

    The detailed derivation of this model can be referred to [6]. Here,the definitions of the five parameters in (1) are briefly given as follows.

    1) N denotes the number of the different network states that constitute a discrete state space S ($S = \{ 1,2,\cdots ,N\}$). These network states are modeled as a hidden Markov chain and comprehensively reflect the whole working status of the network. The network state in the k-th sampling period is denoted by ${s_k}$,and obviously,${s_k} \in S$ holds.

    2) M denotes the number of Gaussian mixture densities. In the SCHMM delay model,the distribution of random delays is described by Gaussian mixture densities. All network states share the same M Gaussian densities,but different states have different mixture weights. The M component Gaussian densities are labeled by the integers from 1 to M,and the number set $\{ 1,2,\cdots ,M\}$ is denoted as L.

    3) ${ \pi}$ is a vector denoting the initial probabilities of the N network states,and ${\pi} = {\{ {\pi _j}\} _{1 \le j \le N}}$,where ${\pi _j} = P({s_1} = j)$ ($j \in S$). There are some constraints for these initial probabilities: ${\pi _j} \ge 0$ and $\sum\nolimits_{j = 1}.N {{\pi _j}} = 1$.

    4) ${A}$ is the state transition matrix of the hidden Markov chain,and ${A} = {\{ {a_{ij}}\} _{1 \le i,j \le N}}$,where ${a_{ij}} = P({s_{k + 1}} = j|{s_k} = i)$ ($i,j \in S$). The element ${a_{ij}}$ indicates the one-step transition probability of going from state i at time k to state j at time $k + 1$. Generally,the hidden Markov chain is time-homogeneous,and then all the elements of matrix ${A}$ are time-independent. There are some constraints for these elements: ${a_{ij}} \ge 0$ and $\sum\nolimits_{j = 1}.N {{a_{ij}}} = 1$.

    5) ${B}$ denotes the combination of the M component Gaussian distributions in the SCHMM delay model,and is defined as

    $B=\left\{ {{b}_{j}}(\tau _{k}.{ca})|{{b}_{j}}(\tau _{k}.{ca})=\sum\limits_{l=1}.{M}{{{c}_{jl}}{{g}_{l}}(\tau _{k}.{ca})} \right\},j\in S,l\in L.$

    (2)

    ${{g}_{l}}(\tau _{k}.{ca})=G(\tau _{k}.{ca}|{{\mu }_{l}},{{\sigma }_{l}})=\frac{1}{\sqrt{2\pi {{\sigma }_{l}}}}{{\text{e}}.{-\frac{{{(\tau _{k}.{ca}-{{\mu }_{l}})}.{2}}}{2{{\sigma }_{l}}}}}.$

    (3)

    In (2),${b_j}(\tau _k.{ca})$ indicates the probability of the event that the current CA delay observation (denoted as ${o_k}$) is $\tau _k.{ca}$ when the current network state (${s_k}$) is j at time k,i.e.,${b_j}(\tau _k.{ca}) = P({o_k} = \tau _k.{ca}|{s_k} = j)$. In (2),${g_l}(\tau _k.{ca})$ and ${c_{jl}}$ define a Gaussian density and its mixture weight. There are also some constraints for these mixture weights: ${c_{jl}} \ge 0$ and $\sum\nolimits_{l = 1}.M {{c_{jl}}} = 1$. Note that the symbol "$\pi $" in (3) denotes the circular constant \emph{pi} (\emph{pi}$\approx$3.14) and is different from the initial state probability vector ${\pi}$ in (1).

    It is worthy to notice that there is an observation process o ($o = \{ {o_r}\} _{r = 1}.{k - 1}$) corresponding to the delay process ${\tau .{ca}}$. The observation variable ${o_k}$ is continuous in the SCHMM delay model but discrete in the DHMM delay model[16, 17]. Actually,the observation variable ${o_k}$ is equivalent to the delay value $\tau _k.{ca}$ in the SCHMM and is only the quantized result of $\tau _k.{ca}$ in the DHMM. This difference has a great contribution to the superiority of the SCHMM over the DHMM in the delay modeling and prediction. In (3),$\tau _k.{ca}$ ($\tau _k.{ca} \in { R}$) is a continuous random variable and can arbitrarily take values from its acceptable range. The mean ${\mu _l}$ and the covariance ${\sigma _l}$ (or the function $G( \cdot )$ as a whole) are irrelevant to the network states. So,in the SCHMM,every mixture density consists of the same M baseline distributions. But,for different network states,the mixture weights of these baseline distributions are different. That is why the parameter ${c_{jl}}$ in (2) is relevant to the network state j.

    Overall,these parameters (N,M,${\pi}$,${A}$ and ${B}$) define the SCHMM delay model as shown in (1). Generally,N and M are known in advance,so the SCHMM delay model ($\lambda $) can be simplified as

    $\lambda =(\pi ,A,B).$

    (4)

    In order to obtain the optimal estimation of these parameters,the expectation maximization (EM) algorithm was proposed in [6] to train the SCHMM delay model. The past CA delay set ${\tau .{ca}}$ was used as the input to the training process and the maximum-likelihood estimations of the parameters ($\pi$,${A}$ and ${B}$) were obtained. The iterative estimation equations in the k-th sampling period are rewritten as follows,which can also be referred to (29a) - (29e) in [6].

    ${{{\hat{\pi }}}_{i}}={{\zeta }_{1}}(i),(i\in S)$

    (5)

    ${{{\hat{a}}}_{ij}}=\frac{\sum\limits_{r=2}.{k-1}{{{\zeta }_{r-1}}(i,j)}}{\sum\limits_{r=2}.{k-1}{{{\zeta }_{r-1}}(i)}},(i,j\in S)$

    (6)

    ${{{\hat{c}}}_{il}}=\frac{\sum\limits_{r=1}.{k-1}{{{\xi }_{r}}(i,l)}}{\sum\limits_{r=1}.{k-1}{{{\zeta }_{r}}(i)}},(i\in S,l\in L)$

    (7)

    ${{{\hat{\mu }}}_{l}}=\frac{\sum\limits_{r=1}.{k-1}{\tau _{r}.{ca}{{\xi }_{r}}(l)}}{\sum\limits_{r=1}.{k-1}{{{\xi }_{r}}(l)}},(l\in L)$

    (8)

    ${{{\hat{\sigma }}}_{l}}=\frac{\sum\limits_{r=1}.{k-1}{{{(\tau _{r}.{ca}-{{{\hat{\mu }}}_{l}})}.{2}}{{\xi }_{r}}(l)}}{\sum\limits_{r=1}.{k-1}{{{\xi }_{r}}(l)}},(l\in L).$

    (9)

    In (5) - (7),${\zeta _r}(i)$ defines the post probability of being in state i at time r given a model $\lambda '$ and a past delay set ${\tau .{ca}}$,and ${\zeta _r}(i,j)$ defines the post probability of transiting from state i at time r to state j at time $r+1$. The definitions of ${\zeta _r}(i)$ and ${\zeta _r}(i,j)$ are the same as (28a) and (28b) in [6],and they can be calculated by introducing the forward variable ${\alpha _r}(i)$ and the backward variable ${\beta _r}(i)$ as defined in Definitions 1 and 2 in [6].

    In (7) - (9),${\xi _r}(i,l)$ defines the probability of selecting in state i the l-th Gaussian mixture component at time r for generating the delay $\tau _r.{ca}$ given a model $\lambda '$ and a past delay set ${\tau .{ca}}$,and ${\xi _r}(l)$ is actually the marginal distribution of ${\xi _r}(i,l)$ by summing over all possible network states. The definitions and calculations of ${\xi _r}(i,l)$ and ${\xi _r}(l)$ can be referred to (28c) and (28d) in [6].

    In summary,based on ${\alpha _r}(i)$,${\beta _r}(i)$,${\zeta _r}(i)$,${\zeta _r}(i,j)$,${\xi _r}(i,l)$ and ${\xi _r}(l)$,we can get the estimations of ${\pi}$,${A}$ and ${B}$ by solving (5) - (9). But,it is worthy to notice that the calculations of ${\alpha _r}(i)$,${\beta _r}(i)$,${\zeta _r}(i)$,${\zeta _r}(i,j)$,${\xi _r}(i,l)$ and ${\xi _r}(l)$ are all based on the previous estimations of $\pi $,A and B. Iteratively using these estimation equations (5) - (9),we can finally get the optimal estimations of these parameters ${\lambda .*}$ (${\lambda .*} = ({{\pi} .*},{{A}.*},{{B}.*})$). The detailed iterative training procedure can be referred to Fig. 2 in [6].

    Based on the derived optimal parameters of the SCHMM,we can predict the current CA delay (denoted by $\tilde \tau _k.{ca}$) through the following three steps.

    Step 1. Estimate the optimum network state set $\tilde s$ that produces the past CA delay set with maximal posterior probability (i.e.,$\tilde s = \arg \mathop {\max }\limits_s P(s|{\tau .{ca}},{\lambda .*})$,where s denotes the network state set ($s = \{ {s_{k - 1}},\cdots ,{s_1}\} $,${s_i} \in S$) corresponding to the past CA delay set). This step can be achieved by using the Viterbi algorithm,see Fig. 3 in [6].

    Step 2. Estimate the optimal network state in the current (i.e.,k-th) sampling period by using (34) in [6],i.e.,${\tilde s_k} = \arg \mathop {\max }\limits_j a_{{{\tilde s}_{k - 1}}j}.*$.

    Step 3. Predict the current CA delay through calculating the extremum of ${b_{{{\tilde s}_k}}}(\tau _k.{ca})$(see (35) - (36) in [6]) and finally obtain the prediction $\tilde \tau _k.{ca}$.

    Now,the predicted current CA delay $\tilde \tau _k.{ca}$ together with the measured current SC delay $\tau _k.{sc}$ can be used to design the current control law ${u_k}$,which will be able to compensate these two delays. How to design such a control law will be discussed in the next section.

  • In this section,an optimal controller is designed to compensate the SC and CA delays,and moreover,to guarantee the exponential mean square stability of the NCS.

    The plant considered in the NCS is described by the following continuous-time dynamics.

    $\left\{ \begin{align} & \dot{x}(t)={{A}_{1}}x(t)+{{A}_{2}}u(t)+v(t) \\ & y(t)={{A}_{3}}x(t)+\omega (t). \\ \end{align} \right.$

    (10)

    In (10),$x(t)\in {{\bf R}.n}$ is the state vector,$u(t)\in {{\bf R}.m}$ is the input vector,$y(t)\in {{\bf R}.z}$ is the output vector,and ${{A}_1}$,${{A}_2}$,${{A}_3}$ are some constant matrices with appropriate dimensions. $v(t)\in {{\bf R}.n}$ is the system white noise and $\omega (t)\in {{\bf R}.z}$ is the measurement white noise. These two noises are mutually independent. Considering that the sum of the current SC and CA delays is not more than one sampling period,the continuous-time dynamics (10) in the r-th ($0 \le r < k$) sampling period can be discretized as

    $\left\{ \begin{align} & {{x}_{r+1}}={{\Omega }_{1}}{{x}_{r}}+{{\Omega }_{2}}(\tau _{r}.{sc},\tilde{\tau }_{r}.{ca}){{u}_{r}}+{{\Omega }_{3}}(\tau _{r}.{sc},\tilde{\tau }_{r}.{ca}){{u}_{r-1}}+{{v}_{r}} \\ & {{y}_{r}}={{A}_{3}}{{x}_{r}}+{{\omega }_{r}} \\ \end{align} \right.$

    (11)

    where ${{\Omega} _1} = {{\rm e}.{{{A}_1}h}}$,${{\Omega} _2}(\tau _r.{sc},\tilde \tau _r.{ca}) = \int_0.{h - \tau _r.{sc} - \tilde \tau _r.{ca}} {{{\rm e}.{{{A}_1}t}}{\rm d}t{{A}_2}} $,and ${{\Omega} _3}(\tau _r.{sc},\tilde \tau _r.{ca}) = \int_{h - \tau _r.{sc} - \tilde \tau _r.{ca}}.h {{{\rm e}.{{{A}_1}t}}{\rm d}t{{A}_2}} $. ${v_r}$ and ${\omega _r}$ are still mutually independent white noise with zero mean. Furthermore,it is assumed that ${v_r}$ and ${\omega _r}$ obey Gaussian distribution ${N}(0,{R_1})$ and ${N}(0,{R_2})$,respectively.

    Obviously,both the predicted CA delay ($\tilde \tau _r.{ca}$) and the measured SC delay ($\tau _r.{sc}$) are combined into the discrete-time system model (11). Then the optimal control law ${u_r}$ to be designed can compensate these two delays.

    The cost function for the optimal controller is given as follows.

    ${{J}_{0}}=\text{E}\left[ x_{k}.{\text{T}}{{\Psi }_{1}}{{x}_{k}}+\sum\limits_{r=0}.{k-1}{\left( x_{r}.{\text{T}}{{\Psi }_{2}}{{x}_{r}}+u_{r}.{\text{T}}{{\Psi }_{3}}{{u}_{r}} \right)} \right].$

    (12)

    In (12),${{\Psi} _1}$ and ${{\Psi} _2}$ are both symmetric positive semi-definite matrices,and ${{\Psi} _3}$ is a symmetric positive definite matrix. The superscript "${\rm{T}}$" denotes the transpose of the vector (or matrix),and the operator "E" denotes the calculation of mathematical expectation.

    Defining an augmented state vector ${{z}_{r}}$ (${{z}_{r}}={{\left( \begin{matrix} x_{r}.{\text{T}} & u_{r-1}.{\text{T}} \\ \end{matrix} \right)}.{\text{T}}}\in {{\mathbf{R}}.{n+m}}$),(11) and (12) can be rewritten as

    $\left\{ \begin{align} & {{z}_{r+1}}={{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}}+\Lambda {{v}_{r}} \\ & {{y}_{r}}=F{{z}_{r}}+{{\omega }_{r}} \\ \end{align} \right.$

    (13)

    ${{J}_{0}}=\text{E}\left[ z_{k}.{\text{T}}{{\Theta }_{1}}{{z}_{k}}+\sum\limits_{r=0}.{k-1}{\left( z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}} \right)} \right].$

    (14)

    In (13),${{\Phi }_{r}}=\left( \begin{matrix} {{\Omega }_{1}} & {{\Omega }_{3}}(\tau _{r}.{sc},\tilde{\tau }_{r}.{ca}) \\ 0 & 0 \\ \end{matrix} \right)$,${{\Gamma }_{r}}=\left( \begin{matrix} {{\Omega }_{2}}(\tau _{r}.{sc},\tilde{\tau }_{r}.{ca}) \\ I \\ \end{matrix} \right)$,$\Lambda =\left( \begin{matrix} \begin{align} & I \\ & 0 \\ \end{align} \\ \end{matrix} \right)$ ($I$ denotes an identity matrix of appropriate dimension),and $F=\left( \begin{matrix} {{A}_{3}} & 0 \\ \end{matrix} \right)$.

    In (14),${{\Theta }_{1}}=\left( \begin{matrix} {{\Psi }_{1}} & 0 \\ 0 & \frac{1}{2}{{\Psi }_{3}} \\ \end{matrix} \right)$,${{\Theta }_{2}}=\left( \begin{matrix} {{\Psi }_{2}} & 0 \\ 0 & \frac{1}{2}{{\Psi }_{3}} \\ \end{matrix} \right)$,${{\Theta} _3} = \frac{1}{2}{{\Psi} _3}$,and ${u_{ - 1}} = 0$.

    Obviously,the optimal controller for system (11) under the cost function (12) is the same as the optimal controller for system (13) under the cost function (14).

    Now,we will design an optimal controller to compensate both the measured SC delay $\tau _k.{sc}$ and the predicted CA delay $\tau _k.{ca}$ based on the discrete-time system models and the cost functions above.

    First,a lemma is introduced here to serve the proof of the following theorem about the optimal controller design.

    Lemma 1[19] Assume that the function $f(x,y,u)$ has a unique minimum with respect to u ($u \in {U}$) for all x and y. Let ${u.0}(x,y)$ denote the value of u for which the minimum is achieved. Then,we have the following equivalence relation.

    $\underset{u(x,y)}{\mathop{\min }}\,\text{E}\left[ f(x,y,u) \right]=\text{E}\left[ f(x,y,{{u}.{0}}(x,y)) \right]=\text{E}\left[ \underset{u}{\mathop{\min }}\,f(x,y,u) \right].$

    Theorem 1. When the system (11) has full state information,the optimal controller that minimizes the cost function (12) can be designed as

    ${{u}_{r}}=-{{L}_{r}}{{\left[ \begin{matrix} x_{r}.{\text{T}} & u_{r-1}.{\text{T}} \\ \end{matrix} \right]}.{\text{T}}}$

    (15)

    where ${{L}_{r}}={{\left[ \text{E}\left[ \Gamma _{r}.{\text{T}}{{S}_{r+1}}{{\Gamma }_{r}} \right]+{{\Theta }_{3}} \right]}.{-1}}\text{E}\left[ \Gamma _{r}.{\text{T}}{{S}_{r+1}}{{\Phi }_{r}} \right]$,${{S}_{r}}=\text{E}\left[ {{\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right) \right]+L_{r}.{\text{T}}{{\Theta }_{3}}{{L}_{r}}+{{\Theta }_{2}}$,${{S}_{k}}={{\Theta }_{1}}$.

    Meanwhile,the minimum cost function can be obtained as follows.

    $\min {{J}_{0}}=m_{0}.{\text{T}}{{S}_{0}}{{m}_{0}}+\text{tr}\left( {{S}_{0}}{{R}_{0}} \right)+\sum\limits_{r=0}.{k-1}{\text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{r+1}}\Lambda {{R}_{1}} \right)}.$

    (16)

    In (16),${m_0}$ and ${R_0}$ are respectively the mean and the variance of the Gaussian distribution ${N}({m_0},{R_0})$ that the initial augmented state ${z_0}$ obeys. ${R_1}$ is the variance of the Gaussian distribution ${N}(0,{R_1})$ that the system noise ${v_r}$ obeys. The operator "tr" is used to calculate the trace of a matrix.

    It is worthy to point out that the coefficient matrix ${{L}_r}$ in (15) is different from the number set L ($L = \{ 1,2,\cdots ,M\} $) in (1),and the matrix in ${{S}_r}$ (15) is different from the number set S ($S = \{ 1,2,\cdots ,N\}$).

    Proof.

    1) Deriving Bellman equation

    The cost function with initial time $\eta $ ($0 \le \eta \le k$) is defined as

    ${{J}_{\eta }}=\text{E}\left[ z_{k}.{\text{T}}{{\Theta }_{1}}{{z}_{k}}+\sum\limits_{r=\eta }.{k-1}{\left( z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}} \right)} \right].$

    (17)

    When $\eta = 0$,(17) is equivalent to (14). Using Lemma 1,the minimum cost ${J_\eta }$ with respect to the full information of ${u_\eta }$,${u_{\eta + 1}}$,and ${u_{k - 1}}$ can be calculated as

    $\begin{align} & \min {{J}_{\eta }}= \\ & \underset{{{u}_{\eta }},\cdots ,{{u}_{k-1}}}{\mathop{\min }}\,\text{E}\left[ z_{k}.{\text{T}}{{\Theta }_{1}}{{z}_{k}}+\sum\limits_{r=\eta }.{k-1}{\left( z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}} \right)} \right]= \\ & \text{E}\left[ \underset{{{u}_{\eta }},\cdots ,{{u}_{k-1}}}{\mathop{\min }}\,\left[ z_{k}.{\text{T}}{{\Theta }_{1}}{{z}_{k}}+\sum\limits_{r=\eta }.{k-1}{\left( z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}} \right)} \right] \right]= \\ & \text{E}\left[ \underset{{{u}_{\eta }},\cdots ,{{u}_{k-1}}}{\mathop{\min }}\,\text{E}\left[ z_{k}.{\text{T}}{{\Theta }_{1}}{{z}_{k}}+\sum\limits_{r=\eta }.{k-1}{\left( z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}} \right)}|{{X}_{\eta }} \right] \right] \\ \end{align}$

    (18)

    where ${X_\eta }$ is defined as ${X_\eta } = \left\{ {{x_\eta },{x_{\eta - 1}},\cdots ,{x_1}} \right.$,$\left. {{u_{\eta - 1}},\cdots ,{u_1}} \right\}$.

    Based on (13) which actually implies that ${z_{\eta + 1}},\cdots ,$ and ${z_k}$ are only determined by ${z_\eta }$,(18) can be rewritten as

    $\begin{align} & \min {{J}_{\eta }}=\text{E}\left[ \underset{{{u}_{\eta }},\cdots ,{{u}_{k-1}}}{\mathop{\min }}\,\text{E}[z_{k}.{\text{T}}{{\Theta }_{1}}{{z}_{k}} \right.+ \\ & \left. \sum\limits_{r=\eta }.{k-1}{\left( z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}} \right)}|{{z}_{\eta }}] \right]. \\ \end{align}$

    (19)

    The right side of (19) is denoted as ${\rm E}\left[{{V}({z_\eta },\eta )} \right]$ in which ${V}({z_\eta },\eta )$ can be treated as a risk function and has the following definition.

    $\begin{align} & V({{z}_{\eta }},\eta )= \\ & \underset{{{u}_{\eta }},\cdots ,{{u}_{k-1}}}{\mathop{\min }}\,\text{E}\left[ z_{k}.{\text{T}}{{\Theta }_{1}}{{z}_{k}}+\sum\limits_{r=\eta }.{k-1}{\left( z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}} \right)}|{{z}_{\eta }} \right]= \\ & \underset{{{u}_{\eta }}}{\mathop{\min }}\,\text{E}\left[ z_{\eta }.{\text{T}}{{\Theta }_{2}}{{z}_{\eta }}+u_{\eta }.{\text{T}}{{\Theta }_{3}}{{u}_{\eta }}+V({{z}_{\eta +1}},\eta +1)|{{z}_{\eta }} \right]. \\ \end{align}$

    When $k = 0$,we can get $\min {J_0} = {\rm E}\left[{{V}({z_0},0)} \right]$.

    Considering that $z_r.{\rm{T}}$ and ${u_r}$ can be determined by ${z_r}$,we can get the following Bellman equation.

    $V({{z}_{r}},r)=\underset{{{u}_{r}}}{\mathop{\min }}\,\left[ z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}}+\text{E}\left[ V({{z}_{r+1}},r+1)|{{z}_{r}} \right] \right].$

    (20)

    2) Solving Bellman equation

    By using mathematical induction,the solution of (20) can be proved to have the following quadratic form.

    $V({{z}_{i}},i)=z_{i}.{\text{T}}{{S}_{i}}{{z}_{i}}+{{s}_{i}}.$

    (21)

    In (21),${{S}_i}$ and ${{s}_i}$ are the parameters to be solved. The solving process based on the mathematical induction is given as follows:

    Step 1. For $i = k$,we have ${V}({z_k},k) = \min {\rm E}\left( {z_k.{\rm{T}}{{\Theta} _1}{z_k}|{z_k}} \right) = z_k.{\rm{T}}{{\Theta} _1}{z_k}$. Setting ${{S}_k} = {{\Theta} _1}$ and ${{s}_k} = 0$,(21) holds for $i = k$.

    Step 2. Assuming that (21) holds for $i = r + 1$,we can get

    $\begin{align} & V({{z}_{r+1}},r+1)=z_{r+1}.{\text{T}}{{S}_{r+1}}{{z}_{r+1}}+{{s}_{r+1}} \\ & \text{E}\left[ V({{z}_{r+1}},r+1)|{{z}_{r}} \right]=\text{E}\left[ z_{r+1}.{\text{T}}{{S}_{r+1}}{{z}_{r+1}}|{{z}_{r}} \right]+{{s}_{r+1}}. \\ \end{align}$

    (22)

    Substituting (13) into (22),we can get

    $\begin{align} & \text{E}\left[ V({{z}_{r+1}},r+1)|{{z}_{r}} \right]= \\ & \text{E}\left[ {{\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}}+\Lambda {{v}_{r}} \right)}.{\text{T}}} \right. \\ & \left. {{S}_{r+1}}\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}}+\Lambda {{v}_{r}} \right)|{{z}_{r}} \right]+{{s}_{r+1}}. \\ \end{align}$

    (23)

    Considering that ${z_r}$ and ${u_r}$ can be determined by ${z_r}$,(23) can be rewritten as follows.

    $\begin{align} & \text{E}\left[ V({{z}_{r+1}},r+1)|{{z}_{r}} \right]= \\ & \text{E}\left[ {{\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right) \right]+ \\ & \text{E}\left[ {{\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\Lambda {{v}_{r}} \right]+ \\ & \text{E}\left[ {{\left( \Lambda {{v}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right) \right]+ \\ & \text{E}\left[ {{\left( \Lambda {{v}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\Lambda {{v}_{r}} \right]+{{s}_{r+1}}. \\ \end{align}$

    (24)

    Since ${v_r}$ obeys the Gaussian distribution ${N}(0,{R_1})$ and ${v_r}$ is independent of ${z_r}$ and ${u_r}$,the second term and the third term on the right side of (24) are both equal to zero. The fourth term on the right side of (24) is equal to ${\rm tr}\left({{{\Lambda} .{\rm{T}}}{{S}_{r + 1}}{\Lambda} {R_1}} \right)$. So,(24) can be rewritten as follows:

    $\begin{align} & \text{E}\left[ V({{z}_{r+1}},r+1)|{{z}_{r}} \right]= \\ & \text{E}\left[ {{\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right) \right]+ \\ & \text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{r+1}}\Lambda {{R}_{1}} \right)+{{s}_{r+1}}. \\ \end{align}$

    Step 3. For $i = r$,we have the following derivations.

    $\begin{align} & V({{z}_{r}},r)= \\ & \underset{{{u}_{r}}}{\mathop{\min }}\,\left\{ z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}} \right.+\text{E}\left[ {{\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\times \right. \\ & \left. \left. \left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right)|{{z}_{r}} \right]+\text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{r+1}}\Lambda {{R}_{1}} \right)+{{s}_{r+1}} \right\}= \\ & \underset{{{u}_{r}}}{\mathop{\min }}\,z_{r}.{\text{T}}{{\Theta }_{2}}{{z}_{r}}+u_{r}.{\text{T}}{{\Theta }_{3}}{{u}_{r}}+\text{E}\left[ z_{r}.{\text{T}}\Phi _{r}.{\text{T}}{{S}_{r+1}}{{\Phi }_{r}}{{z}_{r}}+z_{r}.{\text{T}}\times \right. \\ & \left. \Phi _{r}.{\text{T}}{{S}_{r+1}}{{\Gamma }_{r}}{{u}_{r}}+u_{r}.{\text{T}}\Gamma _{r}.{\text{T}}{{S}_{r+1}}{{\Phi }_{r}}{{z}_{r}}+u_{r}.{\text{T}}\Gamma _{r}.{\text{T}}{{S}_{r+1}}{{\Gamma }_{r}}{{u}_{r}}|{{z}_{r}} \right]+ \\ & \text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{r+1}}\Lambda {{R}_{1}} \right)+{{s}_{r+1}}= \\ & \underset{{{u}_{r}}}{\mathop{\min }}\,\left\{ z_{r}.{\text{T}}{{S}_{r}}{{z}_{r}}+{{\left( {{u}_{r}}+{{L}_{r}}{{z}_{r}} \right)}.{\text{T}}}\left[ \text{E}\left[ \Gamma _{r}.{\text{T}}{{S}_{r+1}}{{\Gamma }_{r}} \right]+{{\Theta }_{3}} \right] \right.\times \\ & \left. \left( {{u}_{r}}+{{L}_{r}}{{z}_{r}} \right)+\text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{r+1}}\Lambda {{R}_{1}} \right)+{{s}_{r+1}} \right\}. \\ \end{align}$

    (25)

    In (25),we have

    $\begin{align} & {{L}_{r}}={{\left[ \text{E}\left[ \Gamma _{r}.{\text{T}}{{S}_{r+1}}{{\Gamma }_{r}} \right]+{{\Theta }_{3}} \right]}.{-1}}\text{E}\left[ \Gamma _{r}.{\text{T}}{{S}_{r+1}}{{\Phi }_{r}} \right] \\ & {{S}_{r}}=\text{E}\left[ \Phi _{r}.{\text{T}}{{S}_{r+1}}{{\Phi }_{r}} \right]+{{\Theta }_{2}}-L_{r}.{\text{T}}\left[ \text{E}\left[ \Gamma _{r}.{\text{T}}{{S}_{r+1}}{{\Gamma }_{r}} \right]+{{\Theta }_{3}} \right]{{L}_{r}}= \\ & \text{E}\left[ {{\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right) \right]+L_{r}.{\text{T}}{{\Theta }_{3}}{{L}_{r}}+{{\Theta }_{2}} \\ & {{s}_{r}}=\text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{r+1}}\Lambda {{R}_{1}} \right)+{{s}_{r+1}} \\ & {{S}_{k}}={{\Theta }_{1}}\text{and}{{s}_{k}}=0. \\ \end{align}$

    In (25),by setting ${u_r} = - {{L}_r}{z_r}$,we can easily get ${V}({z_r},r) = z_r.{\rm{T}}{{S}_r}{z_r} + {{s}_r}$. The result exactly demonstrates that (21) holds for $i = r$.

    Based on the three steps above,we can draw a conclusion that (21) is the solution of the Bellman (20). At the same time,we can get the optimal control law

    ${{u}_{r}}=-{{L}_{r}}{{z}_{r}}=-{{L}_{r}}{{\left[ \begin{matrix} x_{r}.{\text{T}} & u_{r-1}.{\text{T}} \\ \end{matrix} \right]}.{\text{T}}}.$

    3) Finding the minimum cost function: $\min {J_0}$

    Assuming ${z_0}$ obeys the Gaussian distribution: ${N}({m_0},{R_0})$,we have

    $\begin{align} & \min {{J}_{0}}=\text{E}\left[ V({{z}_{0}},0) \right]=\text{E}\left[ z_{0}.{\text{T}}{{S}_{0}}{{z}_{0}}+{{s}_{0}} \right]= \\ & \text{E}\left[ z_{0}.{\text{T}}{{S}_{0}}{{z}_{0}} \right]+{{s}_{0}}=m_{0}.{\text{T}}{{S}_{0}}{{m}_{0}}+tr\left( {{R}_{0}}{{R}_{0}} \right)+{{s}_{0}}. \\ \end{align}$

    Considering that ${{s}_{0}}=\text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{1}}\Lambda {{R}_{1}} \right)+{{s}_{1}}$,${{s}_{1}}=\text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{2}}\Lambda {{R}_{1}} \right)+{{s}_{2}},\cdots $,${{s}_{k-1}}=\text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{k}}\Lambda {{R}_{1}} \right)+{{s}_{k}}$,and ${{s}_{k}}=0$,we can get

    $\min {{J}_{0}}=m_{0}.{\text{T}}{{S}_{0}}{{m}_{0}}+\text{tr}\left( {{S}_{0}}{{R}_{0}} \right)+\sum\limits_{r=0}.{k-1}{\text{tr}\left( {{\Lambda }.{\text{T}}}{{S}_{r+1}}\Lambda {{R}_{1}} \right)}.$

    Remark 1. If ${{S}_r}$ has a stable solution: ${{S}_\infty } = \mathop {\lim }\nolimits_{r \to \infty } {{S}_r}$,${{L}_r}$ has a stable solution: ${{L}_\infty } = \mathop {\lim }\nolimits_{r \to \infty } {{L}_r}$ that will generate a common control law: ${u_r} = - {{L}_\infty }{z_r}$.

    Theorem 1 presents an optimal control law that can minimize the given cost function. Furthermore,the optimal control law (15) can make the NCS described in (11) exponentially mean square stable. To study the stability of the NCS,the exponential mean square stability definition of the NCS and a lemma about Hermite matrix are given as follows.

    Definition 1. The NCS in (11) is exponentially mean square stable if ${\rm E}\left[{{{\left\| {{x_r}} \right\|}.2}|{x_0},\tau _0.{sc},\tau _0.{ca}} \right] \le \gamma {\vartheta .r}{\left\| {{x_0}} \right\|.2}$ holds for any initial state $({x_0},\tau _0.{sc},\tau _0.{ca})$,where $\gamma > 0$ and $0<\vartheta <1$.

    Lemma 2 (Schur complement lemma). For the Hermite matrix $D = {D.{\rm{H}}} \in {{\bf R}.{n \times n}}$ (the superscript "${\rm{H}}$" stands for the matrix conjugate transposition),the eigenvalue (${e_j}$) and the eigenvector (${{\vec{x}}_{j}}$) are defined as

    $D{{\vec{x}}_{j}}={{e}_{j}}{{\vec{x}}_{j}},j=1,\cdots ,n$

    where ${e_j}$ satisfies that ${e_{\max }}(D) = {e_1} \ge {e_2} \ge \cdots \ge {e_n} = {e_{\min }}(D)$. Then the Rayleigh quotient ${R_a}(x) = \frac{{{x.{\rm{H}}}Dx}}{{{x.{\rm{H}}}x}}$ ($x \ne 0$) of arbitrary nonzero x ($\forall 0 \ne x \in {{\bf R}.n}$) satisfies the following inequality.

    ${{e}_{\min }}(D)\le {{R}_{a}}(x)\le {{e}_{\max }}(D).$

    Based on Definition 1 and Lemma 2,the following theorem will indicate that the NCS in (11) is exponentially mean square stable under the designed control law (15).

    Theorem 2. The control law (15) renders the system (11) exponentially mean square stable when ${v_k} = 0$.

    Proof.

    When ${v_k} = 0$,there is no system noise and ${R_1}{\rm{ = }}0$ holds. A Lyapunov function is defined as ${\Upsilon _r} = z_r.{\rm{T}}{{S}_r}{z_r}$. When ${z_r} \ne 0$,${\Upsilon _r}$ is positive definite since ${{S}_r}$ is positive definite (see Theorem 1). Based on the defined Lyapunov function,the system dynamics (13) and the designed control law in Theorem 1,we can calculate the following difference.

    $\begin{align} & \text{E}\left[ {{\Upsilon }_{r+1}}|{{z}_{r}},\cdots ,{{z}_{0}} \right]-{{\Upsilon }_{r}}= \\ & \text{E}\left[ z_{r+1}.{\text{T}}{{S}_{r+1}}{{z}_{r+1}}|{{z}_{r}},\cdots ,{{z}_{0}} \right]-{{\Upsilon }_{r}}= \\ & \text{E}\left[ {{\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\left( {{\Phi }_{r}}{{z}_{r}}+{{\Gamma }_{r}}{{u}_{r}} \right)|{{z}_{r}},\cdots ,{{z}_{0}} \right]-z_{r}.{\text{T}}{{S}_{r}}{{z}_{r}}= \\ & z_{r}.{\text{T}}\left\{ \text{E}\left[ {{\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right) \right]-{{S}_{r}} \right\}{{z}_{r}}. \\ \end{align}$

    Based on the derivation of ${{S}_r}$ in Theorem 1,the difference above can be further rewritten as

    $\begin{align} & \text{E}\left[ {{\Upsilon }_{r+1}}|{{z}_{r}},\cdots ,{{z}_{0}} \right]-{{\Upsilon }_{r}}= \\ & z_{r}.{\text{T}}\left\{ \text{E}\left[ {{\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right) \right] \right.- \\ & \left. \text{E}\left[ {{\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right)}.{\text{T}}}{{S}_{r+1}}\left( {{\Phi }_{r}}-{{\Gamma }_{r}}{{L}_{r}} \right) \right]-L_{r}.{\text{T}}{{\Theta }_{3}}{{L}_{r}}-{{\Theta }_{2}} \right\}{{z}_{r}}= \\ & -z_{r}.{\text{T}}\left\{ L_{r}.{\text{T}}{{\Theta }_{3}}{{L}_{r}}+{{\Theta }_{2}} \right\}{{z}_{r}}= \\ & -z_{r}.{\text{T}}{{\Xi }_{r}}{{z}_{r}} \\ \end{align}$

    where ${{\Xi }_{r}}=L_{r}.{\text{T}}{{\Theta }_{3}}{{L}_{r}}+{{\Theta }_{2}}.$

    So,we can get

    $\begin{align} & \text{E}\left[ {{\Upsilon }_{r+1}}|{{z}_{r}},\cdots ,{{z}_{0}} \right]={{\Upsilon }_{r}}-z_{r}.{\text{T}}{{\Xi }_{r}}{{z}_{r}}= \\ & \left( 1-\frac{z_{r}.{\text{T}}{{\Xi }_{r}}{{z}_{r}}}{{{\Upsilon }_{r}}} \right){{\Upsilon }_{r}}=\left( 1-\frac{z_{r}.{\text{T}}{{\Xi }_{r}}{{z}_{r}}}{z_{r}.{\text{T}}{{S}_{r}}{{z}_{r}}} \right){{\Upsilon }_{r}}. \\ \end{align}$

    (26)

    Based on Lemma 2,we can get the following inequalities.

    $\begin{align} & 0<{{e}_{\min }}\left( {{\Xi }_{r}} \right)z_{r}.{\text{T}}{{z}_{r}}\le z_{r}.{\text{T}}{{\Xi }_{r}}{{z}_{r}}\le {{e}_{\max }}\left( {{\Xi }_{r}} \right)z_{r}.{\text{T}}{{z}_{r}} \\ & 0<{{e}_{\min }}\left( {{S}_{r}} \right)z_{r}.{\text{T}}{{z}_{r}}\le z_{r}.{\text{T}}{{S}_{r}}{{z}_{r}}\le {{e}_{\max }}\left( {{S}_{r}} \right)z_{r}.{\text{T}}{{z}_{r}} \\ \end{align}$

    (27)

    $\begin{align} & \frac{{{e}_{\min }}\left( {{\Xi }_{r}} \right)}{{{e}_{\max }}\left( {{S}_{r}} \right)}=\frac{{{e}_{\min }}\left( {{\Xi }_{r}} \right)z_{r}.{\text{T}}{{z}_{r}}}{{{e}_{\max }}\left( {{S}_{r}} \right)z_{r}.{\text{T}}{{z}_{r}}}\le \frac{z_{r}.{\text{T}}{{\Xi }_{r}}{{z}_{r}}}{z_{r}.{\text{T}}{{S}_{r}}{{z}_{r}}}\le \\ & \frac{{{e}_{\max }}\left( {{\Xi }_{r}} \right)z_{r}.{\text{T}}{{z}_{r}}}{{{e}_{\min }}\left( {{S}_{r}} \right)z_{r}.{\text{T}}{{z}_{r}}}=\frac{{{e}_{\max }}\left( {{\Xi }_{r}} \right)}{{{e}_{\min }}\left( {{S}_{r}} \right)} \\ & \text{E}\left[ {{\Upsilon }_{r+1}}|{{z}_{r}},\cdots ,{{z}_{0}} \right]\le \\ & \left( 1-\frac{{{e}_{\min }}\left( {{\Xi }_{r}} \right)}{{{e}_{\max }}\left( {{S}_{r}} \right)} \right){{\Upsilon }_{r}}<\left( 1-\frac{{{\varepsilon }_{\Xi }}}{{{\varepsilon }_{S}}} \right){{\Upsilon }_{r}}=\varepsilon {{\Upsilon }_{r}}. \\ \end{align}$

    (28)

    In (28),$0<{{\varepsilon }_{\Xi }}<{{e}_{\min }}\left( {{\Xi }_{r}} \right)$,${{\varepsilon }_{S}}>{{e}_{\max }}\left( {{S}_{r}} \right)$,${{\varepsilon }_{\Xi }}<{{\varepsilon }_{S}}$,and $0<\varepsilon \text{ }<1$.

    The conditional expectations have the following smoothing property[20].

    $\text{E}\left[ {{\Upsilon }_{r}}|{{z}_{r-2}},\cdots ,{{z}_{0}} \right]=\text{E}\left[ \text{E}\left[ {{\Upsilon }_{r}}|{{z}_{r-1}},\cdots ,{{z}_{0}} \right]|{{z}_{r-2}},\cdots ,{{z}_{0}} \right].$

    (29)

    Using (28) and (29),we can get

    $\begin{align} & \text{E}\left[ {{\Upsilon }_{r}}|{{z}_{r-1}},\cdots ,{{z}_{0}} \right]<\varepsilon {{\Upsilon }_{r-1}} \\ & \text{E}\left[ {{\Upsilon }_{r}}|{{z}_{r-2}},\cdots ,{{z}_{0}} \right]<\text{E}\left[ \varepsilon {{\Upsilon }_{r-1}}|{{z}_{r-2}},\cdots ,{{z}_{0}} \right]= \\ & \varepsilon \text{E}\left[ {{\Upsilon }_{r-1}}|{{z}_{r-2}},\cdots ,{{z}_{0}} \right]. \\ \end{align}$

    Furthermore,we can get

    $\begin{align} & \text{E}\left[ {{\Upsilon }_{r-1}}|{{z}_{r-2}},\cdots ,{{z}_{0}} \right]<\varepsilon {{\Upsilon }_{r-2}} \\ & \text{E}\left[ {{\Upsilon }_{r}}|{{z}_{r-2}},\cdots ,{{z}_{0}} \right]<{{\varepsilon }.{2}}{{\Upsilon }_{r-2}}. \\ \end{align}$

    Repeatedly using (29) and (28),we can get

    $\begin{align} & \text{E}\left[ {{\Upsilon }_{r}}|{{z}_{r-3}},\cdots ,{{z}_{0}} \right]=\text{E}\left[ \text{E}\left[ {{\Upsilon }_{r}}|{{z}_{r-2}},\cdots ,{{z}_{0}} \right]|{{z}_{r-3}},\cdots ,{{z}_{0}} \right]< \\ & \text{E}\left[ {{\varepsilon }.{2}}{{\Upsilon }_{r-2}}|{{z}_{r-3}},\cdots ,{{z}_{0}} \right]= \\ & {{\varepsilon }.{2}}\text{E}\left[ {{\Upsilon }_{r-2}}|{{z}_{r-3}},\cdots ,{{z}_{0}} \right]< \\ & {{\varepsilon }.{3}}{{\Upsilon }_{r-3}} \\ & \vdots \\ \end{align}$

    As a whole,we can get

    $\text{E}\left[ z_{r}.{\text{T}}{{S}_{r}}{{z}_{r}} \right]<{{\varepsilon }.{r}}{{\Upsilon }_{0}}.$

    (30)

    Based on (27) and (30),we can get the following inequalities.

    $\begin{align} & {{e}_{\min }}\left( {{S}_{r}} \right)\text{E}\left[ z_{r}.{\text{T}}{{z}_{r}} \right]\le \text{E}\left[ z_{r}.{\text{T}}{{S}_{r}}{{z}_{r}} \right]<{{\varepsilon }.{r}}{{\Upsilon }_{0}}={{\varepsilon }.{r}}z_{0}.{\text{T}}{{S}_{0}}{{z}_{0}} \\ & \text{E}\left[ z_{r}.{\text{T}}{{z}_{r}} \right]<\frac{{{\varepsilon }.{r}}}{{{e}_{\min }}\left( {{S}_{r}} \right)}z_{0}.{\text{T}}{{S}_{0}}{{z}_{0}}\le \frac{{{\varepsilon }.{r}}}{{{e}_{\min }}\left( {{S}_{r}} \right)}{{e}_{\max }}\left( {{S}_{0}} \right)z_{0}.{\text{T}}{{z}_{0}}= \\ & \frac{{{e}_{\max }}\left( {{S}_{0}} \right)}{{{e}_{\min }}\left( {{S}_{r}} \right)}{{\varepsilon }.{r}}z_{0}.{\text{T}}{{z}_{0}}=\varpi {{\varepsilon }.{r}}z_{0}.{\text{T}}{{z}_{0}}. \\ \end{align}$

    (31)

    In (31),$\varpi = \frac{{{e_{\max }}\left( {{{S}_0}} \right)}}{{{e_{\min }}\left( {{{S}_r}} \right)}} > 0.$

    According to the definition of the augmented state ${{z}_{r}}={{\left[ \begin{matrix} x_{r}.{\text{T}} & u_{r-1}.{\text{T}} \\ \end{matrix} \right]}.{\text{T}}}$,we have

    $\begin{align} & \text{E}\left[ x_{r}.{\text{T}}{{x}_{r}} \right]\le \text{E}\left[ z_{r}.{\text{T}}{{z}_{r}} \right] \\ & \text{E}\left[ {{\left\| {{x}_{r}} \right\|}.{2}} \right]=\text{E}\left[ x_{r}.{\text{T}}{{x}_{r}} \right]\le \text{E}\left[ z_{r}.{\text{T}}{{z}_{r}} \right]<\varpi {{\varepsilon }.{r}}z_{0}.{\text{T}}{{z}_{0}}. \\ \end{align}$

    Assuming ${u_{ - 1}} = 0$,$z_0.{\rm{T}}{z_0} = x_0.{\rm{T}}{x_0}$ holds.

    Finally,we can get

    $\begin{align} & \text{E}\left[ {{\left\| {{x}_{r}} \right\|}.{2}} \right]<\varpi {{\varepsilon }.{r}}z_{0}.{\text{T}}{{z}_{0}}=\varpi {{\varepsilon }.{r}}x_{0}.{\text{T}}{{x}_{0}}=\varpi {{\varepsilon }.{r}}{{\left\| {{x}_{0}} \right\|}.{2}} \\ & \text{E}\left[ {{\left\| {{x}_{r}} \right\|}.{2}}|{{x}_{0}},\tau _{0}.{sc},\tau _{0}.{ca} \right]<\varpi {{\varepsilon }.{r}}{{\left\| {{x}_{0}} \right\|}.{2}} \\ \end{align}$

    where $\varpi > 0$ and $0 < \varepsilon < 1$.

    So,according to Definition 1,the NCS (11) is exponentially mean square stable.

    This section presents two theorems. One gives the designing method of an optimal controller to compensate the current SC delay and CA delay. The other indicates that the designed controller can guarantee the exponential mean square stability of the NCS.

    Remark 2. For simplicity,the CA delay considered in this paper is assumed to be less than one sampling period. However,the CA delay in real network may be longer than one sampling period,and then some of the past CA delays are not visible to the controller. For this case,the delay compensation methods proposed in this paper are also feasible after making some modification. The modification focuses on two points. One is how to predict the current CA delay when some of the past CA delays are not visible due to their too long lag time. The other is how to design the optimal controller when the discretization of NCSs becomes much complicated due to long delay. The detailed solution to the former problem can be referred to Remark 6 in [6],and the solution to the latter problem can be referred to [21].

  • In this section,some simulation examples are presented to illustrate the SCHMM-based delay compensation method from two aspects: effectiveness and superiority. The effectiveness will be demonstrated by designing an optimal controller to stabilize the NCS with random delays and to minimize a given cost function. The superiority will be demonstrated by comparing the optimal controller designed based on the SCHMM delay model with the optimal controller designed based on the DHMM delay model[17]. These simulation examples are carried out by using the TrueTime 1.5 toolbox installed in Matlab 7.1[22].

    The plant considered in the NCS (see Fig. 1 for its diagram) is a damped compound pendulum with the following dynamics.

    $\left\{ \begin{align} & \dot{x}(t)={{A}_{1}}x(t)+{{A}_{2}}u(t) \\ & y(t)={{A}_{3}}x(t). \\ \end{align} \right.$

    (32)

    In (32),$x(t)$ ($ \in {{\bf R}.2}$) is the state,$u(t)$ ($ \in {\bf R}$) is the input,$y(t)$ ($ \in {\bf R}$) is the output. The coefficient matrices ${{A}_1}$,${{A}_2}$,${{A}_3}$ are given as follows.

    $\begin{align} & {{A}_{1}}\text{=}\left( \begin{matrix} 0 & 1 \\ -10.77 & -0.039 \\ \end{matrix} \right),{{A}_{2}}\text{=}\left( \begin{matrix} 0 \\ 1.89 \\ \end{matrix} \right) \\ & {{A}_{3}}\text{=}\left( \begin{matrix} 1 & 0 \\ \end{matrix} \right). \\ \end{align}$

    In the NCS,the sensor is time-driven and the sampling period is 0.2 s ($h = 0.2$). Both the controller and the actuator are event-driven. The data rate of Ethernet is $8 \times {10.4}$ bits/s,the minimum packet size is 64 bits,and the loss probability is zero. An interference node is designed to generate stochastic SC and CA delays,and the sum of these two delays in one sampling period is not more than one sampling period (i.e.,$\tau _k.{sc} + \tau _k.{ca} \le 0.2$). All the network nodes are clock-synchronous and the delays can be measured by using the timestamp technology.

    The network-induced SC and CA delays may degrade the system performance and even destabilize the system. We aim to compensate these delays by designing an optimal controller in the experiments. When designing the controller,the current SC delay (e.g.,$\tau _k.{sc}$) can be measured directly,but the current CA delay (e.g.,$\tau _k.{ca}$) needs to be predicted. During each (e.g.,the k-th) sampling period,all the past CA delays (${\tau .{ca}} = \{ \tau _r.{ca}\} _{r = 1}.{k - 1}$) will be used to model the CA delay and predict the current CA delay ($\tilde \tau _k.{ca}$),and then the predicted value can be used to design the optimal controller for compensating the upcoming real CA delay ($\tau _k.{ca}$). The following steps illustrate how to model,predict and compensate the current CA delay by using the SCHMM-based delay model.

    Firstly,we need to initialize the SCHMM-based delay model ${\lambda .0}$ (${\lambda .0} = ({N.0},{M.0},{{\pi} .0},{{A}.0},{{B}.0})$). In order to facilitate comparison,the network in this experiment is also assumed to have three different states as in [6, 17],i.e.,$N = 3$ and $S = \left\{ {1,2,3} \right\}$,and there are five Gaussian mixture densities,i.e.,$M = 5$. The vector of initial state distribution (${{\pi} .0}$) and the matrix of state transition probabilities (${{A}.0}$) can be almost uniformly initialized as follows.

    $\begin{align} & {{\pi }.{0}}=[\begin{matrix} 0.3333 & 0.3333 & 0.3334 \\ \end{matrix}] \\ & {{A}.{0}}=\left[ \begin{matrix} 0.3334 & 0.3333 & 0.3333 \\ 0.3333 & 0.3334 & 0.3333 \\ 0.3333 & 0.3333 & 0.3334 \\ \end{matrix} \right]. \\ \end{align}$

    The Gaussian mixture density ${{B}.0}$ (${{B}.0} = \left\{ {b_j.0(\tau _r.{ca})} \right\}$,$b_j.0(\tau _r.{ca}) = \sum\nolimits_{l = 1}.M {c_{jl}.0G(\tau _r.{ca}|\mu _l.0,\sigma _l.0)} $) can be initialized by using the K-means clustering algorithm with the number of clusters equal to the number M of mixture Gaussian components. For simplicity,the mixture weight $c_{il}.0$ is generally initialized independent of network states,i.e.,the weight $c_{il}.0$ is simplified into $c_l.0$ by leaving out the subscript "i". So,the initialized $c_{il}.0$ is equal to the initialized $c_{jl}.0$ even for the different $j \ne i \in S$. In general,${{B}.0}$ needs to be initialized in advance once that the SCHMM is going to be refreshed during one sampling period. For example,during the 101st sampling period,${{B}.0}$ is initialized as follows by using the K-means clustering algorithm.

    $\begin{align} & {{c}.{0}}=\left( c_{jl}.{0} \right){{|}_{1\le j\le 3,1\le l\le 5}}= \\ & \left[ \begin{matrix} 0.0031 & 0.4976 & 0.2815 & 0.0082 & 0.2096 \\ 0.0031 & 0.4976 & 0.2815 & 0.0082 & 0.2096 \\ 0.0031 & 0.4976 & 0.2815 & 0.0082 & 0.2096 \\ \end{matrix} \right] \\ & {{\mu }.{0}}=\left( \mu _{l}.{0} \right){{|}_{1\le l\le 5}}= \\ & \left[ \begin{matrix} 2.4302 & 0.8761 & 0.9437 & 1.3920 & 0.7974 \\ \end{matrix} \right] \\ & {{\sigma }.{0}}=\left( \sigma _{l}.{0} \right){{|}_{1\le l\le 5}}= \\ & \left[ \begin{matrix} 0.4169 & 0.0004 & 0.0013 & 0.0285 & 0.0008 \\ \end{matrix} \right]. \\ \end{align}$

    Secondly,we can use the EM algorithm to train (refresh) the SCHMM-based delay model based on the initialized model ${\lambda .0}$ and the past CA delay values. During the experiment,the maximum iteration number of the training procedure is set to 40 and the threshold to terminate the procedure is set to $5 \times {10.{ - 4}}$. The definition of the threshold can refer to Remark 4 in [6]. Under these conditions,it is found that the training procedure can always converge after dozens of iterations. For example,during the 101st sampling period,the optimized parameters of the SCHMM-based delay model are estimated as follows.

    $\begin{align} & {{\pi }.{*}}=\left[ \begin{matrix} 0.1759 & 0.5084 & 0.3157 \\ \end{matrix} \right] \\ & {{A}.{*}}=\left[ \begin{matrix} 0.6977 & 0.2016 & 0.1007 \\ 0.2994 & 0.5513 & 0.1493 \\ 0.1025 & 0.4011 & 0.4964 \\ \end{matrix} \right] \\ & {{c}.{*}}=\left[ \begin{matrix} 0.0158 & 0.4614 & 0.2693 & 0.0471 & 0.2064 \\ 0.0149 & 0.4592 & 0.2766 & 0.0512 & 0.1981 \\ 0.0155 & 0.4603 & 0.2811 & 0.0465 & 0.1966 \\ \end{matrix} \right] \\ & {{\mu }.{*}}=\left[ \begin{matrix} 2.1072 & 0.8594 & 0.9318 & 1.1107 & 0.8095 \\ \end{matrix} \right] \\ & {{\sigma }.{*}}=\left[ \begin{matrix} 0.7946 & 0.0006 & 0.0013 & 0.0257 & 0.0015 \\ \end{matrix} \right]. \\ & \\ \end{align}$

    Thirdly,we can use the derived delay model to predict the CA delay through the three steps given in Section 2. The first step uses the Viterbi algorithm to estimate the optimal previous network state. The second step uses the maximum transition probability to estimate the optimal current network state. The third step uses the peak of the Gaussian mixture densities to predict the current CA delay ($\tilde \tau _k.{ca}$). Taking these three steps,the predicted values of CA delays can be obtained. For example,during the 101st sampling period,the predicted CA delay is 0.0387 5 s that is close to the real value 0.03985 s,and the relative error is 2.76 %.

    In this experiment,there are totally 200 sampling periods. All the real CA delays and their corresponding predicted values are shown in Fig. 2,where "$\cdot$" denotes the real values and "$\circ$" denotes the predicted values.

    Figure 2.  Real and predicted CA delays

    In order to validate the modeling and predictive method based on SCHMM,the mean square error (MSE) is introduced as follows to judge the precision of the method.

    $MSE=\frac{1}{k}\sum\limits_{r=1}.{k}{{{\left( \tilde{\tau }_{r}.{ca}-\tau _{r}.{ca} \right)}.{2}}}.$

    After computation,the MSE of the predictive method is $8.6 \times {10.{ - 4}}$ which is small enough to demonstrate the effectiveness and feasibility of the method. It is worthy to notice that the MSE is relatively large at the beginning of the experiment. For example,the MSE of the previous 20 sampling periods is $1.25 \times {10.{ - 3}}$. This is because that the precision of the modeling and prediction is related to the number of the CA delay values used to train the SCHMM. Generally,more CA delay values can render higher precision of the prediction. Nevertheless,too large number of CA delay values will definitely reduce the efficiency of the modeling and prediction. So,we need to make a balance between the precision and the efficiency when the experiment runs for a long time. This is one of our future research works.

    In this experiment,the SC delays are measured by using the timestamp technology and their values are shown in Fig. 3. In order to compensate the CA and SC delays in the NCS,an optimal controller is designed by taking the measured SC delay and the predicted CA delay into account. The controller is designed based on Theorem 1 in this paper to minimize the following cost function.

    Figure 3.  Real SC delays

    ${{J}_{0}}=\text{E}\left[ x_{k}.{\text{T}}{{\Psi }_{1}}{{x}_{k}}+\sum\limits_{r=0}.{k-1}{\left( x_{r}.{\text{T}}{{\Psi }_{2}}{{x}_{r}}+u_{r}.{\text{T}}{{\Psi }_{3}}{{u}_{r}} \right)} \right]$

    where ${{\Psi }_{1}}\text{=}{{\Psi }_{2}}\text{=}\left[ \begin{matrix} 0.5 & 0 \\ 0 & 0.5 \\ \end{matrix} \right]$,${{\Psi} _3}{\rm{ = }}0.1$.

    The response of the NCS under the optimal controller is described by ${y_1}$ in Fig. 4,which obviously demonstrates that the controller can not only guarantee the stability of the NCS but also render the NCS with a good performance. In order to further demonstrate the superiority of the controller over the optimal controller designed based on the DHMM delay model[17],a comparing experiment is carried out in the same simulation environment with three different network states. Using the method proposed in [17],the CA delay interval ((0,0.2]) is uniformly divided into five complete subintervals (i.e.,in the DHMM,there are five different CA delay observations and the observation space is denoted as \{1,2,3,4,5\}),and then all the past CA delays are uniformly quantized to get a discrete observation sequence. Based on the discrete sequence,the DHMM-based CA delay model can be obtained,and the current CA delay can be predicted. Consequently,the predicted CA delay and the measured SC delay can be used to design an optimal controller. The response stimulated by the optimal controller designed based on DHMM is described by ${y_2}$ in Fig. 4,which demonstrates that this controller can also guarantee the stability of the NCS. Furthermore,through comparing ${y_1}$ and ${y_2}$,one can see that the performance of ${y_1}$ is better than that of ${y_2}$ on the whole. The performance merit of ${y_1}$ exactly demonstrates the superiority of the optimal controller designed based on the SCHMM over that on the DHMM. It is worthy to notice that at the beginning of this experiment the performance of ${y_1}$ is worse than that of ${y_2}$. This is because that the training of the SCHMM for high precision needs more CA delay data than that of the DHMM. At the beginning of the experiment,the CA delay data is not enough to provide higher precision for the SCHMM than for the DHMM. But as time goes on,the CA delay data become more and more,and the precision of the modeling and prediction based on the SCHMM becomes higher than that on the DHMM. Meanwhile,the performance of ${y_1}$ becomes better than that of ${y_2}$. So overall,the optimal controller designed based on the SCHMM is superior than that on the DHMM in the compensation of random delays. This result shows again,from the aspect of delay compensation,that the SCHMM-based delay model is superior to the DHMM-based delay model in NCSs.

    Figure 4.  Response of the NCS under different optimal controllers

    It is worthy to notice that the performance of ${y_2}$ in this paper is worse than that in [17]. This is because that the cost function used in this paper is stricter than that used in [17]. That is to say,under the cost function used in this paper,the optimal controller designed in [17] cannot achieve the similar performance like that of ${y_2}$ in 17],but the optimal controller designed in this paper can achieve better performance than that of ${y_2}$ in [17]. This,from another aspect,illustrates the superiority of the SCHMM-based compensation over the DHMM-based compensation.

    Based on the experiments above,the SCHMM-based compensation for the random delays (including SC and CA delays) by designing an optimal controller is demonstrated to be effective and superior to the DHMM-based compensation.

  • Random delay is the main cause for the performance degradation and even instability of networked control systems. Generally,there are two kinds of random delays including SC delay and CA delay. SC delay is known to the controller before designing control law,but CA delay is not since it has not occurred. In order to compensate the CA delay as well as the SC delay by using the control law,the feasible way is to predict the upcoming CA delay. Prediction can be achieved based on the model of CA delay. In order to improve the precision of the CA delay model,the semi-continuous hidden Markov model (SCHMM) is introduced in this paper to model the random CA delay. Compared with the discrete hidden Markov model (DHMM),the SCHMM has the advantages of higher precision and faster convergence. The expectation maximization (EM) algorithm is used to train the SCHMM and obtain the optimal estimation of its parameters. Based on the SCHMM,the current network state and the upcoming CA delay can be predicted with a relatively high precision. Thus,the predicted CA delay and the measured SC delay are used to design an optimal controller. The controller can compensate these two delays as well as guarantee the exponential mean square stability. Some simulation examples are given to demonstrate the effectiveness and superiority of the SCHMM-based CA delay model. By comparison,the SCHMM-based optimal controller shows up better performance in delay compensation than the DHMM-based optimal controller.

Reference (22)

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return