Abstract

This paper presents a semi-adaptive closed-loop control approach to autonomous infusion of medications exhibiting significant transport delay in clinical effects. The basic idea of the approach is to enable stable adaptive control of medication infusion by (1) incorporating transport delay explicitly into control design by way of a Padé approximation while (2) facilitating linear parameterization of control design model by desensitization of nonlinearly parameterized cooperativity constant associated with pharmacodynamics (PD). A novel dynamic dose–response model for control design is presented, in which the cooperativity constant exerts zero influence on the model output in the steady-state. Then, an adaptive pole placement control (APPC) technique was employed to fulfill adaptive control design in the presence of nonminimum phase dynamics associated with the Padé approximation of transport delay. The controller was evaluated in silico using a case study of regulating a cardiovascular variable with a sedative under a wide range of transport delay and pharmacological profiles. The results suggest that adaptation of transport delay and pharmacological characteristics may be beneficial in achieving consistent and robust regulation of medication-elicited clinical effects.

Introduction

Closed-loop autonomous control of medication infusion has been a very active area of research by virtue of its potential as an attractive alternative to manual medication infusion performed by human clinicians. Examples include sedatives and anesthetics [17], opioids and analgesics [810], vasoactive drugs [1113], and even fluids and blood products [1420]. In fact, there is an increasing body of evidence that supports the efficacy of closed-loop medication infusion systems: it has the potential to enable tight regulation of desired clinical effects and relief of clinician workload [5,2126].

One control design challenge related to medication infusion problems is the transport delay between the administration of medication and the onset of resulting clinical effects measured by patient monitors. First, although some medication dose–clinical response pairs exhibit minimal transport delay that may be neglected in the control design process (e.g., hypnosis by anesthetics [27] and respiratory depression by fast-acting opioids [9]), there are medication dose–clinical response pairs, including those widely employed in today's clinical practice (e.g., blood pressure increase by phenylephrine [11]), which exhibit relatively large, variable transport delays. Second, patient monitors often introduce additional (and potentially variable) transport delay due to the computational load and algorithmic nonlinearity [2830]. Transport delay tends to deteriorate stability margin and can even destabilize the control loop. Hence, it must be accounted for with care as part of control design in order to avoid catastrophic risks on patient safety. Nonetheless, transport delay has seldom been treated seriously in control design for medication infusion problems. One reason may be that the vast majority of prior work used traditional pharmacokinetics (PK) and pharmacodynamics (PD) models [3133]. These models do not account for transport delay related to the PD and patient monitor explicitly; rather, transport delay associated with the PD and monitor is often abstracted into lumped first-order phase lag [34]. The other reason may be that transport delay is nonlinearly parameterized and cannot be readily estimated with established recursive estimation techniques. Such a nonlinear parameterization presents technical challenges in designing adaptive closed-loop medication infusion controllers capable of counteracting the impact of transport delay in the PD and monitors. Perhaps due to these reasons, only a small number of prior work has taken the transport delay into full consideration in control design [3539]. Hahn et al. [35] employed a simple first-order Padé approximation of PD transport delay and conducted a two degrees-of-freedom robust control design based on the μ synthesis theory to compensate for the associated approximation errors. Mendez et al. [36] employed a Smith predictor to compensate for transport delay in the PD and a monitor, and incorporated it into proportional-integral control. Sawaguchi et al. [37] estimated transport delay and PD model parameters from the measurements obtained from the initial transients and incorporated them into model predictive control. Ionescu et al. [38] inferred variable transport delay associated with a monitor with cross-correlation analysis and incorporated it into model predictive control in combination with a Smith predictor. Soltesz et al. [39] individualized PD model parameters (including PD transport delay therein) based on the measurements obtained from an initial test bolus and used them in conjunction with novel inversion filters for adapting proportional–integral–derivative (PID) control to individual patients. As can be seen above, there has been rare prior work attempting to formally integrate the estimation of transport delay as a component of closed-loop control algorithm with sound stability proof: transport delay was either handled as uncertainty, or was separately estimated and incorporated into closed-loop control through Smith predictor (which may suffer from possible inaccuracy in transport delay as well as uncertainty and modeling errors other than transport delay) or in other ad hoc ways.

Stable adaptive control of dynamic systems with variable transport delay is, in general, a challenging problem even in the control community, and only limited prior work appears to exist. In one category of prior work, methods to estimate transport delay were investigated [4042]. Despite significance and rigor, these methods involve somewhat restrictive requirements and assumptions (e.g., knowledge of high-frequency gain [40,41] and full state information [42]), which may limit their direct application to closed-loop medication infusion problems. In the other category of prior work, methods to linearize transport delay using Padé approximation and incorporate it into indirect adaptive control design techniques were investigated [43,44].

In addition to transport delay, the control design problem for medication infusion must also account for two other well-known challenges: stiff nonlinearity and large interindividual variability associated with the dose–response relationship. These challenges, in combination with the one associated with rigorous account of transport delay, make closed-loop medication infusion a difficult yet interesting control design problem.

This paper presents a semi-adaptive closed-loop control approach to autonomous infusion of medications exhibiting significant transport delay in clinical effects. The basic idea of the approach is to enable stable adaptive control of medication infusion by (1) incorporating transport delay explicitly into control design by way of the Padé approximation while (2) facilitating linear parameterization of the control design model by desensitization of the nonlinearly parameterized cooperativity constant associated with the PD. The resulting controller is semi-adaptive in that the desensitized cooperativity constant is fixed at a nominal value while the remaining model parameters are on-line updated. A novel dynamic dose–response model for control design is presented, in which the cooperativity constant exerts zero influence on the model output in the steady-state. Then, indirect adaptive pole placement control (APPC) technique was employed to fulfill adaptive control design in the presence of nonminimum phase dynamics associated with the Padé approximation of transport delay. The controller was in silico evaluated using a case study of regulating a cardiovascular variable with a sedative under a wide range of transport delay and pharmacological profiles.

This paper is organized as follows: First, a novel dynamic dose–response model for control design is presented. Second, parametric sensitivity analysis of the dynamic dose–response model is performed to assess the influence of transport delay on the model output in comparison with the other dose–response model parameters. Third, the indirect APPC design for medication infusion problems based on the Padé approximation of transport delay is developed. Fourth, the results are presented and discussed. Finally, the paper is concluded with future directions.

Control Design Model: Direct Dynamic Dose–Response Model

Expanding our prior work [9,45], a novel dynamic dose–response model suited to adaptive control design for medication infusion problems with non-negligible transport delay is developed. The model consists of a low-order lumped parameter latency model (1a) and a modified Hill equation model (1b) 

I˙et=keIet+keutLFIet
(1a)
 
yt=yo1100σ1IeγtIσγ+100σ1Ieγt=HIet
(1b)

where ut is the intravenous medication infusion rate, Iet is the medication infusion rate at the site of action, L is the transport delay between medication infusion and the onset of the intended clinical effect, ke is the rate constant associated with the distribution of medication from intravenous site to the site of action, yt and yo are clinical effect and its nominal (i.e., in the absence of medication infusion) value, σ is the target percentage depression of yt from yo (meaning that target clinical effect rt is specified by rt=((100σ)/100)yo), Iσ is the infusion rate required to maintain yt=rt in the steady-state, and γ is the cooperativity constant. There are a few advantages associated with this dose–response model compared with conventional PKPD models. First, intended to primarily capture the direct relationship between dose and the resulting clinical effects, its dynamic component is much simpler than multicompartmental PKPD models. Second, it explicitly incorporates transport delay, which can be exploited to represent PD and/or patient monitor delays. Third and most importantly, it facilitates semi-adaptive control design based on the parametric sensitivity analysis [9], in which model parameters exerting high sensitivity on the model's behavior are adapted while those with low sensitivity are fixed at nominal values (see Parametric Sensitivity Analysis for details).

Parametric Sensitivity Analysis

In this section, analytical and numerical parametric sensitivity analyses are performed to demonstrate that (1) the proposed dose–response model allows us to minimize the influence of γ with appropriate choice of σ; and (2) the transport delay L needs to be adapted for control efficacy. Details follow.

Desensitization of Cooperativity Constant: Analytical Sensitivity Analysis.

The proposed dose–response model (1) desensitizes the cooperativity constant γ in the steady-state when yt=rt, or equivalently, when Iet=Iσ, making it possible to fix it at a nominal value in the control design process without compromising controller robustness against its uncertainty. To illustrate, regard (1a) and (1b) as the state and output equations for the process dynamics. Then, the following sensitivity functions can be derived:  
SIe˙(t)=FIe(t)SIe(t)+[FkeFγFIσFL]=keSIe(t)+[Ie(t)+u(tL)00keu(tL)(tL)]Sy(t)=HIe(t)SIe(t)+[HkeHγHIσHL]=yoγ(100σ1)IσγIeγ1(t)(Iσγ+(100σ1)Ieγ(t))2SIe(t)+[0yo(100σ1)IσγIeγ(t)(log(Iσ)log(Ie(t)))(Iσγ+(100σ1)Ieγ(t))2yoγ(100σ1)Iσγ1Ieγ(t)(Iσγ+(100σ1)Ieγ(t))20]
(2)
where SIetIetkeIetγIetIσIetL and SytytkeytγytIσytL. Then, the closed-form formula for Syt is given by  
SyT(t)=yoγ100σ1IσγIeγ1(t)Iσγ+100σ1Ieγ(t)20tekeτIe(τ)+uτLdτyo100σ1IσγIeγ(t)logIσlogIe(t)Iσγ+100σ1Ieγ(t)2yoγ100σ1Iσγ1Ieγ(t)Iσγ+100σ1Ieγ(t)2yoγ100σ1IσγIeγ1(t)Iσγ+100σ1Ieγ(t)20tekeτkeuτLτLdτ
(3)

Hence, ((yt)/γ), which is the second element of Syt in Eq. (3), becomes zero when Iet=Iσ. The implication of the zero sensitivity of yt on γ for Iet=Iσ is that the influence of γ on the process behavior can be minimized by setting σ according to the target clinical effect, that is, equal to the percentage difference between yo and rt, i.e., σ=((yort)/yo)×100. Indeed, with this choice of σ, it can be readily shown that the process behavior becomes insensitive to γ under steady-state target tracking condition (yt=rt).

Numerical Sensitivity Analysis.

To confirm the above analytical sensitivity analysis as well as to assess the significance of transport delay L relative to the other parameters in the dose–response model, a numerical parametric perturbation analysis was performed using the regulation of a cardiovascular variable cardiac output (CO) with a sedative propofol as a case scenario. In this analysis, a fine-tuned population-based PID controller was used to regulate CO via propofol infusion in 30 randomly created in silico patients for nominal closed-loop response. Then, the dose–response model parameters (including ke, Iσ, γ, and L) were perturbed, one at a time, by ±25% and ±50% (thus, four perturbations per each parameter), and the perturbed in silico patients were simulated with the same PID controller for perturbed closed-loop responses. Then, the difference between nominal and perturbed responses was quantified in terms of root-mean-squared errors (RMSEs) between the two up to the settling time of the desired response. Finally, a total of 120 RMSEs associated with each parameter was aggregated to compute mean and standard deviation (SD). We used the following ranges of the dose–response model parameters obtained from our prior work [46] and our unpublished in-house experimental data to create 30 in silico patients: 0.02ke0.10min−1, 0.2Ie0.6mg/kg/min, 1γ5, and 50L100 s. In all in silico simulations, y0 was set at 3.0 lpm. A multitude of desired responses, in terms of both magnitude and rate, was employed in the analysis: 2.1, 2.4, 2.7 lpm with the time constants of 5, 2.5, and 1.7 min. For each of these nine desired responses, the aggregated RMSEs associated with the dose–response model parameters were compared to assess if the perturbation in the transport delay makes a large impact on the closed-loop control performance relative to the other dose–response model parameters, in order to determine if the adaptation of transport delay is necessary.

Results from the parametric sensitivity analysis indicated that the impact of transport delay on the closed-loop clinical effect response (as measured by the RMSE between nominal and perturbed responses up to the settling time of the desired response) was larger than cooperativity constant but not as large as Iσ and ke. Figure 1 presents the RMSEs between nominal and perturbed responses associated with each model parameter, which offers several key observations. First, all in all Iσ exerted the largest impact on the model's clinical effect response, followed by ke, and then L, and finally γ. Second, although L was not the most crucial parameter in (1) in terms of average sensitivity, it exhibited a very large variability in sensitivity relative to its average counterpart (i.e., large coefficient of variation) in comparison with the remaining parameters in Eq. (1). This is attributed to a large deviation of the closed-loop controlled clinical effect response from its nominal counterpart when L assumes very large values, since a large L reduces the stability margin associated with the population-based PID control (which does not accommodate the increase in L). Hence, it deemed reasonable to investigate the advantage of its adaptation by comparing an adaptive control equipped with the capability of adapting L versus an adaptive control with L fixed at a nominal value. Third, although the dose–response model (1) by construction exhibits zero sensitivity to γ when Iet=Iσ if σ is specified in accordance with the target clinical effect (i.e., if σ is set equal to the percentage difference between the baseline and target clinical responses), its sensitivity to γ is not zero during transients. Hence, the finding that γ exerts the smallest influence on the dose–response model's clinical response even during transients suggests that all in all the model's sensitivity to γ is the smallest and that γ may indeed be fixed at a nominal value.

Semi-Adaptive Pole Placement Control Design

Our semi-APPC is built upon the results of the parametric sensitivity analysis above: that L must be on-line adapted while γ may be fixed at a nominal value. The proposed semi-APPC consists of the following: (1) Padé approximation of transport delay, (2) linear model parameterization, (3) recursive model parameter adaptation, and (4) its integration into pole placement control. Figure 2 shows the scheme of the sensitivity-based semi-APPC. Details follow.

Padé Approximation of Transport Delay.

For the sake of adaptive control design, the transport delay in the dose–response model (1) was simplified into the Padé approximation. It is obvious that the higher the order of approximation, the more accurate the approximation is. But, a high-order approximation complicates control design by increasing the order of process dynamics. Besides, a relatively low-order approximation may still be appropriate for adaptive control, because the adaptation of transport delay parameter may mitigate the approximation error and still produce correct phase delay [44].

To determine the Padé approximation relevant to the problem at hand, six candidate approximations in Eq. (4) were considered  
M1:Ie(s)=kes+keu(s)M2:Ie(s)=kes+ke11+sLu(s)M3:Ie(s)=kes+ke2sL2+sLu(s)M4:Ie(s)=kes+ke62sL6+4sL+(sL)2u(s)M5:Ie(s)=kes+ke126sL+(sL)212+6sL+(sL)2u(s)M6:Ie(s)=kes+ke6024sL+3(sL)260+36sL+9(sL)2+(sL)3u(s)
(4)

Then, the dose–response model (1) with these candidate approximations equipped with nominal parameter values (associated with 30 in silico subjects) used in the parametric sensitivity analysis in Parametric Sensitivity Analysis was in silico simulated with an escalated multistep propofol dose with five infusion rate levels designed to elicit a wide range of transient and steady-state CO response. The CO responses associated with each of the six candidate approximations were then compared with the response of the original dose–response model without Padé approximation. Specifically, RMSEs between the responses associated with the original and all the Padé-approximated models were computed, and the optimal order of the Padé approximation for the problem was determined based on the trend of the RMSE with respect to the order.

Table 1 shows the RMSE between the responses associated with the original (i.e., Eq. (1)) and all the Padé-approximated (i.e., Eq. (1) with the M1M6) dose–response models. Clearly, M1 (which does not account for transport delay) suffers from the largest error, whereas the models incorporating the approximation of transport delay can largely reduce the error. The amount of reduction in error becomes trivial beyond M3. Hence, we used M3 for semi-APPC design in this study.

Dose–Response Model Parameterization.

For the sake of control design, the dose–response model (1) was parameterized as follows: First, by fixing γ to a nominal value γ¯, Eq. (1b) can be written for Iet as follows:  
Iet=Iσy0yt100σ1ytγ¯Iσqt
(5)
where qt=(y0yt)/(100/σ)1ytγ¯. Then, Eq. (1a) can be written in terms of qt as follows:  
q˙t=1IσdIetdt=keIσqt+keIσutLqs=1Iσkes+keeLsus
(6)
Using the Padé approximation M3 in Eq. (4), qs in Eq. (6) can be written as follows:  
qs=ZpsRpsus=1Iσkes2Ls+kes+2Lus
(7)

For a given desired clinical effect rs, the reference model was specified by qms=am/s+amrs with am>0.

Semi-Adaptive Pole Placement Control.

We aim to design an adaptive controller for Eq. (7). Since Eq. (7) is nonminimum phase, the application of direct adaptive control techniques (e.g., direct model reference adaptive control) is not trivial [47]. Therefore, we pursued indirect APPC due to its ability to handle nonminimum phase plants [47]. Details follow.

On-Line Parameter Estimation.

The adaptive law for on-line parameter estimation was derived from the standard recursive gradient algorithm [47]. The input–output model Eq. (7) can be rewritten as Rpsqs=Zpsus, or equivalently  
s2+ke+2Ls+2keLqs=keIσs+2keIσLus
(8)
Rearranging into linear parametric form  
s2qs=ke+2Lsqs2keLqskeIσsus+2keIσLus=keIσ2keIσLke+2L2keLsusussqsqs
(9)
Multiplying a stable low-pass filter (1/s+λ02) with λ0>0 to avoid differentiation and defining the output as zs=(s2/s+λ02)qs leads to the following linear parametric model:  
zs=s2s+λ02qs=θTϕs=keIσ2keIσLke+2L2keLss+λ02us1s+λ02usss+λ02qs1s+λ02qs
(10)
where  
θθ1θ2θ3θ4T=keIσ2keIσLke+2L2keLT
Then, the following adaptive law can be derived from the recursive gradient algorithm [47]:  
θ̂˙t=Γϕtϵt=Γϕtztθ̂Ttϕt1+ϕTtϕt
(11)
To robustify the adaptive law by preventing the drift of the parameter estimates when ϵt0, (11) was augmented by the following dead zone scheme:  
θ̂˙t=Γϕtϵt,ϵt>ϵ00,ϵtϵ0
(12)

Closed-Loop Control Design.

Here, we derive standard pole placement control law for the dose–response model (7) incorporating the Padé approximation, and implement the APPC by combining the pole placement control law thus derived with the adaptive law (11)(12) by leveraging the certainty equivalence principle. Let dRp the degree of Rps, i.e., dRp=2. The standard pole placement control law is given by [47]  
QmsDsus=Psqsqms=Pses
(13)
where Qms is the internal model of order dQm associated with qms satisfying Qmsqms=0, Ds is a monic polynomial of degree dRp1, and Ps is a polynomial of degree dRp+dQm1. Given that qms=(am/(s+am))rs, and that rt is typically specified as constant set point, Qms=ss+am will satisfy Qmsqms=0. Hence, dQm=2, and thus, Ds=s+l0 and Ps=p3s3+p2s2+p1s+p0, where l0, p3, p2, p1, and p0 are unknown coefficients to be determined via pole placement. Substituting us in Eq. (7) by Eq. (13) yields the following closed-loop transfer function between qs and qms:  
qs=ZpsPsDsQmsRps+PsZpsqms
(14)
Hence, the characteristic equation DsQmsRps+PsZps is a fifth-order polynomial. The objective of pole placement control design is to select Ds and Ps so as to design the characteristic equation  
DsQmsRps+PsZps=A*s
(15)
where A*s is a desired fifth-order polynomial. Denoting A*s=s5+i=04αi*si, the solutions Ds and Ps to Eq. (15) can be found by solving the following algebraic equation:  
SQmRp,ZpTl0p3p2p1p0=α4*θ3+amα3*θ4+amθ3α2*θ4amα1*α0*
(16)
where SQmRp,Zp is the Sylvester matrix associated with QmsRps=s4+θ3+ams3+θ4+amθ3s2+θ4ams and Zps=θ1s+θ2 given by  
SQmRp,Zp=1θ3+amθ4+amθ3θ4am0θ1θ20000θ1θ20000θ1θ20000θ1θ2
(17)
Since Qms (especially the value of am) can be chosen so that QmsRps and Zps are coprime (as long as L>0), the Sylvester matrix (17) has full rank. Therefore, the unknown polynomial coefficients of Ds and Ps in Eq. (16) can be determined by  
l0p3p2p1p0=SQmRp,ZpT1α4*θ3+amα3*θ4+amθ3α2*θ4amα1*α0*
(18)

Hence, Ds and Ps to yield a desired closed-loop characteristic polynomial A*s can be determined if the dose–response model parameters θi, i=1,2,3,4 are known. In APPC, these parameters are provided by the adaptive law (11)(12). In sum, APPC is realized by the pole placement control law (13) with Ds and Ps determined by Eq. (18) with the aid of the adaptive law (11)(12). The stability of the resulting APPC can be established using available procedures [47].

One last consideration is concerned with the stability of control law. Specifically, us=((Pses)/(QmsDs)) and QmsDs is not guaranteed to be Hurwitz. Noting that QmsDsus+Pses=0 from Eq. (13), an alternative realization of us can be obtained as follows:  
us=us1ΛsQmsDsus+Pses=1ΛsΛsQmsDsusPsΛses
(19)

where Λs is a monic Hurwitz polynomial of degree 3 for proper filtering of us and es.

In Silico Implementation and Simulation.

The controller was in silico evaluated using a case study of regulating a cardiovascular variable (CO) with a sedative (propofol), under a wide range of transport delay and pharmacological profiles as well as desired responses.

For the sake of in silico evaluation, we used the 30 randomly created subjects in Parametric Sensitivity Analysis. In all in silico subjects, the baseline CO (y0) was set at 3.0 lpm, while a multitude of desired responses, in terms of both magnitude and rate, was employed: 2.1, 2.4, 2.7 lpm with the time constants of 5, 2.5, and 1.7 min. The parameters associated with the indirect semi-APPC were chosen empirically as follows: Γ=I4×4, A*s=s+η5 with η>0, Λs=s+λ3 with λ=λ0, and ϵ0=0.02. Since the goal of this case study was to examine the efficacy of APPC for medication infusion problems with transport delay, these parameters were not rigorously optimized. The values of λ (and thus λ0 as well) and η were likewise tuned for each desired response. The semi-APPC was then implemented in the discrete-time domain using the zero-order hold method, in which a sampling interval of 5 s was used for control computation. To ensure patient safety against over-dosing, a target-dependent (i.e., σ-dependent) upper bound of infusion rate was augmented to the control law (0.8 mg/kg/min for 2.7 lpm target, 1.2 mg/kg/min for 2.4 lpm target, and 1.6 mg/kg/min for 2.1 lpm target). In this way, a total of 270 in silico simulations was performed and analyzed for semi-APPC.

To investigate the significance of on-line transport delay adaptation, the semi-APPC was compared with controllers without explicit account for transport delay. First, semi-APPC with transport delay L fixed at nominal (i.e., 75 s) and worst-case (i.e., 100 s) values were simulated in the same 30 in silico subjects. Second, the population-based PID controller used in Parametric Sensitivity Analysis was also simulated in the same 30 in silico subjects. Then, the performance of these controllers was quantitatively compared for transient and steady-state behaviors. The transient behavior was quantified by RMSE as well as the Varvel's metrics for the assessment of computerized medication infusion control algorithms [48] up to the settling time of the desired response: median percentage error (MDPE: median value of normalized set point tracking errors, i.e., sample-by-sample errors between the target versus actual CO normalized by target CO), median absolute percentage error (MDAPE: median of normalized absolute set point tracking errors), divergence (time-dependent change in normalized set point tracking errors in terms of slope), and wobble (variability associated with normalized set point tracking errors with respect to MDPE in terms of standard deviation). The steady-state behavior was quantified by the standard deviation of the set point tracking errors between settling time and settling time + 10 min of the desired response.

Results and Discussion.

The objectives of this study were to investigate (1) the impact of nontrivial transport delay on closed-loop medication infusion and (2) the adaptive control design for closed-loop medication infusion problems with on-line adaptation of transport delay. We used the Padé approximation of transport delay and indirect APPC techniques to streamline control design while extending our prior work on semi-adaptive control approach to closed-loop medication infusion to more challenging medication infusion problems involving nontrivial transport delay. The results provided below suggest that adaptation of transport delay may benefit in minimizing the variability in closed-loop response against a wide range of dose–response variability. Details follow.

Comparing semi-APPC, semi-APPC with nominal L, semi-APPC with worst-case L, and population-based PID control, semi-APPC outperformed all the other controllers. Figure 3 presents RMSE, MDAPE, and wobble metrics computed for all four controllers associated with 5 min time constant (i.e., am=0.2 s−1; results for other time constants (am=0.4 s−1 and am=0.6 s−1) exhibited similar trends). Figure 4 shows representative examples of the responses of 30 in silico subjects associated with all four controllers under target CO of 2.7 lpm with 5 min time constant. Figure 5 shows the variability of steady-state set point tracking errors associated with all four controllers. The semi-APPC was in particular superior to all the other controllers in terms of RMSE (by 16%, 12%, and 35% against semi-APPC with nominal L, semi-APPC with worst-case L, and population-based PID control), MDAPE (by 17%, 18%, and 42% against semi-APPC with nominal L, semi-APPC with worst-case L, and population-based PID control), and wobble (by 16%, 16%, and 30% against semi-APPC with nominal L, semi-APPC with worst-case L, and population-based PID control). The semi-APPC, semi-APPC with nominal L, and semi-APPC with worst-case L exhibited comparable MDPE and divergence performance, which was superior to population-based PID control (not shown). The semi-APPC also outperformed all the other controllers in terms of the consistency in steady-state response: semi-APPC consistently exhibited smaller steady-state set point tracking error variability than all the other controllers (by 28%, 30%, and 48% against semi-APPC with nominal L, semi-APPC with worst-case L, and population-based PID control) under all desired clinical effect response. The population-based PID control suffered from the largest average variability and variability thereof, indicating that neglecting nontrivial transport delay in closed-loop medication infusion may yield drastic degradation in performance in case dose–response delay is not negligibly small. Although semi-APPC with nominal and worst-case L were superior to population-based PID control, they fell short of semi-APPC in both transient and steady-state response characteristics, implying that the use of a population-based transport delay is not ideal in closed-loop medication infusion control.

It may be of interest to compare the proposed semi-APPC with existing work on the design of closed-loop medication infusion controllers accounting for transport delay [3539]. Hahn et al. [35] and Mendez et al. [36] used a nominal value of the transport delay. Hence, it may at best perform comparably to semi-APPC with nominal transport delay. Sawaguchi et al. [37] and Soltesz et al. [39] performed one-time, batch individualization of transport delay. Hence, as long as the transport delay does not vary largely in time (e.g., for a short-duration medication therapy), the efficacy of these controllers may be comparable to semi-APPC with nominal worst-case transport delay and even semi-APPC. But, these controllers may suffer from limitations in case the transport delay varies largely in time (e.g., for a long-duration medication therapy), since adverse impact of the inaccuracy associated with the transport delay in the closed-loop control algorithms may become pronounced. Ionescu et al. [38] performed on-line adaptation of transport delay but presumably not the remaining parameters. Hence, it may not be as effective as semi-APPC, which on-line adapts all the high-sensitivity parameters in the plant model.

All in all, the results suggest that (1) transport delay may need to be accounted for in medication infusion control problems, and that (2) transport delay may even need to be adapted to secure robustness in control performance despite uncertainty and variability in dose–response relationship.

Conclusion and Future Work

In this paper, we investigated a semi-adaptive indirect pole placement control for infusion of medications with large transport delay in dose–response dynamics. It was illustrated that (1) incorporating transport delay in control design is beneficial, and that (2) transport delay may even need to be adapted, in case it is large, to ensure consistent clinical effect response against patient variability and uncertainty.

This work presented in this paper has several limitations that must be further investigated in future work. First, the semi-APPC designed in this work was based on an approximated plant model (i.e., dose–response model with Padé-approximated transport delay), which is not identical to the original plant model. Thus, the stability results obtained in this work (which is valid for the approximated plant model but not for the original plant model) are not theoretically rigorous. In this paper, we conducted in silico simulation under a wide range of subjects and desired responses to demonstrate the efficacy and robustness of the proposed semi-APPC (see In Silico Implementation and Simulation for details). Yet, future work must be conducted for more rigorous theoretical investigation of the semi-APPC with transport delay. Second, the proposed semi-APPC was illustrated only in a single case scenario, and its generalizability to a range of medication infusion problems with non-negligible transport delay is yet to be shown. Hence, future work must be conducted to evaluate the efficacy of the proposed semi-APPC in other appropriate medication infusion problems.

Funding Data

  • This work was supported by the U.S. Office of Naval Research (ONR) under the Young Investigator Program Award (ONR #N000141410591, #N000141512018; Funder ID: 10.13039/100000006). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the ONR.

References

References
1.
Heusden
,
K. V.
,
Dumont
,
G. A.
,
Soltesz
,
K.
,
Petersen
,
C. L.
,
Umedaly
,
A.
,
West
,
N.
, and
Ansermino
,
J. M.
,
2014
, “
Design and Clinical Evaluation of Robust PID Control of Propofol Anesthesia in Children
,”
IEEE Trans. Control Syst. Technol.
,
22
(
2
), pp.
491
501
.
2.
De Smet
,
T.
,
Struys
,
M. M. R. F.
,
Neckebroek
,
M. M.
,
Den Hauwe
,
K. V.
,
Bonte
,
S.
, and
Mortier
,
E. P.
,
2008
, “
The Accuracy and Clinical Feasibility of a New Bayesian-Based Closed-Loop Control System for Propofol Administration Using the Bispectral Index as a Controlled Variable
,”
Anesth. Analg.
,
107
(
4
), pp.
1200
1210
.
3.
Niño
,
J.
,
De Keyser
,
R.
,
Syafiie
,
S.
,
Ionescu
,
C.
, and
Struys
,
M.
,
2009
, “
EPSAC-Controlled Anesthesia With Online Gain Adaptation
,”
Int. J. Adapt. Control Signal Process.
,
23
(
5
), pp.
455
471
.
4.
Nunes
,
C. S.
,
Mendonça
,
T.
,
Lemos
,
J. M.
, and
Amorim
,
P.
,
2009
, “
Feedforward Adaptive Control of the Bispectral Index of the EEG Using the Intravenous Anaesthetic Drug Propofol
,”
Int. J. Adapt. Control Signal Process.
,
23
(
5
), pp.
485
503
.
5.
Hemmerling
,
T. M.
,
Charabati
,
S.
,
Zaouter
,
C.
,
Minardi
,
C.
, and
Mathieu
,
P. A.
,
2010
, “
A Randomized Controlled Trial Demonstrates That a Novel Closed-Loop Propofol System Performs Better Hypnosis Control Than Manual Administration
,”
Can. J. Anesth.
,
57
(
8
), pp.
725
735
.
6.
Kharisov
,
E.
,
Beck
,
C. L.
, and
Bloom
,
M.
,
2015
, “
Design of L1 Adaptive Controllers for Human Patient Anesthesia
,”
Control Eng. Pract.
,
44
, pp.
65
77
.
7.
Padmanabhan
,
R.
,
Meskin
,
N.
, and
Haddad
,
W. M.
,
2015
, “
Closed-Loop Control of Anesthesia and Mean Arterial Pressure Using Reinforcement Learning
,”
Biomed. Signal Process. Control
,
22
, pp.
54
64
.
8.
Hemmerling
,
T. M.
,
Charabati
,
S.
,
Salhab
,
E.
,
Bracco
,
D.
, and
Mathieu
,
P. A.
,
2009
, “
The Analgoscore TM: A Novel Score to Monitor Intraoperative Nociception and Its Use for Closed-Loop Application of Remifentanil
,”
J. Comput.
,
4
(
4
), pp.
311
318
.
9.
Jin
,
X.
,
Kim
,
C. S.
,
Dumont
,
G. A.
,
Ansermino
,
J. M.
, and
Hahn
,
J. O.
,
2017
, “
A Semi-Adaptive Control Approach to Closed-Loop Medication Infusion
,”
Int. J. Adapt. Control Signal Process.
,
31
(
2
), pp.
240
254
.
10.
van Heusden
,
K.
,
Ansermino
,
J. M.
, and
Dumont
,
G. A.
,
2017
, “
Robust MISO Control of Propofol-Remifentanil Anesthesia Guided by the NeuroSENSE Monitor
,”
IEEE Trans. Control Syst. Technol.
,
26
(5), pp. 1758–1770.https://ieeexplore.ieee.org/document/8017462
11.
Wassar
,
T.
,
Upendar
,
K. R.
,
Moisi
,
M.
,
Voigt
,
R. B.
,
Marques
,
N. R.
,
Khan
,
M. N.
,
Grigoriadis
,
K. M.
,
Franchek
,
M. A.
, and
Kramer
,
G. C.
,
2014
, “
Automatic Control of Arterial Pressure for Hypotensive Patients Using Phenylephrine
,”
Int. J. Model. Simul.
,
34
(
4
), pp.
187
198
https://www.tandfonline.com/doi/abs/10.2316/Journal.205.2014.4.205-6087.
12.
Ngan Kee
,
W. D.
,
Khaw
,
K. S.
,
Tam
,
Y. H.
,
Ng
,
F. F.
, and
Lee
,
S. W.
,
2017
, “
Performance of a Closed-Loop Feedback Computer-Controlled Infusion System for Maintaining Blood Pressure During Spinal Anaesthesia for Caesarean Section: A Randomized Controlled Comparison of Norepinephrine Versus Phenylephrine
,”
J. Clin. Monit. Comput.
,
31
(
3
), pp.
617
623
.
13.
Luspay
,
T.
, and
Grigoriadis
,
K.
,
2015
, “
Robust Linear Parameter-Varying Control of Blood Pressure Using Vasoactive Drugs
,”
Int. J. Control
,
88
(
10
), pp.
2013
2029
.
14.
Marques
,
N. R.
,
Ford
,
B. J.
,
Khan
,
M. N.
,
Kinsky
,
M.
,
Deyo
,
D. J.
,
Mileski
,
W. J.
,
Ying
,
H.
, and
Kramer
,
G. C.
,
2017
, “
Automated Closed-Loop Resuscitation of Multiple Hemorrhages: A Comparison Between Fuzzy Logic and Decision Table Controllers in a Sheep Model
,”
Disaster Mil. Med.
,
3
(
1
), pp.
1
10
.
15.
Chaisson
,
N. F.
,
Kirschner
,
R. A.
,
Deyo
,
D. J.
,
Lopez
,
J. A.
,
Prough
,
D. S.
, and
Kramer
,
G. C.
,
2003
, “
Near-Infrared Spectroscopy-Guided Closed-Loop Resuscitation of Hemorrhage
,”
J. Trauma
,
54
(
5 Suppl
.), pp.
S183
S192
.
16.
Kramer
,
G. C.
,
Kinsky
,
M. P.
,
Prough
,
D. S.
,
Salinas
,
J.
,
Sondeen
,
J. L.
,
Hazel-Scerbo
,
M. L.
, and
Mitchell
,
C. E.
,
2008
, “
Closed-Loop Control of Fluid Therapy for Treatment of Hypovolemia
,”
J. Trauma
,
64
(
4 Suppl
.), pp.
S333
S341
.
17.
Rinehart
,
J.
,
Lee
,
C.
,
Canales
,
C.
,
Kong
,
A.
,
Kain
,
Z.
, and
Cannesson
,
M.
,
2013
, “
Closed-Loop Fluid Administration Compared to Anesthesiologist Management for Hemodynamic Optimization and Resuscitation During Surgery: An In Vivo Study
,”
Anesth. Analg.
,
117
(
5
), pp.
1119
1129
.
18.
Salinas
,
J.
,
Drew
,
G.
,
Gallagher
,
J.
,
Cancio
,
L. C.
,
Wolf
,
S. E.
,
Wade
,
C. E.
,
Holcomb
,
J. B.
,
Herndon
,
D. N.
, and
Kramer
,
G. C.
,
2008
, “
Closed-Loop and Decision-Assist Resuscitation of Burn Patients
,”
J. Trauma
,
64
(
4 Suppl
.), pp.
S321
S332
.
19.
Joosten
,
A.
,
Huynh
,
T.
,
Suehiro
,
K.
,
Canales
,
C.
,
Cannesson
,
M.
, and
Rinehart
,
J.
,
2015
, “
Goal-Directed Fluid Therapy With Closed-Loop Assistance During Moderate Risk Surgery Using Noninvasive Cardiac Output Monitoring: A Pilot Study
,”
Br. J. Anaesth.
,
114
(
6
), pp.
886
892
.
20.
Rinehart
,
J.
,
Lilot
,
M.
,
Lee
,
C.
,
Joosten
,
A.
,
Huynh
,
T.
,
Canales
,
C.
,
Imagawa
,
D.
,
Demirjian
,
A.
, and
Cannesson
,
M.
,
2015
, “
Closed-Loop Assisted Versus Manual Goal-Directed Fluid Therapy During High-Risk Abdominal Surgery: A Case-Control Study With Propensity Matching
,”
Crit. Care
,
19
(
1
), pp.
1
11
.
21.
Liu
,
N.
,
Chazot
,
T.
,
Genty
,
A.
,
Landais
,
A.
,
Restoux
,
A.
,
McGee
,
K.
,
Laloë
,
P. A.
,
Trillat
,
B.
,
Barvais
,
L.
, and
Fischler
,
M.
,
2006
, “
Titration of Propofol for Anesthetic Induction and Maintenance Guided by the Bispectral Index: Closed-Loop Versus Manual Control: A Prospective, Randomized, Multicenter Study
,”
Anesthesiology
,
104
(
4
), pp.
686
695
.
22.
Agarwal
,
J.
,
Puri
,
G. D.
, and
Mathew
,
P. J.
,
2009
, “
Comparison of Closed Loop vs. Manual Administration of Propofol Using the Bispectral Index in Cardiac Surgery
,”
Acta Anaesthesiol. Scand.
,
53
(
3
), pp.
390
397
.
23.
Locher
,
S.
,
Stadler
,
K. S.
,
Boehlen
,
T.
,
Bouillon
,
T.
,
Leibundgut
,
D.
,
Schumacher
,
P. M.
,
Wymann
,
R.
, and
Zbinden
,
A. M.
,
2004
, “
A New Closed-Loop Control System for Isoflurane Using Bispectral Index Outperforms Manual Control
,”
Anesthesiology
,
101
(
3
), pp.
591
602
.
24.
Dussaussoy
,
C.
,
Peres
,
M.
,
Jaoul
,
V.
,
Liu
,
N.
,
Chazot
,
T.
,
Picquet
,
J.
,
Fischler
,
M.
, and
Beydon
,
L.
,
2014
, “
Automated Titration of Propofol and Remifentanil Decreases the Anesthesiologist's Workload During Vascular or Thoracic Surgery: A Randomized Prospective Study
,”
J. Clin. Monit. Comput.
,
28
(
1
), pp.
35
40
.
25.
Pasin
,
L.
,
Nardelli
,
P.
,
Pintaudi
,
M.
,
Greco
,
M.
,
Zambon
,
M.
,
Cabrini
,
L.
, and
Zangrillo
,
A.
,
2017
, “
Closed-Loop Delivery Systems Versus Manually Controlled Administration of Total IV Anesthesia: A Meta-Analysis of Randomized Clinical Trials
,”
Anesth. Analg.
,
124
(
2
), pp.
456
464
.
26.
Brogi
,
E.
,
Cyr
,
S.
,
Kazan
,
R.
,
Giunta
,
F.
, and
Hemmerling
,
T. M.
,
2017
, “
Clinical Performance and Safety of Closed-Loop Systems: A Systematic Review and Meta-Analysis of Randomized Controlled Trials
,”
Anesth. Analg.
,
124
(
2
), pp.
446
455
.
27.
Khosravi
,
S.
,
Hahn
,
J. O.
,
Dumont
,
G. A.
, and
Ansermino
,
J. M.
,
2012
, “
A Monitor-Decoupled Pharmacodynamic Model of Propofol in Children Using State Entropy as Clinical Endpoint
,”
IEEE Trans. Biomed. Eng.
,
59
(
3
), pp.
736
743
.
28.
Pilge
,
S.
,
Zanner
,
R.
,
Schneider
,
G.
,
Blum
,
J.
,
Kreuzer
,
M.
, and
Kochs
,
E. F.
,
2006
, “
Time Delay of Index Calculation
,”
Anesthesiology
,
104
(
3
), pp.
488
494
.
29.
Kreuzer
,
M.
,
Zanner
,
R.
,
Pilge
,
S.
,
Paprotny
,
S.
,
Kochs
,
E. F.
, and
Schneider
,
G.
,
2012
, “
Time Delay of Monitors of the Hypnotic Component of Anesthesia: Analysis of State Entropy and Index of Consciousness
,”
Anesth. Analg.
,
115
(
2
), pp.
315
319
.
30.
Bibian
,
S.
,
Dumont
,
G. A.
, and
Zikov
,
T.
,
2011
, “
Dynamic Behavior of BIS, M-Entropy and NeuroSENSE Brain Function Monitors
,”
J. Clin. Monit. Comput.
,
25
(
1
), pp.
81
87
.
31.
Ionescu
,
C. M.
,
Keyser
,
R. D.
,
Torrico
,
B. C.
,
Smet
,
T. D.
,
Struys
,
M. M. R. F.
, and
Normey-Rico
,
J. E.
,
2008
, “
Robust Predictive Control Strategy Applied for Propofol Dosing Using BIS as a Controlled Variable During Anesthesia
,”
IEEE Trans. Biomed. Eng.
,
55
(
9
), pp.
2161
2170
.
32.
Yelneedi
,
S.
,
Samavedham
,
L.
, and
Rangaiah
,
G. P.
,
2009
, “
Advanced Control Strategies for the Regulation of Hypnosis With Propofol
,”
Ind. Eng. Chem. Res.
,
48
(
8
), pp.
3880
3897
.
33.
Volyanskyy
,
K.
,
Haddad
,
W. M.
, and
Bailey
,
J. M.
,
2009
, “
Adaptive Disturbance Rejection Control for Compartmental Systems With Application to Intraoperative Anesthesia Influenced by Hemorrhage and Hemodilution Effects
,”
Int. J. Adapt. Control Signal Process.
,
23
(
1
), pp.
1
29
.
34.
Minto
,
C. F.
,
Schnider
,
T. W.
,
Gregg
,
K. M.
,
Henthorn
,
T. K.
, and
Shafer
,
S. L.
,
2003
, “
Using the Time of Maximum Effect Site Concentration to Combine Pharmacokinetics and Pharmacodynamics
,”
Anesthesiology
,
99
(
2
), pp.
324
333
.
35.
Hahn
,
J. O.
,
Dumont
,
G. A.
, and
Ansermino
,
J. M.
,
2012
, “
Robust Closed-Loop Control of Hypnosis With Propofol Using WAV CNS Index as the Controlled Variable
,”
Biomed. Signal Process. Control
,
7
(
5
), pp.
517
524
.
36.
Mendez
,
J. A.
,
Torres
,
S.
,
Reboso
,
J. A.
, and
Reboso
,
H.
,
2009
, “
Adaptive Computer Control of Anesthesia in Humans
,”
Comput. Methods Biomech. Biomed. Eng.
,
12
(
6
), pp.
727
734
.
37.
Sawaguchi
,
Y.
,
Furutani
,
E.
,
Shirakami
,
G.
,
Araki
,
M.
, and
Fukuda
,
K.
,
2008
, “
A Model-Predictive Hypnosis Control System Under Total Intravenous Anesthesia
,”
IEEE Trans. Biomed. Eng.
,
55
(
3
), pp.
874
887
.
38.
Ionescu
,
C. M.
,
Hodrea
,
R.
, and
De Keyser
,
R.
,
2011
, “
Variable Time-Delay Estimation for Anesthesia Control During Intensive Care
,”
IEEE Trans. Biomed. Eng.
,
58
(
2
), pp.
363
369
.
39.
Soltesz
,
K.
,
Hahn
,
J. O.
,
Hägglund
,
T.
,
Dumont
,
G. A.
, and
Ansermino
,
J. M.
,
2013
, “
Individualized Closed-Loop Control of Propofol Anesthesia: A Preliminary Study
,”
Biomed. Signal Process. Control
,
8
(
6
), pp.
500
508
.
40.
Niculescu
,
S. I.
, and
Annaswamy
,
A. M.
,
2003
, “
An Adaptive Smith-Controller for Time-Delay Systems With Relative Degree N* ≤2
,”
Syst. Control Lett.
,
49
(
5
), pp.
347
358
.
41.
Evesque
,
S.
,
Annaswamy
,
A. M.
,
Niculescu
,
S.
, and
Dowling
,
A. P.
,
2003
, “
Adaptive Control of a Class of Time-Delay Systems
,”
ASME J. Dyn. Syst. Meas. Control
,
125
(
2
), p.
186
.
42.
Bresch-Pietri
,
D.
, and
Krstic
,
M.
,
2009
, “
Adaptive Trajectory Tracking Despite Unknown Input Delay and Plant Parameters
,”
Automatica
,
45
(
9
), pp.
2074
2081
.
43.
Souzaf
,
C. E. D. E.
,
Goodwin
,
G. C.
,
Mayne
,
D. Q.
, and
Palaniswami
,
M.
,
1988
, “
An Adaptive Control Algorithm for Linear Systems Having Unknown Time Delay
,”
Automatica
,
24
(
3
), pp.
327
341
.
44.
Krstic
,
M.
, and
Banaszuk
,
A.
,
2006
, “
Multivariable Adaptive Control of Instabilities Arising in Jet Engines
,”
Control Eng. Pract.
,
14
(
7
), pp.
833
842
.
45.
Hahn
,
J. O.
,
Dumont
,
G. A.
, and
Ansermino
,
J. M.
,
2012
, “
A Direct Dynamic Dose-Response Model of Propofol for Individualized Anesthesia Care
,”
IEEE Trans. Biomed. Eng.
,
59
(
2
), pp.
571
578
.
46.
Jin
,
X.
,
Kim
,
C.-S.
,
Shipley
,
S. T.
,
Dumont
,
G. A.
, and
Hahn
,
J.-O.
,
2018
, “
Coordinated Semi-Adaptive Closed-Loop Control for Infusion of Two Interacting Medications
,”
Int. J. Adapt. Control Signal Process.
,
32
(
1
), pp.
134
146
.
47.
Ioannou
,
P.
, and
Sun
,
J.
,
2012
,
Robust Adaptive Control
,
Dover Publications
,
Mineola, NY
.
48.
Varvel
,
J. R.
,
Donoho
,
D. L.
, and
Shafer
,
S. L.
,
1992
, “
Measuring the Predictive Performance of Computer-Controlled Infusion Pumps
,”
J. Pharmacokinet. Biopharm.
,
20
(
1
), pp.
63
94
.