## Abstract

This paper presents a semi-adaptive closed-loop control approach to autonomous infusion of medications exhibiting significant transport delay in clinical effects. The basic idea of the approach is to enable stable adaptive control of medication infusion by (1) incorporating transport delay explicitly into control design by way of a Padé approximation while (2) facilitating linear parameterization of control design model by desensitization of nonlinearly parameterized cooperativity constant associated with pharmacodynamics (PD). A novel dynamic dose–response model for control design is presented, in which the cooperativity constant exerts zero influence on the model output in the steady-state. Then, an adaptive pole placement control (APPC) technique was employed to fulfill adaptive control design in the presence of nonminimum phase dynamics associated with the Padé approximation of transport delay. The controller was evaluated in silico using a case study of regulating a cardiovascular variable with a sedative under a wide range of transport delay and pharmacological profiles. The results suggest that adaptation of transport delay and pharmacological characteristics may be beneficial in achieving consistent and robust regulation of medication-elicited clinical effects.

## Introduction

Closed-loop autonomous control of medication infusion has been a very active area of research by virtue of its potential as an attractive alternative to manual medication infusion performed by human clinicians. Examples include sedatives and anesthetics [1–7], opioids and analgesics [8–10], vasoactive drugs [11–13], and even fluids and blood products [14–20]. In fact, there is an increasing body of evidence that supports the efficacy of closed-loop medication infusion systems: it has the potential to enable tight regulation of desired clinical effects and relief of clinician workload [5,21–26].

One control design challenge related to medication infusion problems is the transport delay between the administration of medication and the onset of resulting clinical effects measured by patient monitors. First, although some medication dose–clinical response pairs exhibit minimal transport delay that may be neglected in the control design process (e.g., hypnosis by anesthetics [27] and respiratory depression by fast-acting opioids [9]), there are medication dose–clinical response pairs, including those widely employed in today's clinical practice (e.g., blood pressure increase by phenylephrine [11]), which exhibit relatively large, variable transport delays. Second, patient monitors often introduce additional (and potentially variable) transport delay due to the computational load and algorithmic nonlinearity [28–30]. Transport delay tends to deteriorate stability margin and can even destabilize the control loop. Hence, it must be accounted for with care as part of control design in order to avoid catastrophic risks on patient safety. Nonetheless, transport delay has seldom been treated seriously in control design for medication infusion problems. One reason may be that the vast majority of prior work used traditional pharmacokinetics (PK) and pharmacodynamics (PD) models [31–33]. These models do not account for transport delay related to the PD and patient monitor explicitly; rather, transport delay associated with the PD and monitor is often abstracted into lumped first-order phase lag [34]. The other reason may be that transport delay is nonlinearly parameterized and cannot be readily estimated with established recursive estimation techniques. Such a nonlinear parameterization presents technical challenges in designing adaptive closed-loop medication infusion controllers capable of counteracting the impact of transport delay in the PD and monitors. Perhaps due to these reasons, only a small number of prior work has taken the transport delay into full consideration in control design [35–39]. Hahn et al. [35] employed a simple first-order Padé approximation of PD transport delay and conducted a two degrees-of-freedom robust control design based on the *μ* synthesis theory to compensate for the associated approximation errors. Mendez et al. [36] employed a Smith predictor to compensate for transport delay in the PD and a monitor, and incorporated it into proportional-integral control. Sawaguchi et al. [37] estimated transport delay and PD model parameters from the measurements obtained from the initial transients and incorporated them into model predictive control. Ionescu et al. [38] inferred variable transport delay associated with a monitor with cross-correlation analysis and incorporated it into model predictive control in combination with a Smith predictor. Soltesz et al. [39] individualized PD model parameters (including PD transport delay therein) based on the measurements obtained from an initial test bolus and used them in conjunction with novel inversion filters for adapting proportional–integral–derivative (PID) control to individual patients. As can be seen above, there has been rare prior work attempting to formally integrate the estimation of transport delay as a component of closed-loop control algorithm with sound stability proof: transport delay was either handled as uncertainty, or was separately estimated and incorporated into closed-loop control through Smith predictor (which may suffer from possible inaccuracy in transport delay as well as uncertainty and modeling errors other than transport delay) or in other ad hoc ways.

Stable adaptive control of dynamic systems with variable transport delay is, in general, a challenging problem even in the control community, and only limited prior work appears to exist. In one category of prior work, methods to estimate transport delay were investigated [40–42]. Despite significance and rigor, these methods involve somewhat restrictive requirements and assumptions (e.g., knowledge of high-frequency gain [40,41] and full state information [42]), which may limit their direct application to closed-loop medication infusion problems. In the other category of prior work, methods to linearize transport delay using Padé approximation and incorporate it into indirect adaptive control design techniques were investigated [43,44].

In addition to transport delay, the control design problem for medication infusion must also account for two other well-known challenges: stiff nonlinearity and large interindividual variability associated with the dose–response relationship. These challenges, in combination with the one associated with rigorous account of transport delay, make closed-loop medication infusion a difficult yet interesting control design problem.

This paper presents a semi-adaptive closed-loop control approach to autonomous infusion of medications exhibiting significant transport delay in clinical effects. The basic idea of the approach is to enable stable adaptive control of medication infusion by (1) incorporating transport delay explicitly into control design by way of the Padé approximation while (2) facilitating linear parameterization of the control design model by desensitization of the nonlinearly parameterized cooperativity constant associated with the PD. The resulting controller is semi-adaptive in that the desensitized cooperativity constant is fixed at a nominal value while the remaining model parameters are on-line updated. A novel dynamic dose–response model for control design is presented, in which the cooperativity constant exerts zero influence on the model output in the steady-state. Then, indirect adaptive pole placement control (APPC) technique was employed to fulfill adaptive control design in the presence of nonminimum phase dynamics associated with the Padé approximation of transport delay. The controller was in silico evaluated using a case study of regulating a cardiovascular variable with a sedative under a wide range of transport delay and pharmacological profiles.

This paper is organized as follows: First, a novel dynamic dose–response model for control design is presented. Second, parametric sensitivity analysis of the dynamic dose–response model is performed to assess the influence of transport delay on the model output in comparison with the other dose–response model parameters. Third, the indirect APPC design for medication infusion problems based on the Padé approximation of transport delay is developed. Fourth, the results are presented and discussed. Finally, the paper is concluded with future directions.

## Control Design Model: Direct Dynamic Dose–Response Model

Expanding our prior work [9,45], a novel dynamic dose–response model suited to adaptive control design for medication infusion problems with non-negligible transport delay is developed. The model consists of a low-order lumped parameter latency model (1*a*) and a modified Hill equation model (1*b*)

where $ut$ is the intravenous medication infusion rate, $Iet$ is the medication infusion rate at the site of action, $L$ is the transport delay between medication infusion and the onset of the intended clinical effect, $ke$ is the rate constant associated with the distribution of medication from intravenous site to the site of action, $yt$ and $yo$ are clinical effect and its nominal (i.e., in the absence of medication infusion) value, $\sigma $ is the target percentage depression of $yt$ from $yo$ (meaning that target clinical effect $rt$ is specified by $rt=((100\u2212\sigma )/100)yo$), $I\sigma $ is the infusion rate required to maintain $yt=rt$ in the steady-state, and $\gamma $ is the cooperativity constant. There are a few advantages associated with this dose–response model compared with conventional PKPD models. First, intended to primarily capture the direct relationship between dose and the resulting clinical effects, its dynamic component is much simpler than multicompartmental PKPD models. Second, it explicitly incorporates transport delay, which can be exploited to represent PD and/or patient monitor delays. Third and most importantly, it facilitates semi-adaptive control design based on the parametric sensitivity analysis [9], in which model parameters exerting high sensitivity on the model's behavior are adapted while those with low sensitivity are fixed at nominal values (see Parametric Sensitivity Analysis for details).

## Parametric Sensitivity Analysis

In this section, analytical and numerical parametric sensitivity analyses are performed to demonstrate that (1) the proposed dose–response model allows us to minimize the influence of $\gamma $ with appropriate choice of $\sigma $; and (2) the transport delay $L$ needs to be adapted for control efficacy. Details follow.

### Desensitization of Cooperativity Constant: Analytical Sensitivity Analysis.

*a*) and (1

*b*) as the state and output equations for the process dynamics. Then, the following sensitivity functions can be derived:

Hence, $((\u2202yt)/\u2202\gamma )$, which is the second element of $Syt$ in Eq. (3), becomes zero when $Iet=I\sigma $. The implication of the zero sensitivity of $yt$ on $\gamma $ for $Iet=I\sigma $ is that the influence of $\gamma $ on the process behavior can be minimized by setting $\sigma $ according to the target clinical effect, that is, equal to the percentage difference between $yo$ and $rt$, i.e., $\sigma =((yo\u2212rt)/yo)\xd7100$. Indeed, with this choice of $\sigma $, it can be readily shown that the process behavior becomes insensitive to $\gamma $ under steady-state target tracking condition ($yt=rt$).

### Numerical Sensitivity Analysis.

To confirm the above analytical sensitivity analysis as well as to assess the significance of transport delay $L$ relative to the other parameters in the dose–response model, a numerical parametric perturbation analysis was performed using the regulation of a cardiovascular variable cardiac output (CO) with a sedative propofol as a case scenario. In this analysis, a fine-tuned population-based PID controller was used to regulate CO via propofol infusion in 30 randomly created in silico patients for nominal closed-loop response. Then, the dose–response model parameters (including $ke$, $I\sigma $, $\gamma $, and $L$) were perturbed, one at a time, by $\xb1$25% and $\xb1$50% (thus, four perturbations per each parameter), and the perturbed in silico patients were simulated with the same PID controller for perturbed closed-loop responses. Then, the difference between nominal and perturbed responses was quantified in terms of root-mean-squared errors (RMSEs) between the two up to the settling time of the desired response. Finally, a total of 120 RMSEs associated with each parameter was aggregated to compute mean and standard deviation (SD). We used the following ranges of the dose–response model parameters obtained from our prior work [46] and our unpublished in-house experimental data to create 30 in silico patients: $0.02\u2264ke\u22640.10\u2009$min^{−1}, $0.2\u2264Ie\u22640.6\u2009$mg/kg/min, $1\u2264\gamma \u2009\u2264\u20095$, and $50\u2264L\u2264100$ s. In all in silico simulations, $y0$ was set at 3.0 lpm. A multitude of desired responses, in terms of both magnitude and rate, was employed in the analysis: 2.1, 2.4, 2.7 lpm with the time constants of 5, 2.5, and 1.7 min. For each of these nine desired responses, the aggregated RMSEs associated with the dose–response model parameters were compared to assess if the perturbation in the transport delay makes a large impact on the closed-loop control performance relative to the other dose–response model parameters, in order to determine if the adaptation of transport delay is necessary.

Results from the parametric sensitivity analysis indicated that the impact of transport delay on the closed-loop clinical effect response (as measured by the RMSE between nominal and perturbed responses up to the settling time of the desired response) was larger than cooperativity constant but not as large as $I\sigma $ and $ke$. Figure 1 presents the RMSEs between nominal and perturbed responses associated with each model parameter, which offers several key observations. First, all in all $I\sigma $ exerted the largest impact on the model's clinical effect response, followed by $ke$, and then $L$, and finally $\gamma $. Second, although $L$ was not the most crucial parameter in (1) in terms of average sensitivity, it exhibited a very large variability in sensitivity relative to its average counterpart (i.e., large coefficient of variation) in comparison with the remaining parameters in Eq. (1). This is attributed to a large deviation of the closed-loop controlled clinical effect response from its nominal counterpart when $L$ assumes very large values, since a large $L$ reduces the stability margin associated with the population-based PID control (which does not accommodate the increase in $L$). Hence, it deemed reasonable to investigate the advantage of its adaptation by comparing an adaptive control equipped with the capability of adapting $L$ versus an adaptive control with $L$ fixed at a nominal value. Third, although the dose–response model (1) by construction exhibits zero sensitivity to $\gamma $ when $Iet=I\sigma $ if $\sigma $ is specified in accordance with the target clinical effect (i.e., if $\sigma $ is set equal to the percentage difference between the baseline and target clinical responses), its sensitivity to $\gamma $ is not zero during transients. Hence, the finding that $\gamma $ exerts the smallest influence on the dose–response model's clinical response even during transients suggests that all in all the model's sensitivity to $\gamma $ is the smallest and that $\gamma $ may indeed be fixed at a nominal value.

## Semi-Adaptive Pole Placement Control Design

Our semi-APPC is built upon the results of the parametric sensitivity analysis above: that $L$ must be on-line adapted while $\gamma $ may be fixed at a nominal value. The proposed semi-APPC consists of the following: (1) Padé approximation of transport delay, (2) linear model parameterization, (3) recursive model parameter adaptation, and (4) its integration into pole placement control. Figure 2 shows the scheme of the sensitivity-based semi-APPC. Details follow.

### Padé Approximation of Transport Delay.

For the sake of adaptive control design, the transport delay in the dose–response model (1) was simplified into the Padé approximation. It is obvious that the higher the order of approximation, the more accurate the approximation is. But, a high-order approximation complicates control design by increasing the order of process dynamics. Besides, a relatively low-order approximation may still be appropriate for adaptive control, because the adaptation of transport delay parameter may mitigate the approximation error and still produce correct phase delay [44].

Then, the dose–response model (1) with these candidate approximations equipped with nominal parameter values (associated with 30 in silico subjects) used in the parametric sensitivity analysis in Parametric Sensitivity Analysis was in silico simulated with an escalated multistep propofol dose with five infusion rate levels designed to elicit a wide range of transient and steady-state CO response. The CO responses associated with each of the six candidate approximations were then compared with the response of the original dose–response model without Padé approximation. Specifically, RMSEs between the responses associated with the original and all the Padé-approximated models were computed, and the optimal order of the Padé approximation for the problem was determined based on the trend of the RMSE with respect to the order.

Table 1 shows the RMSE between the responses associated with the original (i.e., Eq. (1)) and all the Padé-approximated (i.e., Eq. (1) with the *M*_{1}–*M*_{6}) dose–response models. Clearly, *M*_{1} (which does not account for transport delay) suffers from the largest error, whereas the models incorporating the approximation of transport delay can largely reduce the error. The amount of reduction in error becomes trivial beyond *M*_{3}. Hence, we used *M*_{3} for semi-APPC design in this study.

### Dose–Response Model Parameterization.

*b*) can be written for $Iet$ as follows:

*a*) can be written in terms of $qt$ as follows:

For a given desired clinical effect $rs$, the reference model was specified by $qms=am/s+amrs$ with $am>0$.

### Semi-Adaptive Pole Placement Control.

We aim to design an adaptive controller for Eq. (7). Since Eq. (7) is nonminimum phase, the application of direct adaptive control techniques (e.g., direct model reference adaptive control) is not trivial [47]. Therefore, we pursued indirect APPC due to its ability to handle nonminimum phase plants [47]. Details follow.

#### On-Line Parameter Estimation.

#### Closed-Loop Control Design.

Hence, $Ds$ and $Ps$ to yield a desired closed-loop characteristic polynomial $A*s$ can be determined if the dose–response model parameters $\theta i$, $i=1,2,3,4$ are known. In APPC, these parameters are provided by the adaptive law (11)–(12). In sum, APPC is realized by the pole placement control law (13) with $Ds$ and $Ps$ determined by Eq. (18) with the aid of the adaptive law (11)–(12). The stability of the resulting APPC can be established using available procedures [47].

where $\Lambda s$ is a monic Hurwitz polynomial of degree 3 for proper filtering of $us$ and $es$.

### In Silico Implementation and Simulation.

The controller was in silico evaluated using a case study of regulating a cardiovascular variable (CO) with a sedative (propofol), under a wide range of transport delay and pharmacological profiles as well as desired responses.

For the sake of in silico evaluation, we used the 30 randomly created subjects in Parametric Sensitivity Analysis. In all in silico subjects, the baseline CO ($y0$) was set at 3.0 lpm, while a multitude of desired responses, in terms of both magnitude and rate, was employed: 2.1, 2.4, 2.7 lpm with the time constants of 5, 2.5, and 1.7 min. The parameters associated with the indirect semi-APPC were chosen empirically as follows: $\Gamma =I4\xd74$, $A*s=s\u2009+\u2009\eta 5$ with $\eta \u2009>\u20090$, $\Lambda s=s\u2009+\u2009\lambda 3$ with $\lambda =\lambda 0$, and $\u03f50=0.02$. Since the goal of this case study was to examine the efficacy of APPC for medication infusion problems with transport delay, these parameters were not rigorously optimized. The values of $\lambda $ (and thus $\lambda 0$ as well) and $\eta $ were likewise tuned for each desired response. The semi-APPC was then implemented in the discrete-time domain using the zero-order hold method, in which a sampling interval of 5 s was used for control computation. To ensure patient safety against over-dosing, a target-dependent (i.e., $\sigma $-dependent) upper bound of infusion rate was augmented to the control law (0.8 mg/kg/min for 2.7 lpm target, 1.2 mg/kg/min for 2.4 lpm target, and 1.6 mg/kg/min for 2.1 lpm target). In this way, a total of 270 in silico simulations was performed and analyzed for semi-APPC.

To investigate the significance of on-line transport delay adaptation, the semi-APPC was compared with controllers without explicit account for transport delay. First, semi-APPC with transport delay $L$ fixed at nominal (i.e., 75 s) and worst-case (i.e., 100 s) values were simulated in the same 30 in silico subjects. Second, the population-based PID controller used in Parametric Sensitivity Analysis was also simulated in the same 30 in silico subjects. Then, the performance of these controllers was quantitatively compared for transient and steady-state behaviors. The transient behavior was quantified by RMSE as well as the Varvel's metrics for the assessment of computerized medication infusion control algorithms [48] up to the settling time of the desired response: median percentage error (MDPE: median value of normalized set point tracking errors, i.e., sample-by-sample errors between the target versus actual CO normalized by target CO), median absolute percentage error (MDAPE: median of normalized absolute set point tracking errors), divergence (time-dependent change in normalized set point tracking errors in terms of slope), and wobble (variability associated with normalized set point tracking errors with respect to MDPE in terms of standard deviation). The steady-state behavior was quantified by the standard deviation of the set point tracking errors between settling time and settling time + 10 min of the desired response.

### Results and Discussion.

The objectives of this study were to investigate (1) the impact of nontrivial transport delay on closed-loop medication infusion and (2) the adaptive control design for closed-loop medication infusion problems with on-line adaptation of transport delay. We used the Padé approximation of transport delay and indirect APPC techniques to streamline control design while extending our prior work on semi-adaptive control approach to closed-loop medication infusion to more challenging medication infusion problems involving nontrivial transport delay. The results provided below suggest that adaptation of transport delay may benefit in minimizing the variability in closed-loop response against a wide range of dose–response variability. Details follow.

Comparing semi-APPC, semi-APPC with nominal $L$, semi-APPC with worst-case $L$, and population-based PID control, semi-APPC outperformed all the other controllers. Figure 3 presents RMSE, MDAPE, and wobble metrics computed for all four controllers associated with 5 min time constant (i.e., $am=0.2$ s^{−1}; results for other time constants ($am=0.4$ s^{−1} and $am=0.6$ s^{−1}) exhibited similar trends). Figure 4 shows representative examples of the responses of 30 in silico subjects associated with all four controllers under target CO of 2.7 lpm with 5 min time constant. Figure 5 shows the variability of steady-state set point tracking errors associated with all four controllers. The semi-APPC was in particular superior to all the other controllers in terms of RMSE (by 16%, 12%, and 35% against semi-APPC with nominal $L$, semi-APPC with worst-case $L$, and population-based PID control), MDAPE (by 17%, 18%, and 42% against semi-APPC with nominal $L$, semi-APPC with worst-case $L$, and population-based PID control), and wobble (by 16%, 16%, and 30% against semi-APPC with nominal $L$, semi-APPC with worst-case $L$, and population-based PID control). The semi-APPC, semi-APPC with nominal $L$, and semi-APPC with worst-case $L$ exhibited comparable MDPE and divergence performance, which was superior to population-based PID control (not shown). The semi-APPC also outperformed all the other controllers in terms of the consistency in steady-state response: semi-APPC consistently exhibited smaller steady-state set point tracking error variability than all the other controllers (by 28%, 30%, and 48% against semi-APPC with nominal $L$, semi-APPC with worst-case $L$, and population-based PID control) under all desired clinical effect response. The population-based PID control suffered from the largest average variability and variability thereof, indicating that neglecting nontrivial transport delay in closed-loop medication infusion may yield drastic degradation in performance in case dose–response delay is not negligibly small. Although semi-APPC with nominal and worst-case $L$ were superior to population-based PID control, they fell short of semi-APPC in both transient and steady-state response characteristics, implying that the use of a population-based transport delay is not ideal in closed-loop medication infusion control.

It may be of interest to compare the proposed semi-APPC with existing work on the design of closed-loop medication infusion controllers accounting for transport delay [35–39]. Hahn et al. [35] and Mendez et al. [36] used a nominal value of the transport delay. Hence, it may at best perform comparably to semi-APPC with nominal transport delay. Sawaguchi et al. [37] and Soltesz et al. [39] performed one-time, batch individualization of transport delay. Hence, as long as the transport delay does not vary largely in time (e.g., for a short-duration medication therapy), the efficacy of these controllers may be comparable to semi-APPC with nominal worst-case transport delay and even semi-APPC. But, these controllers may suffer from limitations in case the transport delay varies largely in time (e.g., for a long-duration medication therapy), since adverse impact of the inaccuracy associated with the transport delay in the closed-loop control algorithms may become pronounced. Ionescu et al. [38] performed on-line adaptation of transport delay but presumably not the remaining parameters. Hence, it may not be as effective as semi-APPC, which on-line adapts all the high-sensitivity parameters in the plant model.

All in all, the results suggest that (1) transport delay may need to be accounted for in medication infusion control problems, and that (2) transport delay may even need to be adapted to secure robustness in control performance despite uncertainty and variability in dose–response relationship.

## Conclusion and Future Work

In this paper, we investigated a semi-adaptive indirect pole placement control for infusion of medications with large transport delay in dose–response dynamics. It was illustrated that (1) incorporating transport delay in control design is beneficial, and that (2) transport delay may even need to be adapted, in case it is large, to ensure consistent clinical effect response against patient variability and uncertainty.

This work presented in this paper has several limitations that must be further investigated in future work. First, the semi-APPC designed in this work was based on an approximated plant model (i.e., dose–response model with Padé-approximated transport delay), which is not identical to the original plant model. Thus, the stability results obtained in this work (which is valid for the approximated plant model but not for the original plant model) are not theoretically rigorous. In this paper, we conducted in silico simulation under a wide range of subjects and desired responses to demonstrate the efficacy and robustness of the proposed semi-APPC (see In Silico Implementation and Simulation for details). Yet, future work must be conducted for more rigorous theoretical investigation of the semi-APPC with transport delay. Second, the proposed semi-APPC was illustrated only in a single case scenario, and its generalizability to a range of medication infusion problems with non-negligible transport delay is yet to be shown. Hence, future work must be conducted to evaluate the efficacy of the proposed semi-APPC in other appropriate medication infusion problems.

## Funding Data

This work was supported by the U.S. Office of Naval Research (ONR) under the Young Investigator Program Award (ONR #N000141410591, #N000141512018; Funder ID: 10.13039/100000006). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the ONR.

## References

^{*}≤2