A method of controller restructuring is introduced for improved closed-loop control of nonlinear plants. In this method, an initial controller, potentially the linear controller designed according to the linearized model of the plant, is expanded into several candidate nonlinear control structures that are subsequently shaped to achieve a desired closed-loop response. The salient feature of the proposed method is a metric for quantifying structural perturbations to the controllers, which it uses to scale the structural Jacobian for improving its condition number. This improved Jacobian underlies shaping of candidate controllers through gradient-based search. Results obtained from three case studies indicate the success of the proposed restructuring method in finding nonlinear controllers that improve not only the closed-loop response of the nonlinear plant but also its robustness to modeling uncertainty.

## Introduction

When the agility of feedback can compensate for mild plant nonlinearities, linear controllers designed according to the linearized model of the plant will suffice [1]; and in cases when the plant nonlinearities are too severe for a single linear controller across the range of operating points, gain scheduling can be employed to incorporate different linear controllers at different operating points [2]. The leap to nonlinear control can be made, for improved performance, when accurate models of plant nonlinearities exist to allow nonlinear controller design [3–5]. This paper offers an alternative method of empirical controller development wherein a starting, generally linear, controller is expanded into a nonlinear controller with coupled components to attain improved closed-loop performance.

The most common platform for empirical development of nonlinear controllers has been neural networks [6–9]. However, these controllers have a “black box” form precluding analysis that requires the transparency of form/structure. In an attempt to attain transparency, one can use symbolic regression wherein the process variables, inputs, and parameters (constants) are treated as symbols and integrated as blocks to form candidate models. Free of restrictions on the form (structure) of candidate controllers, the search can be conducted by genetic programming (GP) for controllers generating best-fit closed-loop outputs to the desired response [10]. However, symbolic regression is computationally expensive, requiring anywhere from thousands to billions of evaluations. While so many evaluations can be accommodated in open-loop by algebraic manipulation of the time series representing measured observations and their derivatives, they are infeasible in closed-loop wherein the system response needs to be obtained via simulation for each adopted controller. As such, the use of evolutionary and/or genetic algorithms in controls has been confined to parameter optimization [11,12] or search among a limited number of structural components [13,14].

Whereas the method proposed in this paper also restricts the search space to a limited number of candidate controllers, it formulates them by restructuring an initial controller instead of relying on preformulated fixed structures. Furthermore, it incorporates pliability in these restructured controllers by inclusion of exponents that can be adapted toward their suitable form. The adaptation of these exponents, which amounts to a local search around the initial controller, is performed by the model structure adaptation method (MSAM) [15]. A key feature of MSAM, that enables the implementation of gradient-based adaptation as its search mechanism, is its quantification of structural changes to the controllers. MSAM uses this metric to scale the structural sensitivities such that they will remain robust to parametric error during adaptation. The proposed controller restructuring is schematized in Fig. 1, which resembles the strategy used in iterative feedback tuning (IFT) [16–19]. In this scheme, *G* represents the nonlinear plant and *G _{c}* the controller. Whereas in IFT the parameters of

*G*are adjusted/tuned, in MSAM a candidate set of controller formats with pliable structures are considered which are adapted iteratively to produce the desired response

_{c}*y*to the reference input

^{d}*r*. Therefore, MSAM differs from iterative tuning in that it changes the controller structure instead of just its parameters toward the desired response. In Fig. 1,

*u*denotes the control effort,

*n*the measurement noise, and $y\u0303$ represents the error between the closed-loop response of the system $y\u0302$ and its desired response

*y*.

^{d}## Model Structure Adaptation Method

Model structure adaptation method, which is a gradient-based method of symbolic adaptation for dynamic models [15,20], begins with an initial controller. It uses the components of this initial controller and creates pairwise coupling between the individual components, amended by exponents. It then forms a set of restructured candidate controllers from different combinations of coupled components and adapts the exponents of each controller by gradient-based search to optimize the influence of individual couplings. It subsequently evaluates the performance of these candidate controllers in closed-loop to find the best controller that surpasses the initial controller in matching the desired response *y ^{d}*. The salient feature of MSAM is its use of a metric for symbolic changes to the model, which it uses for scaling the structural Jacobian. This scaling is shown to improve the condition number of the structural Jacobian, for reliable implementation of gradient-based adaptation in the symbolic domain.

*M*, as

_{i}**M**= [

*M*

_{1},…,

*M*]

_{Q}^{T}comprises components

*M*that are products of combinations of state variables

_{i}*x*included in the state vector

_{i}**x**= [

*x*

_{1},…,

*x*]

_{n}^{T}. For instance, with a proportional-integral-derivative controller as the initial controller $M\Theta \u0303=Kp\u03f5(t)+Ki\u222b\u03f5(t)dt+Kdd\u03f5/dt;\u2009\u03f5(t)=r(t)\u2212y\u0302(t);\u2009M=[M1\u2009M2\u2009M3]T=[\u03f5(t),\u222b\u03f5(t)dt,\u2009d\u03f5/dt]T$ with the corresponding parameter values $\Theta =[\theta 1\u2009\theta 2\u2009\theta 3]T=[Kp,\u2009Ki,\u2009Kd]T$. The fidelity of the controller can be evaluated by how closely the closed-loop response of the nonlinear plant matches the desired response

*y*, as represented by their difference $y\u0303M\u0302$ where $M\u0302$ denotes the candidate controller. The fitness function in MSAM is often defined as

^{d}where $\rho (y\u0302,yd)$ denotes the correlation coefficient between the closed-loop response $y\u0302$ and the desired response *y ^{d}*, computed as $\rho (y\u0302,yd)=Cy\u0302yd/\sigma y\u0302\sigma yd$ where $Cy\u0302yd$ is the covariance of $y\u0302$ and

*y*, and

^{d}*σ*denotes standard deviation. The larger the fitness value, the closer the closed-loop response is to its target; therefore, this fitness function is used primarily to evaluate the fitness of various candidate controllers in the first stage of adaptation by MSAM. It should be noted here that $y\u0302$ is not only a function of the controller structure $M\u0302$ and its parameters $\Theta $ but also the reference

*r*, the plant

*G*, and noise

*n*. Given that $y\u0303$, in addition to its role in the fitness function, is the basis for adaptation of the candidate controller $M\u0302$, it is imperative to have persistency of excitation [6] by $y\u0303(t)$.

With the commonality of *r*, *G*, and *y ^{d}* among the candidate controllers, the output error $y\u0303$ is a function of the candidate controller $M\u0302$ and its parameters $\Theta $. If one assumes that an ideal controller

**M**

^{*}with the ideal parameters $\Theta *$ exists that could generate the desired response

*y*, then the output error $y\u0303$ is mainly caused by the structural mismatch; i.e., $M\u0303\u2260M*$ as well as the parametric error $\Delta \Theta \u0303=\Theta *\u2212\Theta \u0303$. In IFT [16,21,22], the controller form is assumed correct and the model parameters are tuned to reduce $y\u0303$. However, when the controller form is incorrect (i.e., $M\u0303\u2260M*$), parameter tuning will be superficial. Since structural accuracy of the controller transcends its parametric accuracy, MSAM focuses on structural adaptation of

^{d}*G*.

_{c}where $M\u0302=[M\u03031f\u03021(x)\gamma 1,\u2009\u2026,\u2009M\u0303Qf\u0302Q(x)\gamma Q]T$, the $f\u0302i$ are functions of individual state variables, such as $|xi|,\u2009sign(xi),\u2009\u2009and\u2009\u2009cos(xi)$, considered to improve the controller form, and the $\gamma i\u2208\u211c$ are exponents to achieve two goals: (i) to mitigate the discrete nature of the introduced model change and (ii) to provide a mechanism for calibrating the degree of change to individual model components for higher granularity. For instance, to restructure a proportional integral derivative controller into the nonlinear form $Kp\u03f5(t)|d\u03f5/dt|\gamma +Ki\u222b\u03f5(t)dt+Kdd\u03f5/dt$, the first component $M\u03031=\u03f5(t)$ needs to be changed to $M\u03021=\u03f5(t)|d\u03f5/dt|\gamma $. Assuming that the ideal controller structure **M**^{*} can be reached by the introduction of adjustments $f\u0302$ to the initial controller structure $M\u0303$, the ideal controller will have the form $M*=[M\u03031f1*(x)\gamma 1*,\u2009\u2026,\u2009M\u0303QfQ*(x)\gamma Q*]T$.

Hence, the adaptation strategy entails applying adjustments of the form (3) to individual components of the initial controller $M\u0303$ during a round robin stage, and then adapting the exponent *γ _{i}* to fine-tune the controller structure. The goal of MSAM is to mainly find the form $f*=[f1*(x),\u2026,fQ*(x)]T$, in the first stage of adaptation, called round robin, and then fine-tune the exponents

*γ*, to achieve $\Gamma =[\gamma 1,\u2026,\gamma Q]T\u21d2\Gamma *=[\gamma 1*,\u2026,\gamma Q*]T$. For illustration purposes, selection of the best candidate controller in the first stage, followed by its adaptation in the second stage, is shown in Fig. 2. The plots in the first stage represent the fitness values of the candidate controllers during the first 15 iterations of adaptation. The inferior controllers are discarded for the second stage where adaptation is continued toward fine-tuning the exponents of the best-fit controller.

_{i}*θ*,

_{i}*γ*pairs often hinders their concurrent adaptation, only the exponents are adapted iteratively for their larger influence on the error (in the absence of bifurcation) [15,20]. Here, a key contribution of MSAM [15] is its introduction of the “model perturbation magnitude”

_{i}*δM*to quantify model changes affected by perturbations to the exponents

_{i}*γ*in Eq. (3), as

_{i}in lieu of *δγ _{i}* in the denominator of the finite difference approximation of the output sensitivity.

*γ*according to nonlinear least-squares, as

_{i}where $y\u0303N$ is the vector of sampled output error, *q* is the iteration number, and *μ*(*q*) is the adaptation step size, determined at each iteration.

## Study Platforms

Three closed-loop platforms are considered for studying the feasibility of MSAM. The first platform, depicted in Ref. [2], consists of a linear plant that is actuated by a nonlinear valve, representing a compartmentalized plant nonlinearity. Åström and Wittenmark [2] capitalize on knowledge of the actuator nonlinearity to cascade the linear (proportional plus integral (PI)) controller with the inverse function of the actuator model so as to neutralize/compensate for its nonlinearity. The PI controller was restructured by MSAM to replace the controller and cascaded inverse function. The second platform is the benchmark control of an inverted pendulum on a cart which presents an inherently nonlinear and unstable plant commonly controlled within small deviations from the vertical position. These two platforms are used to study the characteristics of the restructured controllers. Since it is also prudent to compare the performance of restructured controllers to those of nonlinear controllers, control of an inverted pendulum is also considered as the third platform. For this platform, a nonlinear control solution according to feedback linearization is available from Ref. [5] to provide a basis for evaluating the restructured controller's performance against that of a nonlinear controller.

### Nonlinear Actuator.

The first platform, adopted from Ref. [2], is shown in Fig. 3 where the plant consists of a nonlinear actuator, preceded by a linear process. The customized controller discussed in Ref. [2] is a PI controller with the parameters *K _{p}* = 0.1 and

*T*= 0.1 cascaded with a nonlinear function that approximates the inverse of the actuator model. The nonlinear actuator model, the transfer function of the process, and the inverse actuator model used in Ref. [2] are shown in Table 1.

_{i}Nonlinear actuator | $v=f(u)=u4$ |

Process | $G0(s)=1(s+1)3$ |

Inverse model | $f\u22121(c)={0.433cif\u20090\u2264c<30.0538c+1.139if\u20093\u2264c\u226416$ |

Nonlinear actuator | $v=f(u)=u4$ |

Process | $G0(s)=1(s+1)3$ |

Inverse model | $f\u22121(c)={0.433cif\u20090\u2264c<30.0538c+1.139if\u20093\u2264c\u226416$ |

As discussed in Ref. [2], and shown in Fig. 4, the above closed-loop system generates different responses at different reference values, representing the limitation of the inverse approximation $f\u0302\u22121$ in neutralizing the actuator nonlinearity *f*(*u*) at different reference values. A drawback of this solution, therefore, is rooted in the deviation of $f(f\u0302\u22121(c))$ from the ideal value of 1 at different reference values, except at *r* = 1 where the inverse function is exact and the response obtained is desired. Another drawback of this solution is its dependence on the accuracy of the modeled nonlinearity. To evaluate the significance of this dependence, the closed-loop step responses of the system at different reference values are compared in Fig. 5 with the step responses of two other systems representing slightly different actuator nonlinearities: *f*(*u*) = *u*^{3.5} and *f*(*u*) = *u*^{4.5}. The results clearly indicate the considerable influence of misrepresented nonlinearity on the responses of the customized solution, particularly at higher reference values.

### Inverted Pendulum on a Cart.

The second platform, obtained from Ref. [23], is the classical inverted pendulum on a cart, as shown in Fig. 6 and modeled in Table 2. In this model, *x*(*t*) denotes the position of the cart in the *x* direction, *θ*(*t*) denotes the angle of the pendulum from vertical, and *u*(*t*) is the force applied to the cart. This model was simulated with the cart mass $m\u2032=0.9\u2009kg$, the pendulum mass at the end of the massless rod represented as *m* = 0.1 kg, and the pendulum length represented as *l* = 0.235 m.

$x\xa8=u+ml(sin(\theta ))\theta \u02d92\u2212mg\u2009cos(\theta )sin(\theta )m\u2032+m\u2212m\u2009cos2(\theta )\theta \xa8=u\u2009cos(\theta )\u2212(m\u2032+m)g\u2009sin(\theta )+ml(cos(\theta )sin(\theta ))\theta \u02d9ml\u2009cos2(\theta )\u2212(m\u2032+m)l$ |

$x\xa8=u+ml(sin(\theta ))\theta \u02d92\u2212mg\u2009cos(\theta )sin(\theta )m\u2032+m\u2212m\u2009cos2(\theta )\theta \xa8=u\u2009cos(\theta )\u2212(m\u2032+m)g\u2009sin(\theta )+ml(cos(\theta )sin(\theta ))\theta \u02d9ml\u2009cos2(\theta )\u2212(m\u2032+m)l$ |

The feature of interest to our study in this platform is the effectiveness of restructured controller in coping with plant nonlinearity beyond angles regulated by the linear controller. At small *θ* values, like those caused by low magnitude impulses to the pendulum, a linear controller, by state feedback, for example, can maintain the upward position of the pendulum. But the prevalence of nonlinearity at larger *θ* values will disturb the performance of linear control. This point is illustrated for a linear state-feedback controller of the form $u(t)=\u2212K1x\u2212K2x\u02d9\u2212K3\theta \u2212K4\theta \u02d9$ with the gains [*K*_{1}, *K*_{2}, *K*_{3}, *K*_{4}] = [−2.00, −3.84, 33.84, 7.22] locating the closed-loop poles at *s*_{1,2,3,4} = −1, −2, −4.73, −4.73 according to the linearized model of the pendulum. The closed-loop impulse responses of the pendulum to different impulse magnitudes applied to the pendulum using this controller are shown in Fig. 7. They clearly indicate the effect of nonlinearity on the performance of the linear controller at higher impulse magnitudes.

### Inverted Pendulum.

The model of the inverted pendulum and the nonlinear controller developed by feedback linearization in Ref. [5] are shown in Table 3. In this model, *u* is the input, $a=(\omega /\Omega )2=1,\u2009\omega 2=mgl/(J+ml2)$ is the oscillation frequency, *J* is the moment of inertia of the pendulum, and Ω is a nominal value of *ω*.

## Restructured Controllers

Controllers were restructured by MSAM for the three platforms according to the configuration in Fig. 1. The desired response *y ^{d}* used for the nonlinear actuator was the step response of a standard second-order model, the one for the inverted pendulum on a cart was the impulse response of the linear controller to the lowest magnitude impulse (

*δ*= 15 N) applied to the pendulum, and the one for the inverted pendulum was an exponentially decayed response of the linear controller. The coupling functions $f\u0302i$ in Eq. (3) for the first and second platforms were the absolute values of the state variables, to avoid imaginary numbers due to exponentiation of negative numbers, and for the third platform were the sine and cosine of the state variables. The restructured controllers obtained for the above platforms are discussed separately.

### Controller for the Nonlinear Actuator.

A feature of restructured controllers is their case specificity, which is rooted in the search mechanism for the exponents *γ _{i}* in Eq. (3). As in any gradient-based search, the robustness of the solution and its form depend not only on the convexity of the error surface presented during training, but also on the search mechanism (nonlinear least squares, in this case). As such, the choice of the desired response

*y*plays a central role in the formulation of the solution. It is observed, for instance, that the more distant is the target from the initial closed-loop response, the better chance there is of finding a radically restructured controller. For case specificity of restructured controllers, consider the controllers obtained at different reference magnitudes for the nonlinear actuator in Table 4. Here we arbitrarily used the step response of a standard second-order model (

^{d}*ζ*= 1,

*ω*= 0.17) as the desired response and the PI controller: $Kp\u03f5(t)+Ki\u222b\u03f5(t)dt$ as the initial controller amended with the functions $[f1,f2]=[|\u03f5|,|\u222b\u03f5dt|]$ in Eq. (3) for its restructuring. Each candidate controller was adapted for 15 iterations in the round robin phase and the best controller was further adapted for 20 more iterations in the final phase. Although the forms of the restructured controllers in Table 4 are the same for reference magnitudes of 1, 2, and 4, in one form, and for reference magnitudes of 3 and 5, in another form, they are not uniform across all reference magnitudes.

_{n}Reference value | Restructured controller |
---|---|

1 | $Kp\u03f5(|\u222b\u03f5dt|)0.27+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.80$ |

2 | $Kp\u03f5(|\u222b\u03f5dt|)0.19+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.82$ |

3 | $Kpsgn(\u03f5)|\u03f5|1.15+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.81$ |

4 | $Kp\u03f5(|\u222b\u03f5dt|)0.15+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.78$ |

5 | $Kpsgn(\u03f5)|\u03f5|1.08+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.78$ |

Reference value | Restructured controller |
---|---|

1 | $Kp\u03f5(|\u222b\u03f5dt|)0.27+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.80$ |

2 | $Kp\u03f5(|\u222b\u03f5dt|)0.19+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.82$ |

3 | $Kpsgn(\u03f5)|\u03f5|1.15+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.81$ |

4 | $Kp\u03f5(|\u222b\u03f5dt|)0.15+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.78$ |

5 | $Kpsgn(\u03f5)|\u03f5|1.08+Kisgn(\u222b\u03f5dt)(|\u222b\u03f5dt|)0.78$ |

To ameliorate their uniformity, restructuring of the controller for the first platform was performed with a staircase reference profile that included three reference magnitudes, as shown in Fig. 8. Controller restructuring resulted in

with its response named “restructured” in Fig. 8. The response of the restructured controller is compared in Fig. 9 with those of the initial (PI) and customized (PI controller cascaded with the inverse model of the actuator) controllers. The results indicate more consistent rise times of the initial and restructured controllers than the customized controller. They also indicate the far smaller overshoot of the restructured controller than the initial controller's, as the result of restructuring toward the desired response.

As discussed earlier, an important feature of MSAM is the use of *δM _{i}* in Eq. (5) for scaling the columns of $\Phi \gamma $ in Eq. (6). A direct ramification of this scaling is ought to be the better quality of $\Phi \gamma $, that results in improved estimates of $\Delta \Gamma \u0302$ when used in Eq. (7). The quality of $\Phi \gamma $ is illustrated by the range of condition numbers (

*λ*

_{max}/

*λ*

_{min}) of $\Phi \gamma $ in Table 5, computed with and without scaling by

*δM*at different reference magnitudes with the nonlinear actuator. Since the closer is the condition number to unity the more separate (less collinear) are the columns of the matrix [24], the smaller condition numbers in Table 5 for $\Phi \gamma $ when scaled by

_{i}*δM*should result in improved restructured controllers. This is verified by the smaller lowest absolute output error sums in Table 5 obtained during adaptation by scaling. Supported by these results, the solutions shown henceforth are obtained with scaled $\Phi \gamma $.

_{i}Condition number of Φ _{γ} | Lowest error ($min\u2211i=1N|y\u0303(ti)|$) | |||
---|---|---|---|---|

Reference Magnitude | Unscaled | Scaled | Unscaled | Scaled |

1 | 1.61–12.16 | 2.02–2.07 | 2.61 | 1.35 |

2 | 1.69–6.95 | 1.80–2.68 | 4.37 | 2.50 |

3 | 2.13–4.94 | 1.07–4.69 | 6.10 | 2.65 |

4 | 10.03–14.05 | 1.09–2.67 | 8.18 | 3.99 |

5 | 13.37–13.53 | 1.09–4.48 | 11.38 | 6.10 |

Condition number of Φ _{γ} | Lowest error ($min\u2211i=1N|y\u0303(ti)|$) | |||
---|---|---|---|---|

Reference Magnitude | Unscaled | Scaled | Unscaled | Scaled |

1 | 1.61–12.16 | 2.02–2.07 | 2.61 | 1.35 |

2 | 1.69–6.95 | 1.80–2.68 | 4.37 | 2.50 |

3 | 2.13–4.94 | 1.07–4.69 | 6.10 | 2.65 |

4 | 10.03–14.05 | 1.09–2.67 | 8.18 | 3.99 |

5 | 13.37–13.53 | 1.09–4.48 | 11.38 | 6.10 |

### Controller for the Inverted Pendulum on a Cart.

*δ*= 18 (see Fig. 7) was applied to the cart, using the closed-loop response of the linear controller to an impulse magnitude of

*δ*= 15 as the desired response. Each candidate controller was adapted for 15 iterations in the round robin phase and the best controller was adapted for 50 iterations in the final phase. The restructured controller had the form

The responses and control efforts of the restructured and linear controllers at the impulse magnitude of *δ* = 18 are shown in Fig. 10 along with the desired response. They indicate the more rapid response than its linear counterpart of the restructured controller in stabilizing the pendulum.

As benchmark, the impulse responses of the inverted pendulum on a cart with the restructured controller (Eq. (10)) are compared with those of the linear controller at different impulse magnitudes in Fig. 11. Both the responses and control efforts of the restructured controller are significantly more robust than those of the linear controller at different impulse magnitudes. This robustness is due in part to the quicker response of the restructured controller to state changes in the system, providing the capacity to cope with impulses of higher magnitude, as discussed in Sec. 5.

### Controller for the Inverted Pendulum.

*y*

_{lin}being the closed-loop response of the linear controller. The candidate controllers were generated from the linear controller: $u=\u2212K1x1\u2212K2x2$, yielding the restructured controller of the form

The stabilizing responses of the linear and restructured controllers and their control efforts to an initial pendulum displacement are shown in Fig. 12. The restructured controller response indicates that the desired response cannot be achieved in this case and that not much improvement is attained by restructuring the linear controller. However, as will be illustrated later, this controller proves to be considerably more robust than the linear controller in response to larger initial displacements.

## Analysis

The case study results obtained can be used to analyze several aspects of the restructured controllers by MSAM. One such aspect is the response of the restructured controllers to conditions absent in training, such as measurement noise, disturbances, and reference magnitudes beyond those used for training. A second aspect is the sensitivity of the restructured controllers to training conditions. A third aspect is the form and behavior of restructured components of the controllers in comparison to their initial counterparts.

### Unrepresented Conditions.

To evaluate the performance of restructured controllers in the presence of noise, band-limited noise at the signal-to-noise ratio of 18 (at *r* = 1) to 33 (at *r* = 5) was added to the output of the plant in the nonlinear actuator platform. Controller responses were tested ten times for different random noise cases, as shown in Fig. 13. The results indicate similarly affected closed-loop responses by measurement noise of both the restructured and customized controllers with smaller variations observed in the control efforts.

The disturbance rejection capacity of the controllers was evaluated in platform one with unit step disturbances applied before and after the plant *G*_{0}(*s*) in Fig. 3. The closed-loop responses of both the restructured and customized controllers are shown in Fig. 14. The results indicate much more agile disturbance rejection by the restructured controller at higher reference magnitudes, replicating the faster step response of these controllers at higher reference magnitudes in Fig. 9.

To evaluate the controllers' regulation capacity of the first platform for levels not encountered in training, the closed-loop step responses of the restructured controller are compared to those of the customized controller at step sizes of 6–15 for the nonlinear actuator in Fig. 15. The results indicate that the restructured controller starts having oscillatory behavior at step sizes of nine and higher, while the customized solution provides continually increasing sluggish response at these higher steps. Similarly, the closed-loop impulse responses of the inverted pendulum on a cart with the restructured and linear controllers were obtained at impulse magnitudes of 21–33. The linear controller was found to be deficient in maintaining upward position for the pendulum for impulse magnitudes of 27 and higher. The responses obtained with the restructured controller for impulse magnitudes of 27–33 are shown in Fig. 16. The results in Fig. 16 reveal the ability of the restructured controller in maintaining a stable response under conditions beyond the capacity of linear control.

Consistent with these results are those obtained for the third platform. Shown in Fig. 17 are the responses and control efforts of the linear, restructured, and feedback linearized controllers for the inverted pendulum to a larger initial displacement of *θ*_{0} = 1.4 rad than that used for restructuring (*θ*_{0} = 1 rad). The results clearly indicate the robustness of the restructured controller in regulating the inverted pendulum, on par with the feedback linearized controller, and far superior to that of the linear controller.

Also of interest is the robustness of the restructured controllers to modeling uncertainty. To evaluate their robustness, the closed-loop responses for the nonlinear actuator platform were generated first with the actuator nonlinearities of *f*(*u*) = *u*^{3.5} and *f*(*u*) = *u*^{4.5}, as shown in Fig. 18. The responses of the restructured controller in Fig. 18 are quite similar, unlike those of the customized controller, even though the controller was restructured for the nominal actuator model of *f*(*u*) = *u*^{4.0}. The similarity of these responses indicates the robustness of the restructured controller to modeling uncertainty of actuator nonlinearity. Second, closed-loop responses of the inverted pendulum on a cart were obtained with 10%, 20%, and 30% smaller pendulum mass with the linear and restructured controllers, as shown in Fig. 19. The responses with the restructured controller in Fig. 19 are very close for different pendulum masses, particularly in comparison to those with the linear controller. They, like those for the nonlinear actuator, indicate the lower sensitivity of the restructured controllers to modeling uncertainty.

### Sensitivity to Training Conditions.

As was discussed earlier and depicted by the controller forms in Table 4, the training conditions influence the controller forms. For the first platform, sensitivity to training conditions was remedied by adopting a staircase format for restructuring the controllers for the nonlinear actuator. It, therefore, behooves us to examine the sensitivity of the controller forms to different staircase scenarios. Similarly, the restructured controller for the inverted pendulum on a cart was obtained at one impulse magnitude (*δ* = 18). So, it raises the question as how the controller forms differ at different impulse magnitudes. To this end, the controller forms obtained for the nonlinear actuator and inverted pendulum from different training cases are shown in Table 6. The results indicate two controller forms found across the ten different staircase combinations (e.g., 1, 2, 3; 1, 3, 5; 2, 3, 4; etc.) for the nonlinear actuator and three controller forms for the inverted pendulum at three different impulse magnitudes. The difference between the controller forms for the nonlinear actuator is in the first component wherein the *ϵ* is coupled with itself, in the first case, and with its integral, in the second case. The restructured controller forms for the inverted pendulum on a cart, however, are quite diverse and can be compared better through their simulated behavior, as presented below.

Restructured controller | |

Step sizes | Nonlinear actuator |

1, 2, 5 | $Kpsgn(\u03f5(t))|\u03f5(t)|(\gamma 1+1)+Kisgn(\u222b\u03f5dt)|\u222b\u03f5dt|(\gamma 2+1)$ |

All others | $Kp\u03f5(t)|\u222b(\u03f5dt)|\gamma 1+Kisgn(\u222b\u03f5dt)|(\u222b\u03f5dt)|(\gamma 2+1)$ |

Impulse magnitude | Inverted pendulum on a cart |

δ = 18 | $K1x(t)|\theta \u02d9(t)|\gamma 1+K2x\u02d9(t)|\theta \u02d9(t)|\gamma 2$ |

$+K3sgn\theta (t)|\theta (t)|\gamma 3+1$ | |

$+K4sgn(\theta \u02d9(t))|\theta \u02d9(t)|\gamma 4+1$ | |

δ = 19 | $K1x(t)|\theta (t)|\gamma 1+K2sgn(x\u02d9(t))|x\u02d9(t)|(\gamma 2+1)$ |

$+K3sgn(\theta (t))|\theta (t)|(\gamma 3+1)$ | |

$+K4\theta \u02d9(t)|x\u02d9(t)|\gamma 4$ | |

δ = 20 | $K1x(t)|\theta \u02d9(t)|\gamma 1+K2x\u02d9(t)|\theta (t)|\gamma 2+K3\theta (t)|x\u02d9(t)|\gamma 3$ |

$+K4\theta \u02d9(t)|\theta (t)|\gamma 4$ |

Restructured controller | |

Step sizes | Nonlinear actuator |

1, 2, 5 | $Kpsgn(\u03f5(t))|\u03f5(t)|(\gamma 1+1)+Kisgn(\u222b\u03f5dt)|\u222b\u03f5dt|(\gamma 2+1)$ |

All others | $Kp\u03f5(t)|\u222b(\u03f5dt)|\gamma 1+Kisgn(\u222b\u03f5dt)|(\u222b\u03f5dt)|(\gamma 2+1)$ |

Impulse magnitude | Inverted pendulum on a cart |

δ = 18 | $K1x(t)|\theta \u02d9(t)|\gamma 1+K2x\u02d9(t)|\theta \u02d9(t)|\gamma 2$ |

$+K3sgn\theta (t)|\theta (t)|\gamma 3+1$ | |

$+K4sgn(\theta \u02d9(t))|\theta \u02d9(t)|\gamma 4+1$ | |

δ = 19 | $K1x(t)|\theta (t)|\gamma 1+K2sgn(x\u02d9(t))|x\u02d9(t)|(\gamma 2+1)$ |

$+K3sgn(\theta (t))|\theta (t)|(\gamma 3+1)$ | |

$+K4\theta \u02d9(t)|x\u02d9(t)|\gamma 4$ | |

δ = 20 | $K1x(t)|\theta \u02d9(t)|\gamma 1+K2x\u02d9(t)|\theta (t)|\gamma 2+K3\theta (t)|x\u02d9(t)|\gamma 3$ |

$+K4\theta \u02d9(t)|\theta (t)|\gamma 4$ |

### Controller Components.

The different forms obtained for the restructured controllers raise two important questions: (1) how different are the individual components of the controller from each other in different forms and from their counterparts in the initial controller? and (2) how differently do they contribute to the total control effort? To address these questions, the numerical values of the individual components in Table 6 were obtained from simulation, as shown in Fig. 20 for the nonlinear actuator and in Fig. 21 for the inverted pendulum on a cart. The results in Fig. 20 indicate that the proportional effect “$Kpsgn(\u03f5(t))|\u03f5(t)|(\gamma 1+1)$” provides a smaller portion of the overall effort than “$Kp\u03f5(t)|\u222b\u03f5dt)|\gamma 1$,” and that it has a nonzero initial value because of its entire dependence on the “*ϵ*(*t*).” Its counterpart, however, is initially null due to its dependence on “$\u222b\u03f5dt$” before it rises rapidly to its maximum value. The integral components, which have the same form, only differ slightly due to differences in the magnitude of $\u222b\u03f5dt$ in the two simulation runs.

The results in Fig. 21, however, show a much more nuanced difference of the controller components. They differ not only in form but also in coefficient and exponent values. For instance, consider the similarity in form “*x* effort” of the restructured controller at the impulse magnitudes of *δ* = 18 and *δ* = 20, simulated in the first row of Fig. 21 (columns 1 and 3). Despite their identical form, their behavior is more different from those at *δ* = 18 and *δ* = 19 (columns 1 and 2), that are different in form. This difference is presumed to be attributed to the confluence of the other components. Another observation of interest from Fig. 21 is the similarity between the total control efforts, shown in the last row of this figure, despite the very different behavior of individual components.

## Discussion

*Stability:*As with any controller design, of concern is the stability of the closed-loop systems with restructured controllers. Fortunately, a fundamental benefit of the proposed restructuring format, as schematized in Fig. 1, is its intrinsic evaluation of the candidate controllers in simulation. Since MSAM is designed to produce a controller that is at least better than the initial controller, it disregards any candidate controllers that are inferior in performance to other candidate controllers or the initial controller. Given that the instability of the system is a natural criterion in this performance evaluation, the solutions delivered by MSAM are guaranteed to be closed-loop stable within the bounds of simulation incorporated in restructuring. Outside these bounds, analysis such as that presented in Sec. 5.1 can be used to identify instabilities unrepresented during restructuring. Analytical approaches to stability can also be used though they are outside the breadth of present study.*Reachability:*In general, MSAM is additive by nature, designed to adapt a potentially inadequate initial controller by adding coupling to its individual components. Accordingly, this method is suited to restructuring initial controllers that are simple in form, as the restructured controllers are guaranteed to be more complex than their initial version. Furthermore, MSAM operates with the assumption that a potentially superior restructured controller is reachable by prescribed adjustments to the components of the initial controller. To this end, the selection of the adjustments $f\u0302i$ in Eq. (3) is of paramount importance.*Scalability:*The scalability of MSAM depends on the number of candidate controllers considered during the round robin phase. Given that with*n*adjustments applied to*Q*components,*Q*candidate controllers need to be examined during the round robin phase, the selection process can become overwhelming if the controllers are examined sequentially. Fortunately, the examination of individual candidate controllers is independent of the others; therefore, this phase can be run in parallel, reducing the computation time to^{n}*Q*/^{n}*p*, with*p*denoting the number of processors. For large-scale problems that cannot be exhaustively searched, one can choose a subset of round robin controllers that are mechanistically plausible.*Algorithmic issues:*As with any other gradient-based search routine, the search process may be sensitive to several parameters. One such parameter is the size of the perturbation*δγ*in Eq. (5) used for computing the structural sensitivities. Another is the initial value of_{i}*μ*in Eq. (8) that is adjusted at each iteration step. A third parameter is the perturbation size of the individual parameters used for computing $\u2202y\u0302/\u2202\Theta $ in Eq. (5). Yet a fourth parameter is the fitness function used to evaluate the candidate models, currently formulated to consider the size of the error as well as the correlation of the candidate output with its target. Since the sensitivity of the search process to these parameters will depend upon the convexity of the error surface, they need to be evaluated in the context of each problem.

## Conclusion

A method of restructuring is introduced for controllers of nonlinear plants. It generates controllers that are intelligible in form, but more complex than an initial controller that is potentially inferior in performance. This method benefits from a metric for quantifying structural perturbations to controllers, which it uses to enable its reliable gradient-based adaptation of candidate controllers derived from the initial controller. The method is demonstrated in application to three benchmark problems, rendering solutions that are more effective in coping with plant nonlinearities and more robust to modeling uncertainties. They are also found to be more robust to conditions not introduced in training, including unseen reference magnitudes, noise, and disturbances.

## Acknowledgment

This work is partially supported by the NSF-sponsored IGERT: Offshore Wind Energy Engineering, Environmental Science, and Policy (Grant No. 1068864).