Abstract

This paper proposes a control algorithm for stable implementation of asynchronous parallel quadratic programing (PQP) through a dual decomposition technique. In general, distributed and parallel optimization requires synchronization of data at each iteration step due to the interdependency of data. The synchronization latency may incur a large amount of waiting time caused by an idle process during computation. We aim to mitigate this synchronization penalty in PQP problems by implementing asynchronous updates of the dual variable. The price to pay for adopting asynchronous computing algorithms is the unpredictability of the solution, resulting in a tradeoff between speedup and accuracy. In the worst case, the state of interest may become unstable owing to the stochastic behavior of asynchrony. The stability condition of asynchronous PQP problems is investigated by employing the switched system framework. A formal algorithm is provided to ensure the asymptotic stability of dual variables. Furthermore, it is shown that the implementation of the proposed algorithm guarantees the uniqueness of optimal solutions, irrespective of asynchronous behavior. To verify the validity of the proposed methods, simulation results are presented.

Issue Section:
Research Papers
Keywords:
control of communication networks, delay systems, networked control systems, switched systems
Topics:
Algorithms, Control algorithms, Stability, Computation, Delays, Optimization, Simulation results, Synchronization

References

1.
Dantzig
,
G. B.
, and
Wolfe
,
P.
,
1960
, “
Decomposition Principle for Linear Programs
,”
Oper. Res.
,
8
(
1
), pp.
101
–
111
. 10.1287/opre.8.1.101
2.
Benders
,
J. F.
,
1962
, “
Partitioning Procedures for Solving Mixed-Variables Programming Problems
,”
Numer. Math.
,
4
(
1
), pp.
238
–
252
. 10.1007/BF01386316
3.
Fortin
,
M.
, and
Glowinski
,
R.
,
2000
,
Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems
,
Elsevier
,
New York
.
4.
Aguiar
,
P.
,
Xing
,
E. P.
,
Figueiredo
,
M.
,
Smith
,
N. A.
, and
Martins
,
A.
,
2011
, “
An Augmented Lagrangian Approach to Constrained Map Inference
,”
Proceedings of the 28th International Conference on Machine Learning (ICML-11)
,
Bellevue, WA
,
June 28–July 2
, pp.
169
–
176
.
5.
Shi
,
W.
,
Ling
,
Q.
,
Yuan
,
K.
,
Wu
,
G.
, and
Yin
,
W.
,
2014
, “
On the Linear Convergence of the ADMM in Decentralized Consensus Optimization
,”
IEEE Trans. Signal Process.
,
62
(
7
), pp.
1750
–
1761
. 10.1109/TSP.2014.2304432
6.
Chen
,
C.
,
He
,
B.
,
Ye
,
Y.
, and
Yuan
,
X.
,
2016
, “
The Direct Extension of ADMM for Multi-Block Convex Minimization Problems is Not Necessarily Convergent
,”
Math. Program.
,
155
(
1–2
), pp.
57
–
79
. 10.1007/s10107-014-0826-5
7.
Bregman
,
L. M.
,
1967
, “
The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming
,”
USSR Comput. Math. Math. Phys.
,
7
(
3
), pp.
200
–
217
. 10.1016/0041-5553(67)90040-7
8.
Yin
,
W.
,
Osher
,
S.
,
Goldfarb
,
D.
, and
Darbon
,
J.
,
2008
, “
Bregman Iterative Algorithms for ℓ1-Minimization With Applications to Compressed Sensing
,”
SIAM J. Imag. Sci.
,
1
(
1
), pp.
143
–
168
. 10.1137/070703983
9.
Combettes
,
P. L.
, and
Pesquet
,
J.-C.
,
2007
, “
A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery
,”
IEEE J. Sel. Top. Signal Process.
,
1
(
4
), pp.
564
–
574
. 10.1109/JSTSP.2007.910264
10.
Tseng
,
P.
,
1997
, “
Alternating Projection-Proximal Methods for Convex Programming and Variational Inequalities
,”
SIAM J. Optim.
,
7
(
4
), pp.
951
–
965
. 10.1137/S1052623495279797
11.
Boyd
,
S.
,
Parikh
,
N.
,
Chu
,
E.
,
Peleato
,
B.
, and
Eckstein
,
J.
,
2011
, “
Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers
,”
Found. Trends® Mach. Learn.
,
3
(
1
), pp.
1
–
122
.
12.
Bertsekas
,
D. P.
, and
Tsitsiklis
,
J. N.
,
1989
,
Parallel and Distributed Computation: Numerical Methods
,
Prentice-Hall, Inc.
,
Englewood Cliffs, NJ
.
13.
Liu
,
J.
,
Wright
,
S. J.
,
Ré
,
C.
,
Bittorf
,
V.
, and
Sridhar
,
S.
,
2015
, “
An Asynchronous Parallel Stochastic Coordinate Descent Algorithm
,”
J. Mach. Learn. Res.
,
16
(
1
), pp.
285
–
322
.
14.
Lee
,
K.
,
Bhattacharya
,
R.
,
Dass
,
J.
,
Sakuru
,
V. P.
, and
Mahapatra
,
R. N.
,
2016
, “
A Relaxed Synchronization Approach for Solving Parallel Quadratic Programming Problems With Guaranteed Convergence
,”
2016 IEEE International Parallel and Distributed Processing Symposium
,
Chicago, IL
,
May 23–27
,
IEEE
, pp.
182
–
191
.
15.
Lee
,
K.
, and
Bhattacharya
,
R.
,
2016
, “
On the Relaxed Synchronization for Massively Parallel Numerical Algorithms
,”
2016 American Control Conference (ACC)
,
Boston, MA
,
July 6–8
,
IEEE
, pp.
3334
–
3339
.
16.
Ghosh
,
S.
,
Lu
,
J.
,
Gupta
,
V.
, and
Tryggvason
,
G.
,
2018
, “
Fast Parallel Computation Using Periodic Synchronization
,”
2018 Annual American Control Conference (ACC)
,
Milwaukee, WI
,
June 27–29
,
IEEE
, pp.
1659
–
1664
.
17.
Fang
,
Y.
, and
Loparo
,
K. A.
,
2002
, “
Stochastic Stability of Jump Linear Systems
,”
IEEE Trans. Autom. Control
,
47
(
7
), pp.
1204
–
1208
. 10.1109/TAC.2002.800674
18.
Fang
,
Y.
, and
Loparo
,
K. A.
,
2002
, “
Stabilization of Continuous-Time Jump Linear Systems
,”
IEEE Trans. Autom. Control
,
47
(
10
), pp.
1590
–
1603
. 10.1109/TAC.2002.803528
19.
Lee
,
K.
, and
Bhattacharya
,
R.
,
2018
, “
Optimal Controller Switching for Resource-Constrained Dynamical Systems
,”
Int. J. Control Autom. Syst.
,
16
(
3
), pp.
1323
–
1331
. 10.1007/s12555-017-0530-3
20.
Lee
,
K.
,
Halder
,
A.
, and
Bhattacharya
,
R.
,
2015
, “
Performance and Robustness Analysis of Stochastic Jump Linear Systems Using Wasserstein Metric
,”
Automatica
,
51
(
1
), pp.
341
–
347
. 10.1016/j.automatica.2014.10.080
21.
Lee
,
K.
,
Bhattacharya
,
R.
, and
Gupta
,
V.
,
2015
, “
A Switched Dynamical System Framework for Analysis of Massively Parallel Asynchronous Numerical Algorithms
,”
2015 American Control Conference (ACC)
,
Chicago, IL
,
July 1–3
,
IEEE
, pp.
2638
–
2643
.
22.
Lee
,
K.
, and
Bhattacharya
,
R.
,
2015
, “
Stability Analysis of Large-Scale Distributed Networked Control Systems With Random Communication Delays: A Switched System Approach
,”
Syst. Control Lett.
,
85
(
11
), pp.
77
–
83
. 10.1016/j.sysconle.2015.08.011
23.
Feng
,
X.
,
Loparo
,
K. A.
,
Ji
,
Y.
, and
Chizeck
,
H. J.
,
1992
, “
Stochastic Stability Properties of Jump Linear Systems
,”
IEEE Trans. Autom. Control
,
37
(
1
), pp.
38
–
53
. 10.1109/9.109637
24.
Costa
,
O. L.
, and
Fragoso
,
M. D.
,
1993
, “
Stability Results for Discrete-Time Linear Systems With Markovian Jumping Parameters
,”
J. Math. Anal. Appl.
,
179
(
1
), pp.
154
–
178
. 10.1006/jmaa.1993.1341
25.
Lee
,
K.
, and
Bhattacharya
,
R.
,
2019
, “
On the Uniqueness of Stationary Solutions of an Asynchronous Parallel and Distributed Algorithm for Diffusion Equations
,”
2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC)
,
Las Vegas, NV
,
Jan. 7–9
,
IEEE
, pp.
0062
–
0068
.
Copyright © 2020 by ASME
You do not currently have access to this content.