Abstract

This paper proposes a control algorithm for stable implementation of asynchronous parallel quadratic programing (PQP) through a dual decomposition technique. In general, distributed and parallel optimization requires synchronization of data at each iteration step due to the interdependency of data. The synchronization latency may incur a large amount of waiting time caused by an idle process during computation. We aim to mitigate this synchronization penalty in PQP problems by implementing asynchronous updates of the dual variable. The price to pay for adopting asynchronous computing algorithms is the unpredictability of the solution, resulting in a tradeoff between speedup and accuracy. In the worst case, the state of interest may become unstable owing to the stochastic behavior of asynchrony. The stability condition of asynchronous PQP problems is investigated by employing the switched system framework. A formal algorithm is provided to ensure the asymptotic stability of dual variables. Furthermore, it is shown that the implementation of the proposed algorithm guarantees the uniqueness of optimal solutions, irrespective of asynchronous behavior. To verify the validity of the proposed methods, simulation results are presented.

References

References
1.
Dantzig
,
G. B.
, and
Wolfe
,
P.
,
1960
, “
Decomposition Principle for Linear Programs
,”
Oper. Res.
,
8
(
1
), pp.
101
111
. 10.1287/opre.8.1.101
2.
Benders
,
J. F.
,
1962
, “
Partitioning Procedures for Solving Mixed-Variables Programming Problems
,”
Numer. Math.
,
4
(
1
), pp.
238
252
. 10.1007/BF01386316
3.
Fortin
,
M.
, and
Glowinski
,
R.
,
2000
,
Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems
,
Elsevier
,
New York
.
4.
Aguiar
,
P.
,
Xing
,
E. P.
,
Figueiredo
,
M.
,
Smith
,
N. A.
, and
Martins
,
A.
,
2011
, “
An Augmented Lagrangian Approach to Constrained Map Inference
,”
Proceedings of the 28th International Conference on Machine Learning (ICML-11)
,
Bellevue, WA
,
June 28–July 2
, pp.
169
176
.
5.
Shi
,
W.
,
Ling
,
Q.
,
Yuan
,
K.
,
Wu
,
G.
, and
Yin
,
W.
,
2014
, “
On the Linear Convergence of the ADMM in Decentralized Consensus Optimization
,”
IEEE Trans. Signal Process.
,
62
(
7
), pp.
1750
1761
. 10.1109/TSP.2014.2304432
6.
Chen
,
C.
,
He
,
B.
,
Ye
,
Y.
, and
Yuan
,
X.
,
2016
, “
The Direct Extension of ADMM for Multi-Block Convex Minimization Problems is Not Necessarily Convergent
,”
Math. Program.
,
155
(
1–2
), pp.
57
79
. 10.1007/s10107-014-0826-5
7.
Bregman
,
L. M.
,
1967
, “
The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming
,”
USSR Comput. Math. Math. Phys.
,
7
(
3
), pp.
200
217
. 10.1016/0041-5553(67)90040-7
8.
Yin
,
W.
,
Osher
,
S.
,
Goldfarb
,
D.
, and
Darbon
,
J.
,
2008
, “
Bregman Iterative Algorithms for ℓ1-Minimization With Applications to Compressed Sensing
,”
SIAM J. Imag. Sci.
,
1
(
1
), pp.
143
168
. 10.1137/070703983
9.
Combettes
,
P. L.
, and
Pesquet
,
J.-C.
,
2007
, “
A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery
,”
IEEE J. Sel. Top. Signal Process.
,
1
(
4
), pp.
564
574
. 10.1109/JSTSP.2007.910264
10.
Tseng
,
P.
,
1997
, “
Alternating Projection-Proximal Methods for Convex Programming and Variational Inequalities
,”
SIAM J. Optim.
,
7
(
4
), pp.
951
965
. 10.1137/S1052623495279797
11.
Boyd
,
S.
,
Parikh
,
N.
,
Chu
,
E.
,
Peleato
,
B.
, and
Eckstein
,
J.
,
2011
, “
Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers
,”
Found. Trends® Mach. Learn.
,
3
(
1
), pp.
1
122
.
12.
Bertsekas
,
D. P.
, and
Tsitsiklis
,
J. N.
,
1989
,
Parallel and Distributed Computation: Numerical Methods
,
Prentice-Hall, Inc.
,
Englewood Cliffs, NJ
.
13.
Liu
,
J.
,
Wright
,
S. J.
,
,
C.
,
Bittorf
,
V.
, and
Sridhar
,
S.
,
2015
, “
An Asynchronous Parallel Stochastic Coordinate Descent Algorithm
,”
J. Mach. Learn. Res.
,
16
(
1
), pp.
285
322
.
14.
Lee
,
K.
,
Bhattacharya
,
R.
,
Dass
,
J.
,
Sakuru
,
V. P.
, and
Mahapatra
,
R. N.
,
2016
, “
A Relaxed Synchronization Approach for Solving Parallel Quadratic Programming Problems With Guaranteed Convergence
,”
2016 IEEE International Parallel and Distributed Processing Symposium
,
Chicago, IL
,
May 23–27
,
IEEE
, pp.
182
191
.
15.
Lee
,
K.
, and
Bhattacharya
,
R.
,
2016
, “
On the Relaxed Synchronization for Massively Parallel Numerical Algorithms
,”
2016 American Control Conference (ACC)
,
Boston, MA
,
July 6–8
,
IEEE
, pp.
3334
3339
.
16.
Ghosh
,
S.
,
Lu
,
J.
,
Gupta
,
V.
, and
Tryggvason
,
G.
,
2018
, “
Fast Parallel Computation Using Periodic Synchronization
,”
2018 Annual American Control Conference (ACC)
,
Milwaukee, WI
,
June 27–29
,
IEEE
, pp.
1659
1664
.
17.
Fang
,
Y.
, and
Loparo
,
K. A.
,
2002
, “
Stochastic Stability of Jump Linear Systems
,”
IEEE Trans. Autom. Control
,
47
(
7
), pp.
1204
1208
. 10.1109/TAC.2002.800674
18.
Fang
,
Y.
, and
Loparo
,
K. A.
,
2002
, “
Stabilization of Continuous-Time Jump Linear Systems
,”
IEEE Trans. Autom. Control
,
47
(
10
), pp.
1590
1603
. 10.1109/TAC.2002.803528
19.
Lee
,
K.
, and
Bhattacharya
,
R.
,
2018
, “
Optimal Controller Switching for Resource-Constrained Dynamical Systems
,”
Int. J. Control Autom. Syst.
,
16
(
3
), pp.
1323
1331
. 10.1007/s12555-017-0530-3
20.
Lee
,
K.
,
Halder
,
A.
, and
Bhattacharya
,
R.
,
2015
, “
Performance and Robustness Analysis of Stochastic Jump Linear Systems Using Wasserstein Metric
,”
Automatica
,
51
(
1
), pp.
341
347
. 10.1016/j.automatica.2014.10.080
21.
Lee
,
K.
,
Bhattacharya
,
R.
, and
Gupta
,
V.
,
2015
, “
A Switched Dynamical System Framework for Analysis of Massively Parallel Asynchronous Numerical Algorithms
,”
2015 American Control Conference (ACC)
,
Chicago, IL
,
July 1–3
,
IEEE
, pp.
2638
2643
.
22.
Lee
,
K.
, and
Bhattacharya
,
R.
,
2015
, “
Stability Analysis of Large-Scale Distributed Networked Control Systems With Random Communication Delays: A Switched System Approach
,”
Syst. Control Lett.
,
85
(
11
), pp.
77
83
. 10.1016/j.sysconle.2015.08.011
23.
Feng
,
X.
,
Loparo
,
K. A.
,
Ji
,
Y.
, and
Chizeck
,
H. J.
,
1992
, “
Stochastic Stability Properties of Jump Linear Systems
,”
IEEE Trans. Autom. Control
,
37
(
1
), pp.
38
53
. 10.1109/9.109637
24.
Costa
,
O. L.
, and
Fragoso
,
M. D.
,
1993
, “
Stability Results for Discrete-Time Linear Systems With Markovian Jumping Parameters
,”
J. Math. Anal. Appl.
,
179
(
1
), pp.
154
178
. 10.1006/jmaa.1993.1341
25.
Lee
,
K.
, and
Bhattacharya
,
R.
,
2019
, “
On the Uniqueness of Stationary Solutions of an Asynchronous Parallel and Distributed Algorithm for Diffusion Equations
,”
2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC)
,
Las Vegas, NV
,
Jan. 7–9
,
IEEE
, pp.
0062
0068
.
You do not currently have access to this content.