This paper presents a major step in the development and validation of a systematic prototype-based methodology for designing multilayer feedforward neural networks to model nonlinearities common in engineering mechanics. The applications of this work include (but are not limited to) system identification of nonlinear dynamic systems and neural-network-based damage detection. In this and previous studies (Pei, J. S., 2001, “Parametric and Nonparametric Identification of Nonlinear Systems,” Ph.D. thesis, Columbia University; Pei, J. S., and Smyth, A. W., 2006, “A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces. Part I: Formulation,” J. Eng. Mech., 132(12), pp. 1290–1300; Pei, J. S., and Smyth, A. W., 2006, “A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces. Part II: Applications,” J. Eng. Mech., 132(12), pp. 1301–1312; Pei, J. S., Wright, J. P., and Smyth, A. W., 2005, “Mapping Polynomial Fitting Into Feedforward Neural Networks for Modeling Nonlinear Dynamic Systems and Beyond,” Comput. Methods Appl. Mech. Eng., 194(42–44), pp. 4481–4505), the authors do not presume to provide a universal method to approximate any arbitrary function. Rather the focus is given to the development of a procedure which will consistently lead to successful approximations of nonlinear functions within the specified field. This is done by examining the dominant features of the function to be approximated and exploiting the strength of the sigmoidal basis function. As a result, a greater efficiency and understanding of both neural network architecture (e.g., the number of hidden nodes) as well as weight and bias values is achieved. Through the use of illuminating mathematical insights and a large number of training examples, this study demonstrates the simplicity, power, and versatility of the proposed prototype-based initialization methodology. A clear procedure for initializing neural networks to model various nonlinear functions commonly seen in engineering mechanics is provided. The proposed methodology is compared with the widely used Nguyen–Widrow initialization to demonstrate its robustness and efficiency in the specified applications. Future work is also identified.

1.
Denoeux
,
T.
, and
Lengellé
,
R.
, 1993, “
Initializing Back Propagation Networks With Prototypes
,”
Neural Networks
0893-6080,
6
, pp.
351
363
.
2.
Sandberg
,
I. W.
,
Lo
,
J. T.
,
Fancourt
,
C. L.
,
Principe
,
J. C.
,
Katagiri
,
S.
, and
Haykin
,
S.
, 2001,
Nonlinear Dynamical Systems: Feedforward Neural Network Perspectives
,
Wiley
,
New York
.
3.
Cybenko
,
G.
, 1989, “
Approximation by Superpositions of Sigmoidal Function
,”
Math. Control, Signals, Syst.
,
2
, pp.
303
314
. 0932-4194
4.
Hornik
,
K.
,
Stinchcombe
,
M.
, and
White
,
H.
, 1989, “
Multilayer Feedforward Networks are Universal Approximators
,”
Neural Networks
0893-6080,
2
,
359
366
.
5.
Jones
,
L. K.
, 1990, “
Constructive Approximations for Neural Networks by Sigmoidal Functions
,”
Proc. IEEE
0018-9219,
78
(
10
), pp.
1586
1589
.
6.
Nguyen
,
D.
, and
Widrow
,
B.
, 1990, “
Improving the Learning Speed of Two-Layer Neural Networks by Choosing Initial Values of the Adaptive Weights
,”
Proceedings of the International Joint Conference on Neural Networks
, July, Vol.
III
, pp.
21
26
.
7.
Burrows
,
T. L.
, and
Niranjan
,
M.
, 1993, “
Feed-Forward and Recurrent Neural Networks for System Identification
,”
Cambridge University Engineering Department
, Technical Report No. CUED/F-INFENG/TR158.
8.
Osowski
,
S.
, 1993, “
New Approach to Selection of Initial Values of Weights in Neural Function Approximation
,”
Electron. Lett.
,
29
(
3
), pp.
313
315
. 0013-5194
9.
Costa
,
P.
, and
Larzabal
,
P.
, 1999, “
Initialization of Supervised Training for Parametric Estimation
,”
Neural Processing Letters
, Vol.
9
,
Kluwer
,
Netherlands
, pp.
53
61
.
10.
Lapedes
,
A.
, and
Farber
,
R.
, 1988, “
How Neural Nets Work
,”
Neural Information Processing Systems
,
D.
Anderson
, ed.,
American Institute of Physics
,
New York
, pp.
442
456
.
11.
Ma
,
L.
, and
Khorasani
,
K.
, 2004, “
New Training Strategies for Constructive Neural Networks With Application to Regression Problems
,”
Neural Networks
,
17
, pp.
589
609
. 0893-6080
12.
Lehtokangas
,
M.
, 1999, “
Fast Initialization for Cascade-Corrolation Learning
,”
IEEE Trans. Neural Netw.
1045-9227,
10
(
2
), pp.
410
414
.
13.
Yam
,
J. Y. F.
, and
Chow
,
T. W. S.
, 2001, “
Feedforward Networks Training Speed Enhancement by Optimal Initialization of the Synaptic Coefficients
,”
IEEE Trans. Neural Netw.
,
12
(
2
), pp.
430
434
. 1045-9227
14.
Pei
,
J. S.
,
Wright
,
J. P.
, and
Smyth
,
A. W.
, 2005, “
Neural Network Initialization With Prototypes—A Case Study in Function Approximation
,”
Proceedings of the International Joint Conference on Neural Networks 2005 (IJCNN’05)
,
Montreal, Canada
, Jul. 31–Aug. 4, pp.
1377
1382
.
15.
Pei
,
J. S.
, 2001, “
Parametric and Nonparametric Identification of Nonlinear Systems
,” Ph.D. thesis, Columbia University, New York.
16.
Pei
,
J. S.
, and
Smyth
,
A. W.
, 2006, “
A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces. Part I: Formulation
,”
J. Eng. Mech.
0733-9399,
132
(
12
), pp.
1290
1300
.
17.
Pei
,
J. S.
, and
Smyth
,
A. W.
, 2006, “
A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces. Part II: Applications
,”
J. Eng. Mech.
,
132
(
12
), pp.
1301
1312
. 0733-9399
18.
Pei
,
J. S.
,
Wright
,
J. P.
, and
Smyth
,
A. W.
, 2005, “
Mapping Polynomial Fitting Into Feedforward Neural Networks for Modeling Nonlinear Dynamic Systems and Beyond
,”
Comput. Methods Appl. Mech. Eng.
0045-7825,
194
(
42–44
), pp.
4481
4505
.
19.
Hagan
,
M. T.
,
Demuth
,
H. B.
, and
Beale
,
M.
, 1995,
Neural Network Design
,
PWS
,
Boston
.
20.
Pei
,
J. S.
,
Smyth
,
A. W.
, and
Kosmatopoulos
,
E. B.
, 2004, “
Analysis and Modification of Volterra/Wiener Neural Networks for Identification of Nonlinear Hysteretic Dynamic Systems
,”
J. Sound Vib.
0022-460X,
275
(
3–5
),
693
718
.
21.
Nelles
,
O.
, 2000,
Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models
,
Springer-Verlag
,
Berlin
.
22.
Adams
,
D. E.
, and
Allemang
,
R. J.
, 2000, “
Non-Linear Vibrations Classnotes
,” Course No. 20-263-781.
23.
Worden
,
K.
, and
Tomlinson
,
G. R.
, 2001,
Nonlinearity in Structural Dynamics: Detection, Identification and Modelling
,
Institute of Physics
,
New York
, p.
680
.
24.
Wright
,
J. P.
,
Pei
,
J. S.
, and
Mai
,
E. C.
, 2008, “
A Growing Neural Network Approach in Engineering Mechanics Applications
,”
Comput. Methods Appl. Mech. Eng.
0045-7825, to be submitted.
25.
Pei
,
J. S.
, and
Mai
,
E. C.
, 2006, “
Neural Network Initialization for Modeling Nonlinear Functions in Engineering Mechanics
,”
Proceedings of the 24th International Modal Analysis Conference (IMAC XXIV)
.
26.
Pei
,
J. S.
, and
Mai
,
E. C.
, 2006, “
A Heuristic Neural Network Initialization Scheme for Modeling Nonlinear Functions in Engineering Mechanics
,” SPIE International Symposia Smart Structures, Materials/NDE.
27.
Pei
,
J. S.
,
Mai
,
E. C.
, and
Piyawat
,
K.
, 2006, “
Multilayer Feedforward Neural Network Initialization Methodology for Modeling Nonlinear Restoring Forces and Beyond
,” Fourth World Conference on Structural Control and Monitoring.
28.
Haykin
,
S. S.
, 1998,
Neural Networks: A Comprehensive Foundation
,
2nd ed.
,
Prentice-Hall
,
Englewood Cliffs, NJ
, p.
842
.
29.
Masri
,
S. F.
, and
Caughey
,
T. K.
, 1979, “
A Nonparametric Identification Technique for Nonlinear Dynamic Problems
,”
J. Appl. Mech.
,
46
, pp.
433
447
. 0021-8936
30.
Masri
,
S. F.
,
Caffrey
,
J. P.
,
Caughey
,
T. K.
,
Smyth
,
A. W.
, and
Chassiakos
,
A. G.
, 2004, “
Identification of the State Equation in Complex Non-Linear Systems
,”
Int. J. Non-Linear Mech.
0020-7462,
39
, pp.
1111
1127
.
31.
Wessels
,
L. F. A.
, and
Barnard
,
E.
, 1992, “
Avoiding False Local Minima by Proper Initialization of Connections
,”
IEEE Trans. Neural Netw.
,
3
(
6
), pp.
899
905
. 1045-9227
You do not currently have access to this content.