A fully automatic feedforward neural network structural and weight learning algorithm is described. The Augmentation by Training with Residuals, ATR, requires neither guess of initial weight values nor the number of neurons in the hidden layer from users. The algorithm takes an incremental approach in which a hidden neuron is trained to model the mapping between the input and output of current exemplars, and is augmented to the existing network. The exemplars are then made orthogonal to the newly identified hidden neuron and used for the training of next hidden neuron. The improvement continues until a desired accuracy is reached. This new structural and weight learning algorithm is applied to the identification of a two-degree-of-freedom planar robot, a Van der Pol oscillator and a Mackay-Glass equation. The algorithm is shown to be effective in modeling all three systems and is far superior to a linear modeling scheme in the case of the robot.

1.
Hecht-Nielsen, R., 1989, “Theory of the Backpropagation Neural Networks,” Proc. Int. Joint Conference on Neural Networks, Washington D.C.
2.
Beigi, H. S. M., and Li, C. J., 1990, “A New Set of Learning Algorithms for Neural Networks,” Proc. of ISMM Conference on Computer Applications in Design, Simulation and Analysis, New Orleans, LA, Mar 5–7, pp. 277–280.
3.
Beigi
H. S. M.
, and
Li
C. J.
,
1993
, “
Learning Algorithms for Neural Networks based on Quasi-Newton Methods with Self-Scaling
,”
ASME JOURNAL OF DYNAMIC SYSTEMS, MEASUREMENT, AND CONTROL
, Vol.
115
, No.
1
, pp.
38
43
.
4.
Korenberg
M.
,
Billings
S. A.
,
Liu
Y. P.
, and
Mcllroy
P. J.
,
1988
, “
Orthogonal Parameter Estimation Algorithm for Non-Linear Stochastic Systems
,”
International J. of Control
, Vol.
48
, No.
1
, pp.
193
210
.
5.
Lapedes, A., and Farber, R., 1987, “Nonlinear Signal Processing Using Neural Networks: Prediction and System Modeling,” Los Alamos National Laboratory, Technical Report LA-UR-87–2662.
6.
Li, C. J., Kim, T., and Nickerson, G. W., Dec. 1991, “Linear Model Based Fault Detection and Isolation for a Screw Compressor,” Proc. of ASME 1991 WAM Symposium on Sensors, Controls and Quality Issues in Manufacturing, Atlanta, GA. PED-Vol. 55, pp. 95–106.
7.
Oren
S. S.
, and
Spedicato
E.
,
1976
, “
Optimal Conditioning of Self-Scaling Variable Metric Algorithms
,”
Mathematical Programming
, Vol.
10
, pp.
70
90
.
8.
Pearson
J. D.
,
1969
, “
Variable Metric Methods of Minimization
,”
Computer Journal
, Vol.
13
, pp.
171
178
.
This content is only available via PDF.
You do not currently have access to this content.