A theoretical and experimental approach to the use of information theory in input space selection for modeling and diagnostic applications is examined. The assumptions and test cases used throughout the paper are specifically tailored to diesel engine diagnostic and modeling applications. This work seeks to quantify the amount of information about an output contained within an input space. The information theoretic quantity, conditional entropy, is shown to be an accurate predictor of model and diagnostic algorithm performance and therefore is a good choice for an input vector selection metric. Methods of estimating conditional entropy from collected data, including the amount of needed data, are also discussed.
Issue Section:Technical Briefs
Keywords:diesel engines, fault diagnosis, entropy
Topics:Diesel engines, Entropy, Probability, Algorithms, Modeling, Errors
Introduction to Statistical Pattern Recognition,
D., 1989, “
What Size Net Gives Valid Generalization?,”
R., 1992, “
Neural Networks and the Bias/Variance Dilemma,”
R., 1994, “
Using Mutual Information for Selecting Features in Supervised Neural Net Learning,”
IEEE Trans. Neural Netw.1045-9227,
H., 1995, “
Input Variable Selection for Neural Networks: Application to Predicting the U. S. Business Cycle,”
IEEE/IAFE Conference on Computational Intelligence for Financial Engineering, New York, NY.
J., 1995, “
Neural Networks Summer School,”
University of Cambridge Programme for Industry, Short Course, July 24–25.
B., 1996, “
Nonparametric Selection of Input Variables for Connectionist Learning,” Ph.D. Thesis, University of Colorado.
J., 1999, “
Feature Selection Based on Joint Mutual Information,”
Proceedings of International ICSC Symposium on Advances in Intelligent Data Analysis,
G., 2000, “
Input Vector Identification and System Model Construction by Average Mutual Information,”
Proceedings of the ASME Dynamic Systems and Control Division, Vol.
C., 1948, “
A Mathematical Theory of Communication,”
Bell Syst. Tech. J.0005-8580,
Elements of Information Theory,
E., 1962, “
On the Estimation of a Probability Density Function and Mode,”
Ann. Math. Stat.0003-4851,
Multivariate Density Estimation Theory, Practice, and Visualization,
Introduction to Information Theory,
Englewood Cliffs, NJ.
M., 2001, MATLAB Neural Network Toolbox, Rev. 1.7, Rel. 4.0.1.
K., 1992, “
LVQ̱PAK: A Program Package for the Correct Application of Learning Vector Quantization Algorithms,”
Proceedings of the International Joint Conference on Neural Networks,
Baltimore, MD, pp.
Copyright © 2007
by American Society of Mechanical Engineers