A variety of metamodeling techniques have been developed in the past decade to reduce the computational expense of computer-based analysis and simulation codes. Metamodeling is the process of building a “model of a model” to provide a fast surrogate for a computationally expensive computer code. Common metamodeling techniques include response surface methodology, kriging, radial basis functions, and multivariate adaptive regression splines. In this paper, we investigate support vector regression (SVR) as an alternative technique for approximating complex engineering analyses. The computationally efficient theory behind SVR is reviewed, and SVR approximations are compared against the aforementioned four metamodeling techniques using a test bed of 26 engineering analysis functions. SVR achieves more accurate and more robust function approximations than the four metamodeling techniques, and shows great potential for metamodeling applications, adding to the growing body of promising empirical performance of SVR.
1.
Kleijnen
, J. P. C.
, 1987, Statistical Tools for Simulation Practitioners
, Marcel Dekker
, New York.2.
Jin
, R.
, Chen
, W.
, and Simpson
, T. W.
, 2001, “Comparative Studies of Metamodeling Techniques Under Multiple Modeling Criteria
,” Struct. Multidiscip. Optim.
1615-147X, 23
(1
), pp. 1
–13
.3.
Haykin
, S.
, 1999, Neural Networks: A Comprehensive Foundation
, 2nd Edition, Prentice Hall
, Upper Saddle River, NJ.4.
Cristianni
, N.
, and Shawe-Taylor
, J.
, 2000, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods
, Cambridge University Press
, Cambridge, UK.5.
Hearst
, M. A.
, 1998, “Trends Controversies: Support Vector Machines
,” IEEE Intell. Syst.
1094-7167, 13
(4
), pp. 18
–28
.6.
Takeuchi
, K.
, and Collier
, N.
, 2002, “Use of Support Vector Machines in Extended Named Entity
,” Proc. of Sixth Conference on Natural Language Learning (CoNLL-2002)
, D.
Roth
and A.
van den Bosch
, eds., Taipei, Taiwan
, Association for Computational Linguistics
, New Brunswick, NJ, pp. 119
–125
.7.
Dumais
, S. T.
, Platt
, J.
, Heckerman
, D.
, and Saharni
, M.
, 1998, “Inductive Learning Algorithms and Representations for Text Categorization
,” Proc. of 7th Int. Conference on Information and Knowledge Management
, Bethesda
, MD, ACM, New York, pp. 148
–155
.8.
Prakasvudhisarn
, C.
, Trafalis
, T. B.
, and Raman
, S.
, 2003, “Support Vector Regression for Determination of Minimum Zone
,” ASME J. Manuf. Sci. Eng.
1087-1357, 125
(4
), pp. 736
–739
.9.
Vapnik
, V.
, and Lerner
, A.
, 1963, “Pattern Recognition Using Generalized Portrait Method
,” Autom. Remote Control (Engl. Transl.)
0005-1179, 24
(6
), pp. 774
–780
.10.
Vapnik
, V.
, 1995, The Nature of Statistical Learning Theory
, Springer
, New York.11.
Smola
, A. J.
, and Schölkopf
, B.
, 1998, “A Tutorial on Support Vector Regression
,” NeuroCOLT2 Technical Report Series, NC2-TR-1998-030
, Berlin
, Germany.12.
Vapnik
, V.
, Golowich
, S.
, and Smola
, A.
, 1997, “Support Vector Method for Function Approximation, Regression Estimation, and Signal Processing
,” Advances in Neural Information Processing Systems
, M.
Mozer
, M.
Jordan
, and T.
Petsche
, eds., MIT Press
, Cambridge, MA, pp. 281
–287
.13.
Gunn
, S. R.
, 1997, “Support Vector Machines for Classification and Regression
,” Technical Report, Image Speech and Intelligent Systems Research Group, University of Southampton
, UK.14.
Simpson
, T. W.
, Peplinski
, J.
, Koch
, P. N.
, and Allen
, J. K.
, 2001, “Metamodels for Computer-Based Engineering Design: Survey and Recommendations
,” Eng. Comput.
0177-0667, 17
(2
), pp. 129
–150
.15.
Myers
, R. H.
, and Montgomery
, D. C.
, 1995, Response Surface Methodology: Process and Product Optimization Using Designed Experiments
, Wiley
, New York.16.
Cappelleri
, D. J.
, Frecker
, M. I.
, Simpson
, T. W.
, and Snyder
, A.
, 2002, “A Metamodel-Based Approach for Optimal Design of a PZT Bimorph Actuator for Minimally Invasive Surgery
,” J. Mech. Des.
1050-0472, 124
(2
), pp. 354
–357
.17.
Chen
, W.
, Allen
, J. K.
, Tsui
, K.-L.
, and Mistree
, F.
, 1996, “A Procedure for Robust Design: Minimizing Variations Caused by Noise and Control Factors
,” J. Mech. Des.
1050-0472, 118
(4
), pp. 478
–485
.18.
Korngold
, J. C.
, and Gabriele
, G. A.
, 1997, “Multidisciplinary Analysis and Optimization of Discrete Problems Using Response Surface Methods
,” J. Mech. Des.
1050-0472, 119
(4
), pp. 427
–433
.19.
Wang
, G.
, 2003, “Adaptive Response Surface Method Using Inherited Latin Hypercube Designs
,” J. Mech. Des.
1050-0472, 125
(2
), pp. 210
–220
.20.
Hernandez
, G.
, Simpson
, T. W.
, Allen
, J. K.
, Bascaran
, E.
, Avila
, L. F.
, and Salinas
, F.
, 2001, “Robust Design of Families of Products With Production Modeling and Evaluation
,” J. Mech. Des.
1050-0472, 123
(2
), pp. 183
–190
.21.
Lewis
, K.
, and Mistree
, F.
, 1998, “Collaborative, Sequential, and Isolated Decisions in Design
,” J. Mech. Des.
1050-0472, 120
(4
), pp. 643
–652
.22.
Dyn
, N.
, Levin
, D.
, and Rippa
, S.
, 1986, “Numerical Procedures for Surface Fitting of Scattered Data by Radial Basis Functions
,” SIAM (Soc. Ind. Appl. Math.) J. Sci. Stat. Comput.
0196-5204, 7
(2
), pp. 639
–659
.23.
Powell
, M. J. D.
, 1987, “Radial Basis Functions for Multivariable Interpolation: A Review
,” Algorithms for Approximation
, J. C.
Mason
and M. G.
Cox
, eds., Oxford University Press
, London, pp. 143
–167
.24.
Tu
, C.
, and Barton
, R. R.
, 1997, “Production Yield Estimation by the Metamodel Method with a Boundary-Focused Experiment Design
,” ASME Design Engineering Technical Conferences-Design Theory and Methodology
, Sacramento
, CA, ASME, Paper No. DETC97/DTM-3870.25.
Meckesheimer
, M.
, Barton
, R. R.
, Simpson
, T. W.
, Limayem
, F.
, and Yannou
, B.
, 2001, “Metamodeling of Combined Discrete/Continuous Responses
,” AIAA J.
0001-1452, 39
(10
), pp. 1955
–1959
.26.
Sacks
, J.
, Welch
, W. J.
, Mitchell
, T. J.
, and Wynn
, H. P.
, 1989, “Design and Analysis of Computer Experiments
,” Stat. Sci.
0883-4237, 4
(4
), pp. 409
–435
.27.
Koehler
, J. R.
, and Owen
, A. B.
, 1996, “Computer Experiments
,” Handbook of Statistics
, S.
Ghosh
and C. R.
Rao
, eds., Elsevier Science
, New York, pp. 261
–308
.28.
Simpson
, T. W.
, Mauery
, T. M.
, Korte
, J. J.
, and Mistree
, F.
, 2001, “Kriging Metamodels for Global Approximation in Simulation-Based Multidisciplinary Design Optimization
,” AIAA J.
0001-1452, 39
(12
), pp. 2233
–2241
.29.
Pacheco
, J. E.
, Amon
, C. H.
, and Finger
, S.
, 2003, “Bayesian Surrogates Applied to Conceptual Stages of the Engineering Design Process
,” J. Mech. Des.
1050-0472, 125
(4
), pp. 664
–672
.30.
Friedman
, J. H.
, 1991, “Multivariate Adaptive Regression Splines
,” Ann. Stat.
0090-5364, 19
(1
), pp. 1
–67
.31.
Chen
, V. C. P.
, Ruppert
, D.
, and Shoemaker
, C. A.
, 1999, “Applying Experimental Design and Regression Splines to High-Dimensional Continuous-State Stochastic Dynamic Programming
,” Oper. Res.
0030-364X, 47
, pp. 38
–53
.32.
Chen
, V. C. P.
, 1999, “Application of MARS and Orthogonal Arrays to Inventory Forecasting Stochastic Dynamic Programs
,” Comput. Stat. Data Anal.
0167-9473, 30
, pp. 317
–341
.33.
Smola
, A. J.
, Schölkopf
, B.
, and Müller
, K. R.
, 1998, “The Connection Between Regularization Operators and Support Vector Kernels
,” Neural Networks
0893-6080, 11
(4
), pp. 637
–649
.34.
Schölkopf
, B.
, and Smola
, A. J.
, 2002, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
, MIT Press
, Cambridge, MA.35.
Su
, J.
, and Renaud
, J. E.
, 1997, “Automatic Differentiation in Robust Optimization
,” AIAA J.
0001-1452, 35
(6
), pp. 1072
–1079
.36.
Markowetz
, F.
, 2001, “Support Vector Machines in Bioinformatics
,” Diploma Thesis in Mathematics, University of Heidelberg, Germany.37.
Simpson
, T. W.
, 1998, “A Concept Exploration Method for Product Family Design
,” Ph.D. Dissertation, G.W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA.38.
Schmit
, L. A.
, 1981, “Structural Synthesis—Its Genesis and Development
,” AIAA J.
0001-1452, 19
(10
), pp. 1249
–1263
.39.
Arora
, J. S.
, 1989, Introduction to Optimum Design
, McGraw-Hill
, New York.40.
Sandgren
, E.
, 1990, “Nonlinear Integer and Discrete Programming in Mechanical Design Optimization
,” J. Mech. Des.
1050-0472, 112
(2
), pp. 223
–229
.41.
Ragsdell
, K. M.
, and Phillips
, D. T.
, 1976, “Optimal Design of a Class of Welded Structures Using Geometric Programming
,” ASME J. Eng. Ind.
0022-0817, 98
(3
), pp. 1021
–1025
.42.
Simpson
, T. W.
, Lin
, D. K. J.
, and Chen
, W.
, 2001, “Sampling Strategies for Computer Experiments: Design and Analysis
,” Int. J. Reliab. Appl.
1598-0073, 2
(3
), pp. 209
–240
.43.
McKay
, M. D.
, Beckman
, R. J.
, and Conover
, W. J.
, 1979, “A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code
,” Technometrics
0040-1706, 21
(2
), pp. 239
–245
.44.
Kalagnanam
, J. R.
, and Diwekar
, U. M.
, 1997, “An Efficient Sampling Technique for Off-Line Quality Control
,” Technometrics
0040-1706, 39
(3
), pp. 308
–319
.45.
Tang
, B.
, 1993, “Orthogonal Array-Based Latin Hypercubes
,” J. Am. Stat. Assoc.
0162-1459, 88
(424
), pp. 1392
–1397
.46.
Fang
, K.-T.
, Lin
, D. K. J.
, Winker
, P.
, and Zhang
, Y.
, 2000, “Uniform Design: Theory and Application
,” Technometrics
0040-1706, 42
, pp. 237
–248
.47.
Wang
, W. J.
, Xu
, Z. B.
, Lu
, W. Z.
, and Zhang
, X. Y.
, 2003, “Determination of the Spread Parameter in the Gaussian Kernel for Classification and Regression
,” Neurocomputing
0925-2312, 55
(1
), pp. 643
–663
.48.
Cameron
, A. C.
, and Windmeijer
, F. A. G.
, 1997, “An R-Squared Measure of Goodness of Fit for Some Common Nonlinear Regression Models
,” J. Econometr.
0304-4076, 77
(2
), pp. 329
–342
.49.
Martin
, J. D.
, and Simpson
, T. W.
, 2004, “On the Use of Kriging Models to Approximate Deterministic Computer Models
,” ASME Design Engineering Technical Conferences-Design Automation Conference
, Salt Lake City
, W.
Chen
, ed., ASME
, New York, ASME Paper No. DETC2004/DAC-57300.Copyright © 2005
by American Society of Mechanical Engineers
You do not currently have access to this content.
