Designers are continuously challenged by complexity, as well as by the excessive instantiation and execution times of models, particularly in the context of integrated product and materials design. In order to manage these challenges, a systematic strategy for evaluating and selecting models is presented in this paper. The systematic strategy is based on value-of-information for design decision making. It consists of a (i) process performance indicator (PPI) to quantify the impact of model refinement from a decision-centric perspective and (ii) a method involving model evaluation. Using this method, a least complex but valid model is evaluated, and, only if necessary, gradually refined it until the most appropriate one is selected. The systematic approach is particularly well suited for integrated product and materials design, and all other scenarios where the perfect knowledge of the true system behavior and bounds of error are not available throughout the design space. The proposed strategy is applied to the design of photonic crystal waveguides for use in a next-generation optoelectronic communication system. In this paper, it is shown that the systematic strategy based on the PPI is useful for evaluating and selecting models particularly when accuracy of the prediction or the associated error bounds are not known.

1.
Box
,
G. E. P.
, 1979, “
Robustness in the Strategy of Scientific Model Building
,”
Robustness in Statistics
,
R. L.
Launer
and
G. N.
Wiklinson
, eds.,
Academic
,
New York
, pp.
201
236
.
2.
Radhakrishnan
,
R.
, and
McAdams
,
D. A.
, 2005, “
A Methodology for Model Selection in Engineering Design
,”
ASME J. Mech. Des.
0161-8458,
127
, pp.
378
387
.
3.
McDowell
,
D. L.
, 1998, “
New Directions in Materials Design Science and Engineering
,” Report of a Workshop Sponsored by the U.S. National Science Foundation, Georgia Institute of Technology and Morehouse College, Atlanta, GA.
4.
Pahl
,
G.
, and
Beitz
,
W.
, 1996,
Engineering Design: A Systematic Approach
,
Springer-Verlag
,
New York
.
5.
Sobek
,
D. K.
, and
Ward
,
A. C.
, 1996, “
Principles From Toyota’s Set-Based Concurrent Engineering Process
,”
ASME
Paper No. 96-DETC/DTM-1510.
6.
Mistree
,
F.
,
Hughes
,
O. F.
, and
Bras
,
B. A.
, 1992, “
The Compromise Decision Support Problem and the Adaptive Linear Programming Algorithm
,”
Structural Optimization: Status and Promise
,
M. P.
Kamat
, ed.,
AIAA
,
Washington, DC
, Vol.
150
, pp.
251
290
.
7.
Mistree
,
F.
,
Muster
,
D.
,
Shupe
,
J. A.
, and
Allen
,
J. K.
, 1989, “
A Decision-Based Perspective for the Design of Methods for Systems Design
,”
NASA
Report No. CP3031.
8.
Panchal
,
J. H.
, 2005, “
A Framework for Simulation-Based Integrated Design of Multiscale Products and Design Processes
,” Ph.D. thesis, G.W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA.
9.
Bradley
,
S. R.
, and
Agogino
,
A. M.
, 1994, “
An Intelligent Real Time Design Methodology for Component Selection: An Approach to Managing Uncertainty
,”
ASME J. Mech. Des.
0161-8458,
116
, pp.
980
988
.
10.
Buede
,
D. M.
, 1999,
The Engineering Design of Systems: Models and Methods
,
Wiley-Interscience
,
New York
.
11.
Eppinger
,
S. D.
, and
Salminen
,
V.
, 2001, “
Patterns of Product Development Interactions
,”
International Conference on Engineering Design (ICED)
, Glasgow, UK.
12.
Panchal
,
J.
,
Paredis
,
C. J. J.
,
Allen
,
J. K.
, and
Mistree
,
F.
, 2008, “
A Value-of-Information Based Approach to Simulation Model Refinement
,”
Eng. Optimiz.
0305-215X,
40
(
3
), pp.
223
251
.
13.
Malak
,
R. J.
, and
Paredis
,
C. J. J.
, 2007, “
Validating Behavioral Models for Reuse
,”
Res. Eng. Des.
0934-9839,
18
(
3
), pp.
111
128
.
14.
Howard
,
R.
, 1966, “
Information Value Theory
,”
IEEE Trans. Syst. Sci. Cybern.
0536-1567,
SSC-2
(
1
), pp.
779
783
.
15.
Ling
,
J. M.
,
Aughenbaugh
,
J. M.
, and
Paredis
,
C. J. J.
, 2006, “
Managing the Collection of Information Under Uncertainty Using Information Economics
,”
ASME J. Mech. Des.
0161-8458,
128
(
4
), pp.
980
991
.
16.
Webster
,
N.
, 2000,
Merriam-Webster`s Collegiate Dictionary
,
Merriam-Webster
,
Springfield, MA
.
17.
Simpson
,
T. W.
,
Lautenschlager
,
U.
, and
Mistree
,
F.
, 1998, “
Mass Customization in the Age of Information: The Case for Open Engineering Systems
,”
The Information Revolution: Current and Future Consequences
,
W.
Read
and
A.
Porter
, eds.,
Ablex
,
Greenwich, CT
, pp.
49
71
.
18.
Kusiak
,
A.
,
Larson
,
N.
, and
Wang
,
J.
, 1994, “
Reengineering of Design and Manufacturing Processes
,”
Comput. Ind. Eng.
0360-8352,
26
(
3
), pp.
521
536
.
19.
Pimmler
,
T. U.
, and
Eppinger
,
S. D.
, 1994, “
Integration Analysis of Product Decompositions
,”
ASME
Paper No. DETC1994-47821.
20.
Bloebaum
,
C. L.
, 1995, “
Coupling Strength-Based System Reduction for Complex Engineering Design
,”
Struct. Optim.
0934-4373,
10
(
2
), pp.
113
121
.
21.
Bloebaum
,
C. L.
,
Hajela
,
P.
, and
Sobieszczanski-Sobieski
,
J.
, 1992, “
Non-Hierarchic System Decomposition in Structural Optimization
,”
Eng. Optimiz.
0305-215X,
19
, pp.
171
186
.
22.
Lawrence
,
D. B.
, 1999,
The Economic Value of Information
,
Springer
,
New York
.
23.
Aughenbaugh
,
J. M.
, and
Paredis
,
C. J. J.
, 2006, “
The Value of Using Imprecise Probabilities in Engineering Design
,”
ASME J. Mech. Des.
0161-8458,
128
(
4
), pp.
969
980
.
24.
Ling
,
J.
, and
Paredis
,
C. J. J.
, 2006, “
An Information Economic Approach for Model Selection in Engineering Design
,”
ASME
Paper No. IMECE2006-14535.
25.
Doraiswamy
,
S.
, and
Krishnamurty
,
S.
, 2000, “
Bayesian Analysis in Engineering Model Assessment
,”
ASME
Paper No. DETC/DTM-21699.
26.
Cherkassky
,
V.
, 2002, “
Model Complexity Control and Statistical Learning Theory
,”
Nat. Comput.
1567-7818,
1
, pp.
109
133
.
27.
Messer
,
M.
, 2008, “
A Systematic Approach for Integrated Product, Materials and Design-Process Design
,” Ph.D. thesis, G.W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA.
28.
Chen
,
W.
,
Allen
,
J. K.
,
Mavris
,
D.
, and
Mistree
,
F.
, 1996, “
A Concept Exploration Method for Determining Robust Top-Level Specifications
,”
Eng. Optimiz.
0305-215X,
26
(
2
), pp.
137
158
.
29.
Taguchi
,
G.
, 1986,
Introduction to Quality Engineering
,
UNIPUB
,
New York
.
30.
Seepersad
,
C. C.
,
Mistree
,
F.
, and
Allen
,
J. K.
, 2005, “
Designing Evolving Families of Products Using Utility-Based Compromise Decision Support Problem
,”
International Journal of Mass Customization
,
1
(
1
), pp.
37
64
.
31.
Panchal
,
J.
,
Fernandez
,
M. G.
,
Paredis
,
C. J. J.
,
Allen
,
J. K.
, and
Mistree
,
F.
, 2009, “
A Modular Decision Centric Approach for Reusable Design Processes
,”
Concurr. Eng. Res. Appl.
1063-293X,
17
(
5
), pp.
5
19
.
32.
Borel
,
P. I.
,
Harpoth
,
A.
,
Frandsen
,
L. H.
, and
Kristensen
,
M.
, 2004, “
Topology Optimization and Fabrication of Photonic Crystal Structures
,”
Opt. Express
1094-4087,
12
(
9
), pp.
1996
2001
.
33.
Hughes
,
S.
,
Ramunno
,
L.
,
Young
,
J. F.
, and
Sipe
,
J. E.
, 2005, “
Extrinsic Optical Scattering Loss in Photonic Crystal Waveguides: Role of Fabrication Disorder and Photon Group Velocity
,”
Phys. Rev. Lett.
0031-9007,
94
, p.
033903
.
34.
Gaylord
,
T. K.
, and
Moharam
,
M. G.
, 1985, “
Analysis and Applications of Optical Diffraction by Gratings
,”
Proc. IEEE
0018-9219,
73
, pp.
894
937
.
35.
Lalanne
,
P.
, 2002, “
Electromagnetic Analysis of Photonic Crystal Waveguides Operating Above the Light Cone
,”
IEEE J. Quantum Electron.
0018-9197,
38
, pp.
800
804
.
36.
Dems
,
M.
,
Kotynskiand
,
R.
, and
Panajotov
,
K.
, 2005, “
Planewave Admittance Method—A Novel Approach for Determining the Electromagnetic Modes in Photonic Structures
,”
Opt. Express
1094-4087,
13
, pp.
3196
3207
.
37.
Helfert
,
S. F.
, and
Pregla
,
R.
, 1998, “
Efficient Analysis of Periodic Structures
,”
J. Lightwave Technol.
0733-8724,
16
, pp.
1694
1702
.
38.
White
,
R. P.
,
Kuhlmey
,
B. T.
,
McPhedran
,
R. C.
,
Maystre
,
D.
,
Renversez
,
G.
,
DeSterke
,
C. M.
, and
Botten
,
L. C.
, 2002, “
Multipole Method for Microstructured Optical Fibers. I. Formulation
,”
J. Opt. Soc. Am. B
0740-3224,
19
, pp.
2322
2330
.
39.
Bienstman
,
P.
, and
Baets
,
R.
, 2002, “
“Rigorous and Efficient Optical VCSEL Model Based on Vectorial Eigenmode Expansion and Perfectly Matched Layers
,”
IEE Proc.: Optoelectron.
1350-2433,
149
, pp.
161
165
.
40.
Bienstman
,
P.
, and
Baets
,
R.
, 2001, “
Optical Modeling of Photonic Crystals and VCSELs Using Eigenmode Expansion and Perfectly Matched Layers
,”
Opt. Quantum Electron.
0306-8919,
33
, pp.
327
341
.
41.
Richardson
,
L. F.
, 1910, “
The Approximate Arithmetical Solution by Finite Differences of Physical Problems Involving Differential Equations With an Application to the Stresses in a Masonry Dam
,”
Transactions of the Royal Society of London, Series A
,
210
, pp.
307
357
.
You do not currently have access to this content.