Thanks to recent advances in computing power and speed, engineers can now generate a wealth of data on demand to support design decision-making. These advances have enabled new approaches to search multidimensional trade spaces through interactive data visualization and exploration. In this paper, we investigate the effectiveness and efficiency of interactive trade space exploration strategies by conducting human subject experiments with novice and expert users. A single objective, constrained design optimization problem involving the sizing of an engine combustion chamber is used for this study. Effectiveness is measured by comparing the best feasible design obtained by each user, and efficiency is assessed based on the percentage of feasible designs generated by each user. Results indicate that novices who watch a 5-min training video before the experiment obtain results that are not significantly different from those obtained by expert users, and both groups are statistically better than the novices without the training video in terms of effectiveness and efficiency. Frequency and ordering of the visualization and exploration tools are also compared to understand the differences in each group’s search strategy. The implications of the results are discussed along with future work.

References

References
1.
Wang
,
G. G.
, and
Shan
,
S.
, 2007, “
Review of Metamodeling Techniques in Support of Engineering Design Optimization
,”
ASME J. Mech. Des.
,
129
(
4
), pp.
370
380
.
2.
Papalambros
,
P. Y.
, and
Wilde
,
D. J.
, 2000.
Principles of Optimal Design: Modeling and Computation
,
Cambridge University
,
New York
.
3.
Balling
,
R.
, 1999, “
Design by Shopping: A New Paradigm?
,”
Proceedings of the Third World Congress of Structural and Multidisciplinary Optimization (WCSMO-3)
,
University at Buffalo
,
Buffalo, NY
, pp.
295
297
.
4.
Stump
,
G.
,
Lego
,
S.
,
Yukish
,
M.
,
Simpson
,
T. W.
, and
Donndelinger
,
J. A.
, 2009, “
Visual Steering Commands for Trade Space Exploration: User-Guided Sampling With Example
,”
ASME J. Comput. Inf. Sci. Eng.
,
9
(
4
), p.
044501
.
5.
Winer
,
E. H.
, and
Bloebaum
,
C. L.
, 2001, “
Visual Design Steering for Optimization Solution Improvement
,”
Struct. Optim.
,
22
(
3
), pp.
219
229
.
6.
Kesavadas
,
T.
, and
Sudhir
,
A.
, 2000, “
Computational Steering in Simulation of Manufacturing Systems
,”
Proceedings of the 2000 IEEE International Conference on Robotics and Automation
,
IEEE
,
San Francisco, CA
, pp.
2654
2658
.
7.
Carlsen
,
D.
,
Malone
,
M.
,
Kollat
,
J.
, and
Simpson
,
T. W.
, 2008, “
Evaluating the Performance of Visual Steering Commands for User-Guided Pareto Frontier Sampling During Trade Space Exploration
,”
ASME Design Engineering Technical Conferences - Design Automation Conference
,
ASME
,
New York
, Paper No. DETC2008/DAC-49681.
8.
Stump
,
G. M.
,
Yukish
,
M.
,
Simpson
,
T. W.
, and
Bennett
,
L.
, 2002, “
Multidimensional Visualization and Its Application to a Design by Shopping Paradigm
,” AIAA Paper No.
2002
5622
.
9.
Stump
,
G.
,
Yukish
,
M.
,
Simpson
,
T. W.
, and
Harris
,
E. N.
, 2003, “
Design Space Visualization and Its Application to a Design by Shopping Paradigm
,”
ASME Design Engineering Technical Conferences - Design Automation Conference
,
ASME
,
Chicago, IL
, Paper No. DETC2003/DAC-48785.
10.
Simpson
,
T. W.
,
Spencer
,
D. B.
, and
Yukish
,
M. A.
, 2008, “
Visual Steering Commands and Test Problems to Support Research in Trade Space Exploration
,” AIAA Paper No. 2008–6085.
11.
Seo
,
J.
, and
Shneiderman
,
B.
, 2005, “
A Rank-By-Feature Framework for Interactive Exploration of Multidimensional Data
,”
Inf. Visualization
,
4
, pp.
96
113
.
12.
Klein
,
G.
, 1998,
Sources of Power: How People Make Decisions
,
MIT
,
Cambridge, MA
.
13.
Randel
,
J. M.
,
Pugh
,
H. L.
, and
Reed
,
S. K.
, 1996, “
Differences in Expert and Novice Situation Awareness in Naturalistic Decision Making
,”
Int. J. Hum.-Comput. Stud.
,
45
, pp.
579
597
.
14.
Jiang
,
X.
,
Gramopadhye
,
A. K.
, and
Melloy
,
B. J.
, 2004, “
Theoretical Issues in the Design of Visual Inspection Systems
,”
Theor. Issues Ergon. Sci.
,
5
(
3
), pp.
232
247
.
15.
Petre
,
M.
, and
Green
,
T. R. G.
, 1993, “
Learning to Read Graphics: Some Evidence That ‘Seeing’ an Information Display is an Acquired Skill
,”
J. Visual Lang. Comput.
,
4
, pp.
55
70
.
16.
Zhu
,
Y.
, 2007, “
Measuring Effective Data Visualization
,”
Advance in Visual Computing
,
Springer
,
Berlin
, pp.
652
661
.
17.
Simpson
,
T. W.
,
Spencer
,
D. B.
, and
Yukish
,
M. A.
, 2008, “
Visual Steering Commands and Test Problems to Support Research in Trade Space Exploration
,”
12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference
,
Victoria, British Columbia, Canada, AIAA
, September 10–12, Paper No. AIAA-2008-6085.
18.
Orasanu
,
J.
, and
Connoly
,
T.
, 1993, “
The Reinvention of Decision Making
,”
Decision Making in Action: Models and Methods
,
G. A.
Klein
,
J.
Orasanu
,
R.
Calderwood
, and
C. E.
Zsambok
, eds.,
Ablex Publishing
,
Norwood, CT
, pp.
3
20
.
19.
Gilmour
,
P.
, and
Corner
,
J.
, 1998, “
The Role of the Expert’s Decision Making Skills in Management
,”
33rd Annual Operational Research Society of New Zealand
,
Ablex Publishing
,
Auckland
, pp.
195
204
.
20.
Bouwman
,
M. J.
, 1984, “
Expert vs Novice Decision Making in Accounting: A Summary
,”
Account., Organ., Soc.
,
9
(
3/4
), pp.
325
327
.
21.
Anderson
,
J. R.
, 1982, “
Acquisition of Cognitive Skill
,”
Psychol. Rev.
,
89
, pp.
369
406
.
22.
Ritter
,
F. E.
, and
Bibby
,
P. A.
, 2008, “
Modeling How, When, and What is Learned in a Simple Fault-Finding Task
,”
Cogn. Sci.
,
32
, pp.
862
892
.
23.
Wiedenbeck
,
S.
, 1986, “
Organization of Programming Knowledge of Novices and Experts
,”
J. Am. Soc. Inf. Sci.
,
37
(
5
), pp.
284
299
.
24.
Jones
,
D. G.
, and
Endsley
,
M. R.
, 1996, “
Sources of Situation Awareness Errors in Aviation
,”
Aviat., Space Environ. Med.
,
67
(
6
), pp.
507
512
.
25.
Shneiderman
,
B.
, 1998,
Designing the User Interface: Strategies for Effective Human-Computer Interaction
, Addison-Wesley, Reading.
26.
Buja
,
A.
,
McDonald
,
J. A.
,
Michalak
,
J.
, and
Stuetzle
,
W.
, 1991, “
Interactive Data Visualization Using Focusing and Linking
,”
Proceedings of the IEEE Conference on Visualization
,
IEEE Society
,
San Diego, CA
, pp.
156
163
.
27.
Becker
,
R. A.
, and
Cleveland
,
W. S.
, 1987, “
Brushing Scatterplots
,”
Technometrics
,
29
(
1
), pp.
127
142
.
28.
Stump
,
G. M.
,
Yukish
,
M. A.
,
Martin
,
J. D.
, and
Simpson
,
T. W.
, 2004, “
The ARL Trade Space Visualizer: An Engineering Decision-Making Tool
,”
10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference
,
AIAA
,
Albany, NY
.
29.
Law
,
A. M.
, and
Kelton
,
W. D.
, 2000,
Simulation Modeling and Analysis
,
McGraw-Hill
,
Boston, MA
.
30.
Price
,
K.
,
Storn
,
R.
, and
Lampinen
,
J.
, 2005,
Differential Evolution - A Practical Approach to Global Optimization
,
Springer-Verlag
,
Berlin
.
31.
Robic
,
T.
, and
Filipic
,
B.
, 2005, “
DEMO: Evolution for Multiobjective Optimization
,”
Third International Conference on Evolutionary Multi-Criterion Optimization
,
Springer
,
Guanajuato, Mexico
, pp.
520
533
.
32.
Stump
,
G. M.
,
Simpson
,
T. W.
,
Yukish
,
M.
, and
O’Hara
,
J. J.
, 2004, “
Trade Space Exploration of Satellite Datasets Using a Design by Shopping Paradigm
,”
2004 IEEE Aerospace Conference Proceedings
,
IEEE
,
Big Sky, MT
.
33.
Neilson
,
J.
, and
Mack
,
R. L.
, 1994,
Usability Inspection Methods
,
Wiley
,
New York
.
34.
Wagner
,
T. C.
, and
Papalambros
,
P. Y.
, 1991, “
Optimal Engine Design Using nonlinear Programming and the Engine System Assessment Model
,” Ford Motor Company, Technical Report No. SR-91-154.
35.
McAllister
,
C. D.
, and
Simpson
,
T. W.
, 2001, “
Multidisciplinary Robust Design Optimization of an Internal Combustion Engine
,”
ASME J. Mech. Des.
,
125
(
1
), pp.
124
130
.
36.
ANSI/ASME PTC 19.1-1998, Measurement Uncertainty, Part I, ASME, New York, NY.
37.
Wolf
,
D.
,
Simpson
,
T. W.
, and
Zhang
,
X.
, 2009, “
A Preliminary Study of Novice and Expert Users’ Decision-Making Procedures During Visual Trade Space Exploration
,”
ASME Design Engineering Technical Conferences—Design Automation Conference
,
ASME
,
San Diego, CA
, Paper No. DETC2009/DAC-87294.
38.
Devore
,
J. L.
, 1995,
Probability and Statistics for Engineering and the Sciences
,
Wadsworth
,
Belmont, CA
.
39.
Booth
,
T.
, 1967,
Sequential Machines and Automata Theory
,
Wiley
,
New York
.
40.
Wolf
,
D. R.
, 2009, “
An Assessment of Novice Users’ Decision-Making Strategies During Visual Trade Sapce Exploration
,” M.Sc. Thesis, Mechanical Engineering, Penn State University, University Park.
41.
Wu
,
J.
, and
Azarm
,
S.
, 2001, “
Metrics for Quality Assessment of a Multiobjective Design Optimization Solution Set
,”
ASME J. Mech. Des.
,
123
(
1
), pp.
18
25
.
You do not currently have access to this content.