Abstract

Concept screening is one of the gatekeepers of innovation process and thus is considered a vital component of engineering design. Yet, we know very little about how decisions are made during concept screening or the factors that inform these decisions. This is due, in part, to the fact that most prior work on concept screening in engineering design has focused on student populations or on industry professionals in an experimental setting which is not indicative of the risks and consequences professionals face in their daily work—particularly when it comes to innovative design process. Thus, the current study was developed to identify how the environmental settings (i.e., experimental versus naturalistic) and the role of the professionals in the design process (i.e., idea generators versus executives) impacts the criteria used to screen design ideas. Two studies were conducted including a workshop study with 45 design professionals from two companies in an experimental setting and a participatory ethnographic study with seven design professionals from a small electromechanical company in a naturalistic setting. The results showed stark differences in the criteria used to screen ideas between naturalistic and experimental practices and between idea generators and company executives. In addition, the results showed differences in the factors considered during concept screening between naturalistic and experimental environments. These results are used to identify opportunities for tools and methods that encourage the consideration of creative ideas in the engineering design industry and encourage appropriate risk-taking in engineering design.

References

References
1.
Hernández
,
H.
,
Grassano
,
N.
,
Tübke
,
A.
,
Potters
,
L.
,
Gkotsis
,
P.
, and
Vezzani
,
A.
,
2018
, “
The 2018 EU Industrial R&D Investment Scoreboard
,”
No. EUR 29450 EN, Luxembourg
.
2.
Brands
,
R. F.
, and
Kleinman
,
M. J.
,
2010
,
Robert’s Rules of Innovation: A 10-Step Program for Corporate Survival
,
John Wiley & Sons
,
New York
.
3.
Leubsdorf
,
B.
,
2017
, “
R&D Spending is a Rare Bright Spot in Business Investment
,” https://www.wsj.com/articles/r-d-spending-is-a-rare-bright-spot-in-business-investment-1477576800#livefyre-toggle-SB10660112401764113944404582394472151524232, Accessed August 1, 2019.
4.
Pahl
,
G.
,
Beitz
,
W.
,
Feldhusen
,
J.
, and
Grote
,
K.
,
2007
,
Engineering Design: A Systematic Approach
, 3rd ed.,
Springer Science+ Business Media Deutschland GmbH
,
Berlin
, 2007, p.
632
.
5.
Sheppard
,
B.
,
Sarrazin
,
H.
,
G.
,
K.
, and
Dore
,
F.
,
2018
, “
The Business Value of Design
,” https://www.mckinsey.com/business-functions/mckinsey-design/our-insights/the-business-value-of-design
6.
Liedtka
,
J.
,
2014
, “
Innovative Ways Companies are Using Design Thinking
,”
Strategy Leadership
,
42
(
2
), pp.
40
45
. 10.1108/SL-01-2014-0004
7.
Rietzschel
,
E. F.
,
Nijstad
,
B. A.
, and
Stroebe
,
W.
,
2006
, “
Productivity is Not Enough: A Comparison of Interactive and Nominal Brainstorming Groups on Idea Generation and Selection
,”
J. Exp. Soc. Psychol.
,
42
(
2
), pp.
244
251
. 10.1016/j.jesp.2005.04.005
8.
Cooper
,
R. G.
,
2008
, “
Perspective: The Stage-Gate® Idea-to-Launch Process—Update, What’s New, and Nexgen Systems
,”
J. Prod. Innovat. Manag.
,
25
(
3
), pp.
213
232
. 10.1111/j.1540-5885.2008.00296.x
9.
Schmidt
,
J. B.
, and
Calantone
,
R. J.
,
2002
, “
Escalation of Commitment During New Product Development
,”
J. Acad. Mark. Sci.
,
30
(
2
), pp.
103
118
. 10.1177/03079459994362
10.
Onarheim
,
B.
, and
Christensen
,
B. T.
,
2012
, “
Distributed Idea Screening in Stage-Gate Development Processes
,”
J. Eng. Des.
,
23
(
9
), pp.
660
673
. 10.1080/09544828.2011.649426
11.
Zheng
,
X.
, and
Miller
,
S. R.
,
2018
,”
Uncovering Ownership Bias: The Influence of Idea Goodness and Creativity on Design Professionals’ Concept Selection Practices
,
IDETC Quebec City, Quebec, Canada
,
DETC2018-85964
.
12.
Rietzschel
,
E.
,
Nijstad
,
B. A.
, and
Stroebe
,
W.
,
2010
, “
The Selection of Creative Ideas After Individual Idea Generation: Choosing Between Creativity and Impact
,”
Br. J. Psychol.
,
101
(
1
), pp.
47
68
. 10.1348/000712609X414204
13.
Mueller
,
J. S.
,
Melwani
,
S.
, and
Goncalo
,
J. A.
,
2012
, “
The Bias Against Creativity: Why People Desire but Reject Creative Ideas
,”
Psychol. Sci.
,
23
(
1
), pp.
13
17
. 10.1177/0956797611421018
14.
Amabile
,
T. M.
,
1996
,
Creativity in Context: Update to “The Social Psychology of Creativity
”,
Westview Press
,
New York
.
15.
Sternberg
,
R. J.
, and
Lubart
,
T. I.
,
1995
,
Defying the Crowd: Simple Solutions to the Most Common Relationship Problems
,
Free Press
,
New York
.
16.
Jones
,
J.
, and
Smith
,
J.
, 2017, “
Ethnography: Challenges and Opportunities
,”
Royal College of Nursing
.
17.
Nikander
,
J. B.
,
Liikkanen
,
L. A.
, and
Laakso
,
M.
,
2014
, “
The Preference Effect in Design Concept Evaluation
,”
Des. Stud.
,
35
(
5
), pp.
473
499
. 10.1016/j.destud.2014.02.006
18.
Toh
,
C.
, and
Miller
,
S.
,
2014
, “
The Role of Individual Risk Attitudes on the Selection of Creative Concepts in Engineering Design
,”
ASME Design Engineering Technical Conferences Buffalo
,
NY
,
DETC2014-35106
.
19.
Toh
,
C. A.
, and
Miller
,
S. R.
,
2016
, “
Creativity in Design Teams: The Influence of Personality Traits and Risk Attitudes on Creative Concept Selection
,”
Res. Eng. Des.
,
27
(
1
), pp.
73
89
. 10.1007/s00163-015-0207-y
20.
Toh
,
C. A.
, and
Miller
,
S. R.
,
2016
, “
Choosing Creativity: The Role of Individual Risk and Ambiguity Aversion on Creative Concept Selection in Engineering Design
,”
Res. Eng. Des.
,
27
(
3
), pp.
195
219
. 10.1007/s00163-015-0212-1
21.
Toh
,
C.
,
Strohmetz
,
A.
, and
Miller
,
S.
,
2016
, “
The Effects of Gender and Idea Goodness on Ownership Bias in Engineering Design
,”
ASME J. Mech. Des.
,
138
(
10
), p.
101105
. 10.1115/1.4034107
22.
Nikander
,
J. B.
,
Liikkanen
,
L. A.
, and
Laakso
,
M.
,
2013
, “
Naturally Emerging Decision Criteria in Product Concept Evaluation
,”
Proc. DS 75-7: Proceedings of the 19th International Conference on Engineering Design (ICED13), Design for Harmonies, Vol. 7: Human Behaviour in Design
,
Seoul, Korea
, 19-22.08.
23.
Ahmed
,
S.
, and
Christensen
,
B. T.
,
2009
, “
An in Situ Study of Analogical Reasoning in Novice and Experienced Design Engineers
,”
ASME J. Mech. Des.
,
131
(
11
), p.
111004
. 10.1115/1.3184693
24.
Toh
,
C.
,
Patel
,
A.
,
Strohmetz
,
A.
, and
Miller
,
S.
,
2015
, “
My Idea is Best! Ownership Bias and Its Influence in Engineering Concept Selection
,”
Design Engineering Technical Conferences
,
Boston, MA
,
DETC2015-46478
.
25.
Kijkuit
,
B.
, and
van der Ende
,
J.
,
2007
, “
The Organizational Life of an Idea: Integrating Social Network, Creativity and Decision-Making Perspectives
,”
J. Manag. Stud.
,
44
(
6
), pp.
863
882
. 10.1111/j.1467-6486.2007.00695.x
26.
Cooper
,
R. G.
,
1990
, “
Stage-gate Systems: A New Tool for Managing New Products
,”
Bus. Horiz.
,
33
(
3
), pp.
44
54
. 10.1016/0007-6813(90)90040-I
27.
Zheng
,
X.
, and
Miller
,
S. R.
,
2019
, “
Is Ownership Bias Bad? The Influence of Idea Goodness and Creativity on Design Professionals Concept Selection Practices
,”
ASME J. Mech. Des.
,
141
(
2
), p.
021106
. 10.1115/1.4042081
28.
Cooper
,
A. C.
,
Woo
,
C. Y.
, and
Dunkelberg
,
W. C.
,
1988
, “
Entrepreneurs’ Perceived Chances for Success
,”
J. Bus. Ventur.
,
3
(
2
), pp.
97
108
. 10.1016/0883-9026(88)90020-1
29.
Gilmore
,
A.
,
Carson
,
D.
, and
O'Donnell
,
A.
,
2004
, “
Small Business Owner-Managers and Their Attitude to Risk
,”
Market. Intell. Plan.
,
22
(
3
), pp.
349
360
. 10.1108/02634500410536920
30.
Kudrowitz
,
B. M.
, and
Wallace
,
D.
,
2013
, “
Assessing the Quality of Ideas From Prolific, Early-Stage Product Ideation
,”
J. Eng. Des.
,
24
(
2
), pp.
120
139
. 10.1080/09544828.2012.676633
31.
Huang
,
H.-Z.
,
Liu
,
Y.
,
Li
,
Y.
,
Xue
,
L.
, and
Wang
,
Z.
,
2013
, “
New Evaluation Methods for Conceptual Design Selection Using Computational Intelligence Techniques
,”
J. Mech. Sci. Technol.
,
27
(
3
), pp.
733
746
. 10.1007/s12206-013-0123-x
32.
Baucus
,
M. S.
,
Norton
,
M. I.
,
Baucus
,
D. A.
, and
Human
,
S. E.
,
2007
, “
Fostering Creativity and Innovation Without Encouraging Unethical Behavior
,”
J. Bus. Ethics
,
81
(
1
), pp.
97
115
. 10.1007/s10551-007-9483-4
33.
Maurer
,
C.
, and
Widmann
,
J.
,
2012
, “
Conceptual Design Theory in Education Versus Practice in Industry: A Comparison Between Germany and the United States
,”
Design Engineering and Technical Conferences
,
Chicago, IL
,
Aug. 12–15
, pp.
277
283
.
34.
Pugh
,
S.
,
1991
,
Total Design: Integrated Methods for Successful Product Engineering
,
Addison-Wesley
,
Workingham
.
35.
Akao
,
Y.
,
1994
, “
Development History of Quality Function Deployment
,”
The Customer Driven Approach to Quality Planning and Deployment
,
Asian Productivity Organization
,
Minato, Tokyo
, p.
339
.
36.
Marsh
,
E. R.
,
Slocum
,
A. H.
, and
Otto
,
K. N.
,
1993
,
Hierarchical Decision Making in Machine Design
,
MIT Precision Engineering Research Center
,
Cambridge, MA
.
37.
Saaty
,
T. L.
,
1994
, “
How to Make a Decision: The Analytic Hierarchy Process
,”
Interfaces
,
24
(
6
), pp.
19
43
. 10.1287/inte.24.6.19
38.
López-Mesa
,
B.
, and
Bylund
,
N.
,
2011
, “
A Study of the Use of Concept Selection Methods From Inside a Company
,”
Res. Eng. Design
,
22
(
1
), pp.
7
27
. 10.1007/s00163-010-0093-2
39.
Toh
,
C.
,
Miele
,
L.
, and
Miller
,
S.
,
2015
, “
Which One Should I Pick? Concept Selection in Engineering Design Industry
,”
Design Engineering Technical Conferences
,
Boston, MA
,
DETC2015-46522
.
40.
Salonen
,
M.
, and
Perttula
,
M.
,
2005
, “
Utilization of Concept Selection Methods: A Survey of Finnish Industry
,”
ASME Design Engineering Technical Conferences
,
Long Beach, CA
, pp.
527
535
.
41.
Dym
,
C. L.
,
Wood
,
W. H.
, and
Scott
,
M. J.
,
2002
, “
Rank Ordering Engineering Designs: Pairwise Comparison Charts and Borda Counts
,”
Res. Eng. Des.
,
13
(
1
), pp.
236
242
. 10.1007/s00163-002-0019-8
42.
Toh
,
C. A.
, and
Miller
,
S. R.
,
2015
, “
How Engineering Teams Select Design Concepts: A View Through the Lens of Creativity
,”
Des. Stud.
,
38
, pp.
111
138
. 10.1016/j.destud.2015.03.001
43.
Starkey
,
E.
,
Toh
,
C. A.
, and
Miller
,
S. R.
,
2016
, “
Abandoning Creativity: The Evolution of Creative Ideas in Engineering Design Course Projects
,”
Des. Stud.
,
47
, pp.
47
72
. 10.1016/j.destud.2016.08.003
44.
Paulus
,
P. B.
,
2000
, “
Groups, Teams, and Creativity: The Creative Potential of Idea-Generating Groups
,”
Appl. Psychol.
,
49
(
2
), pp.
237
262
. 10.1111/1464-0597.00013
45.
Naquin
,
C. E.
, and
Tynan
,
R. O.
,
2003
, “
The Team Halo Effect: Why Teams are Not Blamed for Their Failures
,”
J. Appl. Psychol.
,
88
(
2
), pp.
332
340
. 10.1037/0021-9010.88.2.332
46.
Alnuaimi
,
O. A.
,
Robert
,
L. P.
, and
Maruping
,
L. M.
,
2010
, “
Team Size, Dispersion, and Social Loafing in Technology-Supported Teams: A Perspective on the Theory of Moral Disengagement
,”
J. Manag. Inform. Syst.
,
27
(
1
), pp.
203
230
. 10.2753/MIS0742-1222270109
47.
Larey
,
T. S.
, and
Paulus
,
P. B.
,
1999
, “
Group Preference and Convergent Tendencies in Small Groups: A Content Analysis of Group Brainstorming Performance
,”
Creat. Res. J.
,
12
(
3
), pp.
175
184
. 10.1207/s15326934crj1203_2
48.
Cagan
,
J.
, and
Vogel
,
C. M.
,
2002
,
Creating Breakthrough Products: Innovation From Product Planning to Program Approval
,
Ft Press
.
49.
Cooper
,
R. G.
, and
Kleinschmidt
,
E. J.
,
1996
, “
Winning Businesses in Product Development: The Critical Success Factors
,”
Res. Technol. Manag.
,
39
(
4
), pp.
18
29
. 10.1080/08956308.1996.11671073
50.
Cushman
,
W. H.
, and
Rosenberg
,
D. J.
,
1991
,
Advances in Human Factors/Ergonomics
,
Elsevier
.
51.
Danneels
,
E.
, and
Kleinschmidtb
,
E. J.
,
2001
, “
Product Innovativeness From the Firm’s Perspective: Its Dimensions and Their Relation With Project Selection and Performance
,”
J. Prod. Innov. Manage.
,
18
(
6
), pp.
357
373
. 10.1111/1540-5885.1860357
52.
Bylund
,
N.
, “
Concept Selection in the Automotive Industry With Examples
,”
Proc. DS 36: Proceedings DESIGN 2006, The 9th International Design Conference
,
Dubrovnik, Croatia
.
53.
Zhang
,
P.
,
Sadler
,
C. M.
,
Lyon
,
S. A.
, and
Martonosi
,
M.
,
2004
, “
Hardware Design Experiences in ZebraNet
,”
Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems
,
Baltimore, MD
,
November
, ACM, pp.
227
238
.
54.
Ha
,
L.
, and
Ganahl
,
R.
,
2004
, “
Webcasting Business Models of Clicks-and-Bricks and Pure-Play Media: A Comparative Study of Leading Webcasters in South Korea and the United States
,”
Int. J. Media Manage.
,
6
(
1–2
), pp.
74
87
. 10.1207/s14241250ijmm0601&2_10
55.
Batory
,
D.
, and
O'malley
,
S.
,
1992
, “
The Design and Implementation of Hierarchical Software Systems With Reusable Components
,”
ACM Trans. Soft. Eng. Methodol.
,
1
(
4
), pp.
355
398
. 10.1145/136586.136587
56.
Simonson
,
I.
,
1993
, “
Get Closer to Your Customers by Understanding How They Make
,”
Calif. Manage. Rev.
,
35
(
4
), pp.
68
84
. 10.2307/41166755
57.
Kim
,
E.
,
Chung
,
J.
,
Beckman
,
S.
, and
Agogino
,
A. M.
,
2016
, “
Design Roadmapping: A Framework and Case Study on Planning Development of High-Tech Products in Silicon Valley
,”
ASME J. Mech. Des.
,
138
(
10
), p.
101106
. 10.1115/1.4034221
58.
Arnold
,
H.
,
Erner
,
M.
,
Mckel
,
P.
, and
Schlf˜fer
,
C.
,
2010
,
Applied Technology and Innovation Management
,
Springer
,
New York
.
59.
Fischer
,
G.
,
2013
, “Learning, Social Creativity, and Cultures of Participation,”
Learning and Collective Creativity: Activity-Theoretical and Sociocultural Studies
,
A.
Sannino
, and
V.
Ellis
, eds.,
Routledge
,
New York, NY
, p.
198
.
60.
Shah
,
J. J.
,
Vargas-Hernandez
,
N.
, and
Smith
,
S. M.
,
2003
, “
Metrics for Measuring Ideation Effectiveness
,”
Des. Stud.
,
24
(
1
), pp.
111
134
. 10.1016/S0142-694X(02)00034-0
61.
Goel
,
P. S.
, and
Singh
,
N.
,
1998
, “
Creativity and Innovation in Durable Product Development
,”
Comput. Ind. Eng.
,
35
(
1
), pp.
5
8
. 10.1016/S0360-8352(98)00006-0
62.
Abbey
,
A.
, and
Dickson
,
J. W.
,
1983
, “
R&D Work Climate and Innovation in Semiconductors
,”
Acad. Manage. J.
,
26
(
2
), pp.
362
368
.
63.
Unsworth
,
K. L.
, and
Clegg
,
C. W.
,
2010
, “
Why do Employees Undertake Creative Action?
,”
J. Occup. Organ. Psychol.
,
83
(
1
), pp.
77
99
. 10.1348/096317908X398377
64.
Weber
,
E. U.
, and
Milliman
,
R. A.
,
1997
, “
Perceived Risk Attitudes: Relating Risk Perception to Risky Choice
,”
Manage. Sci.
,
43
(
2
), pp.
123
144
. 10.1287/mnsc.43.2.123
65.
Mellers
,
B. A.
,
Schwartz
,
A.
, and
Weber
,
E. U.
,
1997
, “Do Risk Attitudes Reflect in the Eye of the Beholder?”
Choice, Decision, and Measurement: Essays in Honor of R. Duncan Luce
,
A. A. J.
Marley
, ed.,
Routledge
,
New York
, pp.
57
71
.
66.
Schwartz
,
S.
, and
Griffin
,
T.
,
1986
, “Evaluating Medical Information,”
Medical Thinking
,
Springer
,
New York
, pp.
56
96
.
67.
Pommer
,
A.
,
2015
, “
How do Firms Evaluate and Select Idea
,”
Masters of Science, Master’s thesis
,
University of Zurich
.
68.
Elo
,
S.
, and
Kyngäs
,
H.
,
2008
, “
The Qualitative Content Analysis Process
,”
J. Adv. Nurs.
,
62
(
1
), pp.
107
115
. 10.1111/j.1365-2648.2007.04569.x
69.
QSR, I. P. L.
,
2015
, “
NVivo Qualitative Data Analysis Software Version 11.1.1
”.
70.
Landis
,
J. R.
, and
Koch
,
G. G.
,
1977
, “
The Measurement of Observer Agreement for Categorical Data
,”
Biometrics
,
33
(
1
), pp.
159
174
. 10.2307/2529310
71.
West
G. P.
, III
,
2007
, “
Collective Cognition: When Entrepreneurial Teams, Not Individuals, Make Decisions
,”
Entrep. Theory Pract.
,
31
(
1
), pp.
77
102
. 10.1111/j.1540-6520.2007.00164.x
72.
QSR, I. P. L.
,
2018
, “
NVivo Qualitative Data Analysis Software Version 12
”.
73.
IBM
, Released
2017
, “
IBM SPSS Statistics for Mac, Version 25.0
,”
IBM Corp.
,
Armonk, NY
.
74.
Bradley
,
S.
,
2010
, “
Designing for a Hierarchy of Needs
”, https://www.smashingmagazine.com/2010/04/designing-for-a-hierarchy-of-needs/
75.
Kennedy
,
P.
,
1998
,
A Guide to Economics
,
MIT Press
,
Cambridge, MA
.
76.
Brown
,
D.
,
2013
, “
Developing Computational Design Creativity Systems
,”
Int. J. Des. Creat. Innov.
,
1
(
1
), pp.
43
55
. 10.1080/21650349.2013.754651
77.
Dewar
,
R. D.
, and
Dutton
,
J. E.
,
1986
, “
The Adoption of Radical and Incremental Innovations: An Empirical Analysis
,”
Manage. Sci.
,
32
(
11
), pp.
1422
1433
. 10.1287/mnsc.32.11.1422
78.
Jehn
,
K. A.
,
Northcraft
,
G. B.
, and
Neale
,
M. A.
,
1999
, “
Why Differences Make a Difference: A Field Study of Diversity, Conflict and Performance in Workgroups
,”
Adm. Sci. Q.
,
44
(
4
), pp.
741
763
. 10.2307/2667054
You do not currently have access to this content.