Many tools, techniques, and other interventions have been developed to support idea generation within the design process. In previous research, we explored the separate effects of three such design interventions: teaming, problem framing, and design heuristics. In the teaming intervention, participants discussed a design prompt together but recorded their own ideas separately. In problem framing, multiple versions (framings) of each design prompt were used to elicit different solutions. In design heuristics, participants used specially designed cards to prompt new ways of thinking about the given design problem. In the current work, we compared the effects of these three interventions on students' design ideas with respect to one idea attribute in particular—quality. In total, 1088 design concepts were collected from 171 undergraduate students in engineering and industrial design from two universities. Individual cognitive style was also assessed using Kirton's Adaption–Innovation inventory (KAI). Six metrics taken from the design literature were used to assess the quality of each concept, namely: acceptability, applicability, clarity, effectiveness, implementability, and implicational explicitness. Paired t-tests and Pearson correlations were used to assess differences in quality between concepts generated with and without the three interventions; in addition, secondary effects were sought based on the cognitive styles and academic standings of the participants. Statistically significant differences were observed in design concept quality for the teaming and design heuristics interventions over the full sample and for some subgroups separated by cognitive style and academic standing. These results have implications for how educators teach design interventions and how students choose and apply interventions to affect the quality of their own design solutions.

References

References
1.
Shah
,
J. J.
,
Kulkarni
,
S. V.
, and
Vargas-Hernandez
,
N.
,
2000
, “
Evaluation of Idea Generation Methods for Conceptual Design: Effectiveness Metrics and Design of Experiments
,”
ASME J Mech. Des.
,
122
(
4
), pp.
377
384
.
2.
Daly
,
S. R.
,
Christian
,
J. L.
,
Yilmaz
,
S.
,
Seifert
,
C. M.
, and
Gonzalez
,
R.
,
2012
, “
Assessing Design Heuristics for Idea Generation in an Introductory Engineering Course
,”
Int. J. Eng. Educ.
,
28
(2), pp. 463–473.https://lib.dr.iastate.edu/industrialdesign_pubs/5/
3.
Silk
,
E. M.
,
Daly
,
S. R.
,
Jablokow
,
K. W.
,
Yilmaz
,
S.
, and
Rosenberg
,
M.
,
2014
, “
The Design Problem Framework: Using Adaption-Innovation Theory to Construct Design Problem Statements
,”
ASEE Annual Conference
, Indianapolis, IN, June 15–18, Paper No. 8781.https://lib.dr.iastate.edu/industrialdesign_conf/7/
4.
Yilmaz
,
S.
,
Daly
,
S. R.
,
Jablokow
,
K.
,
Silk
,
E. M.
, and
Rosenberg
,
M.
,
2014
, “
Investigating Impacts on the Ideation Flexibility of Engineers
,”
ASEE Annual Conference
,
Indianapolis, IN
,
June 15–18
, Paper No.
8808
https://www.researchgate.net/publication/286743303_Investigating_impact_on_the_ideation_flexibility_of_engineers.
5.
Rechkemmer
,
A.
,
Makhlouf
,
M.
,
Wenger
,
J. M.
,
Silk
,
E. M.
,
Daly
,
S. R.
,
McKilligan
,
S.
, and
Jablokow
,
K. W.
,
2017
, “
Examining the Effect of a Paradigm-Relatedness Problem-Framing Tool on Idea Generation
,”
ASEE Annual Conference
,
Columbus, OH
,
June 25–28
, Paper No. 18507.
6.
Wright
,
S.
,
Silk
,
E. M.
,
Daly
,
S. R.
,
Jablokow
,
K. W.
,
Yilmaz
,
S.
, and
Teerlink
,
W.
,
2015
, “
Exploring the Effects of Problem Framing on Solution Shifts: A Case Analysis
,”
ASEE Annual Conference
,
Seattle, WA
,
June 14–17
, Paper No.
11638
https://lib.dr.iastate.edu/industrialdesign_conf/11/.
7.
Dean
,
D. L.
,
Hender
,
J. M.
,
Rodgers
,
T. L.
, and
Santanen
,
E. L.
,
2006
, “
Identifying Quality, Novel, and Creative Ideas: Constructs and Scales for Idea Evaluation
,”
J. Assoc. Inf. Syst.
,
7
(
10
), pp.
649
699
.https://aisel.aisnet.org/jais/vol7/iss10/30/
8.
Maher
,
M.
, and
Poon
,
J.
,
1996
, “
Modeling Design Exploration as Co-Evolution
,”
Comput. - Aided Civ. Infrastruct. Eng.
,
11
(
3
), pp.
195
209
.
9.
Cross
,
N.
,
2004
, “
Expertise in Design: An Overview
,”
Des. Stud.
,
25
(
5
), pp.
427
441
.
10.
Dorst
,
K.
, and
Cross
,
N.
,
2001
, “
Creativity in the Design Process: Co-Evolution of Problem-Solution
,”
Des. Stud.
,
22
(
5
), pp.
425
437
.
11.
Liu
,
Y.
,
Bligh
,
T.
, and
Chakrabarti
,
A.
,
2003
, “
Towards an 'Ideal' Approach for Concept Generation
,”
Des. Stud.
,
24
(
4
), pp.
341
355
.
12.
Yilmaz
,
S.
,
Seifert
,
C.
,
Daly
,
S. R.
, and
Gonzalez
,
R.
,
2016
, “
Design Heuristics in Innovative Products
,”
ASME J. Mech. Des.
,
138
(
7
), p.
071102
.
13.
Purcell
,
A. T.
, and
Gero
,
J. S.
,
1996
, “
Design and Other Types of Fixation
,”
Des. Stud.
,
17
(
4
), pp.
363
383
.
14.
Viswanathan
,
V. K.
, and
Linsey
,
J. S.
,
2013
, “
Design Fixation and Its Mitigation: A Study on the Role of Expertise
,”
ASME J. Mech. Des.
,
135
(
5
), p.
051008
.
15.
Jablokow
,
K.
, and
Kirton
,
M. J.
,
2009
, “
Problem Solving, Creativity, and the Level-Style Distinction
,”
Perspectives on the Nature of Intellectual Styles
,
L.-F.
Zhang
and
R. J.
Sternberg
, eds.,
Springer
,
New York
, pp.
137
168
.
16.
Kirton
,
M. J.
,
2011
,
Adaption-Innovation in the Context of Diversity and Change
,
Routledge
,
London
.
17.
Sternberg
,
R. J.
,
1988
,
The Triarchic Mind: A New Theory of Intelligence
,
Viking Press
,
New York
.
18.
Sternberg
,
R. J.
,
1997
,
Thinking Styles
,
Cambridge University Press
,
New York
.
19.
Sternberg
,
R. J.
,
2003
,
Cognitive Psychology
,
3rd ed.
,
Thomson Wadsworth
,
Belmont, CA
.
20.
Sternberg
,
R. J.
, and
Grigorenko
,
E.
,
1997
, “
Are Cognitive Styles Still in Style?
,”
Am. Psychol.
,
52
(
7
), pp.
700
712
.
21.
Sternberg
,
R. J.
, and
Grigorenko
,
E. L.
,
2001
, “
A Capsule History of Theory and Research on Styles
.,”
Perspectives on Thinking, Learning, and Cognitive Styles
,
R. J.
Sternberg
and
L.-F.
Zhang
, eds.,
Lawrence Erlbaum Associates
,
Mahwah, NJ
, pp.
1
22
.
22.
Abernathy
,
W. J.
, and
Utterback
,
J. M.
,
1978
, “
Patterns of Industrial Innovation
,”
Technol. Rev.
,
80
(7), pp.
40
47
.http://www.academia.edu/23341473/Patterns_of_Industrial_Innovation
23.
Jablokow
,
K. W.
,
2008
, “
Developing Problem Solving Leadership: A Cognitive Approach
,”
Int. J. Eng. Educ.
,
24
(5), pp.
936
954
.https://www.researchgate.net/publication/233691102_Developing_Problem_Solving_Leadership_A_Cognitive_Approach
24.
Samuel
,
P.
, and
Jablokow
,
K. W.
,
2011
, “
Toward an Adaption-Innovation Strategy for Engineering Design
,”
International Conference on Engineering Design
(
ICED
),
Copenhagen, Denmark
,
Aug. 15–18
, pp.
377
386
.https://www.designsociety.org/download-publication/30488/TOWARD+AN+ADAPTION-INNOVATION+STRATEGY+FOR+ENGINEERING+DESIGN
25.
Valle
,
S.
, and
Vasquez-Bustelo
,
D.
,
2009
, “
Concurrent Engineering Performance: Incremental Versus Radical Innovation
,”
Int. J. Prod. Econ.
,
119
(
1
), pp.
136
148
.
26.
Carberry
,
A. R.
,
Lee
,
H.-S.
, and
Ohland
,
M. W.
,
2010
, “
Measuring Engineering Design Self-Efficacy
,”
J. Eng. Educ.
,
99
(
1
), pp.
71
79
.
27.
Felder
,
R. M.
, and
Brent
,
R.
,
2005
, “
Understanding Student Differences
,”
J. Eng. Educ.
,
94
(
1
), pp.
57
72
.
28.
Felder
,
R. M.
,
Felder
,
G. N.
, and
Dietz
,
E. J.
,
2002
, “
The Effects of Personality Type on Engineering Student Performance and Attitudes
,”
J. Eng. Educ.
,
91
(
1
), pp.
3
17
.
29.
Kirton
,
M. J.
, and
de Ciantis
,
S. M.
,
1986
, “
Cognitive Style and Personality: The Kirton Adaption–Innovation and Cattell's Sixteen Personality Factor Inventories
,”
Pers. Individ. Differ.
,
7
(2), pp.
14
146
.
30.
Goldsmith
,
R. E.
,
1987
, “
Creative Level and Creative Style
,”
Br. J. Soc. Psychol.
,
26
(
4
), pp.
317
323
.
31.
Hammerschmidt
,
P. K.
, and
Jennings
,
A. C.
,
1992
, “
The Impact of Personality Characteristics on Leadership Effectiveness Ratings
,”
Impact of Leadership
,
K. E.
Clark
,
M. E.
Clark
, and
D. P.
Campbell
, eds.
Center for Creative Leadership
,
Greensboro, NC
, pp.
469
475
.
32.
Buttner
,
E. H.
,
Gryskiewicz
,
N.
, and
Hidore
,
S.
,
1999
, “
The Relationship Between Styles of Creativity and Management Skills Assessment
,”
Br. J. Manage.
,
10
(
3
), pp.
228
238
.
33.
Silk
,
E. M.
,
Daly
,
S. R.
,
Jablokow
,
K. W.
,
Yilmaz
,
S.
,
Rechkemmer
,
A.
, and
Wenger
,
J. M.
,
2016
, “
Using Paradigm-Relatedness to Measure Design Ideation Shifts
,”
ASEE Annual Conference
,
New Orleans, LA
,
June 26–29
, Paper No.
17004
https://www.asee.org/public/conferences/64/papers/17004/view.
34.
Jablokow
,
K. W.
,
Teerlink
,
W.
,
Yilmaz
,
S.
,
Daly
,
S.
, and
Silk
,
E.
,
2015
, “
The Impact of Teaming and Cognitive Style on Student Perceptions of Design Ideation Outcomes
,”
ASEE Annual Conference
,
Seattle, WA
,
June 14–17
, Paper No.
11855
https://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1011&context=industrialdesign_conf.
35.
Jablokow
,
K.
,
Teerlink
,
W.
,
Yilmaz
,
S.
,
Daly
,
S.
,
Silk
,
E.
, and
Wehr
,
C.
,
2015
, “
Ideation Variety in Mechanical Design: Examining the Effects of Cognitive Style and Design Heuristics
,”
ASME
Paper No. DETC2015-46334
.
36.
DeFranco
,
J. F.
,
Jablokow
,
K. W.
,
Bilén
,
S. G.
, and
Gordon
,
A.
,
2012
, “
The Impact of Cognitive Style on Concept Mapping: Visualizing Variations in the Structure of Ideas
,”
ASEE Annual Conference
,
San Antonio, TX
,
June 10–13
, Paper No. 3118.
37.
Austin-Breneman
,
J.
,
Honda
,
T.
, and
Yang
,
M. C.
,
2012
, “
A Study of Student Design Team Behaviors in Complex System Design
,”
ASME J. Mech. Des.
,
134
(
12
), p.
124504
.
38.
Dinar
,
M.
,
Shah
,
J.
,
Cagan
,
J.
,
Leifer
,
L. J.
,
Linsey
,
J. S.
,
Smith
,
S. M.
, and
Vargas-Hernandez
,
N.
,
2015
, “
Empirical Studies of Designer Thinking: Past, Present, and Future
,”
ASME J. Mech. Des.
,
137
(
2
), p.
021101
.
39.
Fu
,
K.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2010
, “
Design Team Convergence: The Influence of Example Solution Quality
,”
ASME J. Mech. Des.
,
132
(
11
), p.
111005
.
40.
Sosa
,
M. E.
, and
Marle
,
F.
,
2013
, “
Assembling Creative Teams in New Product Development Using Creative Team Familiarity
,”
ASME J. Mech. Des.
,
135
(
8
), p.
081009
.
41.
Page
,
S.
,
2007
,
The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies
,
Princeton University Press
,
Princeton, NJ
.
42.
Paulus
,
P. B.
, and
Yang
,
H. C.
,
2000
, “
Idea Generation in Groups: A Basis for Creativity in Organizations
,”
Organ. Behav. Human Decis. Processes
,
82
(
1
), pp.
76
87
.
43.
Shergadwala
,
M.
,
Bilionis
,
I.
,
Kannan
,
K.
, and
Panchal
,
J. H.
,
2018
, “
Quantifying the Impact of Domain Knowledge and Problem Framing on Sequential Decisions in Engineering Design
,”
ASME J. Mech. Des.
,
140
(
10
), p.
101402
.
44.
Yilmaz
,
S.
,
Rosenberg
,
M. N.
,
Daly
,
S. R.
,
Jablokow
,
K. W.
,
Silk
,
E. M.
, and
Teerlink
,
W.
,
2015
, “
Impact of Problem Contexts on the Diversity of Design Solutions: An Exploratory Case Study
,”
ASEE Annual Conference
,
Seattle, WA
,
June 14–17
, Paper No.
11206
https://www.researchgate.net/publication/285057385_Impact_of_problem_contexts_on_the_diversity_of_design_solutions_An_exploratory_case_study?ev=prf_high.
45.
Daly
,
S. R.
,
Seifert
,
C. M.
,
Yilmaz
,
S.
, and
Gonzalez
,
R.
,
2016
, “
Comparing Ideation Techniques for Beginning Designers
,”
ASME J. Mech. Des.
,
138
(
10
), p.
101108
.
46.
Linsey
,
J. S.
,
Clauss
,
E. F.
,
Kurtoglu
,
T.
,
Murphy
,
J. T.
,
Wood
,
K. L.
, and
Markman
,
A. B.
,
2011
, “
An Experimental Study of Group Idea Generation Techniques: Understanding the Roles of Idea Representation and Viewing Methods
,”
ASME J. Mech. Des.
,
133
(
3
), p.
031008
.
47.
Zheng
,
X.
,
Ritter
,
S. C.
, and
Miller
,
S. R.
,
2018
, “
How Concept Selection Tools Impact the Development of Creative Ideas in Engineering Design Education
,”
ASME J. Mech. Des.
,
140
(
5
), p.
052002
.
48.
Daly
,
S. R.
,
Yilmaz
,
S.
,
Christian
,
J. L.
,
Seifert
,
C. M.
, and
Gonzalez
,
R.
,
2012
, “
Design Heuristics in Engineering Concept Generation
,”
J. Eng. Educ.
,
101
(
4
), pp.
601
629
.
49.
Yilmaz
,
S.
,
Daly
,
S. R.
,
Christian
,
J. L.
,
Seifert
,
C. M.
, and
Gonzalez
,
R.
, 2014, “
Design Heuristics: Strategies to Inspire Ideas
,” Design Heuristics, Des Moines, IA, accessed Nov. 17, 2017, www.designheuristics.com
50.
Yilmaz
,
S.
,
Daly
,
S. R.
, and
Seifert
,
C. M.
, 2014, “
Design Heuristics: An Evidence-Based Tool to Improve Innovation
,” ASEE Annual Conference, Indianapolis, IN, June 15–18, Paper No.
8959
.https://lib.dr.iastate.edu/cgi/viewcontent.cgi?referer=https://www.google.co.in/&httpsredir=1&article=1003&context=industrialdesign_conf
51.
Shah
,
J. J.
,
Smith
,
S. M.
, and
Vargas-Hernandez
,
N.
,
2003
, “
Metrics for Measuring Ideation Effectiveness
,”
Des. Stud.
,
24
(
2
), pp.
111
134
.
52.
Besemer
,
S.
,
1998
, “
Creative Product Analysis Matrix: Testing the Model Structure and a Comparison Among Products-Three Novel Chairs
,”
Creativity Res. J.
,
11
(
4
), pp.
333
346
.
53.
Ahmed
,
F.
, and
Fuge
,
M.
,
2017
, “
Ranking Ideas for Diversity and Quality
,”
ASME J. Mech. Des.
,
140
(
1
), p.
011101
.
54.
Henderson
,
D.
,
Helm
,
K.
,
Jablokow
,
K. W.
,
McKilligan
,
S.
,
Daly
,
S. R.
, and
Silk
,
E. M.
,
2017
, “
A Comparison of Variety Metrics in Engineering Design
,”
ASME
Paper No. DETC2017-67502
.
55.
Helm
,
K. C.
,
Jablokow
,
K. W.
,
Daly
,
S. R.
,
Silk
,
E. M.
,
Yilmaz
,
S.
, and
Suero
,
R.
,
2016
, “
Evaluating the Impacts of Different Interventions on Quality in Concept Generation
,”
ASEE Annual Conference
,
New Orleans, LA
,
June 26–29
, Paper No.
15648
https://www.asee.org/public/conferences/64/papers/15648/view.
56.
Edelman
,
J. A.
,
2011
, “
Understanding Radical Breaks: Media and Behavior in Small Teams Engaged in Redesign Scenarios
,” Ph.D. thesis, Stanford University, Stanford, CA.
57.
Sarkar
,
P.
, and
Chakrabarti
,
A.
,
2011
, “
Assessing Design Creativity
,”
Des. Stud.
,
32
(
4
), pp.
348
383
.
58.
Yilmaz
,
S.
,
Daly
,
S. R.
,
Christian
,
J. L.
,
Seifert
,
C. M.
, and
Gonzalez
,
R.
,
2014
, “
Can Experienced Designers Learn From New Tools? A Case Study of Idea Generation in a Professional Engineering Team
,”
Int. J. Des. Creativity Innovation
,
2
(
2
), pp.
82
96
.
59.
Sonalkar
,
N.
,
Mabogunje
,
A.
, and
Cutkosky
,
M.
,
2018
, “
Quadratic Model of Reciprocal Causation for Monitoring, Improving, and Reflecting on Design Team Performance
,”
Design Thinking Research, Understanding Innovation
,
H.
Plattner
ed.,
Springer
,
New York
, pp.
43
57
.
60.
Buffington
,
K. W.
,
Jablokow
,
K. W.
, and
Martin
,
K. A.
,
2002
, “
Project Team Dynamics and Cognitive Style
,”
Eng. Manage. J.
,
14
, pp.
25
33
.
61.
DeFranco
,
J.
, and
Neill
,
C.
,
2011
, “
Problem-Solving Style and Its Impact on Engineering Team Effectiveness
,”
Annual Conference on Systems Engineering Research
,
Los Angeles, CA
,
Apr. 14–16
, pp. 1–13.
62.
Vercellone-Smith
,
P.
,
Jablokow
,
K. W.
, and
Friedel
,
C.
,
2012
, “
Characterizing Communication Networks in a Web-Based Classrom: Cognitive Styles and Linguistic Behavior of Self-Organizing Groups in Online Discussions
,”
Comput. Educ.
,
59
, pp.
222
235
.
63.
Baer
,
J.
,
2014
,
Creativity and Divergent Thinking: A Task-Specific Approach
,
Psychology Press
,
London
.
64.
Niku
,
S. B.
,
2008
,
Creative Design of Products and Systems
,
Wiley
,
Hoboken, NJ
.
65.
Teerlink
,
W.
,
2015
, “
Team Ideation: A Detailed Investigation Into Ideation Effectiveness
,”
Master's thesis
, The Pennsylvania State University, University Park, PA.https://etda.libraries.psu.edu/catalog/26388
66.
Laughlin
,
P.
,
2002
, “
Groups Perform Better Than the Best Individuals on Letters-to-Numbers Problems
,”
Organ. Behav. Human Decis. Processes
,
88
(
2
), pp.
605
620
.
67.
Linsey
,
J. S.
,
Murphy
,
J. T.
, and
Markman
,
A. B.
,
2006
, “
Representing Analogies: Increasing the Probability of Innovation
,”
ASME
Paper No. DETC2006-99383
.
68.
Diehl
,
M.
, and
Stroebe
,
W.
,
1987
, “
Productivity Loss in Brainstorming Groups: Toward the Solution of a Riddle
,”
J. Pers. Soc. Psychol.
,
53
(
3
), pp.
497
509
.
69.
Mullen
,
B.
,
Johnson
,
C.
, and
Salas
,
E.
,
1991
, “
Productivity Loss in Brainstorming Groups: A Meta-Analytic Integration
,”
Basic Appl. Soc. Psychol.
,
12
(
1
), pp.
3
23
.
70.
Choo
,
P. K.
,
Lou
,
Z. N.
,
Camburn
,
B. A.
,
Wood
,
K. L.
,
Koo
,
B.
, and
Grey
,
F.
,
2014
, “
Ideation Methods: A First Study on Measured Outcomes With Personality Type
,”
ASME
Paper No. DETC2014-34954
.
71.
Berthelsdorf
,
F. A.
, and
Stone
,
R. B.
,
2017
, “
Creativity of Concept Ideation as Affected by Team Personality
,”
ASME
Paper No. DETC2017-67974
.
72.
Jablokow
,
K. W.
, and
Booth
,
D. E.
,
2006
, “
The Impact and Management of Cognitive Gap in High Performance Product Development Organizations
,”
J. Eng. Technol. Manage.
,
23
(
4
), pp.
313
336
.
You do not currently have access to this content.