Abstract

Design thinking is often hidden and implicit, so empirical approach based on experiments and data-driven methods has been the primary way of doing such research. In support of empirical studies, design behavioral data which reflects design thinking becomes crucial, especially with the recent advances in data mining and machine learning techniques. In this paper, a research platform that supports data-driven design thinking studies is introduced based on a computer-aided design (cad) software for solar energy systems, energy3d, developed by the team. We demonstrate several key features of energy3d including a fine-grained design process logger, embedded design experiment and tutorials, and interactive cad interfaces and dashboard. These features make energy3d a capable testbed for a variety of research related to engineering design thinking and design theory, such as search strategies, design decision-making, artificial intelligent (AI) in design, and design cognition. Using a case study on an energy-plus home design challenge, we demonstrate how such a platform enables a complete research cycle of studying designers” sequential decision-making behaviors based on fine-grained design action data and unsupervised clustering methods. The results validate the utility of energy3d as a research platform and testbed in supporting future design thinking studies and provide domain-specific insights into new ways of integrating clustering methods and design process models (e.g., the function–behavior–structure model) for automatically clustering sequential design behaviors.

References

1.
Braha
,
D.
, and
Maimon
,
O.
,
1997
, “
The Design Process: Properties, Paradigms, and Structure
,”
IEEE Trans. Syst. Man. Cybern., Part A Syst. Hum.
,
27
(
2
), pp.
146
166
. 10.1109/3468.554679
2.
Mostow
,
J.
,
1985
, “
Toward Better Models of the Design Process
,”
AI Mag.
,
6
(
1
), p.
44
. 10.1609/aimag.v6i1.468
3.
Cross
,
N.
, and
Roy
,
R.
,
1989
,
Engineering Design Methods
, Vol.
4
,
Wiley
,
New York
.
4.
Ishino
,
Y.
, and
Jin
,
Y.
,
2002
, “
Acquiring Engineering Knowledge From Design Processes
,”
AI EDAM
,
16
(
2
), pp.
73
91
. 10.1017/S0890060402020073
5.
Herrmann
,
J. W.
,
2010
, “
Progressive Design Processes and Bounded Rational Designers
,”
ASME J. Mech. Des.
,
132
(
8
), p.
081005
. 10.1115/1.4001902
6.
Dym
,
C. L.
,
Agogino
,
A. M.
,
Eris
,
O.
,
Frey
,
D. D.
, and
Leifer
,
L. J.
,
2005
, “
Engineering Design Thinking, Teaching, and Learning
,”
J. Eng. Educ.
,
94
(
1
), pp.
103
120
. 10.1002/j.2168-9830.2005.tb00832.x
7.
Panchal
,
J. H.
,
Sha
,
Z.
, and
Kannan
,
K. N.
,
2017
, “
Understanding Design Decisions Under Competition Using Games With Information Acquisition and a Behavioral Experiment
,”
ASME J. Mech. Des.
,
139
(
9
), p.
091402
. 10.1115/1.4037253
8.
Sexton
,
T.
, and
Ren
,
M. Y.
,
2017
, “
Learning an Optimization Algorithm Through Human Design Iterations
,”
ASME J. Mech. Des.
,
139
(
10
), p.
101404
. 10.1115/1.4037344
9.
Brockmann
,
E. N.
, and
Anthony
,
W. P.
,
1998
, “
The Influence of Tacit Knowledge and Collective Mind on Strategic Planning
,”
J. Manag. Issues
,
10
(
2
), pp.
204
222
.
10.
Ishino
,
Y.
, and
Jin
,
Y.
,
2001
, “Data Mining for Knowledge Acquisition in Engineering Design,”
Data Mining for Design and Manufacturing
,
D.
Braha
, ed., Vol.
3
,
Massive Computing, Springer
,
Boston, MA
, pp.
145
160
.
11.
Toh
,
C. A.
, and
Miller
,
S. R.
,
2014
, “
The Impact of Example Modality and Physical Interactions on Design Creativity
,”
ASME J. Mech. Des.
,
136
(
9
), p.
091004
. 10.1115/1.4027639
12.
Toh
,
C. A.
, and
Miller
,
S. R.
,
2016
, “
Creativity in Design Teams: The Influence of Personality Traits and Risk Attitudes on Creative Concept Selection
,”
Res. Eng. Des.
,
27
(
1
), pp.
73
89
. 10.1007/s00163-015-0207-y
13.
Toh
,
C. A.
, and
Miller
,
S. R.
,
2016
, “
Choosing Creativity: The Role of Individual Risk and Ambiguity Aversion on Creative Concept Selection in Engineering Design
,”
Res. Eng. Des.
,
27
(
3
), pp.
195
219
. 10.1007/s00163-015-0212-1
14.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2017
, “Utilizing Markov Chains to Understand Operation Sequencing in Design Tasks,”
Design Computing and Cognition’16
,
J. S.
Gero
, ed.,
Springer
,
New York
, pp.
401
418
.
15.
Sha
,
Z.
,
Kannan
,
K. N.
, and
Panchal
,
J. H.
,
2015
, “
Behavioral Experimentation and Game Theory in Engineering Systems Design
,”
ASME J. Mech. Des.
,
137
(
5
), p.
051405
. 10.1115/1.4029767
16.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2015
, “
Lifting the Veil: Drawing Insights About Design Teams From a Cognitively-Inspired Computational Model
,”
Des. Stud.
,
40
, pp.
119
142
. 10.1016/j.destud.2015.06.005
17.
Yu
,
B. Y.
,
Honda
,
T.
,
Sharqawy
,
M.
, and
Yang
,
M.
,
2016
, “
Human Behavior and Domain Knowledge in Parameter Design of Complex Systems
,”
Des. Stud.
,
45
(
Part B
), pp.
242
267
. 10.1016/j.destud.2016.04.005
18.
Egan
,
P.
,
Schunn
,
C.
,
Cagan
,
J.
, and
LeDuc
,
P.
,
2015
, “
Improving Human Understanding and Design of Complex Multi-Level Systems With Animation and Parametric Relationship Supports
,”
Des. Sci.
,
1
, p.
e3
. 10.1017/dsj.2015.3
19.
Jin
,
Y.
, and
Ishino
,
Y.
,
2006
, “
DAKA: Design Activity Knowledge Acquisition Through Data-Mining
,”
Int. J. Prod. Res.
,
44
(
14
), pp.
2813
2837
. 10.1080/00207540600654533
20.
Gopsill
,
J.
,
Snider
,
C.
,
Shi
,
L.
, and
Ben
,
H.
,
2016
, “
Computer Aided Design User Interaction as a Sensor for Monitoring Engineers and the Engineering Design Process
,”
Proceedings of the Design 2016 14th International Design Conference
,
Cavtat, Dubrovnik, Croatia
,
May
, pp.
1707
1718
.
21.
Ritchie
,
J. M.
,
Lim
,
T.
,
Sung
,
R. C. W.
,
Corney
,
J. R.
,
Rea
,
H.
,
2008
, “The Analysis of Design and Manufacturing Tasks Using Haptic and Immersive VR—Some Case Studies,”
Product Engineering
,
D.
Talaba
, ed.,
Springer
,
New York
, pp.
507
522
.
22.
Sivanathan
,
A.
,
Lim
,
T.
,
Ritchie
,
J.
,
Sung
,
R.
,
Kosmadoudi
,
Z.
, and
Liu
,
Y.
,
2015
, “
The Application of Ubiquitous Multimodal Synchronous Data Capture in CAD
,”
Comput. Aided Des.
,
59
, pp.
176
191
. 10.1016/j.cad.2013.10.001
23.
Sen
,
C.
,
Parrish
,
Q.
, and
Galil
,
O.
,
2017
, “
Measuring Information Content of Freehand Sketches Using a Cognitive Chunk Visualization Protocol
,”
ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Volume 7: 29th International Conference on Design Theory and Methodology
,
Cleveland, OH
,
Aug. 6–9
, ASME Paper No. V007T06A035.
24.
Gero
,
J.
,
Yu
,
R.
, and
Wells
,
J.
,
2018
, “
Creative Design Cognition Differences Between High School Students With and Without Design Education
,”
The Fifth International Conference on Design Creativity
,
Bath, UK
,
Jan. 31– Feb. 2
, pp.
240
247
.
25.
Xie
,
C.
,
Schimpf
,
C.
,
Chao
,
J.
,
Nourian
,
S.
, and
Massicotte
,
J.
,
2018
, “
Learning and Teaching Engineering Design Through Modeling and Simulation on a CAD Platform
,”
Comput. Appl. Eng. Educ.
,
26
(
4
), pp.
824
840
. 10.1002/cae.21920
26.
Rahman
,
M. H.
,
Gashler
,
M.
,
Xie
,
C.
, and
Sha
,
Z.
,
2018
, “
Automatic Clustering of Sequential Design Behaviors
,”
ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Volume 1B: 38th Computers and Information in Engineering Conference
,
Quebec City, Quebec, Canada
,
Aug. 26–29
, ASME Paper No. V01BT02A041.
27.
Dinar
,
M.
,
Shah
,
J. J.
,
Cagan
,
J.
,
Leifer
,
L.
,
Linsey
,
J.
,
Smith
,
S. M.
, and
Hernandez
,
N. V.
,
2015
, “
Empirical Studies of Designer Thinking: Past, Present, and Future
,”
ASME J. Mech. Des.
,
137
(
2
), p.
021101
. 10.1115/1.4029025
28.
Gero
,
J. S.
, and
Mc Neill
,
T.
,
1998
, “
An Approach to the Analysis of Design Protocols
,”
Des. Stud.
,
19
(
1
), pp.
21
61
. 10.1016/S0142-694X(97)00015-X
29.
Cross
,
N.
,
2001
,
Design Knowing and Learning: Cognition in Design Education
,
C. M.
Eastman
,
W. M.
McCracken
, and
W. C.
Newstetter
, eds.,
Elsevier
, pp.
79
103
.
30.
Moreno
,
D. P.
,
Hernández
,
A. A.
,
Yang
,
M. C.
, and
Wood
,
K. L.
,
2014
, “
Creativity in Transactional Design Problems: Non-Intuitive Findings of an Expert Study Using Scamper
,”
13th International Design Conference, Human Behavior and Design
,
Dubrovnik, Croatia
,
May 19–22
, pp
569
578
.
31.
Maher
,
M. L.
,
Lee
,
L.
,
Gero
,
J.
,
Yu
,
R.
, and
Clausne
,
T.
,
2017
, “Characterizing Tangible Interaction During a Creative Combination Task,”
Design Computing and Cognition’16
,
J. S.
Gero
, ed.,
Springer
,
New York
, pp.
39
58
.
32.
Toh
,
C. A.
,
Miller
,
S. R.
, and
Kremer
,
G. E. O.
,
2013
, “
The Role of Personality and Team-Based Product Dissection on Fixation Effects
,”
Adv. Eng. Educ.
,
3
(
4
), p.
1
.
33.
Moreno
,
D. P.
,
Blessing
,
L. T.
,
Yang
,
M. C.
,
Hernández
,
A. A.
, and
Wood
,
K. L.
,
2016
, “
Overcoming Design Fixation: Design by Analogy Studies and Nonintuitive Findings
,”
AI EDAM
,
30
(
2
), pp.
185
199
.
34.
Viswanathan
,
V.
, and
Linsey
,
J.
,
2013
, “Mitigation of Design Fixation in Engineering Idea Generation: A Study on the Role of Defixation Instructions,”
ICoRD’13
,
A.
Chakrabarti
, and
R. V.
Prakash
, eds.,
Springer
,
New York
, pp.
113
124
.
35.
Viswanathan
,
V.
, and
Linsey
,
J.
,
2013
, “
Examining Design Fixation in Engineering Idea Generation: The Role of Example Modality
,”
Int. J. Des. Creativ. Innovat.
,
1
(
2
), pp.
109
129
. 10.1080/21650349.2013.774689
36.
Linsey
,
J.
,
Wood
,
K. L.
, and
Markman
,
A. B.
,
2008
, “
Modality and Representation in Analogy
,”
AI EDAM
,
22
(
2
), pp.
85
100
.
37.
Bao
,
Q.
,
Faas
,
D.
, and
Yang
,
M.
,
2018
, “
Interplay of Sketching & Prototyping in Early Stage Product Design
,”
Int. J. Des. Creativ. Innovat.
,
6
(
3–4
), pp.
146
168
. 10.1080/21650349.2018.1429318
38.
Bilda
,
Z.
,
Gero
,
J. S.
, and
Purcell
,
T.
,
2006
, “
To Sketch or Not to Sketch? That Is the Question
,”
Des. Stud.
,
27
(
5
), pp.
587
613
. 10.1016/j.destud.2006.02.002
39.
Tang
,
H.
,
Lee
,
Y.
, and
Gero
,
J.
,
2011
, “
Comparing Collaborative Co-Located and Distributed Design Processes in Digital and Traditional Sketching Environments: A Protocol Study Using the Function–Behaviour–Structure Coding Scheme
,”
Des. Stud.
,
32
(
1
), pp.
1
29
. 10.1016/j.destud.2010.06.004
40.
Menezes
,
A.
, and
Lawson
,
B.
,
2006
, “
How Designers Perceive Sketches
,”
Des. Stud.
,
27
(
5
), pp.
571
585
. 10.1016/j.destud.2006.02.001
41.
Kavakli
,
M.
, and
Gero
,
J. S.
,
2002
, “
The Structure of Concurrent Cognitive Actions: A Case Study on Novice and Expert Designers
,”
Des. Stud.
,
23
(
1
), pp.
25
40
. 10.1016/S0142-694X(01)00021-7
42.
Ball
,
L. J.
,
Ormerod
,
T. C.
, and
Morley
,
N. J.
,
2004
, “
Spontaneous Analogising in Engineering Design: A Comparative Analysis of Experts and Novices
,”
Des. Stud.
,
25
(
5
), pp.
495
508
. 10.1016/j.destud.2004.05.004
43.
Ahmed
,
S.
, and
Christensen
,
B. T.
,
2009
, “
An In Situ Study of Analogical Reasoning in Novice and Experienced Design Engineers
,”
ASME J. Mech. Des.
,
131
(
11
), p.
111004
. 10.1115/1.3184693
44.
Atman
,
C. J.
,
Cardella
,
M. E.
,
Turns
,
J.
, and
Adams
,
R.
,
2005
, “
Comparing Freshman and Senior Engineering Design Processes: An In-Depth Follow-Up Study
,”
Des. Stud.
,
26
(
4
), pp.
325
357
. 10.1016/j.destud.2004.09.005
45.
Gero
,
J. S.
,
1990
, “
Design Prototypes: A Knowledge Representation Schema for Design
,”
AI Mag.
,
11
(
4
), p.
26
.
46.
Goel
,
A. K.
,
Rugaber
,
S.
, and
Vattam
,
S.
,
2009
, “
Structure, Behavior, and Function of Complex Systems: The Structure, Behavior, and Function Modeling Language
,”
AI EDAM
,
23
(
1
), pp.
23
35
.
47.
Shah
,
J. J.
,
Vargas-Hernandez
,
N.
,
Summers
,
J. D.
, and
Kulkarni
,
S.
,
2001
, “
Collaborative Sketching (C-Sketch)—An Idea Generation Technique for Engineering Design
,”
J. Creativ. Behav.
,
35
(
3
), pp.
168
198
. 10.1002/j.2162-6057.2001.tb01045.x
48.
Panchal
,
J. H.
, and
Szajnfarber
,
Z.
,
2017
, “
Experiments in Systems Engineering and Design Research
,”
Syst. Eng.
,
20
(
6
), pp.
529
541
. 10.1002/sys.21415
49.
Shergadwala
,
M.
,
Kannan
,
K. N.
, and
Panchal
,
J. H.
,
2016
, “
Understanding the Impact of Expertise on Design Outcome: An Approach Based on Concept Inventories and Item Response Theory
,”
ASME 2016 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Volume 3
,
Charlotte, NC
,
Aug. 21–24
, ASME Paper No. V003T04A016.
50.
Gosnell
,
C. A.
, and
Miller
,
S. R.
,
2016
, “
But Is It Creative? Delineating the Impact of Expertise and Concept Ratings on Creative Concept Selection
,”
ASME J. Mech. Des.
,
138
(
2
), p.
021101
. 10.1115/1.4031904
51.
Sutera
,
J.
,
Yang
,
M. C.
, and
Elsen
,
C.
,
2014
, “
The Impact of Expertise on the Capture of Sketched Intentions: Perspectives for Remote Cooperative Design
,”
International Conference on Cooperative Design, Visualization and Engineering
,
Seattle, WA
,
Sept. 14–17
, pp.
245
252
,
Springer
,
New York
.
52.
Cross
,
N.
,
2004
, “
Expertise in Design: An Overview
,”
Des. Stud.
,
25
(
5
), pp.
427
441
. 10.1016/j.destud.2004.06.002
53.
Toh
,
C. A.
, and
Miller
,
S. R.
,
2015
, “
How Engineering Teams Select Design Concepts: A View Through the Lens of Creativity
,”
Des. Stud.
,
38
, pp.
111
138
. 10.1016/j.destud.2015.03.001
54.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2015
, “
Rolling With the Punches: An Examination of Team Performance in a Design Task Subject to Drastic Changes
,”
Des. Stud.
,
36
, pp.
99
121
. 10.1016/j.destud.2014.10.001
55.
Ball
,
L. J.
, and
Christensen
,
B. T.
,
2009
, “
Analogical Reasoning and Mental Simulation in Design: Two Strategies Linked to Uncertainty Resolution
,”
Des. Stud.
,
30
(
2
), pp.
169
186
. 10.1016/j.destud.2008.12.005
56.
Visser
,
W.
,
1996
, “
Two Functions of Analogical Reasoning in Design: A Cognitive-Psychology Approach
,”
Des. Stud.
,
17
(
4
), pp.
417
434
. 10.1016/S0142-694X(96)00020-8
57.
Hernandez
,
N. V.
,
Shah
,
J. J.
, and
Smith
,
S. M.
,
2010
, “
Understanding Design Ideation Mechanisms Through Multilevel Aligned Empirical Studies
,”
Des. Stud.
,
31
(
4
), pp.
382
410
. 10.1016/j.destud.2010.04.001
58.
Shah
,
J. J.
,
1998
, “
Experimental Investigation of Progressive Idea Generation Techniques in Engineering Design
,”
Proceedings of ASME DETC
,
Atlanta, GA
,
Sept. 13–16
.
59.
Grogan
,
P. T.
, and
de Weck
,
O. L.
,
2016
, “
Collaboration and Complexity: An Experiment on the Effect of Multi-Actor Coupled Design
,”
Res. Eng. Des.
,
27
(
3
), pp.
221
235
. 10.1007/s00163-016-0214-7
60.
Alelyani
,
T.
,
Yang
,
Y.
, and
Grogan
,
P. T.
,
2017
, “
Understanding Designers Behavior in Parameter Design Activities
,”
ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Volume 7: 29th International Conference on Design Theory and Methodology
,
Cleveland, OH
,
Aug. 6–9
, ASME Paper No. V007T06A030.
61.
Yao
,
H. H.
, and
Ren
,
M. Y.
,
2016
, “
Impressionist: A 3D Peekaboo Game for Crowdsourcing Shape Saliency
,”
ASME 2016 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Volume 7: 28th International Conference on Design Theory and Methodology
,
Charlotte, NC
,
Aug. 21–24
, ASME Paper No. V007T06A025.
62.
Grogan
,
P. T.
, and
de Weck
,
O. L.
,
2015
, “
Interactive Simulation Games to Assess Federated Satellite System Concepts
,”
Aerospace Conference
,
Big Sky, MT
,
Mar. 7–14
, pp.
1
13
.
63.
Bayrak
,
A. E.
, and
Papalambros
,
P. Y.
,
2016
, “
EcoRacer: Game-Based Optimal Electric Vehicle Design and Driver Control Using Human Players
,”
ASME J. Mech. Des.
,
138
(
6
), p.
061407
. 10.1115/1.4033426
64.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2017
, “
Mining Process Heuristics From Designer Action Data via Hidden Markov Models
,”
ASME J. Mech. Des.
,
139
(
11
), p.
111412
. 10.1115/1.4037308
65.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2016
, “
Drawing Inspiration From Human Design Teams for Better Search and Optimization: The Heterogeneous Simulated Annealing Teams Algorithm
,”
ASME J. Mech. Des.
,
138
(
4
), p.
044501
. 10.1115/1.4032810
66.
Gero
,
J. S.
, and
Peng
,
W.
,
2009
, “
Understanding Behaviors of a Constructive Memory Agent: A Markov Chain Analysis
,”
Knowl. Base. Syst.
,
22
(
8
), pp.
610
621
. 10.1016/j.knosys.2009.05.006
67.
Sung
,
R. C. W.
,
Corney
,
J. R.
, and
Clark
,
D. E.
,
2001
, “
Automatic Assembly Feature Recognition and Disassembly Sequence Generation
,”
ASME J. Comput. Inf. Sci. Eng.
,
1
(
4
), pp.
291
299
. 10.1115/1.1429931
68.
Ritchie
,
J. M.
,
Sung
,
R. C. W.
,
Rea
,
H.
,
Lim
,
T.
,
Corney
,
J. R.
, and
Howley
,
I.
,
2008
, “
The Use of Non-Intrusive User Logging to Capture Engineering Rationale, Knowledge and Intent During the Product Life Cycle
,”
Portland International Conference on Management of Engineering & Technology, PICMET 2008
,
Cape Town, South Africa
,
July 27–31
, pp.
981
989
.
69.
Finger
,
S.
, and
Dixon
,
J. R.
,
1989
, “
A Review of Research in Mechanical Engineering Design. Part I: Descriptive, Prescriptive, and Computer-Based Models of Design Processes
,”
Res. Eng. Des.
,
1
(
1
), pp.
51
67
. 10.1007/BF01580003
70.
Purzer
,
Ş.
,
Goldstein
,
M. H.
,
Adams
,
R. S.
,
Xie
,
C.
, and
Nourian
,
S.
,
2015
, “
An Exploratory Study of Informed Engineering Design Behaviors Associated With Scientific Explanations
,”
Int. J. STEM Educ.
,
2
(
1
), pp.
1
12
. 10.1186/s40594-015-0019-7
71.
Dong
,
A.
,
Hill
,
A. W.
, and
Agogino
,
A. M.
,
2004
, “
A Document Analysis Method for Characterizing Design Team Performance
,”
ASME J. Mech. Des.
,
126
(
3
), pp.
378
385
. 10.1115/1.1711818
72.
Coley
,
F.
,
Houseman
,
O.
, and
Roy
,
R.
,
2007
, “
An Introduction to Capturing and Understanding the Cognitive Behaviour of Design Engineers
,”
J. Eng. Des.
,
18
(
4
), pp.
311
325
. 10.1080/09544820600963412
73.
Simon
,
H. A.
,
1972
, “
Theories of Bounded Rationality
,”
Decis. Organ.
,
1
(
1
), pp.
161
176
.
74.
Greene
,
M. T.
,
Gonzalez
,
R.
,
Papalambros
,
P. Y.
,
McGowan
,
A.-M.
,
2017
, “
Design Thinking Versus Systems Thinking for Engineering Design: What’s the Difference?
,”
21st International Conference on Engineering Design, ICED 2017
,
Vancouver, Canada
,
Aug. 21–25
, Vol.
2
, pp.
467
476
.
75.
Gajewski
,
R.
, and
Kułakowski
,
T.
,
2018
, “
Towards Optimal Design of Energy Efficient Buildings
,”
Arch. Civ. Eng.
,
64
(
4
), pp.
135
153
. 10.2478/ace-2018-0067
76.
Gajewski
,
R.
, and
Pienikazek
,
P.
,
2017
, “
Building Energy Modelling and Simulations: Qualitative and Quantitative Analysis
,”
MATEC Web of Conferences
, Vol.
117
, Article: 00051.
77.
Xie
,
C.
,
2019
, “
Engineering Design Projects for Students
,” http://energy.concord.org/energy3d/projects.html, Accessed June 15, 2019.
78.
Sha
,
Z.
,
2019
, “
Human-Subject Experiment for Design Research
,” https://sidilab.wordpress.com/sidi-resources/human-subject-experiment-for-design-research/, Accessed June 15, 2019.
79.
De Mauro
,
A.
,
Greco
,
M.
, and
Grimaldi
,
M.
,
2015
, “
What Is Big Data? A Consensual Definition and a Review of Key Research Topics
,”
AIP Conf. Proc.
,
1644
(
1
), pp.
97
104
. 10.1063/1.4907823
80.
Xie
,
C.
,
Zhang
,
Z.
,
Nourian
,
S.
, and
Pallant
,
A. R.
,
2014
, “
On the Instructional Sensitivity of CAD Logs
,”
Int. J. Eng. Educ.
,
30
(
4
), pp.
760
778
.
81.
Xie
,
C.
,
Zhang
,
Z.
,
Nourian
,
S.
,
Pallant
,
A.
, and
Hazzard
,
E.
,
2014
, “
A Time Series Analysis Method for Assessing Engineering Design Processes Using a CAD Tool
,”
Int. J. Eng. Educ.
,
30
(
1
), pp.
218
230
.
82.
Adams
,
R. S.
,
Goldstein
,
M.
,
Purzer
,
Ş.
,
Chao
,
J.
,
Xie
,
C.
, and
Nourian
,
S.
,
2017
, “
Traversing the Barriers to Using Big Data in Understating How High School Students Design
,”
Design Computing and Cognition’16
,
J. S.
Gero
, ed.,
Springer
,
New York
, pp.
613
631
.
83.
Stroock
,
D. W.
,
2013
,
An Introduction to Markov Processes
, Vol.
230
,
Springer Science, Business Media
,
Berlin, Germany
.
84.
James
,
G.
,
Witten
,
D.
,
Hastie
,
T.
, and
Tibshirani
,
R.
,
2013
,
An Introduction to Statistical Learning
, Vol.
112
,
Springer
,
New York
.
85.
Schaeffer
,
S. E.
,
2007
, “
Graph Clustering
,”
Comput. Sci. Rev.
,
1
(
1
), pp.
27
64
. 10.1016/j.cosrev.2007.05.001
86.
Singh
,
V. K.
,
Tiwari
,
N.
, and
Garg
,
S.
,
2011
, “
Document Clustering Using k-Means, Heuristic k-Means and Fuzzy c-Means
,”
2011 International Conference on Computational Intelligence and Communication Networks (CICN)
,
Gwalia, India
,
Oct. 7–9
, pp.
279
301
.
87.
Kodinariya
,
T. M.
, and
Makwana
,
P. R.
,
2013
, “
Review on Determining Number of Cluster in K-Means Clustering
,”
Int. J.
,
1
(
6
), pp.
90
95
.
88.
Steinbach
,
M.
,
Karypis
,
G.
, and
Kumar
,
V.
,
2000
, “
A Comparison of Document Clustering Techniques
,”
KDD Workshop on Text Mining
,
Boston, MA
,
Aug. 20–23
, Vol.
400
(
1
), pp.
525
526
.
89.
Chen
,
M.
,
Kuzmin
,
K.
, and
Szymanski
,
B. K.
,
2014
, “
Community Detection via Maximization of Modularity and its Variants
,”
IEEE Trans. Comput. Soc. Syst.
,
1
(
1
), pp.
46
65
. 10.1109/TCSS.2014.2307458
90.
Brandes
,
U.
,
Delling
,
D.
,
Gaertler
,
M.
,
Gorke
,
R.
,
Hoefer
,
M.
,
Nikoloski
,
Z.
, and
Wagner
,
D.
,
2008
, “
On Modularity Clustering
,
IEEE Trans. Knowl. Data Eng.
,
20
(
2
), pp.
172
188
. 10.1109/TKDE.2007.190689
91.
Meila
,
M.
,
2007
, “
Comparing Clusterings—An Information Based Distance
,”
J. Multivariate Anal.
,
98
(
5
), pp.
873
895
. 10.1016/j.jmva.2006.11.013
92.
Xie
,
C.
,
2015
,
Visual Process Analytics in @CONCORD
, The Concord Consortium: Concord.org, pp.
4
6
.
You do not currently have access to this content.