An important part of the engineering design process is prototyping, where designers build and test their designs. This process is typically iterative, time consuming, and manual in nature. For a given task, there are multiple objects that can be used, each with different time units associated with accomplishing the task. Current methods for reducing time spent during the prototyping process have focused primarily on optimizing designer to designer interactions, as opposed to designer to tool interactions. Advancements in commercially available sensing systems (e.g., the Kinect) and machine learning algorithms have opened the pathway toward real-time observation of designer's behavior in engineering workspaces during prototype construction. Toward this end, this work hypothesizes that an object O being used for task i is distinguishable from object O being used for task j, where i is the correct task and j is the incorrect task. The contributions of this work are: (i) the ability to recognize these objects in a free roaming engineering workshop environment and (ii) the ability to distinguish between the correct and incorrect use of objects used during a prototyping task. By distinguishing the difference between correct and incorrect uses, incorrect behavior (which often results in wasted time and materials) can be detected and quickly corrected. The method presented in this work learns as designers use objects, and infers the proper way to use them during prototyping. In order to demonstrate the effectiveness of the proposed method, a case study is presented in which participants in an engineering design workshop are asked to perform correct and incorrect tasks with a tool. The participants' movements are analyzed by an unsupervised clustering algorithm to determine if there is a statistical difference between tasks being performed correctly and incorrectly. Clusters which are a plurality incorrect are found to be significantly distinct for each node considered by the method, each with p ≪ 0.001.

References

References
1.
Tuarob
,
S.
, and
Tucker
,
C. S.
,
2015
, “
Automated Discovery of Lead Users and Latent Product Features by Mining Large Scale Social Media Networks
,”
ASME J. Mech. Des.
,
137
(
7
), p.
071402
.
2.
Tuarob
,
S.
, and
Tucker
,
C. S.
,
2015
, “
Quantifying Product Favorability and Extracting Notable Product Features Using Large Scale Social Media Data
,”
ASME J. Comput. Inf. Sci. Eng.
,
15
(
3
), p.
031003
.
3.
Wassenaar
,
H. J.
,
Chen
,
W.
,
Cheng
,
J.
, and
Sudjianto
,
A.
,
2005
, “
Enhancing Discrete Choice Demand Modeling for Decision-Based Design
,”
ASME J. Mech. Des.
,
127
(
4
), pp.
514
523
.
4.
Hoyle
,
C.
,
Chen
,
W.
,
Ankenman
,
B.
, and
Wang
,
N.
,
2009
, “
Optimal Experimental Design of Human Appraisals for Modeling Consumer Preferences in Engineering Design
,”
ASME J. Mech. Des.
,
131
(
7
), p.
071008
.
5.
Agard
,
B.
, and
Kusiak
,
A.
,
2004
, “
Data-Mining-Based Methodology for the Design of Product Families
,”
Int. J. Prod. Res.
,
42
(
15
), pp.
2955
2969
.
6.
Kusiak
,
A.
, and
Smith
,
M.
,
2007
, “
Data Mining in Design of Products and Production Systems
,”
Annu. Rev. Control
,
31
(
1
), pp.
147
156
.
7.
Tucker
,
C. S.
, and
Kim
,
H. M.
,
2011
, “
Trend Mining for Predictive Product Design
,”
ASME J. Mech. Des.
,
133
(
11
), p.
111008
.
8.
Gershenson
,
J. K.
,
Prasad
,
G. J.
, and
Zhang
,
Y.
,
2004
, “
Product Modularity: Measures and Design Methods
,”
J. Eng. Des.
,
15
(
1
), pp.
33
51
.
9.
Kurtoglu
,
T.
,
Campbell
,
M.
I
.
,
Arnold
,
C. B.
,
Stone
,
R. B.
, and
Mcadams
,
D. A.
,
2009
, “
A Component Taxonomy as a Framework for Computational Design Synthesis
,”
ASME J. Comput. Inf. Sci. Eng.
,
9
(
1
), p.
011007
.
10.
Bonjour
,
E.
,
Deniaud
,
S.
,
Dulmet
,
M.
, and
Harmel
,
G.
,
2009
, “
A Fuzzy Method for Propagating Functional Architecture Constraints to Physical Architecture
,”
ASME J. Mech. Des.
,
131
(
6
), p.
061002
.
11.
Wang
,
G. G.
, and
Shan
,
S.
,
2007
, “
Review of Metamodeling Techniques in Support of Engineering Design Optimization
,”
ASME J. Mech. Des.
,
129
(
4
), pp.
370
380
.
12.
Apley
,
D. W.
,
Liu
,
J.
, and
Chen
,
W.
,
2006
, “
Understanding the Effects of Model Uncertainty in Robust Design With Computer Experiments
,”
ASME J. Mech. Des.
,
128
(
4
), pp.
945
958
.
13.
Hannah
,
R.
,
Joshi
,
S.
, and
Summers
,
J. D.
,
2012
, “
A User Study of Interpretability of Engineering Design Representations
,”
J. Eng. Des.
,
23
(
6
), pp.
443
468
.
14.
Gerber
,
E.
, and
Carroll
,
M.
,
2012
, “
The Psychological Experience of Prototyping
,”
Des. Stud.
,
33
(
1
), pp.
64
84
.
15.
Yang
,
M. C.
,
2005
, “
A Study of Prototypes, Design Activity, and Design Outcome
,”
Des. Stud.
,
26
(
6
), pp.
649
669
.
16.
Houde
,
S.
, and
Hill
,
C.
,
1997
, “
What Do Prototypes Prototype
,”
Handbook of Human-Computer Interaction
, Vol.
2
, Elsevier, Amsterdam, The Netherlands, pp.
367
381
.
17.
Eppinger
,
S.
, and
Whitney
,
D.
,
1995
, “
Accelerating Product Development by the Exchange of Preliminary Product Design Information
,”
ASME J. Mech. Des.
,
117
(
4
), pp.
491
498
.
18.
Teizer
,
J.
,
Venugopal
,
M.
, and
Walia
,
A.
,
2008
, “
Ultrawideband for Automated Real-Time Three-Dimensional Location Sensing for Workforce, Equipment, and Material Positioning and Tracking
,”
Transp. Res. Rec.
,
2081
, pp.
56
64
.
19.
Lim
,
Y.-K.
,
Stolterman
,
E.
, and
Tenenberg
,
J.
,
2008
, “
The Anatomy of Prototypes: Prototypes as Filters, Prototypes as Manifestations of Design Ideas
,”
ACM Trans. Comput.-Hum. Interact.
,
15
(
2
), p.
7
.
20.
Dalal
,
N.
, and
Triggs
,
B.
,
2005
, “
Histograms of Oriented Gradients for Human Detection
,”
IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(
CVPR
), San Diego, CA, June 20–25, pp.
886
893
.
21.
Rublee
,
E.
,
Rabaud
,
V.
,
Konolige
,
K.
, and
Bradski
,
G.
,
2011
, “
ORB: An Efficient Alternative to SIFT or SURF
,”
IEEE International Conference on Computer Vision
(
ICCV
), Barcelona, Spain, Nov. 6–13, pp.
2564
2571
.
22.
Chi
,
S.
, and
Caldas
,
C. H.
,
2011
, “
Automated Object Identification Using Optical Video Cameras on Construction Sites
,”
Comput.-Aided Civ. Infrastruct. Eng.
,
26
(
5
), pp.
368
380
.
23.
Bo
,
L.
,
Ren
,
X.
, and
Fox
,
D.
,
2011
, “
Depth Kernel Descriptors for Object Recognition
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
(
IROS
), San Francisco, CA, Sept. 25–30, pp.
821
826
.
24.
Ren
,
X.
,
Bo
,
L.
, and
Fox
,
D.
,
2012
, “
RGB-(D) Scene Labeling: Features and Algorithms
,”
IEEE Conference on Computer Vision and Pattern Recognition
(
CVPR
), Providence, RI, June 16–21, pp.
2759
2766
.
25.
Russakovsky
,
O.
,
Deng
,
J.
,
Su
,
H.
,
Krause
,
J.
,
Satheesh
,
S.
,
Ma
,
S.
,
Huang
,
Z.
,
Karpathy
,
A.
,
Khosla
,
A.
,
Bernstein
,
M.
,
Berg
,
A. C.
, and
Fei-Fei
,
L.
,
2015
, “
ImageNet Large Scale Visual Recognition Challenge
,”
Int. J. Comput. Vision
,
115
(
3
), pp.
211
252
.
26.
Krizhevsky
,
A.
,
Sutskever
,
I.
, and
Hinton
,
G. E.
,
2012
, “
ImageNet Classification With Deep Convolutional Neural Networks
,” 25th International Conference on Neural Information Processing Systems (
NIPS
), Lake Tahoe, NV, Dec. 3–6, pp.
1097
1105
.http://dl.acm.org/citation.cfm?id=2999257
27.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2014
, “
Very Deep Convolutional Networks for Large-Scale Image Recognition
,” preprint
arXiv:1409.1556
.
28.
Dunleavy
,
M.
, and
Dede
,
C.
,
2014
, “
Augmented Reality Teaching and Learning
,”
Handbook of Research on Educational Communications and Technology
,
Springer
,
New York
, pp.
735
745
.
29.
Kosmadoudi
,
Z.
,
Lim
,
T.
,
Ritchie
,
J.
,
Louchart
,
S.
,
Liu
,
Y.
, and
Sung
,
R.
,
2013
, “
Engineering Design Using Game-Enhanced CAD: The Potential to Augment the User Experience With Game Elements
,”
Comput.-Aided Des.
,
45
(
3
), pp.
777
795
.
30.
Jezernik
,
A.
, and
Hren
,
G.
,
2003
, “
A Solution to Integrate Computer-Aided Design (CAD) and Virtual Reality (VR) Databases in Design and Manufacturing Processes
,”
Int. J. Adv. Manuf. Technol.
,
22
(
11–12
), pp.
768
774
.
31.
Bourdot
,
P.
,
Convard
,
T.
,
Picon
,
F.
,
Ammi
,
M.
,
Touraine
,
D.
, and
Vézien
,
J.-M.
,
2010
, “
VR–CAD Integration: Multimodal Immersive Interaction and Advanced Haptic Paradigms for Implicit Edition of CAD Models
,”
Comput.-Aided Des.
,
42
(
5
), pp.
445
461
.
32.
Verlinden
,
J.
, and
Horváth
,
I.
,
2009
, “
Analyzing Opportunities for Using Interactive Augmented Prototyping in Design Practice
,”
J. Artif. Intell. Eng. Des. Anal. Manuf.
,
23
(
3
), pp.
289
303
.
33.
Fiorentino
,
M.
,
Uva
,
A. E.
,
Monno
,
G.
, and
Radkowski
,
R.
,
2012
, “
Augmented Technical Drawings: A Novel Technique for Natural Interactive Visualization of Computer-Aided Design Models
,”
ASME J. Comput. Inf. Sci. Eng.
,
12
(
2
), p.
024503
.
34.
Vélaz
,
Y.
,
Arce
,
J. R.
,
Gutiérrez
,
T.
,
Lozano-Rodero
,
A.
, and
Suescun
,
A.
,
2014
, “
The Influence of Interaction Technology on the Learning of Assembly Tasks Using Virtual Reality
,”
ASME J. Comput. Inf. Sci. Eng.
,
14
(
4
), p.
041007
.
35.
Bradski
,
G. R.
, and
Davis
,
J. W.
,
2002
, “
Motion Segmentation and Pose Recognition With Motion History Gradients
,”
Mach. Vision Appl.
,
13
(
3
), pp.
174
184
.
36.
Ribeiro
,
P. C.
,
Santos-Victor
,
J.
, and
Lisboa
,
P.
,
2005
, “
Human Activity Recognition From Video: Modeling, Feature Selection and Classification Architecture
,”
International Workshop on Human Activity Recognition and Modelling
(
HAREM
), Oxford, UK, Sept. 9, pp.
61
78
.https://www.researchgate.net/publication/237448747_Human_Activity_Recognition_from_Video_modeling_feature_selection_and_classification_architecture
37.
Li
,
W.
,
Zhang
,
Z.
, and
Liu
,
Z.
,
2010
, “
Action Recognition Based on a Bag of 3D Points
,”
IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
(
CVPRW
), San Francisco, CA, June 13–18, pp.
9
14
.
38.
Morato
,
C.
,
Kaipa
,
K. N.
,
Zhao
,
B.
, and
Gupta
,
S. K.
,
2014
, “
Toward Safe Human Robot Collaboration by Using Multiple Kinects Based Real-Time Human Tracking
,”
ASME J. Comput. Inf. Sci. Eng.
,
14
(
1
), p.
011006
.
39.
Behoora
,
I.
, and
Tucker
,
C. S.
,
2014
, “
Quantifying Emotional States Based on Body Language Data Using Non Invasive Sensors
,”
ASME
Paper No. DETC2014-34770.
40.
Tucker
,
C.
, and
Kumara
,
S.
,
2015
, “
An Automated Object-Task Mining Model for Providing Students With Real Time Performance Feedback
,”
American Society for Engineering Education Annual Conference and Exposition
(
ASEE
), Seattle, WA, June 14–17.https://www.google.co.in/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0ahUKEwi3i6ig4-fVAhWR2YMKHeNwA1EQFggwMAE&url=https%3A%2F%2Fwww.asee.org%2Fpublic%2Fconferences%2F56%2Fpapers%2F13126%2Fdownload&usg=AFQjCNEmkbMFMpGjZUWtqYfK-2gnAoQtfw
41.
Lopes
,
M.
, and
Santos-Victor
,
J.
,
2005
, “
Visual Learning by Imitation With Motor Representations
,”
IEEE Trans. Syst. Man Cybern., Part B
,
35
(
3
), pp.
438
449
.
42.
Jiang
,
Y.
,
Lim
,
M.
, and
Saxena
,
A.
,
2012
, “
Learning Object Arrangements in 3D Scenes Using Human Context
,” preprint
arXiv:1206.6462
.
43.
Koppula
,
H. S.
,
Gupta
,
R.
, and
Saxena
,
A.
,
2013
, “
Learning Human Activities and Object Affordances From RGB-D Videos
,”
Int. J. Rob. Res.
,
32
(
8
), pp.
951
970
.
44.
Yu
,
G.
,
Liu
,
Z.
, and
Yuan
,
J.
,
2014
, “
Discriminative Orderlet Mining for Real-Time Recognition of Human-Object Interaction
,”
Asian Conference on Computer Vision
(
ACCV
), Singapore, Nov. 1–5, pp.
50
65
.https://eeeweba.ntu.edu.sg/computervision/Research%20Papers/2014/Discriminative%20Orderlet%20Mining%20For%20Real-time%20Recognition%20of%20Human-Object%20Interaction.pdf
45.
Wieling
,
M.
, and
Hofman
,
W.
,
2010
, “
The Impact of Online Video Lecture Recordings and Automated Feedback on Student Performance
,”
Comput. Educ.
,
54
(
4
), pp.
992
998
.
46.
Chen
,
P. M.
,
2004
, “
An Automated Feedback System for Computer Organization Projects
,”
IEEE Trans. Educ.
,
47
(
2
), pp.
232
240
.
47.
Calvo
,
R. A.
, and
Ellis
,
R. A.
,
2010
, “
Students' Conceptions of Tutor and Automated Feedback in Professional Writing
,”
J. Eng. Educ.
,
99
(
4
), pp.
427
438
.
48.
Patel
,
R. A.
,
Hartzler
,
A.
,
Pratt
,
W.
,
Back
,
A.
,
Czerwinski
,
M.
, and
Roseway
,
A.
,
2013
, “
Visual Feedback on Nonverbal Communication: A Design Exploration With Healthcare Professionals
,”
Seventh International Conference on Pervasive Computing Technologies for Healthcare
(
PervasiveHealth
), Venice, Italy, May 5–8, pp.
105
112
.
49.
Lamancusa
,
J. S.
,
2006
, “
The Reincarnation of the Engineering “Shop”
,”
ASME
Paper No. DETC2006-99723.
50.
Le
,
Q.
V
.
,
2013
, “
Building High-Level Features Using Large Scale Unsupervised Learning
,”
IEEE International Conference on Acoustics, Speech and Signal Processing
(
ICASSP
), Vancouver, BC, Canada, May 26–31, pp.
8595
8598
.
51.
Massey
,
F. J.
, Jr.
,
1951
, “
The Kolmogorov-Smirnov Test for Goodness of Fit
,”
J. Am. Stat. Assoc.
,
46
(
253
), pp.
68
78
.
You do not currently have access to this content.