Abstract

Design knowledge in the vast amount of design reports and documents can be an excellent resource for designers in their practice. However, capturing such domain-specific information embedded in long-length unstructured texts is always time-consuming and sometimes tricky. Therefore, it is highly desirable for a computer system to automatically extract the main knowledge points and their corresponding inner structures from given documents. In this study of document understanding for design support (DocUDS), a design-perspective knowledge extraction approach is proposed that uses phrase-level domain-specific labeled datasets to finetune a Bidirectional Encoder Representation from Transformers (BERT) model so that it can extract design knowledge from documents. The BERT model finetuning attempts to blend in the domain-specific knowledge of well-recognized domain concepts and is based on the datasets generated from design reports. The model is utilized to map the captured sentences to the main design entities <reguirement>, <function>, and <solution>. In addition, this approach uncovers inner relationships among the sentences and constructs overall structures of documents to enhance understanding. The definitions of design perspectives, inter-perspective relations, and intra-perspective relations are introduced, which together capture the main design knowledge points and their relations and constitute an understanding of the design domain knowledge of a text. The case study results have demonstrated the proposed approach's effectiveness in understanding and extracting relevant design knowledge points.

References

1.
Fleuren
,
W. W.
, and
Alkema
,
W.
,
2015
, “
Application of Text Mining in the Biomedical Domain
,”
Methods
,
74
, pp.
97
106
.
2.
Ferreira
,
R.
,
de Souza Cabral
,
L.
,
Lins
,
R. D.
,
e Silva
,
G. P.
,
Freitas
,
F.
,
Cavalcanti
,
G. D.
,
Lima
,
R.
,
Simske
,
S. J.
, and
Favaro
,
L.
,
2013
, “
Assessing Sentence Scoring Techniques for Extractive Text Summarization
,”
Expert Syst. Appl.
,
40
(
14
), pp.
5755
5764
.
3.
Mishra
,
R.
,
Bian
,
J.
,
Fiszman
,
M.
,
Weir
,
C. R.
,
Jonnalagadda
,
S.
,
Mostafa
,
J.
, and
Del Fiol
,
G.
,
2014
, “
Text Summarization in the Biomedical Domain: A Systematic Review of Recent Research
,”
J. Biomed. Inform.
,
52
, pp.
457
467
.
4.
Camacho-Collados
,
J.
, and
Pilehvar
,
M. T.
,
2018
, “
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning
,”
J. Artif. Intell. Res.
,
63
(
1
), pp.
743
788
.
5.
Qiu
,
Y.
, and
Jin
,
Y.
,
2022
, “
Engineering Document Summarization: A Bidirectional Language Model-Based Approach
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
6
), p.
061004
.
6.
Beigbeder
,
M.
, and
Mercier
,
A.
,
2005
, “
An Information Retrieval Model Using the Fuzzy Proximity Degree of Term Occurrences
,”
Proceedings of the 2005 ACM Symposium on Applied Computing
,
Santa Fe, NM
,
Mar. 13–17
, pp.
1018
1022
.
7.
Castells
,
P.
,
Fernandez
,
M.
, and
Vallet
,
D.
,
2006
, “
An Adaptation of the Vector-Space Model for Ontology-Based Information Retrieval
,”
IEEE Trans. Knowl. Data Eng.
,
19
(
2
), pp.
261
272
.
8.
Zhang
,
X.
,
Hou
,
X.
,
Chen
,
X.
, and
Zhuang
,
T.
,
2013
, “
Ontology-Based Semantic Retrieval for Engineering Domain Knowledge
,”
Neurocomputing
,
116
(
9
), pp.
382
391
.
9.
Shi
,
F.
,
Chen
,
L.
,
Han
,
J.
, and
Childs
,
P.
,
2017
, “
A Data-Driven Text Mining and Semantic Network Analysis for Design Information Retrieval
,”
ASME J. Mech. Des.
,
139
(
11
), p.
111402
.
10.
Martinez-Rodriguez
,
J. L.
,
López-Arévalo
,
I.
, and
Rios-Alvarado
,
A. B.
,
2018
, “
Openie-Based Approach for Knowledge Graph Construction From Text
,”
Expert Syst. Appl.
,
113
(
12
), pp.
339
355
.
11.
Sarica
,
S.
,
Luo
,
J.
, and
Wood
,
K. L.
,
2020
, “
TechNet: Technology Semantic Network Based on Patent Data
,”
Expert Syst. Appl.
,
142
(
3
), p.
112995
.
12.
Siddharth
,
L.
,
Blessing
,
L.
,
Wood
,
K. L.
, and
Luo
,
J.
,
2022
, “
Engineering Knowledge Graph From Patent Database
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
2
), p.
021008
.
13.
Hou
,
T.
,
Yannou
,
B.
,
Leroy
,
Y.
, and
Poirson
,
E.
,
2018
, “
Mining Changes of User Expectations Over Time From Online Reviews
.”
14.
Siddharth
,
L.
,
Blessing
,
L.
, and
Luo
,
J.
,
2022
, “
Natural Language Processing in-and-for Design Research
,”
Des Sci.
,
8
, pp.
E21, 1
76
.
15.
Lin
,
H.
,
Chi
,
N.
, and
Hsieh
,
S.
,
2012
, “
A Concept-Based Information Retrieval Approach for Engineering Domain-Specific Technical Documents
,”
Adv. Eng. Inform.
,
26
(
2
), pp.
349
360
.
16.
Han
,
J.
,
Park
,
D.
,
Forbes
,
H.
, and
Schaefer
,
D.
,
2020
, “
A Computational Approach for Using Social Networking Platforms to Support Creative Idea Generation
,”
Procedia CIRP
,
91
, pp.
382
387
.
17.
Liu
,
Q.
,
Wang
,
K.
,
Li
,
Y.
, and
Liu
,
Y.
,
2020
, “
Data-Driven Concept Network for Inspiring Designers’ Idea Generation
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
3
), p.
031004
.
18.
Han
,
Y.
, and
Moghaddam
,
M.
,
2021
, “
Eliciting Attribute-Level User Needs From Online Reviews With Deep Language Models and Information Extraction
,”
ASME J. Mech. Des.
,
143
(
6
), p.
061403
.
19.
Brisco
,
R.
,
Whitfield
,
R. I.
, and
Grierson
,
H.
,
2020
, “
A Novel Systematic Method to Evaluate Computer-Supported Collaborative Design Technologies
,”
Res. Eng. Des.
,
31
(
1
), pp.
53
81
.
20.
Ye
,
X.
, and
Lu
,
Y.
,
2020
, “
Automatic Extraction of Engineering Rules From Unstructured Text: A Natural Language Processing Approach
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
3
), p.
034501
.
21.
Lester
,
M.
,
Guerrero
,
M.
, and
Burge
,
J.
,
2020
, “
Using Evolutionary Algorithms to Select Text Features for Mining Design Rationale
,”
Artif. Intell. Eng. Des. Anal. Manuf.
,
34
(
2
), pp.
132
146
.
22.
Hubka
,
V.
, and
Eder
,
W. E.
,
2012
,
Theory of Technical Systems: A Total Concept Theory for Engineering Design
,
Springer Science & Business Media
,
Berlin/Heidelberg, Germany
.
23.
Quintana-Amate
,
S.
,
Bermell-Garcia
,
P.
, and
Tiwari
,
A.
,
2015
, “
Transforming Expertise Into Knowledge-Based Engineering Tools: A Survey of Knowledge Sourcing in the Context of Engineering Design
,”
Knowledge Based Syst.
,
84
, pp.
89
97
.
24.
Verhagen
,
W. J. C.
,
Bermell-Garcia
,
P.
,
van Dijk
,
R. E. C.
, and
Curran
,
R.
,
2012
, “
A Critical Review of Knowledge-Based Engineering: An Identification of Research Challenges
,”
Adv. Eng. Inform.
,
26
(
1
), pp.
5
15
.
25.
Howard
,
T. J.
,
Culley
,
S. J.
, and
Dekoninck
,
E.
,
2008
, “
Describing the Creative Design Process by the Integration of Engineering Design and Cognitive Psychology Literature
,”
Des. Stud.
,
29
(
2
), pp.
160
180
.
26.
Pahl
,
G.
, and
Beitz
,
W.
,
2013
,
Engineering Design: A Systematic Approach
,
Springer Science & Business Media
,
London, UK
.
27.
Ullman
,
D. G.
,
1992
,
The Mechanical Design Process
,
McGraw-Hill
,
New York
.
28.
Suh
,
N. P.
,
1995
,
Axiomatic Design of Mechanical Systems
.”
29.
Rocca
,
G. L.
,
2012
, “
Knowledge Based Engineering: Between AI and CAD. Review of a Language Based Technology to Support Engineering Design
,”
Adv. Eng. Inform.
,
26
(
2
), pp.
159
179
.
30.
Bouhana
,
A.
,
Zidi
,
A.
,
Fekih
,
A.
,
Chabchoub
,
H.
, and
Abed
,
M.
,
2015
, “
An Ontology-Based CBR Approach for Personalized Itinerary Search Systems for Sustainable Urban Freight Transport
,”
Expert Syst. Appl.
,
42
(
7
), pp.
3724
3741
.
31.
Morkos
,
B.
,
Mathieson
,
J.
, and
Summers
,
J. D.
,
2014
, “
Comparative Analysis of Requirements Change Prediction Models: Manual, Linguistic, and Neural Network
,”
Res. Eng. Des.
,
25
(
2
), pp.
139
156
.
32.
Zhang
,
Z.
,
Liu
,
L.
,
Wei
,
W.
,
Tao
,
F.
,
Li
,
T.
, and
Liu
,
A.
,
2017
, “
A Systematic Function Recommendation Process for Data-Driven Product and Service Design
,”
ASME J. Mech. Des.
,
139
(
11
), p.
111404
.
33.
Sanya
,
I. O.
, and
Shehab
,
E. M.
,
2015
, “
A Framework for Developing Engineering Design Ontologies Within the Aerospace Industry
,”
Int. J. Prod. Res.
,
53
(
8
), pp.
2383
2409
.
34.
Zhang
,
C.
,
Zhou
,
G.
,
Lu
,
Q.
, and
Chang
,
F.
,
2017
, “
Graph-Based Knowledge Reuse for Supporting Knowledge-Driven Decision-Making in new Product Development
,”
Int. J. Prod. Res.
,
55
(
23
), pp.
7187
7203
.
35.
Sarica
,
S.
, and
Luo
,
J.
,
2021
, “
Design Knowledge Representation With Technology Semantic Network
,”
Proc. Des. Soc.
,
1
(
7
), pp.
1043
1052
.
36.
Akay
,
H.
, and
Kim
,
S.-G.
,
2021
, “
Extracting Functional Requirements From Design Documentation Using Machine Learning
,”
Procedia CIRP
,
100
, pp.
31
36
.
37.
Kintch
,
W.
,
1998
,
Comprehension: A Paradigm of Cognition
,
Cambridge University Press
,
Cambridge, UK
.
38.
Wharton
,
C.
, and
Kintsch
,
W.
,
1991
, “
An Overview of Construction-Integration Model: A Theory of Comprehension as a Foundation for a new Cognitive Architecture
,”
SIGART Bull.
,
2
(
4
), pp.
169
173
.
39.
Mahowald
,
K.
,
Ivanova
,
A. A.
,
Blank
,
I. A.
,
Kanwisher
,
N.
,
Tenenbaum
,
J. B.
, and
Fedorenko
,
E.
,
2023
, “
Dissociating Language and Thought in Large Language Models: A Cognitive Perspective
,”
arXiv preprint
. https://arxiv.org/abs/2301.06627v1
40.
Gero
,
J. S.
, and
Kannengiesser
,
U.
,
2014
, “The Function-Behaviour-Structure Ontology of Design,”
An Anthology of Theories and Models of Design: Philosophy, Approaches and Empirical Explorations
, pp.
263
283
.
41.
Loper
,
E.
, and
Bird
,
S.
,
2002
, “
NLTK: the Natural Language Toolkit
,”
Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics
,
Philadelphia, PA
,
July 7
.
42.
Nakayama
,
H.
,
Kubo
,
T.
,
Kamura
,
J.
,
Taniguchi
,
Y.
, and
Liang
,
X.
,
2018
, “doccano: Text Annotation Tool for Human,” https://Github.Com/Doccano/Doccano
43.
Cohen
,
J.
,
1960
, “
A Coefficient of Agreement for Nominal Scales
,”
Educ. Psychol. Meas.
,
20
(
1
), pp.
37
46
.
44.
McHugh
,
M. L.
,
2012
, “
Interrater Reliability: the Kappa Statistic
,”
Biochem. Med.
,
22
(
3
), pp.
276
282
.
45.
Devlin
,
J.
,
Chang
,
M.
,
Lee
,
K.
, and
Toutanova
,
K.
,
2019
, “
BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding
,”
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
,
Minneapolis, MN
,
June 2–7
.
46.
Vaswani
,
A.
,
Shazeer
,
N.
,
Parmar
,
N.
,
Uszkoreit
,
J.
,
Jones
,
L.
,
Gomez
,
A. N.
,
Kaiser
,
Ł
, and
Polosukhin
,
I.
,
2017
, “
Attention Is All You Need
,”
Proceeding of the 31st Conference on Neural Information Processing Systems (NIPS)
,
Long Beach, CA
,
Dec. 4–9
.
47.
Zhang
,
T.
,
Wu
,
F.
,
Katiyar
,
A.
,
Weinberger
,
K. Q.
, and
Artzi
,
Y.
,
2021
, “
Revisiting Few-sample BERT Fine-tuning
,”
The Ninth International Conference on Learning Representations (ICLR)
,
Virtual
,
May 3–7
.
48.
Kingma
,
D. P.
, and
Ba
,
J.
,
2015
, “
Adam: A Method for Stochastic Optimization
,”
The Third International Conference on Learning Representations (ICLR)
,
San Diego, CA
,
May 7–9
.
49.
Kalyan
,
K. S.
, and
Sangeetha
,
S.
,
2021
, “
BertMCN: Mapping Colloquial Phrases to Standard Medical Concepts Using BERT and Highway Network
,”
Artif. Intell. Med.
,
112
, p.
102008
.
50.
Miller
,
D.
,
2019
, “
Leveraging BERT for Extractive Text Summarization on Lectures
,”
arXiv preprint
. https://arxiv.org/abs/1906.04165
51.
Alghanmi
,
I.
,
Espinosa-Anke
,
L.
, and
Schockaert
,
S.
,
2020
, “
Combining BERT With Static Word Embeddings for Categorizing Social Media
,”
Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020)
,
Virtual
,
Nov. 19
.
52.
Reimers
,
N.
, and
Gurevych
,
I.
,
2019
, “Sentence-Bert: Sentence Embeddings Using Siamese Bert-Networks,”
arXiv Preprint
. https://arxiv.org/abs/1908.10084
53.
Cheng
,
X.
,
2021
, “
Dual-View Distilled Bert for Sentence Embedding
,”
Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
,
New York, NY
,
July 11–15
, pp.
2151
2155
.
54.
Ogot
,
M.
,
2004
, “
EMS Models: Adaptation of Engineering Design Black-Box Models for Use in TRIZ
,”
Proceedings of the ETRIA TRIZ Future Conference
,
Florence, Italy
,
Nov. 3–5
.
55.
Abbas
,
O. A.
,
2008
, “
Comparisons Between Data Clustering Algorithms
,”
Int. Arab J. Inf. Technol.
,
5
(
3
), p.
320
.
56.
Beeferman
,
D.
, and
Berger
,
A.
,
2000
, “
Agglomerative Clustering of a Search Engine Query Log
,”
Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
,
New York, NY
,
Aug. 20–23
, pp.
407
416
.
57.
Van der Maaten
,
L.
, and
Hinton
,
G.
,
2008
, “
Visualizing Data Using t-SNE
,”
J. Mach. Learn. Res.
,
9
(
11
), pp.
2579
2605
.
58.
Zhu
,
Q.
,
Zhang
,
X.
, and
Luo
,
J
.,
2023
, “
Biologically Inspired Design Concept Generation Using Generative Pre-Trained Transformers
,”
ASME. J. Mech. Des.
145
(
4
), p.
041409
.
59.
Qiu
,
Y.
, and
Jin
,
Y.
,
2023
, “
ChatGPT and Finetuned BERT: A Comparative Study for Domain Knowledge Tasks
,”
Intell. Syst. Appli
.
You do not currently have access to this content.