Abstract

Disassembly is an essential process for the recovery of end-of-life (EOL) electronics in remanufacturing sites. Nevertheless, the process remains labor-intensive due to EOL electronics’ high degree of uncertainty and complexity. The robotic technology can assist in improving disassembly efficiency; however, the characteristics of EOL electronics pose difficulties for robot operation, such as removing small components. For such tasks, detecting small objects is critical for robotic disassembly systems. Screws are widely used as fasteners in ordinary electronic products while having small sizes and varying shapes in a scene. To enable robotic systems to disassemble screws, the location information and the required tools need to be predicted. This paper proposes a computer vision framework for detecting screws and recommending related tools for disassembly. First, a YOLOv4 algorithm is used to detect screw targets in EOL electronic devices and a screw image extraction mechanism is executed based on the position coordinates predicted by YOLOv4. Second, after obtaining the screw images, the EfficientNetv2 algorithm is applied for screw shape classification. In addition to proposing a framework for automatic small-object detection, we explore how to modify the object detection algorithm to improve its performance and discuss the sensitivity of tool recommendations to the detection predictions. A case study of three different types of screws in EOL electronics is used to evaluate the performance of the proposed framework.

References

1.
Wang
,
X. V.
, and
Wang
,
L.
,
2018
, “
Digital Twin-Based WEEE Recycling, Recovery and Remanufacturing in the Background of Industry 4.0
,”
Int. J. Prod. Res.
,
57
(
12
), pp.
3892
3902
.
2.
Wurster
,
M.
,
Michel
,
M.
,
May
,
M. C.
,
Kuhnle
,
A.
,
Stricker
,
N.
, and
Lanza
,
G.
,
2022
, “
Modelling and Condition-Based Control of a Flexible and Hybrid Disassembly System With Manual and Autonomous Workstations Using Reinforcement Learning
,”
J. Intell. Manuf.
,
33
(
2
), pp.
575
591
.
3.
Poschmann
,
H.
,
Brüggemann
,
H.
, and
Goldmann
,
D.
,
2020
, “
Disassembly 4.0: A Review on Using Robotics in Disassembly Tasks As a Way of Automation
,”
Chem. Ing. Tech.
,
92
(
4
), pp.
341
359
.
4.
Palmieri
,
G.
,
Marconi
,
M.
,
Corinaldi
,
D.
,
Germani
,
M.
, and
Callegari
,
M.
,
2018
, “
Automated Disassembly of Electronic Components: Feasibility and Technical Implementation
,”
Proceedings of the ASME Design Engineering Technical Conference
,
Quebec City, Quebec, Canada
,
Aug. 26–29
, vol. 51791, p. V004T05A006.
5.
Marconi
,
M.
,
Palmieri
,
G.
,
Callegari
,
M.
, and
Germani
,
M.
,
2019
, “
Feasibility Study and Design of an Automatic System for Electronic Components Disassembly
,”
ASME J. Manuf. Sci. Eng.
,
141
(
2
), p.
021011
.
6.
Liu
,
B.
,
Xu
,
W.
,
Liu
,
J.
,
Yao
,
B.
,
Zhou
,
Z.
, and
Pham
,
D. T.
,
2019
, “
Human–Robot Collaboration for Disassembly Line Balancing Problem in Remanufacturing
,”
ASME 2019 14th International Manufacturing Science and Engineering Conference, MSEC 2019
,
Erie, PA
,
June 10–14
, vol. 58745, p. V001T02A037.
7.
Li
,
K.
,
Liu
,
Q.
,
Xu
,
W.
,
Liu
,
J.
,
Zhou
,
Z.
, and
Feng
,
H.
,
2019
, “
Sequence Planning Considering Human Fatigue for Human–Robot Collaboration in Disassembly
,”
Procedia CIRP
,
83
, pp.
95
104
.
8.
Torres
,
F.
,
Gil
,
P.
,
Puente
,
S. T.
,
Pomares
,
J.
, and
Aracil
,
R.
,
2004
, “
Automatic PC Disassembly for Component Recovery
,”
Int. J. Adv. Manuf. Technol.
,
23
(
1–2
), pp.
39
46
.
9.
Bdiwi
,
M.
,
Rashid
,
A.
, and
Putz
,
M.
,
2016
, “
Autonomous Disassembly of Electric Vehicle Motors Based on Robot Cognition
,”
Proceedings of the IEEE International Conference on Robotics and Automation
,
Stockholm, Sweden
,
May 16–21
, pp.
2500
2505
.
10.
Yildiz
,
E.
, and
Worgotter
,
F.
,
2019
, “
DCNN-Based Screw Detection for Automated Disassembly Processes
,”
Proceedings—15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019
,
Sorrento, Italy
,
Nov. 26–29
, pp.
187
192
.
11.
Jiang
,
Z.
,
Zhao
,
L.
,
Li
,
S.
,
Jia
,
Y.
, and
Liquan
,
Z.
, 2020, “
Real-Time Object Detection Method Based on Improved YOLOv4-Tiny
,”
arXxiv
.
12.
Li
,
Y.
,
Wang
,
H.
,
Dang
,
L. M.
,
Nguyen
,
T. N.
,
Han
,
D.
,
Lee
,
A.
,
Jang
,
I.
, and
Moon
,
H.
,
2020
, “
A Deep Learning-Based Hybrid Framework for Object Detection and Recognition in Autonomous Driving
,”
IEEE Access
,
8
, pp.
194228
194239
.
13.
Sandler
,
M.
,
Howard
,
A.
,
Zhu
,
M.
,
Zhmoginov
,
A.
, and
Chen
,
L.-C.
,
2018
, “
MobileNetV2: Inverted Residuals and Linear Bottlenecks
,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.
4510
4520
.
14.
Jiang
,
J.
,
Fu
,
X.
,
Qin
,
R.
,
Wang
,
X.
, and
Ma
,
Z.
,
2021
, “
High-Speed Lightweight Ship Detection Algorithm Based on YOLO-V4 for Three-Channels RGB SAR Image
,”
Remote Sensing
,
13
(
10
), p.
1909
.
15.
Liu
,
S.
,
Qi
,
L.
,
Qin
,
H.
,
Shi
,
J.
, and
Jia
,
J.
, “
Path Aggregation Network for Instance Segmentation
.” https://github.com/. Accessed November 28, 2021.
16.
Tan
,
M.
, and
Le
,
Q. v.
,
2021
, “
EfficientNetV2: Smaller Models and Faster Training
,”
Proceedings of the 38 th International Conference on Machine Learning
,
Virtual
,
July 18–24
.
17.
Tan
,
M.
, and
Le
,
Q. v.
,
2019
, “
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
,” PMLR, pp.
6105
6114
. https://proceedings.mlr.press/v97/tan19a.html. Accessed November 27, 2021.
18.
Mangold
,
S.
,
Steiner
,
C.
,
Friedmann
,
M.
, and
Fleischer
,
J.
,
2022
, “
Vision-Based Screw Head Detection for Automated Disassembly for Remanufacturing
,”
Procedia CIRP
,
105
, pp.
1
6
.
19.
Wegener
,
K.
,
Chen
,
W. H.
,
Dietrich
,
F.
,
Dröder
,
K.
, and
Kara
,
S.
,
2015
, “
Robot Assisted Disassembly for the Recycling of Electric Vehicle Batteries
,”
Procedia CIRP
,
29
, pp.
716
721
.
20.
Jiang
,
Z.
,
Zhao
,
L.
,
Li
,
S.
,
Jia
,
Y.
, and
Liquan
,
Z.
, 2020, “
Real-Time Object Detection Method Based on Improved YOLOv4-Tiny
,”
arXiv
.
21.
Bochkovskiy
,
A.
,
Wang
,
C.-Y.
, and
Liao
,
H.-Y. M.
,
2020
, “
YOLOv4: Optimal Speed and Accuracy of Object Detection
,”
arXiv
.
22.
Nepal
,
U.
, and
Eslamiat
,
H.
,
2022
, “
Comparing YOLOv3, YOLOv4 and YOLOv5 for Autonomous Landing Spot Detection in Faulty UAVs
,”
Sensors
,
22
(
2
), p.
464
.
23.
Liu
,
S.
,
Qi
,
L.
,
Qin
,
H.
,
Shi
,
J.
, and
Jia
,
J.
,
2018
, “
Path Aggregation Network for Instance Segmentation
,”
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 19–21
, pp.
8759
8768
.
24.
EfficientNetV2: Smaller Models and Faster Training
. http://proceedings.mlr.press/v139/tan21a.html. Accessed March 5, 2022.
25.
Tan
,
M.
, and
Le
,
Q.
,
2019
, “
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
,” In Proceedings of International Conference on Machine Learning (ICML), pp.
6105
6114
.
26.
Deng
,
J.
,
Dong
,
W.
,
Socher
,
R.
,
Li
,
L.-J.
,
Li
,
K.
, and
Fei-Fei
,
L.
,
2010
, “
ImageNet: A Large-Scale Hierarchical Image Database
,”
2009 IEEE Conference on Computer Vision and Pattern Recognition
,
Miami, FL
,
June 20–25
, pp.
248
255
.
27.
Sandler
,
M.
,
Howard
,
A.
,
Zhu
,
M.
,
Zhmoginov
,
A.
, and
Chen
,
L. C.
,
2018
, “
MobileNetV2: Inverted Residuals and Linear Bottlenecks
,”
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 19–21
, pp.
4510
4520
.
28.
Hu
,
J.
,
Shen
,
L.
, and
Sun
,
G.
,
2018
, “
Squeeze-and-Excitation Networks
,” pp.
7132
7141
.
arXiv
.
29.
Pan
,
S. J.
, and
Yang
,
Q.
,
2010
, “
A Survey on Transfer Learning
,”
IEEE Trans. Knowl. Data Eng.
,
22
(
10
), pp.
1345
1359
.
30.
Tzutalin
,
2015
, “
LabelImg
,” Git code. https://github.com/tzutalin/labelImg.
31.
Kingma
,
D. P.
, and
Ba
,
J. L.
,
2014
, “
Adam: A Method for Stochastic Optimization
,”
Third International Conference on Learning Representations, ICLR 2015—Conference Track Proceedings
,
San Diego, CA
,
May 7–9
, pp.
1
13
.
You do not currently have access to this content.