Abstract

In the production of cold-rolled galvanized steel strips used for stamping car body parts, the in-situ and real-time defect detection is crucial for quality control, in which various types of defects inevitably occur. It is challenging to improve the accuracy of defect detection and classification by appropriate means to assist the manual screening process better. Defects under actual production conditions are often not prominent enough in defect characteristics, and there may be a significant similarity between different defect categories. To eliminate this weakness, we propose a data-driven deep learning approach named steel surface faulty detection attention net (SSFDANet) that uses images of the galvanized steel surfaces as input to identify whether the product is qualified and automatic classification of defect types instantaneously. This method can shorten product inspection time and improve the production line automation efficiency. In addition, the attention mechanism is utilized to enhance the performance of SSFDANet. Compared with the baseline ResNet, SSFDANet achieves a noticeable improvement in classification accuracy on test data. The well-trained model can successfully show an improved performance than the baseline models on the multiple types of faulty. Enhanced by SSFDANet with high classification accuracy, the defect rate of products is significantly reduced, and the production speed of the production line is significantly improved. Future prospective studies that are inspired by this article are also discussed.

References

1.
Xu
,
D.
, and
Di
,
X.
,
2004
, “
Research on Surface Defect Classification of Strip Steel Based on LVQ
,”
Chinese Instrumentation
,
24
(
2
), p.
3
.
2.
Hua
,
C.
, and
Zhou
,
H.
,
2016
, “
Clustering and Optimization of Support Vector Machines for Surface Defect Classification of Cold-Rolled Strip Steel
,”
J. Plastic Eng.
,
23
(
5
), p.
6
.
3.
Liu
,
K.
,
Zhang
,
A.
,
Quer
,
Q.
, and
Dong
,
Y.
,
2018
, “
Classification of Surface Defects of Strip Steel Based on Weighted Coding of Local Features
,”
Control Eng.
,
25
(
12
), p.
6
.
4.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2015
, “
Deep Residual Learning for Image Recognition
,”
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on.
,
Las Vegas, NV
,
June 26–July 1
, pp.
770
778
.
5.
Cao
,
C.
,
Liu
,
X.
,
Yang
,
Y.
,
Yu
,
Y.
,
Wang
,
J.
,
Wang
,
Z.
,
Huang
,
Y.
,
Wang
,
L.
,
Huang
,
C.
,
Xu
,
W.
,
Ramanan
,
D.
, and
Huang
,
T. S.
,
2015
,
Look and Think Twice: Capturing Top-Down Visual Attention With Feedback Convolutional Neural Networks
.
6.
Li
,
K.
,
Wu
,
Z.
,
Peng
,
K.-C.
,
Ernst
,
J.
, and
Fu
,
Y.
,
2018
, “
Tell Me Where to Look: Guided Attention Inference Network
,”
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Computer Vision and Pattern Recognition (CVPR), 2018 IEEE/CVF Conference on, CVPR
,
Salt Lake City, UT
,
June 18–22
, pp.
9215
9223
.
7.
Elakkiya
,
E.
,
Selvakumar
,
S.
, and
Leela Velusamy
,
R.
,
2020
, “
Textspamdetector: Textual Content Based Deep Learning Framework for Social Spam Detection Using Conjoint Attention Mechanism
,”
J. Ambient Intell. Humanized Comput.
,
12
(
10
), p.
1
.
8.
Huang
,
R.
,
Gu
,
J.
,
Sun
,
X.
,
Hou
,
Y.
, and
Uddin
,
S.
,
2019
, “
A Rapid Recognition Method for Electronic Components Based on the Improved YOLO-V3 Network
,”
Electronics
,
8
(
8
), p.
825
.
9.
Redmon
,
J.
, and
Farhadi
,
A.
,
2018
, “
YOLOV3: An Incremental Improvement
”. CoRR, abs/1804.02767.
10.
Howard
,
A. G.
,
Zhu
,
M.
,
Chen
,
B.
,
Kalenichenko
,
D.
,
Wang
,
W.
,
Weyand
,
T.
,
Andreetto
,
M.
, and
Adam
,
H.
,
2017
, “
Mobilenets: Efficient Convolutional Neural Networks for Mobile Vision Applications
”. CoRR, abs/1704.04861.
11.
Liu
,
W.
,
Anguelov
,
D.
,
Erhan
,
D.
,
Szegedy
,
C.
,
Reed
,
S. E.
,
Fu
,
C.
, and
Berg
,
A. C.
,
2015
, “
SSD: Single Shot Multibox Detector
”. CoRR, abs/1512.02325.
12.
Girshick
,
R. B.
,
2015
, “
Fast R-CNN
”. CoRR, abs/1504.08083.
13.
Bhatt
,
P. M.
,
Malhan
,
R. K.
,
Rajendran
,
P.
,
Shah
,
B. C.
,
Thakar
,
S.
,
Yoon
,
Y. J.
, and
Gupta
,
S. K.
,
2021
, “
Image-Based Surface Defect Detection Using Deep Learning: A Review
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
4
), p.
040801
.
14.
Wan
,
Q.
,
Gao
,
L.
,
Li
,
X.
, and
Wen
,
L.
,
2022
, “
Unsupervised Image Anomaly Detection and Segmentation Based on Pre-Trained Feature Mapping
,”
IEEE Trans. Ind. Inf.
, pp.
1
10
.
15.
Bergmann
,
P.
,
Fauser
,
M.
,
Sattlegger
,
D.
, and
Steger
,
C.
,
2019
, “
MVTec AD – A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
,”
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
, pp.
9584
9592
.
16.
Napoletano
,
P.
,
Piccoli
,
F.
, and
Schettini
,
R.
,
2018
, “
Anomaly Detection in Nanofibrous Materials by CNN-Based Self-Similarity
,”
Sensors
,
18
(
1
), p.
209
.
17.
Vijai Kumar
,
S.
, and
Vuik
,
C.
,
2021
, “
A Simple and Fast Hole Detection Algorithm for Triangulated Surfaces
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
4
), p.
044502
.
18.
Wong
,
V. W. H.
,
Ferguson
,
M.
,
Law
,
K. H.
,
Lee
,
Y. -T. T.
, and
Witherell
,
P.
,
2021
, “
Segmentation of Additive Manufacturing Defects Using U-Net
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
3
), p.
031005
.
19.
Chiu
,
M.-C.
,
Tsai
,
C.-D.
, and
Li
,
T.-L.
,
2020
, “
An Integrative Machine Learning Method to Improve Fault Detection and Productivity Performance in a Cyber-Physical System
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
2
), p.
021009
.
20.
Ren
,
M.
,
Shen
,
R.
, and
Gong
,
Y.
,
2022
, “
A Surface Defect Detection Method Via Fusing Multi-Level Features
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
5
), p.
051005
.
21.
Konovalenko
,
I.
,
Maruschak
,
P.
, and
Brevus
,
V.
,
2021
, “
Steel Surface Defect Detection Using an Ensemble of Deep Residual Neural Networks
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
1
), p.
014501
.
22.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on
,
Las Vegas, NV
,
June 26–July 1
, pp.
770
778
.
23.
Nie
,
Z.
,
Jiang
,
H.
, and
Kara
,
L. B.
,
2020
, “
Stress Field Prediction in Cantilevered Structures Using Convolutional Neural Networks
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
1
), p.
011002
.
24.
An
,
F.-P.
, and
Liu
,
J.-e.
,
2021
, “
Medical Image Segmentation Algorithm Based on Multilayer Boundary Perception-Self Attention Deep Learning Model
,”
Multim. Tool. Appl.: Int. J.
,
80
(
10
), pp.
15017
15039
.
25.
Choi
,
J.-S.
, and
Kim
,
M.
,
2017
, “
A Deep Convolutional Neural Network With Selection Units for Super-Resolution
,”
2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference on, CVPRW
,
Honolulu
, HI,
July 22–25
, pp.
1150
1156
.
26.
Zhang
,
Y.
,
Li
,
K.
,
Li
,
K.
,
Wang
,
L.
,
Zhong
,
B.
, and
Fu
,
Y.
,
2018
, “
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
.” arXiv.
27.
Iandola
,
F. N.
,
Han
,
S.
,
Moskewicz
,
M. W.
,
Ashraf
,
K.
,
Dally
,
W. J.
, and
Keutzer
,
K.
,
2016
, “
Squeezenet: Alexnet-Level Accuracy With 50x Fewer Parameters and <0.5 mb Model Size
.” arXiv.
28.
Canny
,
J.
,
1986
, “
A Computational Approach to Edge Detection
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
PAMI-8
(
6
), pp.
679
698
.
29.
Johnson
,
J.
,
Alahi
,
A.
, and
Fei-Fei
,
L.
,
2016
, “
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
,”
European conference on computer vision (ECCV)
,
The Netherlands
,
Oct. 11–14
, p.
694
.
30.
Zhang
,
Z.
,
Wang
,
Z.
,
Lin
,
Z.
, and
Qi
,
H.
,
2019
, “
Image Super-Resolution by Neural Texture Transfer
,”
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Computer Vision and Pattern Recognition (CVPR), 2019 IEEE/CVF Conference on
,
Long Beach, CA
,
June 16–20
, pp.
7974
7983
.
31.
Ledig
,
C.
,
Theis
,
L.
,
Huszar
,
F.
,
Caballero
,
J.
,
Cunningham
,
A.
,
Acosta
,
A.
,
Aitken
,
A.
,
Tejani
,
A.
,
Totz
,
J.
,
Wang
,
Z.
, and
Shi
,
W.
,
2016
, “
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
.” arXiv.
32.
Xu
,
Q.
,
Nie
,
Z.
,
Xu
,
H.
,
Zhou
,
H.
,
Attar
,
H. R.
,
Li
,
N.
,
Xie
,
F.
, and
Liu
,
X.-J.
,
2021
, “
SuperMeshing: A New Deep Learning Architecture for Increasing the Mesh Density of Physical Fields in Metal Forming Numerical Simulation
,”
ASME J. Appl. Mech.
,
89
(
1
), p.
011002
.
33.
Park
,
J.
,
Woo
,
S.
,
Lee
,
J.-Y.
, and
Kweon
,
I. S.
,
2018
, “
Bam: Bottleneck Attention Module
.” arXiv.
34.
Janocha
,
K.
, and
Czarnecki
,
W. M.
,
2017
, “
On Loss Functions for Deep Neural Networks in Classification
.” arXiv.
35.
Zhao
,
P.
, and
Lai
,
L.
,
2020
, “
Minimax Optimal Estimation of KL Divergence for Continuous Distributions
,”
IEEE Trans. Inform. Theory
,
66
(
12
), pp.
7787
7811
.
36.
Kingma
,
D. P.
, and
Ba
,
J.
,
2014
, “
Adam: A Method for Stochastic Optimization
,” arXiv preprint arXiv:1412.6980.
37.
Vaswani
,
A.
,
Shazeer
,
N.
,
Parmar
,
N.
,
Uszkoreit
,
J.
,
Jones
,
L.
,
Gomez
,
A. N.
,
Kaiser
,
L.
, and
Polosukhin
,
I.
,
2017
, “
Attention Is All You Need
,”
Neural Information Processing Systems.
38.
Yang
,
F.
,
Yang
,
H.
,
Fu
,
J.
,
Lu
,
H.
, and
Guo
,
B.
,
2020
, “
Learning Texture Transformer Network for Image Super-Resolution
,”
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Computer Vision and Pattern Recognition (CVPR), 2020 IEEE/CVF Conference on, CVPR
,
Virtual
,
June 14–19
, pp.
5790
5799
.
39.
Kasem
,
H.
,
Hung
,
K.
, and
Jiang
,
J.
,
2019
, “
Spatial Transformer Generative Adversarial Network for Robust Image Super-resolution
,”
IEEE Access
,
7
, p.
182993
.
You do not currently have access to this content.