Abstract

Resistance spot welding (RSW) is a widely adopted joining technique in automotive industry. Recent advancement in sensing technology makes it possible to collect thermal videos of the weld nugget during RSW using an infrared (IR) camera. The effective and timely analysis of such thermal videos has the potential of enabling in situ nondestructive evaluation (NDE) of the weld nugget by predicting nugget thickness and diameter. Deep learning (DL) has demonstrated to be effective in analyzing imaging data in many applications. However, the thermal videos in RSW present unique data-level challenges that compromise the effectiveness of most pre-trained DL models. We propose a novel image segmentation method for handling the RSW thermal videos to improve the prediction performance of DL models in RSW. The proposed method transforms raw thermal videos into spatial-temporal instances in four steps: video-wise normalization, removal of uninformative images, watershed segmentation, and spatial-temporal instance construction. The extracted spatial-temporal instances serve as the input data for training a DL-based NDE model. The proposed method is able to extract high-quality data with spatial-temporal correlations in the thermal videos, while being robust to the impact of unknown surface emissivity. Our case studies demonstrate that the proposed method achieves better prediction of nugget thickness and diameter than predicting without the transformation.

References

1.
Manladan
,
S.
,
Yusof
,
F.
,
Ramesh
,
S.
,
Fadzil
,
M.
,
Luo
,
Z.
, and
Ao
,
S.
,
2017
, “
A Review on Resistance Spot Welding of Aluminum Alloys
,”
Int. J. Adv. Manuf. Technol.
,
90
(
1–4
), pp.
605
634
.
2.
Chen
,
C.
,
Kong
,
L.
,
Wang
,
M.
,
Haselhuhn
,
A. S.
,
Sigler
,
D. R.
,
Wang
,
H.-P.
, and
Carlson
,
B. E.
,
2019
, “
The Robustness of Al-Steel Resistance Spot Welding Process
,”
J. Manuf. Process.
,
43
(
A
), pp.
300
310
.
3.
Wan
,
X.
,
Wang
,
Y.
, and
Fang
,
C.
,
2014
, “
Welding Defects Occurrence and Their Effects on Weld Quality in Resistance Spot Welding of AHSS Steel
,”
ISIJ Int.
,
54
(
8
), pp.
1883
1889
.
4.
Chen
,
S.
,
Sun
,
T.
,
Jiang
,
X.
,
Qi
,
J.
, and
Zeng
,
R.
,
2016
, “
Online Monitoring and Evaluation of the Weld Quality of Resistance Spot Welded Titanium Alloy
,”
J. Manuf. Process.
,
23
, pp.
183
191
.
5.
Imani
,
F.
,
Chen
,
R.
,
Diewald
,
E.
,
Reutzel
,
E.
, and
Yang
,
H.
,
2019
, “
Deep Learning of Variant Geometry in Layerwise Imaging Profiles for Additive Manufacturing Quality Control
,”
ASME J. Manuf. Sci. Eng.
,
141
(
11
), p.
111001
.
6.
Zhang
,
B.
,
Liu
,
S.
, and
Shin
,
Y. C.
,
2019
, “
In-Process Monitoring of Porosity During Laser Additive Manufacturing Process
,”
Addit. Manuf.
,
28
, pp.
497
505
.
7.
Francis
,
J.
, and
Bian
,
L.
,
2019
, “
Deep Learning for Distortion Prediction in Laser-Based Additive Manufacturing Using Big Data
,”
Manuf. Lett.
,
20
, pp.
10
14
.
8.
Janssens
,
O.
,
Van de Walle
,
R.
,
Loccufier
,
M.
, and
Van Hoecke
,
S.
,
2018
, “
Deep Learning for Infrared Thermal Image Based Machine Health Monitoring
,”
IEEE/ASME Trans. Mechatron.
,
23
(
1
), pp.
151
159
.
9.
Trefil
,
J. S.
,
2003
,
The Nature of Science: An AZ Guide to the Laws and Principles Governing Our Universe
,
Houghton Mifflin Harcourt
,
Boston, MA
.
10.
Beucher
,
S.
, and
Meyer
,
F.
,
1993
, “The Morphological Approach to Segmentation: The Watershed Transformation Chapter 12,”
Mathematical Morphology in Image Processing
,
E. R.
Dougherty
, ed.,
Marcel Dekker
,
New York
, pp.
433
481
.
11.
Grau
,
V.
,
Mewes
,
A.
,
Alcaniz
,
M.
,
Kikinis
,
R.
, and
Warfield
,
S. K.
,
2004
, “
Improved Watershed Transform for Medical Image Segmentation Using Prior Information
,”
IEEE Trans. Med. Imaging
,
23
(
4
), pp.
447
458
.
12.
Parvati
,
K.
,
Prakasa Rao
,
B. S.
, and
Mariya Das
,
M.
,
2008
, “
Image Segmentation Using Gray-Scale Morphology and Marker-Controlled Watershed Transformation
,”
Discrete Dyn. Nat. Soc.
,
2008
, Article ID 384346, pp.
1
8
.
13.
Lu
,
X.
,
Ma
,
C.
,
Ni
,
B.
,
Yang
,
X.
,
Reid
,
I.
, and
Yang
,
M.-H.
,
2018
, “
Deep Regression Tracking With Shrinkage Loss
,”
Proceedings of the European Conference on Computer Vision (ECCV).
,
Munich, Germany
,
Aug. 8–14
, pp.
353
369
.
14.
Saleem
,
M. H.
,
Potgieter
,
J.
, and
Arif
,
K. M.
,
2020
, “
Plant Disease Classification: A Comparative Evaluation of Convolutional Neural Networks and Deep Learning Optimizers
,”
Plants
,
9
(
10
), p.
1319
.
You do not currently have access to this content.