Abstract

Nowadays, additive manufacturing technologies (AM) suffer from insufficient or lacking methodologies/techniques for quality control. This fact represents a key technological barrier preventing broader industrial adoption of AM, particularly in high-value applications where component failure cannot be accepted. This article presents a real-time melt pool segmentation and monitoring technique applicable to the direct laser metal deposition (LMD) process. An infrared camera with an InSb detector (resolution of 640 × 480, spectral range between 3 and 5 μm) was used. An algorithm, called gravitational superpixels, is presented. This algorithm can group pixels and generate superpixels based on a block generation technique that compares color similarity and temperature in infrared images. Besides, a color similarity correction is applied to reduce uncertainty in segmentation, as well as for eliminating the image background. The task of extracting edges is based on the law of universal gravitation. A quantitative and qualitative algorithm performance analysis, which uses standard metrics, is presented. The analysis demonstrates better versatility than reduction/feature extraction or image segmentation approaches by high-/low-pass filtering. The experimental validation was carried out, extracting and measuring the molten pool geometry and its thermal signature. Then, measures were compared against ground truth and against results obtained by other similar methods. The proposed gravitational superpixel method has higher precision and performance. Our proposal has a significant potential for monitoring industrial AM processes since it requires minimal modifications of commercially available industrial machines.

References

1.
Achanta
,
R.
,
Shaji
,
A.
,
Smith
,
K.
,
Lucchi
,
A.
,
Fua
,
P.
, and
Süsstrunk
,
S.
,
2010
,
EPFL Technical Report No. 149300
.
2.
Arinez
,
J. F.
,
Chang
,
Q.
,
Gao
,
R. X.
,
Xu
,
C.
, and
Zhang
,
J.
,
2020
, “
Artificial Intelligence in Advanced Manufacturing: Current Status and Future Outlook
,”
ASME J. Manuf. Sci. Eng.
,
142
(
11
), p.
110804
.
3.
Bartlett
,
J. L.
,
Heim
,
F. M.
,
Murty
,
Y. V.
, and
Li
,
X.
,
2018
, “
In Situ Defect Detection in Selective Laser Melting Via Full-Field Infrared Thermography
,”
Addit. Manuf.
,
24
, pp.
595
605
.
4.
Beaman
,
J. J.
,
Bourell
,
D. L.
,
Seepersad
,
C. C.
, and
Kovar
,
D.
,
2020
, “
Additive Manufacturing Review: Early Past to Current Practice
,”
ASME J. Manuf. Sci. Eng.
,
142
(
11
), p.
110812
.
5.
Bikas
,
H.
,
Stavropoulos
,
P.
, and
Chryssolouris
,
G.
,
2016
, “
Additive Manufacturing Methods and Modelling Approaches: A Critical Review
,”
Int. J. Adv. Manuf. Technol.
,
83
(
1–4
), pp.
389
405
.
6.
Boone
,
N.
,
Zhu
,
C.
,
Smith
,
C.
,
Todd
,
I.
, and
Willmott
,
J. R.
,
2018
, “
Thermal Near Infrared Monitoring System for Electron Beam Melting With Emissivity Tracking
,”
Addit. Manuf.
,
22
, pp.
601
605
.
7.
Bouguet
,
J.-Y.
,
2002
, “
Camera Calibration Tool-Box for Matlab
,” http://www.vision.caltech.edu/bouguetj/calib_doc/, Accessed October, 2019.
8.
Bruhn
,
A.
,
Weickert
,
J.
, and
Schnörr
,
C.
,
2005
, “
Lucas/Kanade Meets Horn/schunck: Combining Local and Global Optic Flow Methods
,”
Int. J. Comput. Vision
,
61
(
3
), pp.
211
231
.
9.
Caprio
,
L.
,
Demir
,
A. G.
, and
Previtali
,
B.
,
2020
, “
Observing Molten Pool Surface Oscillations During Keyhole Processing in Laser Powder Bed Fusion as a Novel Method to Estimate the Penetration Depth
,”
Addit. Manuf.
,
36
, pp.
1
15
.
10.
Chandrasekhar
,
N.
,
Vasudevan
,
M.
,
Bhaduri
,
A. K.
, and
Jayakumar
,
T.
,
2015
, “
Intelligent Modeling for Estimating Weld Bead Width and Depth of Penetration From Infra-Red Thermal Images of the Weld Pool
,”
J. Intell. Manufact.
,
26
(
1
), pp.
59
71
.
11.
Cheng
,
B.
,
Lei
,
J.
, and
Xiao
,
H.
,
2019
, “
A Photoacoustic Imaging Method for In-Situ Monitoring of Laser Assisted Ceramic Additive Manufacturing
,”
Optics Laser Technol.
,
115
, pp.
459
464
.
12.
Clijsters
,
S.
,
Craeghs
,
T.
,
Buls
,
S.
,
Kempen
,
K.
, and
Kruth
,
J.-P.
,
2014
, “
In Situ Quality Control of the Selective Laser Melting Process Using a High-Speed, Real-Time Melt Pool Monitoring System
,”
Int. J. Adv. Manuf. Technol.
,
75
(
5–8
), pp.
1089
1101
.
13.
Craeghs
,
T.
,
Clijsters
,
S.
,
Kruth
,
J.-P.
,
Bechmann
,
F.
, and
Ebert
,
M.-C.
,
2012
, “
Detection of Process Failures in Layerwise Laser Melting With Optical Process Monitoring
,”
Phys. Procedia.
,
39
, pp.
753
759
.
14.
Everton
,
S. K.
,
Hirsch
,
M.
,
Stravroulakis
,
P.
,
Leach
,
R. K.
, and
Clare
,
A. T.
,
2016
, “
Review of In-Situ Process Monitoring and In-Situ Metrology for Metal Additive Manufacturing
,”
Mater. Des.
,
95
, pp.
431
445
.
15.
Grasso
,
M.
, and
Colosimo
,
B. M.
,
2017
, “
Process Defects and In Situ Monitoring Methods in Metal Powder Bed Fusion: A Review
,”
Meas. Sci. Technol.
,
28
(
4
), p.
044005
.
16.
Hanbury
,
A.
,
2008
,
How Do Superpixels Affect Image Segmentation?
,
J.
Ruiz-Shulcloper
, and
W. G.
Kropatsch
, eds.,
Springer
,
New York
, pp.
178
186
.
17.
Helwig
,
N. E.
, and
Ma
,
P.
,
2015
, “
Fast and Stable Multiple Smoothing Parameter Selection in Smoothing Spline Analysis of Variance Models With Large Samples
,”
J. Computat. Graphical Stat.
,
24
(
3
), pp.
715
732
.
18.
Hooper
,
P. A.
,
2018
, “
Melt Pool Temperature and Cooling Rates in Laser Powder Bed Fusion
,”
Addit. Manuf.
,
22
, pp.
548
559
.
19.
Jurrens
,
K.
, and
Incorporated
,
E.
,
2013
,
Technical Report, National Institute of Standards and Technology
.
20.
Karnati
,
S.
, and
Liou
,
F. F.
,
2020
,
Detection and Tracking of Melt Pool in Blown Powder Deposition Through Image Processing of Infrared Camera Data
,
O.
Sergiyenko
,
W.
Flores-Fuentes
, and
P.
Mercorelli
, eds.,
Springer
,
New York
, pp.
711
732
.
21.
Khanzadeh
,
M.
,
Chowdhury
,
S.
,
Marufuzzaman
,
M.
,
Tschopp
,
M. A.
, and
Bian
,
L.
,
2018
, “
Porosity Prediction: Supervised-Learning of Thermal History for Direct Laser Deposition
,”
J. Manuf. Syst.
,
47
, pp.
69
82
.
22.
Khanzadeh
,
M.
,
Tian
,
W.
,
Yadollahi
,
A.
,
Doude
,
H. R.
,
Tschopp
,
M. A.
, and
Bian
,
L.
,
2018
, “
Dual Process Monitoring of Metal-Based Additive Manufacturing Using Tensor Decomposition of Thermal Image Streams
,”
Addit. Manuf.
,
23
, pp.
443
456
.
23.
Leach
,
R. K.
,
Bourell
,
D.
,
Carmignato
,
S.
,
Donmez
,
A.
,
Senin
,
N.
, and
Dewulf
,
W.
,
2019
, “
Geometrical Metrology for Metal Additive Manufacturing
,”
CIRP. Ann.
,
68
(
2
), pp.
677
700
.
24.
Lee
,
J.
,
Ni
,
J.
,
Singh
,
J.
,
Jiang
,
B.
,
Azamfar
,
M.
, and
Feng
,
J.
,
2020
, “
Intelligent Maintenance Systems and Predictive Manufacturing
,”
ASME J. Manuf. Sci. Eng.
,
142
(
11
), p.
110805
.
25.
Leung
,
C. L. A.
,
Marussi
,
S.
,
Atwood
,
R. C.
,
Towrie
,
M.
,
Withers
,
P. J.
, and
Lee
,
P. D.
,
2018
, “
In Situ X-Ray Imaging of Defect and Molten Pool Dynamics in Laser Additive Manufacturing
,”
Nat. Commun.
,
9
(
1
), pp.
1
9
.
26.
Levinshtein
,
A.
,
Stere
,
A.
,
Kutulakos
,
K. N.
,
Fleet
,
D. J.
,
Dickinson
,
S. J.
, and
Siddiqi
,
K.
,
2009
, “
Turbopixels: Fast Superpixels Using Geometric Flows
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
31
(
12
), pp.
2290
2297
.
27.
Li
,
Z.
, and
Chen
,
J.
,
2015
, “
Superpixel Segmentation Using Linear Spectral Clustering
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Boston, MA
.
28.
Liu
,
Y.-J.
,
Yu
,
C.-C.
,
Yu
,
M.-J.
, and
He
,
Y.
,
2016
, “
Manifold Slic: A Fast Method to Compute Content-Sensitive Superpixels
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Las Vegas, NV
.
29.
Americs
,
Makes
, and
ANSI
,
2017
,
Standardization Roadmap for Additive Manufacturing
,
ANSI Additive Manufacturing Standardization Collaborative
.
30.
Medina
,
J. S.
,
Devesse
,
W.
,
Ertveldt
,
J.
, and
Guillaume
,
P.
,
2020
, “
Comparison of Visual and Hyperspectral Monitoring of the Melt Pool During Laser Metal Deposition
,”
Proc. CIRP
,
94
, pp.
462
468
.
31.
Moralejo
,
S.
,
Penaranda
,
X.
,
Nieto
,
S.
,
Barrios
,
A.
,
Arrizubieta
,
I.
,
Tabernero
,
I.
, and
Figueras
,
J.
,
2017
, “
A Feedforward Controller for Tuning Laser Cladding Melt Pool Geometry in Real Time
,”
Int. J. Adv. Manuf. Technol.
,
89
(
1–4
), pp.
821
831
.
32.
ASTM Committee F42 on Additive Manufacturing Technologies
,
2012
,
Standard Terminology for Additive Manufacturing Technologies
,
ASTM International
.
33.
Pacher
,
M.
,
Mazzoleni
,
L.
,
Caprio
,
L.
,
Demir
,
A. G.
, and
Previtali
,
B.
,
2019
, “
Estimation of Melt Pool Size by Complementary Use of External Illumination and Process Emission in Coaxial Monitoring of Selective Laser Melting
,”
J. Laser. Appl.
,
31
(
2
), p.
022305
.
34.
Papari
,
G.
, and
Petkov
,
N.
,
2011
, “
Edge and Line Oriented Contour Detection: State of the Art
,”
Image Vision Comput.
,
29
(
2–3
), pp.
79
103
.
35.
Rieder
,
H.
,
Dillhöfer
,
A.
,
Spies
,
M.
,
Bamberg
,
J.
, and
Hess
,
T.
,
2014
, “
Online Monitoring of Additive Manufacturing Processes Using Ultrasound
,”
11th European Conference on Non-Destructive Testing (ECNDT)
,
Prague, Czech Republic
,
Oct. 6–10
, pp.
6
10
.
36.
Sampson
,
R.
,
Lancaster
,
R.
,
Sutcliffe
,
M.
,
Carswell
,
D.
,
Hauser
,
C.
, and
Barras
,
J.
,
2020
, “
An Improved Methodology of Melt Pool Monitoring of Direct Energy Deposition Processes
,”
Optics Laser Technol.
,
127
, p.
106194
.
37.
Shevchik
,
S. A.
,
Masinelli
,
G. G.
,
Kenel
,
C.
,
Leinenbach
,
C.
, and
Wasmer
,
K.
,
2019
, “
Deep Learning for In Situ and Real-Time Quality Monitoring in Additive Manufacturing Using Acoustic Emission
,”
IEEE Trans. Indus. Inform.
,
15
(
6
), pp.
5194
5203
.
38.
Song
,
L.
,
Wang
,
F.
,
Li
,
S.
, and
Han
,
X.
,
2017
, “
Phase Congruency Melt Pool Edge Extraction for Laser Additive Manufacturing
,”
J. Mater. Process. Technol.
,
250
, pp.
261
269
.
39.
Taheri
,
H.
,
Koester
,
L. W.
,
Bigelow
,
T. A.
,
Faierson
,
E. J.
, and
Bond
,
L. J.
,
2019
, “
In Situ Additive Manufacturing Process Monitoring With An Acoustic Technique: Clustering Performance Evaluation Using K-Means Algorithm
,”
ASME J. Manuf. Sci. Eng.
,
141
(
4
), p.
041011
.
40.
Uma
,
J.
,
Muniraj
,
C.
, and
Sathya
,
N.
,
2019
, “
Diagnosis of Photovoltaic (PV) Panel Defects Based on Testing and Evaluation of Thermal Image
,”
J. Test. Evaluat.
,
47
(
6
), pp.
4249
4262
.
41.
Van den Bergh
,
M.
,
Boix
,
X.r
,
Roig
,
G.
,
de Capitani
,
B.
, and
Van Gool
,
L.
,
2012
,
Seeds: Superpixels Extracted Via Energy-Driven Sampling
,
A.
Fitzgibbon
,
S.
Lazebnik
,
P.
Perona
,
Y.
Sato
, and
C.
Schmid
, eds.,
Springer
,
New York
, pp.
13
26
.
42.
Veksler
,
O.
,
Boykov
,
Y.
, and
Mehrani
,
P.
,
2010
,
Superpixels and Supervoxels in an Energy Optimization Framework
,
K.
Daniilidis
,
P.
Maragos
, and
N.
Paragios
, eds.,
Springer
,
New York
, pp.
211
224
.
43.
Wang
,
J.
, and
Wang
,
X.
,
2012
, “
Vcells: Simple and Efficient Superpixels Using Edge-Weighted Centroidal Voronoi Tessellations
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
34
(
6
), pp.
1241
1247
.
44.
Wang
,
M.
,
Liu
,
X.
,
Gao
,
Y.
,
Ma
,
X.
, and
Soomro
,
N. Q.
,
2017
, “
Superpixel Segmentation: A Benchmark
,”
Signal Process.: Image Communicat.
,
56
, pp.
28
39
.
45.
Wu
,
J.
,
Li
,
J.
,
Liu
,
J.
, and
Tian
,
J.
,
2004
, “
Infrared Image Segmentation Via Fast Fuzzy C-Means With Spatial Information
,”
2004 IEEE International Conference on Robotics and Biomimetics
,
IEEE
, pp.
742
745
.
46.
Xu
,
C.
, and
Corso
,
J. J.
,
2012
,
Evaluation of Super-Voxel Methods for Early Video Processing
,
IEEE
, pp.
1202
1209
.
47.
Yan
,
Z.
,
Liu
,
W.
,
Tang
,
Z.
,
Liu
,
X.
,
Zhang
,
N.
,
Li
,
M.
, and
Zhang
,
H.
,
2018
, “
Review on Thermal Analysis in Laser-Based Additive Manufacturing
,”
Optics Laser Technol.
,
106
, pp.
427
441
.
48.
Zhang
,
Y.
,
Fuh
,
J. Y. H.
,
Ye
,
D.
, and
Hong
,
G. S.
,
2019
, “
In-Situ Monitoring of Laser-Based Pbf Via Off-Axis Vision and Image Processing Approaches
,”
Addit. Manuf.
,
25
, pp.
263
274
.
You do not currently have access to this content.