Laser-based additive manufacturing (LBAM) provides unrivalled design freedom with the ability to manufacture complicated parts for a wide range of engineering applications. Melt pool is one of the most important signatures in LBAM and is indicative of process anomalies and part defects. High-speed thermal images of the melt pool captured during LBAM make it possible for in situ melt pool monitoring and porosity prediction. This paper aims to broaden current knowledge of the underlying relationship between process and porosity in LBAM and provide new possibilities for efficient and accurate porosity prediction. We present a deep learning-based data fusion method to predict porosity in LBAM parts by leveraging the measured melt pool thermal history and two newly created deep learning neural networks. A PyroNet, based on Convolutional Neural Networks, is developed to correlate in-process pyrometry images with layer-wise porosity; an IRNet, based on Long-term Recurrent Convolutional Networks, is developed to correlate sequential thermal images from an infrared camera with layer-wise porosity. Predictions from PyroNet and IRNet are fused at the decision-level to obtain a more accurate prediction of layer-wise porosity. The model fidelity is validated with LBAM Ti–6Al–4V thin-wall structure. This is the first work that manages to fuse pyrometer data and infrared camera data for metal additive manufacturing (AM). The case study results based on benchmark datasets show that our method can achieve high accuracy with relatively high efficiency, demonstrating the applicability of the method for in situ porosity detection in LBAM.