Multi-laser powder bed fusion (M-LPBF) systems are garnering increased attention in metal additive manufacturing as they promise increased productivity and part size without sacrificing feature resolution or mechanical properties. However, M-LPBF introduces unique problems related to the interaction of multiple moving heat sources not observed in single laser systems, possibly leading to unexpected flaws and other process anomalies. Careful process modeling, planning, and monitoring are required to fully exploit M-LPBF. We present a novel in situ sensing and machine learning-based flaw detection for M-LPBF. Specifically, we consider a configuration where on-axis multi-spectral sensors are integrated and synchronized with each of the three lasers on a 3D Systems DMP Factory 500 printer. Each multi-spectral sensor monitors spectral emissions at two material-dependent wavelengths. The time series data generated from the multiple multi-spectral sensors are converted into a rasterized image per layer to be fed into a supervised deep learning (DL)-based semantic segmentation pipeline. To discriminate nominal process variations from anomalies, we explore a novel framework to incorporate context into the DL model which includes factors such as laser scan direction, processing parameters, and multi-laser proximity. We demonstrate our framework on in situ monitoring data collected during a build of carefully selected specimens seeded with surrogate lack of fusion flaws. Post-build X-ray computed tomography data are registered to the in situ data to generate ground truth labels for training and validation of the DL model.