There are significant quality and reliability problems for components/products made by additive manufacturing (AM) due to various reasons. Selective laser melting (SLM) process is one of the popular AM techniques and it suffers from low quality and reliability issue as well. Among many reasons, the lack of accurate and efficient models to simulate the SLM process could be the most important one because reliability and quality quantification rely on accurate models; otherwise, a large number of experiments should be conducted for reliability and quality assurance. To date, modeling techniques for the SLM process are either computationally expensive based on finite element (FE) modeling or economically expensive requiring a significant amount of experiment data for data-driven modeling. This paper proposes the integration of FE and data-driven modeling with systematic calibration and validation framework for the SLM process based on limited experiment data. Multi-fidelity models are the FE model for the SLM process and a machine learning model constructed based on the FE model instead of real experiment data. The machine learning model, after incorporation of the learned physics from the FE model, is then further improved based on limited real experiment data through the calibration and validation framework. The proposed work enables the development of highly efficient and accurate models for melt pool prediction of the SLM process under various configurations. The effectiveness of the framework is demonstrated by real experiment data under 14 different printing configurations.