Most practical multi-objective optimization problems are often characterized by two or more expensive and conflicting objectives, which require time-consuming simulations. Commonly used algorithms construct a surrogate model of each objective function from a few high-fidelity solutions. In order to further decrease the computational burden, adaptive infilling strategies for multi-objective problems are developed to guide the next infilling design for expensive evaluation and update the surrogate model as well as the Pareto front in an iterative manner. In this paper, a multi-objective infilling strategy integrating the Kriging model with a two-stage infilling framework is proposed, termed as ATKIS. This method allows exploitation and exploration alternately to pinpoint the infilling solution for improving the Pareto set and avoiding local over-exploitation simultaneously. At the local exploitation stage, Kriging-based prediction and uncertainty estimation are combined with Non-dominant Ranking and Minimum Relative Distance theories for determining a new design solution, which has maximum improvement relative to the current Pareto set. At the global exploration stage, Voronoi tessellation theory is employed to search for the sparsest position in the design space for a new evaluation. The proposed method is compared with five recent infilling strategies to investigate the performance of infilling ability using several numerical benchmarks. The experimental results show that the proposed method outperforms the other three strategies in improving both effectiveness and robustness using the improvement of hypervolume as the evaluating indicator. In addition, a lightweight optimization design of hoist sheaves shows that the proposed method can deal with real engineering problems, while significantly reducing the computational time and the number of expensive simulations of samples.