A surrogate-assisted optimization approach is an attractive way to reduce the total computational budget for obtaining optimal solutions. This makes it special for its application to practical optimization problems requiring a large number of expensive evaluations. Unfortunately, all practical applications are affected by measurement noises, and not much work has been done to address the issue of handling stochastic problems with multiple objectives and constraints. This work tries to bridge the gap by demonstrating three different frameworks for performing surrogate-assisted optimization on multi-objective constrained problems with stochastic measurements. To make the algorithms applicable to real-world problems, heteroscedastic (nonuniform) noise is considered for all frameworks. The proposed algorithms are first validated on several multi-objective numerical problems (unconstrained and constrained) to verify their effectiveness and then applied to the diesel engine calibration problem, which is expensive to perform and has measurement noises. A gt-suite model is used to perform the engine calibration study. Three control parameters, namely, variable geometry turbocharger (VGT) vane position, exhaust-gas-recirculating (EGR) valve position, and the start of injection (SOI), are calibrated to obtain the tradeoff between engine fuel efficiency performance (brake specific fuel consumption (BSFC)) and NOx emissions within the constrained design space. The results show that all three proposed extensions can handle the problems well with different measurement noise levels at a reduced evaluation budget. For the engine calibration problem, a good approximation of the optimal region is observed with more than 80% reduction in the evaluation budget for all the proposed methodologies.