Space nuclear power is the most potential power source for future deep space exploration, interstellar navigation, and planetary surface base station. For medium and high-power space missions, an efficient, compact and reliable energy conversion system that converts nuclear reactor thermal energy into mechanical energy or electrical energy is critical for the entire power system. The Stirling cycle that converts thermal energy into mechanical energy through expansion and compression of working fluids has the advantages of strong load adaptation, high conversion efficiency, modular combination, robust reliability, redundancy, etc. Therefore, it is very suitable for future high-power deep space missions. The technical route of space nuclear power proposed by Nanjing University of Aeronautics and Astronautics is based on the modular Stirling thermoelectric conversion and liquid molten salt core scheme. The present study is the simulation prediction of the thermodynamic performance of the Stirling engine. The working fluid’s composition, physical properties, and leakage properties have great influences on the thermal efficiency, operating lifetime, and reliability of the Stirling cycle. In this regard, a comprehensive study on the properties and effects of H2, He, He-Xe mixture, N2 and air as working fluids was carried out in the present study. The modified Stirling thermodynamic model IPD-MSM was used to simulate the Stirling cycle, and the heat and power losses caused by various irreversible factors were analyzed. Moreover, impacts of operating pressure, heat source temperature, and piston frequency on every heat and power loss were discussed. The results show that the pressure loss and the non-ideal heat transfer loss are the dominant losses. However, different working fluids and operating conditions have different performances of power loss. The effect of operating frequency and working pressure are significant, while the effect of operating temperature is relatively small. The present study provides theoretical support for selecting thermoelectric conversion methods for future medium and high-power space missions.