This article investigates numerically welding residual stress distributions of a tube with J-groove weld in control rod drive mechanisms of a pressurized nuclear reactor vessel. Parametric study is performed for the effect of the tube location, tube dimensions, and material's yield strength. It is found that residual stresses increase with increasing the inclination angle of the tube, and the up-hill side is the most critical. For thicker tube, residual stresses decrease. For material's yield strength, both axial and hoop residual stresses tend to increase with increasing the yield strength of Alloy 600. Furthermore, axial stresses tend to increase with increasing yield strength of Alloys 82/182.