The versatile nature of ultrasound imaging finds applications in various fields. A point contact excitation and detection method is generally used for visualizing the acoustic waves in Lead Zirconate Titanate (PZT) ceramics. Such an excitation method with a delta pulse generates a broadband frequency spectrum and wide directional wave vector. The presence of noise in the ultrasonic signals severely degrades the resolution and image quality. Deep learning-based signal and image denoising have been demonstrated recently. This paper bench-marked and compared several state-of-the-art deep learning image denoising methods with the classical denoising methods. The best-performing deep learning models are observed to be performing at par or, in some cases, even better than the classical methods on ultrasonic images. We further demonstrate the effectiveness and versatility of the deep learning-based denoising model for the unexplored domain of ultrasound/ultrasonic data. We conclude with a discussion on selecting the best method for denoising ultrasonic images. The impact of this work may help ultrasound-based defects identification equipment manufacturers to adopt a deep learning-based denoising model for more wider and versatile use.