Over the past two decades, due to the rising energy prices and growing awareness about climate change, significant efforts have been devoted to reducing the energy consumption of various home appliances. However, the energy efficiency of clothes dryers has little improvement. Recent innovations in the direct-contact ultrasonic fabric drying technique offer new opportunities for energy saving. In this technique, high-frequency mechanical vibrations generated by the ultrasonic transducer are utilized to atomize water from a fabric in the liquid form, which demonstrates great potential for reducing energy use and drying time of the fabric drying process. Here, for the first time, fabric drying kinetics under different direct-contact ultrasonic drying conditions were investigated experimentally and analytically. The drying processes of four kinds of fabrics were experimentally tested under different ultrasonic transducer vibration frequency (115, 135, and 155 kHz) and input power (1.2, 2.5, and 4.4 W) conditions. According to the experimental data, five different kinds of models were applied to quantify the drying kinetics of fabrics during direct-contact ultrasonic drying. The models not only incorporated the transducer parameters but also the parameters related to the nature of fabric. Our evaluation results of model prediction performance demonstrated that the two empirical models, i.e., the Weibull model and the Gaussian model, were superior to the three semi-theoretical models for anticipating the drying kinetics of fabrics under direct-contact ultrasonic drying. Furthermore, the Weibull model is more suitable for practical energy-efficient direct-contact ultrasonic fabric drying applications compared with the Gaussian model.