Steady-state and transient behaviors of single-phase natural circulation loop (SPNCL) are investigated using four thermal oils (Therminol VP1, Paratherm CR, Dowtherm A, and Dowtherm Q) and water-based ternary hybrid (various combinations of different nature and shaped nanoparticles: Al2O3, Cu, carbon nanotube (CNT) and graphene) nanofluids as loop fluid. The influences of nanoparticle volume concentration and loop height-to-width ratio on the mass flow rate and total entropy generation rate of SPNCL are investigated. Results disclose that ternary hybrid nanofluids enhance flow initiation, reduce fluctuation and are expected to attain a steady-state faster than water. Steady-state mass flow rate increases/decreases for ternary hybrid nanofluid depending on the shape of the nanoparticle, and the total entropy generation rate decreases as compared to water. Thermal oil shows a higher mass flow rate and total entropy generation rate as compared to water. Al2O3–Cu–CNT–water and Paratherm CR show the best result among all ternary hybrid nanofluids and thermal oils, respectively. The nanoparticle shape decides the optimum nanoparticle volume fraction. Increasing the height-to-width ratio decreases the total entropy generation and upsurges the mass flow rate at specified input power. The optimum height-to-width ratio depends on the loop fluid.