This paper deals with a sensor system based on the magnetostrictive effect, which is installed in a tool holder, for sensing cutting torque in order to monitor the milling process. Changes in the magnetic permeability of ferromagnetic alloy layers formed onto a rotating shaft due to torque are detected by coils as changes in impedance without contacting the rotating shaft. By mounting the magnetostrictive sensor on a tool holder, it became possible to detect cutting torque as close to the cutting point as possible. The accuracy of the proposed sensor system was confirmed through static and dynamic evaluation tests. In order to investigate the relationship between tool failure and cutting torque, side milling tests were performed, which bore out the possibility of estimating flank wear and tool fracture from the cutting torque detected by the proposed sensor system.