On-line detection and active control of chatter vibration have always been important issues in milling process respectively. To some extent, the signals obtained with sensors determine the performance of on-line detection and active control of chatter. However, due to the characteristics of milling process, the obtained signals are mainly consisted with spindle rotation frequency and its harmonics, and the chatter components are usually submerged by these stable harmonics, imposing negative effects for the detection and active control of milling chatter. Then, it is highly needed to design a real-time filter to filter out the spindle rotation frequency and its harmonics. In this paper, an adaptive filter is designed to filter out the spindle speed related components. Moving average (MR) model and adaptive filter theory is utilized to estimate these periodic components. The influence of filter order and step size factor on the filter characteristics are also analyzed. Considering that the filter order needs to be adjusted under different cutting conditions, which will alter the filter’s performance, an improved adaptive filter is proposed. Experiments are also performed and the experimental results show that, not only the spindle speed related components can be filtered out effectively, but the chatter frequency components are amplified with appropriate initial step factor, which is beneficial for the detection of milling chatter at early stage. Meanwhile, the periodic components caused by the installation error and the other spindle speed related components can be effectively filtered out real-timely, preventing the saturation of actuator caused by these stable components.