Sensor Fusion of Wayside Visible and Thermal Imagery for Rail Car Wheel and Bearing Damage Detection

Two major components of rolling stock that are always of great interest when it comes to maintenance and safety related issues are car wheels and bearings. Rail car wheels are subjected to a variety of damage types due to their interaction with the track and brakes. It is important for the rail industry to detect these defects and take proper action at an early stage, before more damage can be caused to the train or possibly the track and to prevent possible safety hazards. Different inspection sensors and systems, such as wheel impact monitors, wheel profile detectors, hotbox detectors and acoustic detection technologies, are employed to detect different types of wheel and bearing defects. Usually no single sensor can accurately detect all kinds of damages and hence a combination of different sensors and systems and manual inspection by experts is used for wheel maintenance purposes and to guarantee train safety. The more complete and accurate the automatic defect detections are, the less manual examination is necessary, leading to potential savings in inspection time/resources and rail car maintenance costs. Wayside thermal and visible spectrum cameras are one option for the automatic wheel and bearing inspection. Each of these sensors has their own strengths and weaknesses. There are some types of defects that are not detectable at an early stage in the images taken by a vision camera, however these defects generate a distinctive heat pattern on the wheel or bearing that is clearly visible in the thermal imagery. On the other hand, other damages might be detectable from the visible spectrum image, but not necessarily have a distinguishable heat pattern in the thermal imagery. Since a thermal image is basically built of solely temperature data, it excludes other critical information, such as texture or color. This makes thermal and visible spectrum imagery complementary and if the images are fused the result will benefit from the strengths of both sensors. In this paper, wavelet decomposition is employed to extract the features of the thermal and vision imagery. Then the two images are merged based on their decompositions and a fused image is composed. The resulting fused image contains more information than each individual image and can be used as an input for image-based wheel and bearing defect detection algorithms. To verify the proposed method and to show an example of this application, it is demonstrated on a real data set from a Union Pacific rail line to identify sliding wheels.