The network in the United States is dominated by freight railroads, which transport over 40% of the nation’s intercity freight. The freight railroad network, including 76 000 bridges, was largely built 100 years ago, and most of these bridges are still in service. Over the last few decades, traffic loads have increased due to higher transport efficiency and demand from the continuously growing economy, subjecting these bridges to loads much greater than what they were designed to carry. This accelerates the deterioration process of the aging bridge structures and poses inspection, maintenance, and management challenges. It is of particular interest to closely monitor the displacement of railroad bridges under trainloads, as excessive displacement not only accelerates fatigue in bridge structures but can potentially cause track instability and loss of contact between the rail and train wheels. It is highly challenging, if not impossible, to install contact-type sensors such as a linear variable differential transformer (LVDT) or a string potentiometer on a bridge crossing a river with high piers, due to the difficulty of connecting the sensor to a stationary reference point. The noncontact computer vision sensor has demonstrated its significant advantages in cost-effective measurement of actual railway bridges, as presented in Section 3.7.