A stereo camera system was installed facing Southeast at the observational tower owned by the University of Tokyo in the Sagami Bay, Japan. The three-dimensional wave fields were reconstructed from the stereo images, which were successfully captured from April 2017 until now, by using an open source software WASS (Waves Acquisition Stereo System). The significant wave heights and periods calculated from the stereo images covering an area of 80 m by 80 m were compared against those derived from the ultra-sonic wave gauge at the tower. Overall, a reasonable agreement is achieved, but the accuracy of the stereo reconstruction degrades with the distance from the camera. Also, the significant wave period derived from stereo imaging tends to be shorter, likely related to the error at high frequencies. The reconstructed wave field will be assimilated into a phase-resolved nonlinear wave model. The ensemble Higher Order Spectral simulations and the implementation of the a4DVAR data assimilation scheme, allowed us to substantially extend the estimated wave field beyond the stereo imaging domain. A field campaign with an ADCP in the field of view of the stereo camera and two surface wave buoys outside of the view were conducted for validation.