It has been shown that model predictive control (MPC) is a promising solution for energy-efficient building operations. However, the deployment of MPC in a large portion of the building stock has not been possible partially because of high installation costs. Every building is unique and requires a tailored MPC solution. The best performing solutions are often based on physics-based modeling, which is, however, computationally expensive and requires dedicated software. A promising direction that tackles this problem is to train a neural network-based optimal control policy to imitate the behavior of physics-based MPC from the simulation data generated offline. The neural networks give control actions that closely approximate those produced by physics-based MPC, but with a fraction of the computational and memory requirements and without the need for licensed software. The main advantage of the proposed approach stems from simple evaluation at execution time, leading to low computational foot-prints and easy deployment on embedded HW platforms. In the case study, we present the energy savings potential of physics-based MPC applied to an office building in Belgium. We demonstrate how neural network approximators can be used to cut the implementation and maintenance costs of MPC deployment without compromising performance. We also critically assess the presented approach by pointing out the remaining challenges and open research questions.