Evaluation of Reinforcement Learning for Optimal Control of Building Active and Passive Thermal Storage Inventory

This paper describes an investigation of machine-learning control for the supervisory control of building active and passive thermal storage inventory. Previous studies show that the utilization of either active or passive, or both can yield significant peak cooling load reduction and associated electrical demand and operational cost savings. In this study, a model-free learning control is investigated for the operation of electrically driven chilled water systems in heavy-mass commercial buildings. The reinforcement learning controller learns to operate the building and cooling plant optimally based on the feedback it receives from past control actions. The learning agent interacts with its environment by commanding the global zone temperature setpoints and TES charging/discharging rate. The controller extracts cues about the environment solely based on the reinforcement feedback it receives, which in this study is the monetary cost of each control action. No prediction or system model is required. Over time and by exploring the environment, the reinforcement learning controller establishes a statistical summary of plant operation, which is continuously updated as operation continues. This presented analysis revealed that learning control is a feasible methodology to find a near-optimal control strategy for exploiting the active and passive building thermal storage capacity, and also shows that the learning performance is affected by the dimensionality of the action and state space, the learning rate and several other factors. Moreover learning speed proved to be relatively low when dealing with tasks associated with large state and action spaces.