In the domain of optimal control for building HVAC systems, the performance of model-based control has been widely investigated and validated. However, the performance of model-based control highly depends on an accurate system performance model and sufficient sensors, which are difficult to obtain for certain buildings. To tackle this problem, a model-free optimal control method based on reinforcement learning is proposed to control the building cooling water system. In the proposed method, the wet bulb temperature and system cooling load are taken as the states, the frequencies of fans and pumps are the actions, and the reward is the system COP (i.e., the comprehensive COP of chillers, cooling water pumps, and cooling towers). The proposed method is based on Q-learning. Validated with the measured data from a real central chilled water system, a three-month measured data-based simulation is conducted under the supervision of four types of controllers: basic controller, local feedback controller, model-based controller, and the proposed model-free controller. Compared with the basic controller, the model-free controller can conserve 11% of the system energy in the first applied cooling season, which is greater than that of the local feedback controller (7%) but less than that of the model-based controller (14%). Moreover, the energy saving rate of the model-free controller could reach 12% in the second applied cooling season, after which the energy saving rate gets stabilized. Although the energy conservation performance of the model-free controller is inferior to that of the model-based controller, the model-free controller requires less a priori knowledge and sensors, which makes it promising for application in buildings for which the lack of accurate system performance models or sensors is an obstacle. Moreover, the results suggest that for a central chilled water system with a designed peak cooling load close to 2000 kW, three months of learning during the cooling season is sufficient to develop a good model-free controller with an acceptable performance.

在建筑物HVAC系统的最佳控制领域，基于模型的控制性能已得到广泛研究和验证。但是，基于模型的控制的性能高度依赖于准确的系统性能模型和足够的传感器，这对于某些建筑物而言很难获得。针对这一问题，提出了一种基于强化学习的无模型最优控制方法来控制建筑冷却水系统。在该方法中，以湿球温度和系统冷却负荷为状态，以风扇和泵的频率作为动作，以系统COP（即冷水机组，冷却水泵的综合COP）和系统的COP为回报。冷却塔）。所提出的方法基于Q学习。经过实际中央冷冻水系统的测量数据验证，在四种类型的控制器的监督下进行了为期三个月的基于测量数据的仿真：基本控制器，本地反馈控制器，基于模型的控制器和拟议的无模型控制器。与基本控制器相比，无模型控制器在第一个应用冷却季节可以节省11％的系统能量，这比本地反馈控制器的能量（7％）大，但比基于模型的控制器的能量少。 （14％）。此外，在第二个冷却季节中，无模型控制器的节能率可达到12％，此后，节能率将趋于稳定。尽管无模型控制器的节能性能不如基于模型的控制器，但无模型控制器需要的先验知识和传感器更少，这使其在缺乏精确的系统性能模型或传感器成为障碍的建筑物中具有广阔的应用前景。此外，结果表明，对于设计峰值制冷负荷接近2000 kW的中央冷冻水系统，在制冷季节学习三个月就足以开发出性能良好的无模型控制器。