基于深度Q神经网络（DQN）的空调冷却水系统无模型优化
Model-free optimization for air conditioning cooling water systems based on deep Q network (DQN)
摘要：
在建筑空调水系统的优化控制领域，基于模型的控制方法得到了广泛的研究和验证。但基于模型的控制很大程度上依赖于精确的系统性能模型和足够的传感器，而这对于某些建筑来说是很难获得的。针对这一问题，本文提出了一种基于深度Q神经网络（DQN）的空调冷却水系统无模型优化方法，该方法以室外空气湿球温度、系统冷负荷及冷水机组开启状态为状态，以冷却塔风机和水泵的频率为动作，以系统性能系数（COP）为奖励。根据实际系统的实测数据进行建模，在模拟环境中使用基于粒子群优化算法的模型优化方法、基于Q值（Q learning）优化的强化学习方法和基于DQN的无模型优化方法进行实验，结果表明基于DQN的无模型优化方法的优化效果最好，有7.68%的平均COP提升与7.15%的节能率，在复杂系统下拥有较好的节能效果。
Abstract：
Model-based control methods have been widely investigated and validated in the domain of optimal control for building air conditioning water systems. However, the performance of model-based control highly depends on accurate system models and enough sensors, which are difficult to obtain in some buildings. To overcome this problem, a model-free optimization method for air conditioning cooling water system based on deep Q network (DQN) is proposed. The wet bulb temperature of outdoor air, system cooling load and chiller on/off states are taken as the states, the frequencies of cooling tower fans and cooling water pumps are taken as the actions, and the reward is the system COP. In the simulation environment built by the measured data of an actual system, the model optimization method based on particle swarm optimization, the reinforcement learning method based on Q-value (Q learning) optimization and the model-free optimization method based on DQN are used to conduct experiment. The results show that the model-free optimization method based on DQN has the best optimization effect with 7.68% average COP improvement and 7.15% energy saving rate, which has a better energy saving effect in complex systems.
Keywords:model-freeoptimization;deepQnetwork(DQN);coolingwatersystem;optimalcontrol;energyconsumption