As environmental pollution becomes increasingly serious and industrial energy consumption continuously rises, an intelligent and efficient industrial energy management policy is urgently needed to reduce costs and maximize the benefits of industrial energy systems. However, modern industrial energy systems are characterized by hybrid industrial equipment actions, diverse objectives, and highly intermittent and stochastically distributed renewable energy sources. Therefore, efficient operation and control are difficult. This article presents a novel, model-free energy management policy using a hybrid action deep reinforcement learning algorithm for energy scheduling of industrial equipments operating in various modes. Specifically, the interaction process between the industrial energy management center and each equipment is modeled as a Markov decision process that minimizes the daily operating cost of the energy system and maximizes the revenue of the production equipment. Then, a double parameterized deep Q-networks that does not require an explicit environmental model is developed to learn the hybrid action signals using actor and critic networks, in which the double Q value mechanism avoids value overestimation and improves the algorithm efficiency. In addition, the policy gradient of the proposed algorithm is derived and its convergence proof is discussed. Finally, numerical studies are conducted using real-world data to evaluate algorithm performance and verify its effectiveness.

中文翻译：

---

一种新型的基于混合行动的工业能源管理深度强化学习


随着环境污染日益严重、工业能源消耗不断上升，迫切需要智能、高效的工业能源管理政策，以降低工业能源系统成本，实现效益最大化。然而，现代工业能源系统的特点是工业设备动作混合、目标多样化、可再生能源高度间歇性和随机分布。因此，高效的运行和控制是困难的。本文提出了一种新颖的无模型能源管理策略，使用混合动作深度强化学习算法来对各种模式下运行的工业设备进行能源调度。具体来说，工业能源管理中心与各设备之间的交互过程被建模为马尔可夫决策过程，使能源系统的日常运行成本最小化，生产设备的收益最大化。然后，开发了一种不需要显式环境模型的双参数化深度 Q 网络，以使用行动者和批评者网络来学习混合动作信号，其中双 Q 值机制避免了价值高估并提高了算法效率。此外，推导了该算法的策略梯度并讨论了其收敛性证明。最后，利用真实数据进行数值研究来评估算法性能并验证其有效性。