Due to the exponential growth of endpoints in the Internet of Things (IoT), new protocols have been proposed to utilize cellular infrastructures, allowing a large amount of IoT devices to communicate through them. These novel protocols make up the Cellular IoT (C-IoT). In C-IoT, the energy efficiency of endpoints is essential in order to reduce both operational cost and required maintenance. One method of energy reduction is discontinuous reception (DRX). DRX allows a device’s radio frequency (RF) circuitry to turn off for brief periods of time. While off, the device experiences a tradeoff between saving energy and an increase in expected latency, which can be tuned by how long the device spends asleep. In this article, we model DRX as a Markov decision process (MDP). This MDP is solved using a low-complexity “DRX-aware” value iteration algorithm, then verified through simulation and analytical analysis. Further, the energy-latency tradeoff is explored by varying the device’s priority on either energy or latency in addition to varying the traffic intensity. Finally, a method of traffic estimation is applied, and the model’s performance in an environment with time-varying traffic intensity is explored. This approach is compared with a reinforcement learning approach, showing that the traffic estimation approach is better suited to the problem of DRX optimization.

中文翻译：

---

一种基于 MDP 的全面方法，用于建模和优化蜂窝物联网网络中的非连续接收 （DRX）


由于物联网 （IoT） 中端点的指数级增长，已经提出了利用蜂窝基础设施的新协议，允许大量物联网设备通过它们进行通信。这些新颖的协议构成了蜂窝物联网 （C-IoT）。在 C-IoT 中，端点的能源效率对于降低运营成本和所需的维护至关重要。一种减少能量的方法是不连续接收 （DRX）。DRX 允许设备的射频 （RF） 电路短暂关闭。关闭时，设备会在节省能源和增加预期延迟之间进行权衡，这可以通过设备休眠的时间来调整。在本文中，我们将 DRX 建模为马尔可夫决策过程 （MDP）。该 MDP 使用低复杂度的“DRX 感知”值迭代算法进行求解，然后通过仿真和分析进行验证。此外，除了改变流量强度之外，还可以通过改变设备对能量或延迟的优先级来探索能量-延迟的权衡。最后，应用了一种流量估计方法，并探讨了模型在具有时变流量强度的环境中的性能。将这种方法与强化学习方法进行了比较，表明流量估计方法更适合 DRX 优化问题。