Adaptive Duty Cycle Scheduling for Battery-Powered IoT Sensors using Reinforcement Learning: A Simulation Study

Abstract

Battery-powered IoT sensors face a fundamental dilemma between energy conservation and QoS responsiveness. Static sleep-wake scheduling (fixed duty cycle) saves energy but wastes active cycles during low traffic, while simple threshold-based scheduling is vulnerable to oscillation under dynamic loads. This study proposes a reinforcement learning framework based on tabular Q-learning for adaptive duty cycle decision-making that balances battery life against packet latency. The environment is simulated as a Markov Decision Process with a three-dimensional state space (battery level, queue load, traffic class) and four discrete sleep interval actions. The agent is trained on a synthetic Poisson traffic generator over 50,000 episodes with epsilon-greedy exploration and evaluated on five test scenarios including traffic surges and low-battery conditions. Simulation results show that the RL agent extends battery lifetime by 28.4% over fixed scheduling and 14.7% over threshold-based scheduling, with average latency remaining below the 2-second QoS limit. This study contributes in three aspects: a compact MDP formulation suitable for TelosB/CC2650-class microcontrollers, demonstration of tabular Q-learning convergence without function approximation, and sensitivity evaluation under traffic variation. Main limitations include offline training and simulation-only validation, with planned next steps covering tile coding adaptation, solar harvest integration, and physical testbed validation.