Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator

Heating, Ventilation, and Air Conditioning (HVAC) systems are among the largest contributors to global building energy consumption, making their optimization vital for sustainability and cost reduction. While reinforcement learning (RL) has emerged as a powerful tool for adaptive HVAC control, its practical application is hindered by the slow training process of physics-based simulators like EnergyPlus, which require computationally expensive thermodynamic calculations at every step.

This study introduces an LSTM-based surrogate simulator that replicates EnergyPlus dynamics to accelerate RL training. By training on a dataset of simulated building conditions—including indoor temperature, humidity, CO₂ concentration, and energy consumption—the surrogate model efficiently approximates system behavior. Unlike EnergyPlus, which operates through CPU-intensive physical simulations, the LSTM model leverages GPU acceleration and generates predictions in real time, cutting execution time by more than 50% while preserving accuracy.

Experimental results show that the LSTM surrogate achieves a 57.6% reduction in training time per epoch compared to EnergyPlus, with only a marginal loss in prediction accuracy. This trade-off makes RL-based HVAC optimization significantly more scalable and practical for real-world applications. Moreover, the framework supports future integration into deep reinforcement learning (DRL) environments, enabling faster policy training and evaluation for smart building climate control.

This work highlights a scalable, data-driven, and computationally efficient approach to building energy management, paving the way for intelligent and sustainable HVAC systems in IoT-enabled smart cities. Future directions include expanding the action space, incorporating richer datasets, and exploring advanced model architectures for further improvements in generalization and accuracy.

For more information, please click here!