Conference2025IEEE

Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator

Mehmet Burak Hekimoğlu, Sajjad Baghaee, Ulaş Filiz, Ilkay Ulusoy, Klaus Werner Schmidt

2025 33rd Signal Processing and Communications Applications Conference (SIU)

Official publicationDOI

Abstract

Reinforcement learning (RL) has shown great potential in optimizing the operation of HVAC systems, improving energy efficiency, and enhancing user comfort. However, the slow input/output operations associated with simulation tools like EnergyPlus significantly hinder the training process. This paper proposes a novel approach to accelerate RL training by using a data-driven LSTM model to replicate the behavior of a building energy simulator. By training the LSTM model on a set of observations and actions, the model learns to approximate the simulator's dynamics, providing a faster and more efficient training environment for RL agents. We demonstrate that using the LSTM-based surrogate simulator leads to substantial reductions in computational time while maintaining the accuracy of the system's behavior.

Selected figures

Figure 1 from “Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator”: Normal and Proposed System Diagram
Fig. 1 · Normal and Proposed System Diagram.
Reproduced from: M. B. Hekimoğlu et al., “Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator,” 2025 33rd Signal Processing and Communications Applications Conference (SIU), 2025. View the original publication.
Figure 2 from “Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator”: Temperature Prediction vs. Ground Truth
Fig. 2 · Temperature Prediction vs. Ground Truth.
Reproduced from: M. B. Hekimoğlu et al., “Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator,” 2025 33rd Signal Processing and Communications Applications Conference (SIU), 2025. View the original publication.

Cite this work

Plain text

M. B. Hekimoğlu, S. Baghaee, U. Filiz, I. Ulusoy and K. W. Schmidt, "Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator," in 2025 33rd Signal Processing and Communications Applications Conference (SIU), pp. 1-4, 2025. doi: 10.1109/SIU66497.2025.11112292.

BibTeX 
@inproceedings{hekimolu2025accelerating,
  author    = {Hekimoğlu, Mehmet Burak and Baghaee, Sajjad and Filiz, Ulaş and Ulusoy, Ilkay and Schmidt, Klaus Werner},
  title     = {Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator},
  booktitle = {2025 33rd Signal Processing and Communications Applications Conference (SIU)},
  pages     = {1-4},
  publisher = {IEEE},
  year      = {2025},
  doi       = {10.1109/SIU66497.2025.11112292},
  url       = {https://ieeexplore.ieee.org/abstract/document/11112292}
}

All publications