Learning Agent for a Heat-Pump Thermostat with a Set-Back Strategy Using Model-Free Reinforcement Learning

Energies

Frederik Ruelens²  Sandro Iacovella²  Bert J. Claessens¹  Ronnie Belmans² 

[1] EnergyVille, Thor park 8300, Genk 3600, Belgium; E-Mail:;Division ELECTA, Department of Electrical Engineering, Faculty of Engineering, KU Leuven, Kasteelpark Arenberg 10, Box 2445, Leuven 3001, Belgium; E-Mails:

关键词: auto-encoder;    batch reinforcement learning;    heat pump;    set-back thermostat;   

DOI  :  10.3390/en8088300

来源: mdpi

【 摘 要 】

The conventional control paradigm for a heat pump with a less efficient auxiliary heating element is to keep its temperature set point constant during the day. This constant temperature set point ensures that the heat pump operates in its more efficient heat-pump mode and minimizes the risk of activating the less efficient auxiliary heating element. As an alternative to a constant set-point strategy, this paper proposes a learning agent for a thermostat with a set-back strategy. This set-back strategy relaxes the set-point temperature during convenient moments, e.g., when the occupants are not at home. Finding an optimal set-back strategy requires solving a sequential decision-making process under uncertainty, which presents two challenges. The first challenge is that for most residential buildings, a description of the thermal characteristics of the building is unavailable and challenging to obtain. The second challenge is that the relevant information on the state, i.e., the building envelope, cannot be measured by the learning agent. In order to overcome these two challenges, our paper proposes an auto-encoder coupled with a batch reinforcement learning technique. The proposed approach is validated for two building types with different thermal characteristics for heating in the winter and cooling in the summer. The simulation results indicate that the proposed learning agent can reduce the energy consumption by 4%–9% during 100 winter days and by 9%–11% during 80 summer days compared to the conventional constant set-point strategy.

【 授权许可】

CC BY   
© 2015 by the authors; licensee MDPI, Basel, Switzerland.