Towards reinforcement learning for vulnerability analysis in power-economic systems

Energy Informatics

Research

Eric MSP Veith¹  Thomas Wolgast²  Astrid Nieße² 

[1] OFFIS – Institute for Information Technology, Escherweg 2, Oldenburg, Germany;University of Oldenburg, Ammerländer Heerstraße 114-118, Oldenburg, Germany;OFFIS – Institute for Information Technology, Escherweg 2, Oldenburg, Germany;

关键词: Ancillary service markets;    Systemic weaknesses;    Reinforcement learning;    Economic attack;    Artificial intelligence;    Machine learning;   

DOI  :  10.1186/s42162-021-00181-5

来源: Springer

【 摘 要 】

Future smart grids can and will be subject of systematic attacks that can result in monetary costs and reduced system stability. These attacks are not necessarily malicious, but can be economically motivated as well. Emerging flexibility markets are of interest here, because they can incite attacks if market design is flawed. The dimension and danger potential of such strategies is still unknown. Automatic analysis tools are required to systematically search for unknown strategies and their respective countermeasures. We propose deep reinforcement learning to learn attack strategies autonomously to identify underlying systemic vulnerabilities this way. As a proof-of-concept, we apply our approach to a reactive power market setting in a distribution grid. In the case study, the attacker learned to exploit the reactive power market by using controllable loads. That was done by systematically inducing constraint violations into the system and then providing countermeasures on the flexibility market to generate profit, thus finding a hitherto unknown attack strategy. As a weak-point, we identified the optimal power flow that was used for market clearing. Our general approach is applicable to detect unknown attack vectors, to analyze a specific power system regarding vulnerabilities, and to systematically evaluate potential countermeasures.

【 授权许可】

CC BY   
© The Author(s) 2021. corrected publication 2023

【 图 表 】

41116_2023_36_Article_IEq347.gif

41116_2023_36_Article_IEq346.gif

41116_2023_36_Article_IEq344.gif

Fig. 5

41116_2023_36_Article_IEq341.gif

41116_2023_36_Article_IEq340.gif