Robust control for anaerobic digestion systems of Tequila vinasses under uncertainty: A Deep Deterministic Policy Gradient Algorithm

Digital Chemical Engineering

Luis A. Ricardez-Sandoval¹  Tannia A. Mendiola-Rodriguez² 

[1] Corresponding author.;Department of Chemical Engineering, University of Waterloo, Waterloo, ON, Canada N2L 3G1;

关键词: Anaerobic Digestion;    Reinforcement learning;    Deep Deterministic Policy Gradient;    Process uncertainty;    Tequila vinasses;   

DOI  :  

来源: DOAJ

【 摘 要 】

The disposal of high concentrated Tequila vinasses is an environmental threat. An alternative to solve this problem is through anaerobic digestion processes to reduce organic matter while producing biogas. Anaerobic digestion is a complex system subject to external perturbations and parametric uncertainty; hence, control strategies are needed to guarantee operational efficiency under uncertainty. This study proposes an approach using an actor-critic model algorithm called Deep Deterministic Policy Gradient (DDPG) for a single-stage and a two-stage anaerobic digestion system to manage Tequila vinasses. To explore the feasibility of this algorithm, various scenarios expected during operation are investigated. To provide further insight, the algorithm's performance is compared between the two systems and compared to a conventional robust optimization formulation. In addition, a robust economic model predictive controller (EMPC) using the DDPG framework is tested to further illustrate their benefits. Results showed that the DDPG algorithm learned an optimal policy to minimize the organic matter content of tequila vinasses while producing biogas despite disturbances and parametric uncertainty. Also, the two-stage model exhibited better performance when compared to the single-stage model since significant improvements in methanogenic biomass and accumulated Chemical Oxygen Demand (COD) reduction were observed. While the DPPG algorithm showed an ability to learn optimal control policies under different scenarios, it still requires further improvements to realize their implementation for online large-scale applications.

【 授权许可】

Unknown