Storage System Management Using Reinforcement Learning Techniques and Nonlinear Models

reinforcement learning;nonlinear models;storage management;reservoir management;System Design Engineering

Mahootchi, Masoud

University of Waterloo

关键词: reinforcement learning;    nonlinear models;    storage management;    reservoir management;    System Design Engineering;   

Others  :  https://uwspace.uwaterloo.ca/bitstream/10012/4213/1/phd_thesis_submitted_1.pdf

瑞士|英语

来源: UWSPACE Waterloo Institutional Repository

【 摘 要 】

In this thesis, modeling and optimization in the field of storage management understochastic condition will be investigated using two different methodologies: SimulationOptimization Techniques (SOT), which are usually categorized in the area of ReinforcementLearning (RL), and Nonlinear Modeling Techniques (NMT).For the first set of methods, simulation plays a fundamental role in evaluating the controlpolicy: learning techniques are used to deliver sub-optimal policies at the end of alearning process. These iterative methods use the interaction of agents with the stochasticenvironment through taking actions and observing different states. To converge tothe steady-state condition where policies and value functions do not change significantlywith the continuation of the learning process, all or most important states must be visitedsufficiently. This might be prohibitively time-consuming for large-scale problems.To make these techniques more efficient both in terms of computation time and robustoptimal policies, the idea of Opposition-Based Learning (OBL-Type I and Type II) isemployed to modify/extend popular RL techniques including Q-Learning, Q(λ), sarsa,and sarsa(λ). Several new algorithms are developed using this idea. It is also illustratedthat, function approximation techniques such as neural networks can contribute to theprocess of learning. The state-of-the-art implementations usually consider the maximizationof expected value of accumulated reward. Extending these techniques to considerrisk and solving some well-known control problems are important contributions of thisthesis.Furthermore, the new nonlinear modeling for reservoir management using indicator functionsand randomized policy introduced by Fletcher and Ponnambalam, is extended tostochastic releases in multi-reservoir systems. In this extension, two different approachesfor defining the release policies are proposed. In addition, the main restriction of consideringthe normal distribution for inflow is relaxed by using a beta-equivalent generaldistribution. A five-reservoir case study from India is used to demonstrate the benefitsof these new developments. Using a warehouse management problem as an example,application of the proposed method to other storage management problems is outlined.