Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures

JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS

卷:443

Article

Anselmi, Jonatha¹  Dufour, Francois^2,3,4  Prieto-Rumeau, Tomas⁵ 

[1] INRIA Bordeaux Sud Ouest, Talence, France

[2] Univ Bordeaux, Inst Polytech Bordeaux, Bordeaux, France

[3] Univ Bordeaux, INRIA Bordeaux Sad Ouest, CQFD, Bordeaux, France

[4] Univ Bordeaux, IMB, Inst Math Bordeaux, Bordeaux, France

[5] UNED, Madrid, Spain

关键词: Continuous-time Markov decision processes;    Piecewise Lipschitz continuous control models;    Approximation of the optimal value function;    epsilon-optimal policy;   

DOI  :  10.1016/j.jmaa.2016.05.055

来源: Elsevier

【 摘 要 】

In this paper, we propose an approach for approximating the value function and an e-optimal policy of continuous-time Markov decision processes with Borel state and action spaces, with possibly unbounded cost and transition rates, under the total expected discounted cost optimality criterion. Under adequate assumptions, which in particular include that the transition rate has a density function with respect to a reference measure, together with piecewise Lipschitz continuity of the elements of the control model, we approximate the original controlled process by a model with finite state and action spaces. The approximation error is related to the 1-Wasserstein distance between suitably defined probability measures and approximating measures with finite support. We also study the case when the reference measure is approximated with empirical distributions and we show that convergence of the approximations takes place at an exponential rate in probability. (C) 2016 Elsevier Inc. All rights reserved.

【 授权许可】

Free