Optimal risk probability for first passage models in semi-Markov decision processes

JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS

卷:359

Article

Huang, Yonghui¹  Guo, Xianping¹ 

[1] Sun Yat Sen Univ, Sch Math & Computat Sci, Guangzhou 510275, Guangdong, Peoples R China

关键词: Semi-Markov decision processes;    Target set;    First passage time;    Risk probability;    Optimal policy;   

DOI  :  10.1016/j.jmaa.2009.05.058

来源: Elsevier

【 摘 要 】

This paper studies the risk minimization problem in semi-Markov decision processes with denumerable states. The criterion to be optimized is the risk probability (or risk function) that a first passage time to some target set doesn't exceed a threshold value. We first characterize such risk functions and the corresponding optimal value function, and prove that the optimal value function satisfies the optimality equation by using a successive approximation technique. Then, we present some properties of optimal policies, and further give conditions for the existence of optimal policies. In addition, a value iteration algorithm and a policy improvement method for obtaining respectively the optimal value function and optimal policies are developed. Finally, two examples are given to illustrate the value iteration procedure and essential characterization of the risk function. Crown Copyright (C) 2009 Published by Elsevier Inc. All rights reserved.

【 授权许可】

Free