Decision-Theoretic Planning under Risk-Sensitive Planning Objectives

Risk attitudes;Utility theory;Markov decision processes;Risk-sensitive planning;Decision-theoretic planning

Liu, Yaxin ; Computing

University:Georgia Institute of Technology

Department:Computing

关键词: Risk attitudes;    Utility theory;    Markov decision processes;    Risk-sensitive planning;    Decision-theoretic planning;   

Others  :  https://smartech.gatech.edu/bitstream/1853/6959/1/liu_yaxin_200505_phd.pdf

美国|英语

来源: SMARTech Repository

【 摘 要 】

Risk attitudes are important for human decision making, especially in scenarios where huge wins or losses are possible, as exemplified by planetary rover navigation, oilspill response, and business applications.Decision-theoretic planners therefore need to take risk aspects into account to serve their users better.However, most existing decision-theoretic planners use simplistic planning objectives that are risk-neutral.The thesis research is the first comprehensive study of how to incorporate risk attitudes into decision-theoretic planners and solve large-scale planning problems represented as Markov decision process models.The thesis consists of three parts.The first part of the thesis work studies risk-sensitive planning in case where exponential utility functions are used to model risk attitudes.I show that existing decision-theoretic planners can be transformed to take risk attitudes into account.Moreover, different versions of the transformation are needed if the transition probabilities are implicitly given, namely, temporally extended probabilities and probabilities given in a factored form. The second part of the thesis work studies risk-sensitive planning in case where general nonlinear utility functions are used to model risk attitudes.I show that a state-augmentation approach can be used to reduce a risk-sensitive planning problem to a risk-neutral planning problem with an augmented state space.I further use a functional interpretation of value functions and approximation methods to solve the planning problems efficiently with value iteration.I also show an exact method for solving risk-sensitive planning problems where one-switch utility functions are used to model risk attitudes.The third part of the thesis work studies risk sensitive planning in case where arbitrary rewards are used.I propose a spectrum of conditions that can be used to constrain the utility function and the planning problem so that the optimal expected utilities exist and are finite.I prove that the existence and finiteness properties hold for stationary plans, where the action to perform in each state does not change over time, under different sets of conditions.