Active Sensing for Partially Observable Markov Decision Processes

Active Sensing;Smart Sensor Selection;POMDP;Markov Decision Process;Probability;Utility;Sensor;Networks;Computer Science

Koltunova, Veronika

University of Waterloo

关键词: Active Sensing;    Smart Sensor Selection;    POMDP;    Markov Decision Process;    Probability;    Utility;    Sensor;    Networks;    Computer Science;   

Others  :  https://uwspace.uwaterloo.ca/bitstream/10012/7222/1/Koltunova_Veronika.pdf

瑞士|英语

来源: UWSPACE Waterloo Institutional Repository

【 摘 要 】

Context information on a smart phone can be used to tailor applications for specific situations (e.g. provide tailored routing advice based on location, gas prices and traffic). However, typical context-aware smart phone applications use very limited context information such as user identity, location and time. In the future, smart phones will need to decide from a wide range of sensors to gather information from in order to best accommodate user needs and preferences in a given context. In this thesis, we present a model for active sensor selection within decision-making processes, in which observational features are selected based on longer-term impact on the decisions made by the smart phone. This thesis formulates the problem as a partially observable Markov decision process (POMDP), and proposes a non-myopic solution to the problem using a state of the art approximate planning algorithm Symbolic Perseus. We have tested our method on a 3 small example domains, comparing different policy types, discount factors and cost settings. The experimental results proved that the proposed approach delivers a better policy in the situation of costly sensors, while at the same time provides the advantage of faster policy computation with less memory usage.