Indoor Scene Knowledge Acquisition using a NaturalLanguage Interface

Workshop on Spatial Knowledge Acquisition with Limited Information Displays 2012

Saranya Kesavan ; Nicholas A. Giudice

Others  :  http://ceur-ws.org/Vol-888/SKALID2012_Kesavan.pdf
PID  :  43375

来源: CEUR

【 摘 要 】

This paper proposes an interface that uses automatically-generated Natural Language (NL) descriptions to describe indoor scenes based on photostaken of that scene from smartphones or other portable camera-equipped mobiledevices. The goal is to develop a non-visual interface based on spatio-linguisticdescriptions which could assist blind people in knowing the contents of an in- door scene (e.g., room structure, furniture, landmarks, etc.) and supporting effi- cient navigation of this space based on these descriptions. In this paper, weconcentrate on understanding the most salient content of a stereotypic indoorscene that is described by an observer, categorizing the description strategiesemployed in this process, and evaluating the best presentation of directional in- formation using NL descriptions in order to support the most accurate spatialbehaviors and mental representations of these scenes by means of human be- havioral experiments. This knowledge will then be used to develop a domainspecific indoor scene ontology, which in turn will be used to generate auto- mated NL descriptions of indoor scenes based on their photographs, which willfinally be integrated into a real-time non-visual scene description system.