Reliable Fusion of Stereo Matching and Depth Sensor for High Quality Dense Depth Maps

Sensors

Jing Liu¹  Chunpeng Li¹  Xuefeng Fan¹  Zhaoqi Wang¹ 

[1] The Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences, No.6 Kexueyuan South Road Zhongguancun, Haidian District, Beijing 100190, China; E-Mails:

关键词: stereo matching;    depth sensor;    multiscale pseudo-two-layer model;    segmentation;    texture constraint;    fusion move;   

DOI  :  10.3390/s150820894

来源: mdpi

【 摘 要 】

Depth estimation is a classical problem in computer vision, which typically relies on either a depth sensor or stereo matching alone. The depth sensor provides real-time estimates in repetitive and textureless regions where stereo matching is not effective. However, stereo matching can obtain more accurate results in rich texture regions and object boundaries where the depth sensor often fails. We fuse stereo matching and the depth sensor using their complementary characteristics to improve the depth estimation. Here, texture information is incorporated as a constraint to restrict the pixel’s scope of potential disparities and to reduce noise in repetitive and textureless regions. Furthermore, a novel pseudo-two-layer model is used to represent the relationship between disparities in different pixels and segments. It is more robust to luminance variation by treating information obtained from a depth sensor as prior knowledge. Segmentation is viewed as a soft constraint to reduce ambiguities caused by under- or over-segmentation. Compared to the average error rate 3.27% of the previous state-of-the-art methods, our method provides an average error rate of 2.61% on the Middlebury datasets, which shows that our method performs almost 20% better than other “fused” algorithms in the aspect of precision.

【 授权许可】

CC BY   
© 2015 by the authors; licensee MDPI, Basel, Switzerland.