In this dissertation, I report the progress towards building a robust and efficient 3Dreconstruction system based on stereo vision. Stereo vision is known to be quitefragile in practice due to specular highlights, lack of texture, lighting variations,image blurring, etc. In this dissertation, I focus on exploiting the relationshipsbetween illuminants, surface reflection and shape to increase the robustness ofstereo vision. I first present a new image transform for matching low-textured regionsand then a robust solution for illumination chromaticity estimation based ona new correspondence matching invariant called Illumination Chromaticity Constancy.I next propose a new framework based on bilateral filtering and loopybelief propagation for simultaneous estimation of surface reflectance and shapewith the assumption that the illumination chromaticity can be correctly estimated.Two new bilateral filtering algorithms with computational complexity invariantto filter kernel size and a new belief propagation with computational complexityinvariant to the disparity search range are then presented to reduce the speed andmemory cost.