The representation of visual depth perception based on the plenoptic function in the retina and its neural computation in visual cortex V1

BMC Neuroscience

Qiu Jun¹  Sun Shousi¹  Liu Xuemin¹  Liu Chang¹  Zou Qi²  Zhao Songnian³ 

[1] Beijing Information Science and Technology University, Beijing 100101, China;Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China;LAPC, Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing 100029, China

关键词: Affine transformation;    Neural computation;    Primary visual cortex;    Retina;    Vanishing point;    Three-dimensional scene;    Visual perception;    Plenoptic function;   

Others  :  799403
DOI  :  10.1186/1471-2202-15-50

 received in 2013-04-12, accepted in 2014-03-25,  发布年份 2014

【 摘 要 】

Background

How it is possible to “faithfully” represent a three-dimensional stereoscopic scene using Cartesian coordinates on a plane, and how three-dimensional perceptions differ between an actual scene and an image of the same scene are questions that have not yet been explored in depth. They seem like commonplace phenomena, but in fact, they are important and difficult issues for visual information processing, neural computation, physics, psychology, cognitive psychology, and neuroscience.

Results

The results of this study show that the use of plenoptic (or all-optical) functions and their dual plane parameterizations can not only explain the nature of information processing from the retina to the primary visual cortex and, in particular, the characteristics of the visual pathway’s optical system and its affine transformation, but they can also clarify the reason why the vanishing point and line exist in a visual image. In addition, they can better explain the reasons why a three-dimensional Cartesian coordinate system can be introduced into the two-dimensional plane to express a real three-dimensional scene.

Conclusions

1. We introduce two different mathematical expressions of the plenoptic functions, P_w and P_v that can describe the objective world. We also analyze the differences between these two functions when describing visual depth perception, that is, the difference between how these two functions obtain the depth information of an external scene.

2. The main results include a basic method for introducing a three-dimensional Cartesian coordinate system into a two-dimensional plane to express the depth of a scene, its constraints, and algorithmic implementation. In particular, we include a method to separate the plenoptic function and proceed with the corresponding transformation in the retina and visual cortex.

3. We propose that size constancy, the vanishing point, and vanishing line form the basis of visual perception of the outside world, and that the introduction of a three-dimensional Cartesian coordinate system into a two dimensional plane reveals a corresponding mapping between a retinal image and the vanishing point and line.

【 授权许可】

   
2014 Songnian et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

Figure 10.

Figure 11.

Figure 12.

Figure 13.

Figure 14.

【 参考文献 】

• [1]Cumming BG, Parker AJ: Binocular neurons in V1 of awake monkeys are selective for absolute, not relative, disparity. J Neurosci 1999, 19(13):5602-5618.
• [2]Neri P, Bridge H, Heeger DJ: Stereoscopic processing of absolute and relative disparity in human visual cortex. J Neurophysiol 2004, 92(3):1880-1891.
• [3]Ohzawa I, DeAngelis GC, Freeman RD: Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. Science 1990, 249(4972):1037-1041.
• [4]Haefner RM, Cumming BG: Adaptation to natural binocular disparities in primate V1 explained by a generalized energy model. Neuron 2008, 57(1):147-158.
• [5]Marr D: Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. New York: Freeman; 1982.
• [6]Julesz B: Binocular depth perception of computer generated patterns. Bell Syst Tech J 1960, 39:1125-1162.
• [7]Julesz B: Stereoscopic vision. Vis Res 1986, 26:1601-1612.
• [8]Crick F: The Astonishing Hypothesis: The Scientific Search for the Soul. New York: Touchstone Rockefeller Center; 1995:23-71.
• [9]Frisby JP, Stone JV: Seeing, The Computational Approach to Biological Vision. 2nd edition. England: The MIT Press London; 2010:539-551.
• [10]Adelson E, Bergen J: The Plenoptic Function and the Elements of Early Vision. In Computational Models of Visual Processing. Cambridge, MA: The MIT Press; 1991:385-394.
• [11]Adelson EH, Wang John YA: Single lens stereo with a plenoptic camera. IEEE Trans Pattern Anal Mach Intell 1992, 14(2):99-106.
• [12]Ren N, Levoy M, Bredif M, Duval G, Horowitz M, Hanrahan P: Light Field Photography With a Handheld Plenoptic Camera. California: Stanford University Computer Science Tech Report CSTR; 2005.
• [13]Levoy M, Ren N, Andrew A, Footer M, Horowitz M: Light field microscopy. ACM Trans Graphics (TOG) 2006, 25(3):924-934.
• [14]McMillan L, Bishop G: Plenoptic modeling: An image-based rendering system. In Computer Graphics of Proceedings ACM SIGGRAPH'95. Los Angeles: SIGGRAPH Press; 1995:899-903.
• [15]Wenger A, Gardner A, Tchou C, Unger J, Hawkins T, Debevec P: Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Trans Graph 2005, 24(3):756-764.
• [16]Schreer O, Kauff P, Sikjora T: 3D videocommunication: Algorithms, concepts and real-time systems in human centred communication. New York: John &Sons, Inc; 2005:110-150.
• [17]Yan Z: A course on techniques of photography. Seven edition. Shanghai: Fudan university press; 2013.
• [18]Sonka M, Havac V, Boyle R: Image Processing, Analysis, and Machine Vision. Second edition. New Jersey: Thomson Learning and PT Press; 1999:310-321.
• [19]Mallot HA: Computational Vision: Information Processing in Perception and Visual Behavior. Cambridge, London, England: The MIT Press; 2000:23-46.
• [20]Koenderink JJ, van Doorn AJ: Representation of local geometry in the visual system. Biol Cybern 1987, 55(6):367-375.
• [21]Faugeras O: Three-Dimensional Computer Vision: A Geometric Viewpoint. Cambridge, London, England: The MIT Press; 1993.
• [22]Faugeras O, Luong QT, Papadopoulo T: The Geometry of Multiple Images: The Laws That Govern the Formation of Multiple Images of a Scene Andsome of Their Applications. Cambridge, London, England: MIT press; 2001.
• [23]Schwartz SH: Geometrical and Visual Optics. New York: McGraw-Hill Medical; 2013.
• [24]Rybak IA, Gusakova VI, Golovan AV, Podladchikova LN, Shevtsova NA: A model of attention-guided visual perception and recognition. Vis Res 1998, 38:2387-2400.
• [25]Chua HF, Boland JE, Nisbett RE: Cultural variation in eye movements during scene perception. PNAS 2005, 102(35):12629-12633.
• [26]Zou Q, Zhao S, Wang Z, Huang Y: A neural computational model for bottom-up attention with invariant and overcomplete representation. BMC Neurosci 2012, 13:145. BioMed Central Full Text
• [27]Zeki S: A Vision of the Brain. Oxford: Blackwell Scientific Pub; 1993.
• [28]Livingstone MS, Hubel DH: Anatomy and physiology of a color system in the primate visual cortex. J Neurosci 1984, 4:309-356.
• [29]Livingstone MS, Hubel DH: Psychophysical evidence for separate channels for the perception of form, color, movement, and depth. J Neurosci 1987, x7:3416-3468.
• [30]Hubel DH, Wiesel TN: Ferrier lecture, functional architecture of macaque monkey visual cortex. Proc R Soc Lond Biol Sci 1977, 198:1-59.
• [31]Hubel DH: Exploration of the primary visual cortex: 1955–1978. Nature 1982, 299:515-524.
• [32]Nicholls JG, Martin AR, Wallace BG, Fuchs PA: From Neuron to Brain. Four edition. Massachusetts: Sinauer Associates, Inc; 2001.
• [33]Regan D: Human Perception of Objects. Sunderland, Mass: Sinauer Associates, Inc; 2000:116-120.
• [34]Rock I: The Logic of Perception. Cambridge, MA: MIT Press; 1983.
• [35]Hershenson M: Visual Space Perception, a Primer. Cambridge, MA: The MIT Press; 2000:78-91.
• [36]Aimin W, De X, Wang H, Wu J: Objects size constancy computation based on visual psychology. Acta Electronica Sin 2006, 34(6):1096-1103.
• [37]Wu A: Application of Visual Psychology in Computer Vision. Beijing: Press of Beijing Jiaotong University; 2006:105-106.
• [38]Shufelt JA: Performance evaluation and analysis of vanishing point detection techniques. IEEE Trans Pattern Anal Mach Intell 2002, 21(3):282-288.
• [39]Almansa A, Desolneux A, Vamech S: Vanishing point detection without any a priori information. IEEE Trans Pattern Anal Mach Intell 2003, 25(4):502-507.
• [40]Kalantari M, Jung F, Guedon J: Precise: Automatic and fast method for vanishing point detection. Photogramm Rec 2009, 24(127):246-263.
• [41]Schaffalitzky F, Zisserman A: Planar grouping for automatic detection of vanishing lines and points. Image Vis Comput 2000, 18(9):647-658.
• [42]Tardif JP: Non-Iterative Approach for Fast and Accurate Vanishing Point Detection. In Proceedings of the 12th IEEE International Conference on Computer Vision. Kyoto, Japan: IEEE; 2009:1250-1257.
• [43]Palmer SE: Vision Science. London, Cambridge, Mass: MIT Press; 1999:120-280.
• [44]Potter MC, Wyble B, Hagmann CE, McCourt ES: Detecting meaning in RSVP at 13 ms per picture. Atten Percept Psychophys 2013, 12:1-10.
• [45]Murray SO, Boyaci H, Kersten D: The representation of perceived angular size in human primary visual cortex. Nat Neurosci 2006, 9:429-434.
• [46]Songnian Z, Qi Z, Zhen J, Guozheng Y, Li Y: Neural computation of visual imaging based on Kronecker product in the primary visual cortex. BMC Neurosci 2010, 11(43):1-14.
• [47]SongNian ZHAO, Li YAO, Zhen JIN, XiaoYun XIONG, Xia WU, Qi ZOU, GuoZheng YAO, XiaoHong CAI, YiJun LIU: Sparse representation of global features of visual images in human primary visual cortex in human primary visual cortex: Evidence from fMRI. Chin Sci Bull 2008, 14(53):2165-2174.
• [48]Songnian ZHAO, Qi ZOU, Zhen JIN, GuoZheng YAO, Li YAO: A computational model of early vision based on synchronized response and inner product operation. Neurocomputing 2010, 73:3229-3241.
• [49]Zhao S, Xiong X, Yao G, Fu Z: A computational model as neurodecoder based on synchronous oscillation in the visual cortex. Neural Comput 2003, 15:2399-2418.
• [50]Zhao S, Zou Q, Jin Z, Xiong X, Yao G, Yao L, Liu Y: A Computational Model that Realizes a Sparse Representation of the Primary Visual Cortex V1. In Software Engineering 2009, WCSE '09. WRI World Congress on Software Engineering, IEEE Computer Society, Xplore. Los Alamitos, CA: Publications office of the IEEE Computer Society; 2009:54-62. Issue Date: 19–21 May
• [51]Jackson AJ, Bailey IL: Visual acuity. Optom Pract 2004, 5:53-70.