Numerical Demultiplexing of Color Image Sensor Measurements via Non-linear Random Forest Modeling

multispectral;random forest;inverse problem;demultiplexing;imaging;microscopy;spectral;photoplethysmographic imaging;camera

Deglint, Jason

University of Waterloo

关键词: multispectral;    random forest;    inverse problem;    demultiplexing;    imaging;    microscopy;    spectral;    photoplethysmographic imaging;    camera;   

Others  :  https://uwspace.uwaterloo.ca/bitstream/10012/10560/3/Deglint_Jason.pdf

瑞士|英语

来源: UWSPACE Waterloo Institutional Repository

【 摘 要 】

Due to recent advancements in technology, consumer digital cameras are becoming cheaper and easier to use.These consumer digital cameras, with Bayer color filter arrays (CFAs), allow for simultaneous capture of the red, green and blue (RGB) channels.To achieve higher spectral resolution, multispectral imaging systems use methods such as filter wheels and tunable filters to capture data in a sequential manner.However, in order to capture transient phenomena, one would need to capture spectral information of a 2D scene in a simultaneous manner.Therefore, there has been an on-going trend towards creating a simultaneous multispectral imaging system that uses a conventional consumer digital camera with a Bayer CFA.Such a system allows for a effective imaging of transient or dynamic phenomena with a low-cost and compact system.Currently, the main method to accomplish this is known as Wiener estimation which uses statistical assumptions of the relationship between the incoming spectra and the RGB measurements. However, these assumptions limit the ability to accurately predict the incoming spectra.Therefore, we leverage a comprehensive framework based on numerical demultiplexing of sensor measurements via spectral characterization of the image sensor CFA and non-linear random forest modeling.To create this numerical demultiplexing system we create a forward model from the spectral sensitivity of the imaging system, which is accomplished with a monochrometer.This forward model is then used to create a mapping of 10,000 randomly generated spectra to their corresponding RGB values.This mapping acts as our training set for our non-linear inverse model which utilizes the random forest modeling framework.Having constructed the numerical demultiplexer, we test the performance against the state-of-the-art Wiener estimation for both quantitative and qualitative experiments.In the first set of experiments, we performed a quantitative performance assessment of the proposed framework within a controlled simulation environment.The second set of experiments, validated the observations made from thefirst set of controlled simulation experiments within a real-world setting. More specifically, we used an icon with different colors as well as a scene of different color flowers to perform quantitative analysis.In these experiments, we show that the proposed numerical demultiplexer outperforms the state-of-the art and is a more robust and reliable way to infer higher spectra from RGB measurements.Having validated the numerical demultiplexer, we use it for two applications which are photoplethysmogrpahic imaging and multispectral microscopy.For photoplethysmogrpahic imaging we found that decomposing the RGB camera measurements into narrow-band spectral information can noticeably improve the prediction of heart rate estimation.In addition, we used the numerical demultiplexer forboth a bright-field multispectral microscope as well as a dark-field fluorescence multispectral microscope, which illustrates its potential as a low-cost, portable, point-of-care system.