Deep Light Field Acquisition Using Learned CodedMask Distributions for Color Filter Array Sensors,
IEEE Trans. on computational imaging, vol. 7, pp. 475-488, 2021, May 2021.
Compressive light field photography enables light field acquisition using a single sensor by utilizing a color coded mask. This approach is very cost effective since consumer-level digital cameras can be turned into a light field camera by simply placing a coded mask between the sensor and the aperture plane and solving an inverse problem to obtain an estimate of the original light field. This paper describes a deep learning architecture for compressive light field acquisition using a color coded mask and a sensor with Color Filter Array (CFA). Unlike previous methods where a fixed mask pattern is used, our deep network learns the optimal distribution of the color coded mask pixels. The proposed solution enables end-to-end learning of the color-coded mask distribution, the CFA, and the reconstruction network.
Consequently, the resulting network can efficiently perform joint demosaicing and light field reconstruction of images acquired with color-coded mask and a CFA sensor. Compared to previous methods based on deep learning with monochrome sensors, as well as traditional compressive sensing approaches using CFA sensors, we obtain superior color reconstruction of the light fields.
Light fields from Wanner, S. Meister, and B. Goldluecke, Datasets and benchmarks for densely sampled 4D light fields,
Conf. on Vision, Modeling &Visualization (VMV), 2013, pp. 225–226.
Note that these light fields come from a very different distribution than the one used for training.