Analysis of preprocessing and rendering light field videos

Abstract

Since light field cameras became available on the consumer market, its versatile functions are more and more known to computer vision communities and photographers. After all, refocusing an image after it was taken is one of the features most wished for in photography. The possibility of refocusing a video after being taken extends this feature to an additional dimension, enabling direct user interaction in videos, which could result in a new form of entertainment and user experience.

This work presents different approaches of preprocessing and rendering light field videos made with a Lytro Illum, a plenoptic camera using microlenses. We compare three differently preprocessed light field videos and reveal the main issues for rendering on consumer hardware. Our results show, that one of the biggest obstacles lies in the transfer of data from light field video files to the main memory of the system. Therefore, the demosaiced and color corrected representation that balances preprocessing and resulting video file size is fastest in rendering light field video files. Rendering a short light field video directly from the main memory allows interaction like refocusing in real time and points out the need for future research on new codecs for light field videos based on non-raw light field images.