A. C. Sankaranarayanan, C. Studer, and R. G. Baraniuk, "CS-MUVI: Video Compressive Sensing for Spatial-Multiplexing Cameras", IEEE International Conference on Computational Photography, Seattle, WA, April, 2012.
Abstract: Compressive sensing (CS)-based spatial-multiplexing cameras (SMCs) sample a scene through a series of coded projections using a spatial light modulator and a few optical sensor elements. SMC architectures are particularly useful when imaging at wavelengths for which full-frame sensors are too cumbersome or expensive. While existing recovery algorithms for SMCs perform well for static images, they typically fail for time-varying scenes (videos). In this paper, we propose a novel CS multi-scale video (CS-MUVI) sensing and recovery framework for SMCs. Our framework features a co-designed video CS sensing matrix and recovery algorithm that provide an efficiently computable low-resolution video preview. We estimate the scene's optical flow from the video preview and feed it into a convex-optimization algorithm to recover the high-resolution video. We demonstrate the performance and capabilities of the CS-MUVI framework for different scenes.
The CS-MUVI concept: The key challenge with sensing videos with cameras such as the single pixel camera (SPC) is that the scene changes with every compressive measurement obtained. Traditional L1-recovery methods fail in the presence of rapid motion. We circumvent this problem by designing special measurement matrices that enable a two-step recovery process; the first step is to estimate motion in the scene and the second step is to recover the scene in full spatial and temporal resolution.