High performance imaging using arrays of inexpensive cameras

Digital cameras are becoming increasingly cheap and ubiquitous, leading researchers to exploit multiple cameras and plentiful processing to create richer and more accurate representations of real settings. This thesis addresses issues of scale in large camera arrays. I present a scalable architecture that continuously streams color video from over 100 inexpensive cameras to disk using four PCs, creating a one gigasample-per-second photometer. It extends prior work in camera arrays by providing as much control over those samples as possible. For example, this system not only ensures that the cameras are frequency-locked, but also allows arbitrary, constant temporal phase shifts between cameras, allowing the application to control the temporal sampling. The flexible mounting system also supports many different configurations, from tightly packed to widely spaced cameras, so applications can specify camera placement. Even greater flexibility is provided by processing power at each camera, including an MPEG2 encoder for video compression, and FPGAs and embedded microcontrollers to perform low-level image processing for real-time applications. I present three novel applications for the camera array that highlight strengths of the architecture and the advantages and feasibility of working with many inexpensive cameras: synthetic aperture videography, high speed videography, and spatiotemporal view interpolation. Synthetic aperture videography uses numerous moderately spaced cameras to emulate a single large-aperture one. Such a camera can see through partially occluding objects like foliage or crowds. I show the first synthetic aperture images and videos of dynamic events, including live video accelerated by image warps performed at each camera. High-speed videography uses densely packed cameras with staggered trigger times to increase the effective frame rate of the system. I show how to compensate for artifacts induced by the electronic rolling shutter commonly used in inexpensive CMOS image sensors and present results streaming 1560 fps video using 52 cameras. Spatiotemporal view interpolation processes images from multiple video cameras to synthesize new views from times and positions not in the captured data. We simultaneously extend imaging performance along two axes by properly staggering the trigger times of many moderately spaced cameras, enabling a novel multiple-camera optical flow variant for spatiotemporal view interpolation.* *This dissertation is a compound document (contains both a paper copy and a CD as part of the dissertation). The CD requires the following system requirements: Windows MediaPlayer or RealPlayer.