Local shape from texture

Human observers can judge the 3D shape and orientation of a surface from a number of different cues such as motion, binocular stereopsis, and texture. All of these cues are based on the information in two or more perspective views of the same surface in the scene. In structure from motion, the relative motion of the observer and the surface generates different views of the surface. In stereopsis, two eyes or cameras give slightly different views of the surface. One can think of shape from in this framework as well. Consider two textured patches of a surface in the scene. Even if the patches have the same pattern, in an image they will appear slightly different because of the slightly different orientation that they have with respect to the observer's eye or camera. Thus we effectively get multiple views in a single, monocular image. This framework suggests that we should treat the shape from problem as a two stage problem, as one would treat stereopsis or structure from motion: (1) Estimate the texture from the image, and (2) Interpret the texture to infer the shape and orientation of the surface. Here, measuring the texture corresponds to finding the binocular disparity in stereopsis, or computing the optical flow in structure from motion. We assume that the has stationary second-order statistics on the surface in the scene. This assumption suggests that one measure the deviation from stationarity in the image, i.e., by estimating the local power spectrum and measuring its distortion from one part of the image to another. We model the distortion locally as an affine transformation between neighboring image patches. We demonstrate two related methods for measuring the local distortion. In the first of these methods, we use a differential method to find the affine transforms explicitly. The differential method bears strong resemblances to differential techniques for finding optical flow. In the second method, we estimate only the second moments of the in the image. In this method we implicitly model the distortion as a set of affine transforms, but we do not explicitly solve for the affine transforms. This latter method is intended for use on more irregular textures. We have derived the relationship between the parameters of these affine transformations and all five local shape and orientation parameters. We use non-linear minimization of a least squares error criterion to estimate the shape and orientation parameters from the distortion, using a simple linear algorithm to obtain an initial guess. Under the assumption that the measurement errors in the affine parameters are independent and normally distributed, we find error bounds on the shape and orientation parameter estimates. This dissertation presents experimental results of images of planar and curved surfaces under perspective projection. We test the method based on our first measure of distortion on fairly regular textures, and we find all five local shape and orientation parameters with no a priori assumptions about the shape of the surface. We test the method based on our second distortion measure on textures ranging from regular to irregular, defined by spatial point processes. (Abstract shortened by UMI.)