The Ecological Statistics of Grouping and Figure-Ground Cues

Visual grouping and figure-ground discrimination were first studied by the Gestalt school of visual perception nearly a century ago. By the use of cleverly constructed examples, they were able to demonstrate the role of factors such as proximity, similarity, curvilinear continuity and common fate in visual grouping and factors such as convexity, size, and symmetry in figure-ground discrimination. However, this left open (at least) three major problems (1) there wasn't a precise operationalization of these factors for general images, (2) the interaction of these cues was ill understood (3) and there was no justification for why these factors might be helpful to an observer interacting with the visual world. In my research group, we have tackled these problems in the framework of what we call “ecological statistics”. We start with a set of natural images and use human observers to mark the perceptual groups and assign figure-ground labels to the various boundary contours. We construct computational models of various grouping and figure-ground factors inspired by corresponding mechanisms in visual cortex. Finally we calibrate and optimally combine the grouping and figure-ground factors by using the principle that vision evolved to be adaptive to the statistics of objects in the natural world. Over the last few years of research in this framework, we have been able to quantitatively characterize the grouping cues of brightness, color, and texture similarity and curvilinear continuity, and figure-ground cues of size, lower-region and convexity. I shall summarize some of these results in my talk; more can be found on http://http.cs.berkeley.edu/projects/vision/grouping/. This research is joint work with Charless Fowlkes, David Martin and Xiaofeng Ren.