A unified probabilistic graphical model and its application to image segmentation

Probabilistic graphical models (PGMs) are a marriage between the probability theories and graph theories. They provide a natural and powerful tool for dealing with two problems that occur throughout applied mathematics and engineering: uncertainty and complexity. In recent years, there has been a gradual yet widespread adoption of PGMs in many areas of computer vision and pattern recognition. Current applications of PGMs in computer vision, however, are limited to modeling either causal or mutually dependent relationships, but not both. For many computer vision problems, the relationships among different image entities are often heterogeneous and of different types. A hybrid probabilistic graphical model is, therefore, needed in order to effectively capture these complex relationships. This thesis aims at developing a hybrid probabilistic graphical model that can exploit both directed links and undirected links for modeling heterogeneous relationships and demonstrating its application to image segmentation problems. We propose a systematic way to deal with the modeling, parameter learning, and inference issues for such a hybrid graphical model. Specifically, we construct the hybrid graphical model structure with different types of links based on the relationships between random variables. We derive the joint probability distribution represented by the hybrid graphical model using the Markov property encoded in the graphical structure. We then propose a joint parameter learning approach to train the hybrid graphical model based on the combination of an analytic solution and a numerical solution through Gibbs sampling. Finally, we propose to convert the hybrid graphical model into a factor graph representation to perform probabilistic inference through principled methods. For the application of PGMs to image segmentation, we first study an undirected graphical model (i.e. Conditional Random Field) that models the spatial relationships among region labels for image segmentation. We then study a directed graphical model (i.e. Bayesian Network) that captures the causality among the fundamental image entities (regions, edges, vertices, etc) and various constraints such as contour smoothness and connectivity for image segmentation. Subsequently, we propose a hybrid graphical model for image segmentation, which consists of both directed links and undirected links to capture the causality and spatial relationships among image entities. We further extend the hybrid graphical model to a multiscale hybrid graphical model that exploits other contextual relationships such as the homogeneity of multiscale labels. The hybrid graphical model allows the systematic modeling and integration of different types of uncertain knowledge including image measurements, contextual knowledge, and subjective human knowledge for effective and robust image segmentation. It also allows combining region-based image segmentation with edge-based image segmentation. The performance of the unified framework is quantitatively evaluated against techniques based on either directed or undirected graphical models alone and against other state-of-the-art image segmentation techniques on commonly used image databases. Our experimental results demonstrate that the hybrid graphical model can be successfully applied to image segmentation and the performance outperforms state-of-the-art approaches due to the integration of multiple image measurements and incorporation of the informative contextual relationships.