Efficient propagation for face annotation in family albums

In this paper, we propose and investigate a new user scenario for face annotation, in which users are allowed to multi-select a group of photographs and assign names to these photographs. The system will then attempt to propagate names from photograph level to face level, i.e. to infer the correspondence between name and face. Given the face similarity measure which combines methodologies from face recognition and content-based image retrieval, we formulate name propagation as an optimization problem. We define the objective function as the sum of similarities between each pair of faces of the same individual in different photographs, and propose an iterative optimization algorithm to infer the optimal correspondence. To make the propagation result reliable, a reject scheme is adopted to reject those with low confidence scores. Furthermore, we investigate the combination and alternation of browsing mode for propagation and viewer mode for annotation, so that each mode can benefit from additional inputs from the other mode. The experimental evaluation has been conducted within a typical family album of over one thousand photographs and the results show that the proposed approach is effective and efficient in automated face annotation in family albums.