LogisticLDA: Regularizing Latent Dirichlet Allocation by Logistic Regression

We present in this paper a supervised topic model for multi-class classification problems. To incorporate supervisory information, we jointly model documents and their labels in a graphical model called LogisticLDA, which mathematically integrates a genera- tive model and a discriminative model in a principled way. By maximizing the posterior of document labels using logistic normal distributions, the model effectively incorporates the supervisory information to maximize inter-class distance in the topic space, while documents still enjoy the interchangeability characteristic for ease of inference. Experimental results on three benchmark datasets demonstrate that the model outperforms state-of-the-art supervised topic models. Compared with support vector machine, the model also achieves compara- ble performance, but meanwhile it discovers a topic space, which is valuable for dimension reduction, topic mining and document retrieval.