Active learning using pessimistic expectation estimators

Active learning is the process in which unlabeled in- stances are dynamically selected for expert labelling, and then a clas- sifier is trained on the labeled data. Active learning is particularly useful when there is a large set of unlabeled instances, and acquir- ing a label is costly. In business scenarios such as direct marketing, active learning can be used to indicate which customer to approach such that the potential benefit from the approached customer can cover the cost of approach. This paper presents a new algorithm for cost-sensitive active learning using a conditional expectation es- timator. The new estimator focuses on acquisitions that are likely to improve the profit. Moreover, we investigate simulated annealing techniques to combine exploration with exploitation in the classi- fier construction. Using five evaluation metrics, we evaluated the algorithm on four benchmark datasets. The results demonstrate the superiority of the proposed method compared to other algorithms.