Classifier evaluation under limited resources

Existing evaluations measures are insufficient when probabilistic classifiers are used for choosing objects to be included in a limited quota. This paper reviews performance measures that suit probabilistic classification and introduce two novel performance measures that can be used effectively for this task. It then investigates when to use each of the measures and what purpose each one of them serves. The use of these measures is demonstrated on a real life dataset obtained from the human resource field and is validated on set of benchmark datasets.