Evaluating Quality of Models via Prediction Information Granules

Numeric models (including fuzzy models) produce numeric results. There are no ideal models that deliver a complete match with the data. In this study, we advocate that a way of evaluating the quality of models can be realized at the higher level of abstraction by developing a concept of granular prediction. In this way, modeling results are expressed in the form of information granules, in particular as intervals or fuzzy sets. The study formulates a general conceptual and algorithmically supported statement: a meaningful evaluation framework to assess the quality of numeric models is the one engaging information granules. This general observation comprises a special case commonly investigated in regression analysis, where the quality of numeric results is expressed via granular constructs, namely, confidence or prediction intervals. The original design of prediction information granules is formulated as an optimization problem, in which the criteria of coverage of data and specificity of granular results are considered. In the optimization process, we also engage some nonlinear transformation of the level of information granularity depending upon the value of the numeric result. The proposed development is model agnostic and can support a variety of modeling architectures; the experimental part of the study is focused on rule-based models. Further generalizations of prediction information granules are covered by involving granular parameters in the design process.