Application of Gradient Boosting in the Design of Fuzzy Rule-Based Regression Models

This study is devoted to the design of gradient boosted fuzzy rule-based models for regression problems. Fuzzy rule-based models are built on the basis of information granules formed in the input and output spaces whose structure involves a family of conditional 'if-then' statements. The architecture of fuzzy rule-based models contributes to the realization of a sound tradeoff between modeling accuracy and interpretability and computing overhead. Gradient boosting paradigm has emerged as a powerful learning method realized through sequentially fitting additive base learners to current residuals in the steepest descent way. However, surprisingly, studies on the design and analysis of gradient boosted fuzzy rule-based models are still lacking. In this study, fuzzy rule-based model is regarded as a base learner. Different loss functions and their influence on the performance of the final models are explored. We also thoroughly investigate an impact of the initial quality of the rule-based model (implied by the number of rules) on the process of gradient boosting. The performance of the proposed approach is illustrated by a series of experimental studies concerning synthetic and publicly available datasets.