Gradient boosting as a tool for solving classification problems in data-constrained environments

Abstract

In machine learning, the question of effective construction of classification models with insufficient amount of educational information has arisen. The purpose of the study was to analyse the possibilities of using gradient boosting to solve classification problems in data-constrained environments. The research methodology was based on a comprehensive analysis of the leading gradient boosting implementations: XGBoost, LightGBM, and HistGradientBoosting. The main focus was on investigating regularisation mechanisms, hyperparameter optimisation strategies, and adaptive learning techniques under small sample conditions. The research was aimed at identifying the architectural features of algorithms that can provide high classification accuracy with a minimum amount of data. It was established that the proposed algorithms have demonstrated a significant potential for effectively solving classification problems. It was found that the mechanisms of shrinkage and subsampling significantly increased the generalising ability of models. The results of the study expanded the theoretical understanding of ensemble machine learning methods and outlined promising areas for adapting algorithms to specific conditions of limited information resources. XGBoost, LightGBM, and HistGradientBoosting have been shown to have unique architectural features that allow working efficiently with different types of data. It was found that the internal regularisation mechanisms of these algorithms provided resistance to retraining and high prediction accuracy. The potential of gradient boosting for solving complex classification problems in medicine, finance, and other industries with limited information resources is shown. The practical significance of the study was to develop methodological recommendations for selecting and configuring gradient boosting algorithms for various types of classification problems. The results obtained will be useful for further development of machine learning methods