Hyperparameter Optimization in Machine Learning Classification Models: A Case Study in the Retail Industry

Abstract

It is now important to forecast customer purchase patterns and classify stores appropriately considering the effects of the global pandemic and changing consumer preferences in the retail sector. Machine learning (ML) algorithms can be used to predict customer buying patterns and classify stores. However, conventional ML algorithms are very sensitive to their parameters. Hyperparameter optimization (HPO) is used for tuning the parameters of the model with grid search and random search algorithms and finding the best parameters. The aim of this study is to analyze which HPO approach performs best in determining parameters in ML-based classification algorithms. In this study, for the store classification in the retail sector, gradient boosting classifier, XGBClassifier, and random forest (RF) methods are selected. Model parameters are determined by using two different HPO methods for each model, and performance metrics are compared. In these three classification models, monthly sales amount, population in the store region, GDP per capita, and warehouse size of the stores are used between January 2017 and December 2021. In the analysis that evaluated the pre-pandemic and pandemic periods together in the study, RF is the best prediction algorithm with an accuracy score of 98%. The RF algorithm has an accuracy rate of 99% for both periods separately. Our analyses evaluated pre-pandemic, pandemic, and both periods together. The HPO that gives the best results is the random search tuning. This study focuses on evaluating the comparative performance of grid search and random search HPO techniques across multiple ML models for retail store classification.