Abstract Book of the 8th International Conference on Research in Management
Year: 2025
[PDF]
Automated Machine Learning in Action: A Performance Evaluation for Predictive Analytics Tasks
Nicolas Leyh
ABSTRACT:
As organizations increasingly focus on data-driven insights, the demand for machine learning (ML) expertise continues to outpace workforce supply. Automated Machine Learning (AutoML) frameworks address this gap by automating complex ML processes, thereby making advanced modeling accessible to non-specialists. This study provides an up-to-date performance evaluation of four prominent AutoML frameworks – Auto-Keras, Auto-Sklearn, H2O, and TPOT – within the domain of predictive analytics. Focusing on both binary and multiclass classification, the research draws on 22 secondary datasets and evaluates effectiveness across 12 data segments, including class imbalance, feature count, and categorical proportions. The results show that H2O delivers strong overall performance, particularly in datasets with high categorical proportions and feature counts. However, no framework performs best across all conditions. TPOT and Auto-Keras underperform in multiclass tasks with small datasets, while Auto-Sklearn and Auto-Keras exhibit variability in binary classification under certain data constraints. The analysis highlights the need for improved feature selection, handling of categorical variables, and resampling strategies to enhance framework robustness. Theoretically, this research extends benchmarking work in the AutoML field by integrating data-centric complexity factors and contributes actionable insights for optimizing AutoML use in business environments. Practically, the findings offer decision-makers guidance on framework selection depending on data characteristics and classification goals. Future research should examine hybrid AutoML approaches combining automation with domain-driven feature engineering to enhance interpretability and robustness. This paper thus offers a timely contribution to both academic discourse and managerial practice in leveraging AutoML for predictive analytics.
Keywords: AutoML frameworks, performance benchmarking, predictive analytics