Kedziora D. AutonoML. Towards an Integrated Framework for Autonomous ML 2024

Textbook in PDF format

Over the last decade, the long-running endeavour to automate high-level processes in machine learning (ML) has risen to mainstream prominence. Beyond this, an even loftier goal is the pursuit of autonomy, which describes the capability of the system to independently adjust an ML solution over a lifetime of changing contexts. This monograph provides an expansive perspective on what constitutes an automated/autonomous ML system. In doing so, the authors survey developments in hyperparameter optimisation, multicomponent models, neural architecture search, automated feature engineering, meta-learning, multi-level ensembling, dynamic adaptation, multi-objective evaluation, resource constraints, flexible user involvement, and the principles of generalisation. Furthermore, they develop a conceptual framework throughout to illustrate one possible way of fusing high-level mechanisms into an autonomous ML system. This monograph lays the groundwork for students and researchers to understand the factors limiting architectural integration, without which the field of automated ML risks stifling both its technical advantages and general uptake.
Introduction
Machine Learning Basics
Algorithm Selection and Hyperparameter Optimisation
Multi-component Pipelines
Neural Architecture Search
Automated Feature Engineering
Meta-knowledge
Ensembles and Bundled Pipelines
Persistence and Adaptation
Definitions of Model Quality
Resource Management
User Interactivity
Towards General Applicability
Discussion
Conclusions