Primary
Hyperparameter Optimization ○｜Definition｜1st｜20260604233205-00-⌔
Hyperparameter optimization - Wikipedia
Hyperparameter optimization
In machine learning, hyperparameter optimization1 or tuning is the problem of choosing a set of optimal hyperparameters for a learning algorithm. A hyperparameter is a parameter whose value is used to control the learning process, which must be configured before the process starts.23
Hyperparameter optimization determines the set of hyperparameters that yields an optimal model which minimizes a predefined loss function on a given data set.4 The objective function takes a set of hyperparameters and returns the associated loss.4 Cross-validation is often used to estimate this generalization performance, and therefore choose the set of values for hyperparameters that maximize it.5
Printed 2026-06-28.
(echo:: @ ᯤ)
Link to original Footnotes
Matthias Feurer and Frank Hutter. Hyperparameter optimization. In: AutoML: Methods, Systems, Challenges, pages 3–38. ↩
Yang, Li (2020). “On hyperparameter optimization of machine learning algorithms: Theory and practice”. Neurocomputing. 415: 295–316. arXiv:2007.15745. doi:10.1016/j.neucom.2020.07.061. ↩
Franceschi L, Donini M, Perrone V, Klein A, Archambeau C, Seeger M, Pontil M, Frasconi P (2024). “Hyperparameter Optimization in Machine Learning”. arXiv:2410.22854 [stat.ML]. ↩
Claesen, Marc; Bart De Moor (2015). “Hyperparameter Search in Machine Learning”. arXiv:1502.02127 [cs.LG]. ↩ ↩2
Bergstra, James; Bengio, Yoshua (2012). “Random Search for Hyper-Parameter Optimization” (PDF). Journal of Machine Learning Research. 13: 281–305. ↩
Secondary
• • •