Hyperparameter Optimization Task

A Hyperparameter Optimization Task is an optimization task that selects optimal model hyperparameter values for a trained machine learning model to optimize an offline model evaluation measure.

• Context:
  • It can be solved by a Hyperparameter Tuning Algorithm (that implements a hyperparameter tuning algorithm).
• Example(s):
  • a Decision Tree-based Model Hyperparameter Optimization Task.
  • a SVM-based Model Hyperparameter Optimization Task.
  • a Neural Network-based Model Hyperparameter Optimization Task.
• See: Model Selection, Cross-Validation (Statistics), Loss Function, Overfitting.

---

---

References

2020

• (Wikipedia, 2020) ⇒ https://en.wikipedia.org/wiki/Hyperparameter_optimization Retrieved:2020-6-5.
  • In machine learning, hyperparameter optimization or tuning is the problem of choosing a set of optimal hyperparameters for a learning algorithm. A hyperparameter is a parameter whose value is used to control the learning process. By contrast, the values of other parameters (typically node weights) are learned.

    The same kind of machine learning model can require different constraints, weights or learning rates to generalize different data patterns. These measures are called hyperparameters, and have to be tuned so that the model can optimally solve the machine learning problem. Hyperparameter optimization finds a tuple of hyperparameters that yields an optimal model which minimizes a predefined loss function on given independent data. The objective function takes a tuple of hyperparameters and returns the associated loss. Cross-validation is often used to estimate this generalization performance.

---