Machine Learning Hyperparameter

A Machine Learning Hyperparameter is a parameter that is set before and that can be tuned during successive runs of the Learning Algorithm.

• Example(s):
  • Learning Rate,
  • Machine Learning Momentum,
  • Regularization Penalty.
  • …
• Counter-Example(s):
  • Model Parameter,
  • Population Parameter,
  • Learnable Parameter.
• See: Parameter, Regularization (Mathematics), Hyperparameter Tuning Algorithm.

---

---

References

2018a

• (Wikipedia, 2018) ⇒ https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning) Retrieved:2018-4-22.
  • In machine learning, a hyperparameter is a parameter whose value is set before the learning process begins. By contrast, the values of other parameters are derived via training.

    Different model training algorithms require different hyperparameters, some simple algorithms (such as ordinary least squares regression) require none. Given these hyperparameters, the training algorithm learns the parameters from the data. For instance, LASSO is an algorithm that adds a regularization hyperparameter to ordinary least squares regression, which has to be set before estimating the parameters through the training algorithm.

2018b

• (Google ML Glossary, 2018) ⇒ (2018). hyperparameter. In: Machine Learning Glossary https://developers.google.com/machine-learning/glossary/ Retrieved: 2018-04-22.
  • QUOTE: The "knobs" that you tweak during successive runs of training a model. For example, learning rate is a hyperparameter. Contrast with parameter.

---