sklearn.gaussian process.GaussianProcessRegressor

An sklearn.gaussian process.GaussianProcessRegressor is a Gaussian Process Regression (GPR) system within sklearn.gaussian_process.

• Example(s):
  • Gaussian Processes regression: basic introductory example
  • Gaussian process regression (GPR) with noise-level estimation
  • Gaussian process regression (GPR) on Mauna Loa CO2 data.
  • …
• Counter-Examples:
  • sklearn.gaussian_process.GaussianProcessClassifier[1].
• See: Gaussian Process, Nonparametric Regression, Kernel Method.

---

---

References

2017a

• http://scikit-learn.org/stable/modules/gaussian_process.html
  • QUOTE: The GaussianProcessRegressor implements Gaussian processes (GP) for regression purposes. For this, the prior of the GP needs to be specified. The prior mean is assumed to be constant and zero (for normalize_y=False) or the training data’s mean (for normalize_y=True). The prior’s covariance is specified by a passing a kernel object. The hyperparameters of the kernel are optimized during fitting of GaussianProcessRegressor by maximizing the log-marginal-likelihood (LML) based on the passed optimizer. As the LML may have multiple local optima, the optimizer can be started repeatedly by specifying n_restarts_optimizer. The first run is always conducted starting from the initial hyperparameter values of the kernel; subsequent runs are conducted from hyperparameter values that have been chosen randomly from the range of allowed values. If the initial hyperparameters should be kept fixed, None can be passed as optimizer.

2017b

• (Scikit Learn, 2017) ⇒ http://scikit-learn.org/stable/modules/generated/sklearn.gaussian_process.GaussianProcessRegressor.html Retrieved: 2017-09-03
  • QUOTE: sklearn.gaussian_process.GaussianProcessRegressor(kernel=None, alpha=1e-10, optimizer=’fmin_l_bfgs_b', n_restarts_optimizer=0, normalize_y=False, copy_X_train=True, random_state=None) source

    Gaussian process regression (GPR).

    The implementation is based on Algorithm 2.1 of Gaussian Processes for Machine Learning (GPML) by Rasmussen and Williams.

    In addition to standard scikit-learn estimator API, GaussianProcessRegressor:

    • allows prediction without prior fitting (based on the GP prior)
    • provides an additional method sample_y(X), which evaluates samples drawn from the GPR (prior or posterior) at given inputs
    • exposes a method log_marginal_likelihood(theta), which can be used externally for other ways of selecting hyperparameters, e.g., via Markov chain Monte Carlo.

    Read more in the User Guide

2017c

• (Scikit Learn, 2017) ⇒ "GPR Examples" in http://scikit-learn.org/stable/modules/gaussian_process.html#gpr-examples Retrieved: 2017-09-03
  • QUOTE: 1.7.2.1. GPR with noise-level estimation

    This example illustrates that GPR with a sum-kernel including a WhiteKernel can estimate the noise level of data. An illustration of the log-marginal-likelihood (LML) landscape shows that there exist two local maxima of LML.

    Code & Plot (1)

    The first corresponds to a model with a high noise level and a large length scale, which explains all variations in the data by noise.

    Code & Plot (2)

    The second one has a smaller noise level and shorter length scale, which explains most of the variation by the noise-free functional relationship. The second model has a higher likelihood; however, depending on the initial value for the hyperparameters, the gradient-based optimization might also converge to the high-noise solution. It is thus important to repeat the optimization several times for different initializations.

    Log-marginal-likelihood Plots (3)

---