ElasticNet Cross-Validation System

An ElasticNet Cross-Validation System is a mixed ElasticNet System that implements an Cross-Validation Algorithm to solve a Linear Regression Task.

• AKA: ElasticNet CV System.
  • …
• Example(s):
  • sklearn.linear_model.ElasticNetCV [1].
  • …
• Counter-Example(s):
  • LASSO Regression System.
  • Ridge Regression System.
  • ElasticNet System.
  • Multi-Task ElasticNet System.
• See: Cross-Validation Task, L1-Norm, L2-norm.

---

---

References

2017B

• (Scikit Learn, 2017) ⇒ hhttp://scikit-learn.org/stable/modules/linear_model.html#elastic-net Retrieved:2017-09-17
  • QUOTE: ElasticNet is a linear regression model trained with L1 and L2 prior as regularizer. This combination allows for learning a sparse model where few of the weights are non-zero like Lasso, while still maintaining the regularization properties of Ridge. We control the convex combination of L1 and L2 using the l1_ratio parameter.

     Elastic-net is useful when there are multiple features which are correlated with one another. Lasso is likely to pick one of these at random, while elastic-net is likely to pick both.

    A practical advantage of trading-off between Lasso and Ridge is it allows Elastic-Net to inherit some of Ridge’s stability under rotation.

    The objective function to minimize is in this case

    [math]\displaystyle{ \underset{w}{min\,} { \frac{1}{2n_{samples}} ||X w - y||_2 ^ 2 + \alpha \rho ||w||_1 + \frac{\alpha(1-\rho)}{2} ||w||_2 ^ 2} }[/math]

    The class ElasticNetCV can be used to set the parameters alphaalpha ([math]\displaystyle{ \alpha }[/math]) and l1_ratio ([math]\displaystyle{ \rho }[/math]) by cross-validation.

2017B

• (Scikit Learn, 2017) ⇒ http://scikit-learn.org/stable/modules/grid_search.html
  • QUOTE: 3.2.4.1. Model specific cross-validation

    Some models can fit data for a range of values of some parameter almost as efficiently as fitting the estimator for a single value of the parameter. This feature can be leveraged to perform a more efficient cross-validation used for model selection of this parameter.

    The most common parameter amenable to this strategy is the parameter encoding the strength of the regularizer. In this case we say that we compute the regularization path of the estimator.

Here is the list of such models:

 :: linear_model.ElasticNetCV([l1_ratio, eps,...]), Elastic Net model with iterative fitting along a regularization path;

linear_model.LarsCV([fit_intercept, ...]), Cross-validated Least Angle Regression model

linear_model.LassoCV([eps, n_alphas, ...]), Lasso linear model with iterative fitting along a regularization path;

linear_model.LassoLarsCV([fit_intercept, ...]), Cross-validated Lasso, using the LARS algorithm

linear_model.LogisticRegressionCV([Cs, ...]), Logistic Regression CV (aka logit, MaxEnt) classifier.

linear_model.MultiTaskElasticNetCV([...]), Multi-task L1/L2 ElasticNet with built-in cross-validation.

linear_model.MultiTaskLassoCV([eps, ...]), Multi-task L1/L2 Lasso with built-in cross-validation.

linear_model.OrthogonalMatchingPursuitCV([...]), Cross-validated Orthogonal Matching Pursuit model (OMP)

linear_model.RidgeCV([alphas, ...]), Ridge regression with built-in cross-validation.

linear_model.RidgeClassifierCV([alphas, ...]), Ridge classifier with built-in cross-validation.

---