Stochastic Gradient Descent System

A Stochastic Gradient Descent System is an numeric optimization system that implements an SGD algorithm to solve an SGD task.

• AKA: SGD Solver.
• Example(s):
  • scikit-learn sdg[1]
    • from sklearn.linear_model import SGDClassifier ;
X = [[0., 0.], [1., 1.]] ;
y = [0, 1] ;
clf = SGDClassifier(loss="hinge", penalty="l2") ;
clf.fit(X, y) ;
SGDClassifier(alpha=0.0001, class_weight=None, eta0=0.0, fit_intercept=True, learning_rate='optimal', loss='hinge', n_iter=5, n_jobs=1, penalty='l2', power_t=0.5, rho=0.85, seed=0, shuffle=False, verbose=0, warm_start=False)
    • from sklearn.linear_model import SGDRegressor ; ...
  • scikit-learn SGDRegressor[2]
• See: Gradient Descent System, CNTK.

---

---

References

2015

• http://en.wikipedia.org/wiki/Stochastic_gradient_descent#Software
  • sgd: an LGPL C++ library which uses stochastic gradient descent to fit SVM and conditional random field models.
  • CRF-ADF A C# toolkit of stochastic gradient descent and its feature-frequency-adaptive variation for training conditional random field models.

---