sklearn.neural network.MLPClassifier

A sklearn.neural_network.MLPClassifier is a Multi-layer Perceptron Classification System within sklearn.neural_network.

• Context
  • Usage:

1) Import MLP Classification System from scikit-learn : from sklearn.neural_network import MLPClassifier

2) Create design matrix X and response vector Y

3) Create Classifier object: clf=MLPClassifier([hidden_layer_sizes=(100, ), activation=’relu’, solver=’adam’, alpha=0.0001, batch_size=’auto’, learning_rate=’constant’, learning_rate_init=0.001,...])

4) Choose method(s):

• fit(X, y), fits the classification model to data matrix X and target(s) y.
• get_params([deep]), retrieves parameters for this estimator.
• predict(X), predicts using the multi-layer perceptron classifier.
• predict_log_proba(X), returns the probability log estimates.
• predict_proba(X), returns probability estimates.
• score(X, y[, sample_weight]), returns the mean accuracy on the given test data and labels.
• set_params(**params), sets the parameters of this estimator.

• Example(s):
  • Varying regularization in Multi-layer Perceptron.
  • Compare Stochastic learning strategies for MLPClassifier
  • Visualization of MLP weights on MNIST
• Counter-Example(s):
  • sklearn.neural network.MLPRegressor
  • sklearn.neural_network.BernoulliRBM
• See: Classification System, Regularization Task, Ridge Regression Task.

---

---

References

2017a

• (Scikit-Learn, 2017A) ⇒ http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html Retrieved:2017-12-17
  • QUOTE: class sklearn.neural_network.MLPClassifier(hidden_layer_sizes=(100, ), activation=’relu’, solver=’adam’, alpha=0.0001, batch_size=’auto’, learning_rate=’constant’, learning_rate_init=0.001, power_t=0.5, max_iter=200, shuffle=True, random_state=None, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True, early_stopping=False, validation_fraction=0.1, beta_1=0.9, beta_2=0.999, epsilon=1e-08)

     Multi-layer Perceptron classifier.

    This model optimizes the log-loss function using LBFGS or stochastic gradient descent.

    (...)

    Notes

     MLPClassifier trains iteratively since at each time step the partial derivatives of the loss function with respect to the model parameters are computed to update the parameters. It can also have a regularization term added to the loss function that shrinks model parameters to prevent overfitting. This implementation works with data represented as dense numpy arrays or sparse scipy arrays of floating point values.

2017b

• (sklearn,2017) ⇒ http://scikit-learn.org/stable/modules/neural_networks_supervised.html#classification Retrieved:2017-12-17.
  • QUOTE: Class MLPClassifier implements a multi-layer perceptron (MLP) algorithm that trains using Backpropagation. MLP trains on two arrays: array X of size (n_samples, n_features), which holds the training samples represented as floating point feature vectors; and array y of size (n_samples,), which holds the target values (class labels) for the training samples:

>>> from sklearn.neural_network import MLPClassifier X = [ [0., 0.], [1., 1.] ] y = [0, 1] clf = MLPClassifier(solver='lbfgs', alpha=1e-5, hidden_layer_sizes=(5, 2), random_state=1) clf.fit(X, y)

After fitting (training), the model can predict labels for new samples:

{| class="wikitable" style="margin-left: 50px;border:1px;background:#f2f2f2"

|style="font-family:monospace; font-size:10pt;font-weight=bold;text-align:left;width:700px;"|>>> clf.predict([ [2., 2.], [-1., -2.] ]) |}

MLP can fit a non-linear model to the training data. clf.coefs_ contains the weight matrices that constitute the model parameters:

>>> [coef.shape for coef in clf.coefs_]

Currently, MLPClassifier supports only the Cross-Entropy loss function, which allows probability estimates by running the predict_proba method. MLP trains using Backpropagation. More precisely, it trains using some form of gradient descent and the gradients are calculated using Backpropagation. For classification, it minimizes the Cross-Entropy loss function, giving a vector of probability estimates P(y|x) per sample x:

>>> clf.predict_proba([ [2., 2.], [1., 2.] ])

MLPClassifier supports multi-class classification by applying Softmax as the output function. Further, the model supports multi-label classification in which a sample can belong to more than one class. For each class, the raw output passes through the logistic function. Values larger or equal to 0.5 are rounded to 1, otherwise to 0. For a predicted output of a sample, the indices where the value is 1 represents the assigned classes of that sample:

>>> X = [ [0., 0.], [1., 1.] ] y = [ [0, 1], [1, 1] ] clf = MLPClassifier(solver='lbfgs', alpha=1e-5, hidden_layer_sizes=(15,), random_state=1) clf.fit(X, y) clf.predict([ [1., 2.] ]) clf.predict([ [0., 0.] ])

---