2010 RectifiedLinearUnitsImproveRest
- (Nair & Hinton, 2010) ⇒ Vinod Nair, and Geoffrey E. Hinton. (2010). “Rectified Linear Units Improve Restricted Boltzmann Machines.” In: Proceedings of the 27th International Conference on Machine Learning (ICML-10).
Subject Headings: Rectified Neural Unit, Rectifier-based NNet.
Notes
Cited By
Quotes
Abstract
Restricted Boltzmann machines were developed using binary stochastic hidden units. These can be generalized by replacing each binary unit by an infinite number of copies that all have the same weights but have progressively more negative biases. The learning and inference rules for these “Stepped Sigmoid Units" are unchanged. They can be approximated efficiently by noisy, rectified linear units. Compared with binary units, these units learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset. Unlike binary units, rectified linear units preserve information about relative intensities as information travels through multiple layers of feature detectors.
References
;
| Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2010 RectifiedLinearUnitsImproveRest | Geoffrey E. Hinton Vinod Nair | Rectified Linear Units Improve Restricted Boltzmann Machines | 2010 |