Feature Learning Algorithm

A Feature Learning Algorithm is a learning algorithm that can be applied by a feature learning system (to solve a feature learning task).

• AKA: Representation Learning Algorithm.
• Context:
  • It can range from being a Supervised Feature Learning Algorithm to being an Unsupervised Feature Learning Algorithm.
  • It can range from being a Single-Task Feature Learning Algorithm to being a Multi-Task Feature Learning Algorithm.
  • …
• Example(s):
  • Autoencoding Algorithm,
  • Convex Multi-Task Feature Learning Algorithm,
  • Dictionary Learning Algorithm,
  • Independent Component Algorithm,
  • Matrix Decomposition Algorithm,
  • Neural Network Training Algorithm,
  • Word Representation Algorithm.
  • …
• Counter-Example(s):
  • Feature Extraction Algorithm.
  • Feature Engineering Algorithm,
  • Object Recognition Algorithm,
  • Manual Feature Extraction,
  • Transfer Learning Algorithm.
• See: Speech Recognition Task, Signal Processing Task, Natural Language Processing Deep Learning Algorithm, Feature Detector, Derived Feature, Cluster Analysis, Multilayer Perceptron, Feature Vector.

---

---

References

2021

• (Wikipedia, 2021) ⇒ https://en.wikipedia.org/wiki/Feature_learning Retrieved:2021-5-29.
  • In machine learning, feature learning or representation learning (Bengio et al., 2013) is a set of techniques that allows a system to automatically discover the representations needed for feature detection or classification from raw data. This replaces manual feature engineering and allows a machine to both learn the features and use them to perform a specific task.

    Feature learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process. However, real-world data such as images, video, and sensor data has not yielded to attempts to algorithmically define specific features. An alternative is to discover such features or representations through examination, without relying on explicit algorithms.

    Feature learning can be either supervised or unsupervised.

    • In supervised feature learning, features are learned using labeled input data. Examples include supervised neural networks, multilayer perceptron and (supervised) dictionary learning.
    • In unsupervised feature learning, features are learned with unlabeled input data. Examples include dictionary learning, independent component analysis, autoencoders, matrix factorization and various forms of clustering.

2013

• (Bengio et al., 2013) ⇒ Yoshua Bengio, Aaron Courville, and Pascal Vincent. (2013). “Representation Learning: A Review and New Perspectives.” In: IEEE Transactions on Pattern Analysis and Machine Intelligence Journal, 35(8). doi:10.1109/TPAMI.2013.50
  • QUOTE: This paper is about representation learning, i.e., learning representations of the data that make it easier to extract useful information when building classifiers or other predictors.

In the case of probabilistic models, a good representation is often one that captures the posterior distribution of the underlying explanatory factors for the observed input. A good representation is also one that is useful as input to a supervised predictor. Among the various ways of learning representations, this paper focuses on deep learning methods: those that are formed by the composition of multiple non-linear transformations, with the goal of yielding more abstract- and ultimately more useful - representations.

2008

• (Argyriou et al., 2008) ⇒ Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil. (2008). “Convex Multi-task Feature Learning.” In: Machine Learning Journal, 73(3). doi:10.1007/s10994-007-5040-8
  • QUOTE: We present a method for learning sparse representations shared across multiple tasks. … Learning common sparse representations across multiple tasks or datasets may also be of interest for example for data compression. While the problem of learning (or selecting) sparse representations has been extensively studied either for single-task supervised learning (e.g., using 1-norm regularization) or for unsupervised learning (e.g., using principal component analysis (PCA) or independent component analysis (ICA)), there has been only limited work [3, 9, 31, 48] in the multi-task supervised learning setting.

---