Dataset Feature Sampling Task

A Dataset Feature Sampling Task is a dimensionality reduction task that is a sampling task (requires the selection of features in a training Set that are most informative to the learning task).

• AKA: Variable Selection, Feature Reduction, Attribute Selection.
• Context:
  • Input: a Labeled Training Set.
    • optional: an Unlabeled Training Set.
  • output: a Feature Subset.
  • Performance:
    • effect on Accuracy Metric.
    • Time Complexity and Space Complexity.
  • It can range from being a Supervised Feature Selection Task to being an Unsupervised Feature Selection Task.
  • It can be supported by a Feature Ranking Task.
  • It can be solved by a Feature Selection System (that implements a Feature Subset Selection algorithm).
  • …
• Counter-Example(s):
  • a Dataset Item Subset Selection Task.
  • a Dataset Feature Compression Task.
  • a Feature Engineering Task.
  • a Feature Discretization Task.
  • a Feature Space Transformation Task.
  • a Feature Ablation Study.
• See: Predictive Function; Classification; Clustering; Cross Validation; Curse of Dimensionality; Dimensionality Reduction; Semi-Supervised Learning.

---

---

References

2016

• (Li et al., 2016) ⇒ Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P. Trevino, Jiliang Tang, and Huan Liu. (2016). “Feature Selection: {A} Data Perspective.” In: CoRR, abs/1601.07996.
  • QUOTE: Dimensionality reduction is one of the most powerful tools to address the previously described issues. It can be categorized mainly into into two main components: feature extraction and feature selection. Feature extraction projects original high dimensional feature space to a new feature space with low dimensionality. The new constructed feature space is usually a linear or nonlinear combination of the original feature space. .... Feature selection, on the other hand, directly selects a subset of relevant features for the use model construction. …

    Both feature extraction and feature selection have the advantage of improving learning performance, increasing computational efficiency, decreasing memory storage requirements, and building better generalization models. However, since feature extraction builds a set of new features, further analysis is problematic as we cannot get the physical meaning of these features in the transformed space. In contrast, by keeping some original features, feature selection maintains physical meanings of original features, and gives models better readability and interpretability. Therefore, feature selection is often preferred in many realworld applications such as text mining and genetic analysis compared to feature extraction.

2011

• (Liu, 2011) ⇒ Huan Liu. (2011). “Feature Selection.” In: (Sammut & Webb, 2011) p.402

2009

• (Wikipedia, 2009) ⇒ http://en.wikipedia.org/wiki/Feature_selection
  • Feature selection, also known as variable selection, feature reduction, attribute selection or variable subset selection, is the technique, commonly used in machine learning, of selecting a subset of relevant features for building robust learning models. When applied in biology domain, the technique is also called discriminative gene selection, which detects influential genes based on DNA microarray experiments. By removing most irrelevant and redundant features from the data, feature selection helps improve the performance of learning models by:
    • Alleviating the effect of the curse of dimensionality.
    • Enhancing generalization capability.
    • Speeding up learning process.
    • Improving model interpretability.
  • Feature selection also helps people to acquire better understanding about their data by telling them which are the important features and how they are related with each other.

2007

• (Zhao & Liu, 2007) ⇒ Zheng Zhao, and Huan Liu. (2007). “Spectral feature selection for supervised and unsupervised learning.” In: Proceedings of the 24th International Conference on Machine learning (ICML 2007).
  • Feature selection aims to reduce dimensionality for building comprehensible learning models with good generalization performance. Feature selection algorithms are largely studied separately according to the type of learning: supervised or unsupervised. This work exploits intrinsic properties underlying supervised and feature selection algorithms, and proposes a unified framework for feature selection based on spectral graph theory. The proposed framework is able to generate families of algorithms for both supervised and unsupervised feature selection. And we show that existing powerful algorithms such as ReliefF (supervised) and Laplacian Score (unsupervised) are special cases of the proposed framework. To the best of our knowledge, this work is the first attempt to unify supervised and unsupervised feature selection, and enable their joint study under a general framework. Experiments demonstrated the efficacy of the novel algorithms derived from the framework.
• (Sewell, 2007a) ⇒ Martin Sewell. (2007). “Feature Selection.”
  • Feature selection (also known as subset selection) is a process commonly used in machine learning, wherein a subset of the features available from the data are selected for application of a learning algorithm. The best subset contains the least number of dimensions that most contribute to accuracy; we discard the remaining, unimportant dimensions. This is an important stage of pre-processing and is one of two ways of avoiding the curse of dimensionality (the other is feature extraction). …

2004

• (Dy & Brodley, 2004) ⇒ J. G. Dy, and C. E. Brodley. (2004). “Feature Selection for Unsupervised Learning.” In: Journal of Machine Learning Research, 5.

2003

• (Guyon & Elisseeff, 2003) ⇒ Isabelle M. Guyon, and André Elisseeff. (2003). “An Introduction to Variable and Feature Selection.” In: The Journal of Machine Learning Research, 3.
  • QUOTE: Variable and feature selection have become the focus of much research in areas of application for which datasets with tens or hundreds of thousands of variables are available. These areas include text processing of internet documents, gene expression array analysis, and combinatorial chemistry. The objective of variable selection is three-fold: improving the prediction performance of the predictors, providing faster and more cost-effective predictors, and providing a better understanding of the underlying process that generated the data.

1998

• (Liu & Motoda, 1998) ⇒ Huan Liu, and Hiroshi Motoda. (1998). “Feature Selection for Knowledge Discovery and Data Mining." Kluwer Academic.

1997

• (Yang & Pedersen, 1997) ⇒ Yiming Yang, and Jan P. Pedersen. (1997). “A Comparative Study on Feature Selection in Text Categorization.” In: Proceedings of the Fourteenth International Conference on Machine Learning (ICML 1997).
  • Cited by ~2,285 http://scholar.google.com/scholar?cites=14250908352503652390
• (Kohavi & John, 1997) ⇒ Ron Kohavi, and George John. “Wrappers for Feature Selection.” In: Artificial Intelligence, 97(1). doi:10.1016/S0004-3702(97)00043-X
• (Blum & Langley, 1997) ⇒ Avrim L. Blum, and Pat Langley. (1998). “Selection of Relevant Features and Examples in Machine Learning.” In: Artificial Intelligence, 97(1-2). doi:10.1016/S0004-3702(97)00063-5

1996

• (Koller & Sahami, 1996) ⇒ Daphne Koller and Mehran Sahami. (1996). “Toward Optimal Feature Selection.” In: Proceedings of the International Conference on Machine Learning (ICML 1996).

---