Dataset Dimensionality Compression Task
A Dataset Dimensionality Compression Task is a dimensionality reduction task that is a compression task.
- Context:
- Output (optional): a Tuple Compression Function.
- It can range from being an Unsupervised Feature Space Compression Task to being a Supervised Feature Space Compression Task.
- It can be solved by a Dataset Dimensionality Compression System (that implements a dataset dimensionality compression algorithm).
- Example(s):
- a PCA Task.
- an SVD Task.
- a Feature Compression Task.
- …
- Counter-Example(s)::
- See: Exploratory Factor Analysis, Eigenvalue.
References
2016
- (Li et al., 2016) ⇒ Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P. Trevino, Jiliang Tang, and Huan Liu. (2016). “Feature Selection: {A} Data Perspective.” In: CoRR, abs/1601.07996.
- QUOTE: Dimensionality reduction is one of the most powerful tools to address the previously described issues. It can be categorized mainly into into two main components: feature extraction and feature selection. Feature extraction projects original high dimensional feature space to a new feature space with low dimensionality. The new constructed feature space is usually a linear or nonlinear combination of the original feature space. Examples of feature extraction methods include Principle Component Analysis (PCA) (Jolliffe, 2002), Linear Discriminant Analysis (LDA) (Scholkopft and Mullert, 1999), Canonical Correlation Analysis (CCA) (Hardoon et al., 2004), Singular Value Decomposition (Golub and Van Loan, 2012), ISOMAP (Tenenbaum et al., 2000) and Locally Linear Embedding (LLE) (Roweis and Saul, 2000). Feature selection, on the other hand, directly selects a subset of relevant features for the use model construction. Lasso (Tibshirani, 1996), Information Gain (Cover and Thomas, 2012), Relief (Kira and Rendell, 1992a), MRMR (Peng et al., 2005), Fisher Score (Duda et al., 2012), Laplacian Score (He et al., 2005), and SPEC (Zhao and Liu, 2007) are some of the well known feature selection techniques.
Both feature extraction and feature selection have the advantage of improving learning performance, increasing computational efficiency, decreasing memory storage requirements, and building better generalization models. However, since feature extraction builds a set of new features, further analysis is problematic as we cannot get the physical meaning of these features in the transformed space. In contrast, by keeping some original features, feature selection maintains physical meanings of original features, and gives models better readability and interpretability. Therefore, feature selection is often preferred in many realworld applications such as text mining and genetic analysis compared to feature extraction.
- QUOTE: Dimensionality reduction is one of the most powerful tools to address the previously described issues. It can be categorized mainly into into two main components: feature extraction and feature selection. Feature extraction projects original high dimensional feature space to a new feature space with low dimensionality. The new constructed feature space is usually a linear or nonlinear combination of the original feature space. Examples of feature extraction methods include Principle Component Analysis (PCA) (Jolliffe, 2002), Linear Discriminant Analysis (LDA) (Scholkopft and Mullert, 1999), Canonical Correlation Analysis (CCA) (Hardoon et al., 2004), Singular Value Decomposition (Golub and Van Loan, 2012), ISOMAP (Tenenbaum et al., 2000) and Locally Linear Embedding (LLE) (Roweis and Saul, 2000). Feature selection, on the other hand, directly selects a subset of relevant features for the use model construction. Lasso (Tibshirani, 1996), Information Gain (Cover and Thomas, 2012), Relief (Kira and Rendell, 1992a), MRMR (Peng et al., 2005), Fisher Score (Duda et al., 2012), Laplacian Score (He et al., 2005), and SPEC (Zhao and Liu, 2007) are some of the well known feature selection techniques.
2014
- (Wikipedia, 2014) ⇒ http://en.wikipedia.org/wiki/Dimensionality_reduction#Dimension_reduction Retrieved:2014-5-17.
- For high-dimensional dataset s (i.e. with number of dimensions more than 10), dimension reduction is usually performed prior to applying a K-nearest neighbors algorithm(k-NN) in order to avoid the effects of the curse of dimensionality.
[1] Feature extraction and dimension reduction can be combined in one step using principal component analysis (PCA), linear discriminant analysis (LDA), or canonical correlation analysis (CCA) techniques as a pre-processing step followed by clustering by K-NN on feature vectors in reduced-dimension space. In machine learning this process is also called low-dimensional embedding. For very-high-dimensional datasets (e.g. when performing similarity search on live video streams, DNA data or high-dimensional Time series) running a fast approximate K-NN search using locality sensitive hashing, "random projections", "sketches" [2] or other high-dimensional similarity search techniques from the VLDB toolbox might be the only feasible option.
- For high-dimensional dataset s (i.e. with number of dimensions more than 10), dimension reduction is usually performed prior to applying a K-nearest neighbors algorithm(k-NN) in order to avoid the effects of the curse of dimensionality.
- ↑ Kevin Beyer , Jonathan Goldstein , Raghu Ramakrishnan , Uri Shaft (1999) "When is “nearest neighbor” meaningful?". Database Theory — ICDT99, 217-235
- ↑ Shasha, D High (2004) Performance Discovery in Time Series Berlin: Springer. ISBN 0-387-00857-8
2012
- http://en.wikipedia.org/wiki/Dimensionality_reduction#Feature_extraction
- Feature extraction transforms the data in the high-dimensional space to a space of fewer dimensions. The data transformation may be linear, as in principal component analysis (PCA), but many nonlinear dimensionality reduction techniques also exist.
2006a
- (Samet, 2006) ⇒ H. Samet. (2006). “Foundations of Multidimensional and Metric Data Structures." Morgan Kaufmann. ISBN:0123694469
2006b
- (Hinton & Salakhutdinov, 2006) ⇒ Geoffrey E. Hinton, and Ruslan R. Salakhutdinov. (2006). “Reducing the Dimensionality of Data with Neural Networks.” In: Science, 313(5786). doi:10.1126/science.1127647
- QUOTE: Dimensionality reduction facilitates the classification, visualization, communication, and storage of high-dimensional data.
2001
- (Ding et al., 2002) ⇒ C. Ding, X. He , H. Zha , H.D. Simon. (2002). “Adaptive Dimension Reduction for Clustering High Dimensional Data.” In: Proceedings of International Conference on Data Mining.