Unsupervised Dimensionality Reduction Algorithm

An Unsupervised Dimensionality Reduction Algorithm is a dimensionality reduction algorithm (to solve a dimensionality reduction task) that is a unsupervised learning algorithm.

• Context:
  • It can be implemented into an Unsupervised Dimensionality Reduction System.
• Example(s):
  • a Latent Semantic Indexing Algorithm (Deerwester, Dumais et al., 1990).
  • a Class-based Clustering (Brown, deSouza et al., 1992).
  • a Feature Selection by Mutual Information Algorithm (Yang & Pedersen, 1997).
  • a Markov-Blanket-based Feature Selection Algorithm (Koller & Sahami, 1996).
  • …
• Counter-Example(s):
  • an Supervised Dimensionality Reduction Algorithm.
• See: Compression.

---

---

Context

1998

• (Baker & McCallum, 1998) ⇒ L. Douglas Baker, and Andrew McCallum. (1998). “Distributional Clustering of Words for Text Classification.” In: Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval. ISBN:1-58113-015-5 doi:10.1145/290941.290970
  • QUOTE: ... unlike some other unsupervised dimensionality-reduction techniques, such as Latent Semantic Indexing, we are able to compress the feature space much more aggressively, while still maintaining high document classification accuracy. ... significantly better than Latent Semantic Indexing [6], class-based clustering [1], feature selection by mutual information [23] or Markov-blanket-based feature selection [13].

1996

• (Koller & Sahami, 1996) ⇒ Daphne Koller and Mehran Sahami. (1996). “Toward Optimal Feature Selection.” In: Proceedings of the International Conference on Machine Learning (ICML 1996).
  • ABSTRACT: In this paper, we examine a method for feature subset selection based on Information Theory. Initially, a framework for defining the theoretically optimal, but computationally intractable, method for feature subset selection is presented. We show that our goal should be to eliminate a feature if it gives us little or no additional information beyond that subsumed by the remaining features. In particular, this will be the case for both irrelevant and redundant features. We then give an efficient algorithm for feature selection which computes an approximation to the optimal feature selection criterion. The conditions under which the approximate algorithm is successful are examined. Empirical results are given on a number of data sets, showing that the algorithm effectively handles datasets with large numbers of features.

1990

• (Deerwester et al., 1990) ⇒ Scott C. Deerwester, Susan T. Dumais, Thomas K. Landauer, George W. Furnas, and Richard A. Harshman. (1990). “Indexing by Latent Semantic Analysis.” In: JASIS, 41(6).

---