Feature Generation Algorithm

A Feature Generation Algorithm is a data processing algorithm that can be implemented by a feature generation system to solve feature generation task (sot create new ML features).

• AKA: Feature Extraction Algorithm.
• Context:
  • It can support ML Predictive Quality Improvements and Data Preprocessing Processing Improvements.
  • It can involve techniques such as Feature Extraction, Feature Selection, and Dimensionality Reduction.
  • It can be critical in domains where raw data is complex and high-dimensional.
  • It can use domain knowledge to create more relevant features to specific Machine Learning Tasks.
  • It can range from being a Heuristic Feature Creation Algorithm to being a Data-Driven Feature Creation Algorithm.
  • It can range from being a Low-Level Feature Creation Algorithm to being a High-Level Feature Creation Algorithm.
  • ...
• Example(s):
  • A Text Classification Feature Generation Algorithm, that extracts n-grams or sentiment scores.
  • In finance, it might generate features like moving averages or volatility measures from stock price data.
  • In image recognition, algorithms could generate features by identifying edges, textures, or color histograms in images.
  • ...
• Counter-Example(s):
  • a Feature Normalization Algorithm.
  • a Data Cleaning Algorithm.
  • a Feature Dimensionality-Reduction Algorithm, such as a feature selection algorithm.
• See: Feature Engineering, Machine Learning Pipeline, Data Preprocessing, Feature Weighting Algorithm, Text Item Feature.

---

---

References

2003

• (Torkkola, 2003) ⇒ Kari Torkkola. (2003). “Feature Extraction by Non Parametric Mutual Information Maximization.” In: The Journal of Machine Learning Research, 3.
  • QUOTE: We present a method for learning discriminative feature transforms using as criterion the mutual information between class labels and transformed features. Instead of a commonly used mutual information measure based on Kullback-Leibler divergence, we use a quadratic divergence measure, which allows us to make an efficient non-parametric implementation and requires no prior assumptions about class densities.

---