Learning Algorithm

A Learning Algorithm is an algorithm of a learning method that can be implemented into a learning system to solve a learning task (to improve task performance on some learning task over multiple algorithm runs).

• AKA: Induction Algorithm, Formal Learning Process, Data-Driven Algorithm, Data-Centric Algorithm.
• Context:
  • It can typically perform Pattern Recognition through statistical analysis.
  • It can typically handle Model Training through iterative optimization.
  • It can typically manage Error Minimization through loss function optimization.
  • It can typically support Feature Extraction through data transformation.
  • It can typically implement Performance Measurement through validation metrics.
  • ...
  • It can often process Noisy Data through noise-tolerant search.
  • It can often handle Continuous Attributes through attribute processing.
  • It can often support Search Space Pruning through optimization rules.
  • It can often maintain Model Compactness through rule set optimization.
  • ...
  • It can range from being a Small Training Set Learning Algorithm to being a Large Training Set Learning Algorithm, depending on its data requirements.
  • It can range from being a Batch Learning Algorithm to being an Online Learning Algorithm, depending on its update method.
  • It can range from being a Black-Box Learning Algorithm to being a Symbolic Learning Algorithm, depending on its interpretability.
  • ...
  • It can integrate with Tree Kernel Function for structural learning.
  • It can connect to Schema Matching System for ontology mapping.
  • It can support Word Similarity Computation for semantic analysis.
  • ...
• Examples:
  • Core Learning Algorithms, such as:
    • Probabilistic Learning Algorithms, such as:
      • Parametric Statistical Modeling Algorithm for distribution learning.
      • Log-Likelihood Ratio-based Algorithm for statistical inference.
    • Logic-Based Learnings, such as:
      • Inductive Logic Algorithm for rule learning.
      • Markov Logic Learning Algorithm for probabilistic logic.
  • Task-Specific Algorithms, such as:
    • Natural Language Processings, such as:
      • Word Similarity Learning Algorithm for semantic relationships.
      • Transformation-based Learning Algorithm for language tagging.
    • Graph-Based Learnings, such as:
      • Graph Learning Algorithm for network analysis.
      • Tree Learning Algorithm for hierarchical learning.
  • Strategy Learning Algorithms, such as:
    • Feedback-Based Learning Algorithms, such as:
      • Rocchio Relevance Feedback Algorithm for information retrieval.
      • Regret Minimization Learning Algorithm for strategy optimization.
    • Ensemble-based Learning Algorithms, such as:
      • Ensemble-based Learning Algorithm for model combination.
      • Association Rule Learning Algorithm for pattern mining.
  • ...
• Counter-Examples:
  • Deductive Inference Algorithm, which uses explicit rules.
  • Rule-Based System, which lacks learning capability.
  • Static Lookup System, which uses fixed mappings.
• See: Learning Method, Learning Theory, Function Approximation Algorithm, Testing Corpus, P-Value, Conditional Likelihood Estimation, Synonym Relation Learning Algorithm.

---

---

References

2009

• http://en.wikipedia.org/wiki/Learning
  • Learning is acquiring new knowledge, behaviors, skills, values, preferences or understanding, and may involve synthesizing different types of information. The ability to learn is possessed by humans, animals and some machines. Progress over time tends to follow learning curves.
  • Human learning may occur as part of education or personal development. It may be goal-oriented and may be aided by motivation. The study of how learning occurs is part of neuropsychology, educational psychology, learning theory, and pedagogy.
  • Learning may occur as a result of habituation or classical conditioning, seen in many animal species, or as a result of more complex activities such as play, seen only in relatively intelligent animals

2006

• (Mitchell, 2006) ⇒ Tom M. Mitchell. (2006). “The Discipline of Machine Learning." Machine Learning Department technical report CMU-ML-06-108, Carnegie Mellon University.
  • A scientific field is best defined by the central question it studies. The field of Machine Learning seeks to answer the question “How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?”

1998

• (Kohavi & Provost, 1998) ⇒ Ron Kohavi, and Foster Provost. (1998). “Glossary of Terms.” In: Machine Leanring 30(2-3).
  • Inducer / induction algorithm: An algorithm that takes as input specific instances and produces a model that generalizes beyond these instances.

---