Multi-Agent Learning (MAL) Algorithm

Context:
- It can range from being a simple Multi-Agent Learning Algorithm to being a Multi-Agent Reinforcent Learning Algorithm.
- It can range from being a Cooperative Multi-Agent Learning Algorithm to being a Competitive Multi-Agent Learning Algorithm.
Example(s):
Counter-Example(s):
- Single-Agent Learning Algorithm.
See: Game Theory, Nash Equilibrium, Machine Learning System, Q-Learning, Reinforcent Learning, Multi-Agent Learning Testbed (MALT).

References

(Shoham & Powers, 2017) ⇒ Yoav Shoham, and Rob Powers (2017). “Multi-Agent Learning Algorithms”. In: (Sammut & Webb, 2017)
- QUOTE: Multi-agent learning (MAL) refers to settings in which multiple agents learn simultaneously. Usually defined in a game theoretic setting, specifically in repeated games or stochastic games, the key feature that distinguishes MAL from single-agent learning is that in the former the learning of one agent impacts the learning of others. As a result, neither the problem definition for multi-agent learning nor the algorithms offered follow in a straightforward way from the single-agent case. In this second of two entries on the subject, we focus on algorithms.

(Zawadzki et al., 2014) ⇒ Erik Zawadzki, Asher Lipson, and Kevin Leyton-Brown. (2014). “Empirically Evaluating Multiagent Learning Algorithms.” arXiv:1401.8074
- QUOTE: We have developed a new suite of tools for running multiagent experiments: the MultiAgent Learning Testbed (MALT). These tools are designed to facilitate larger and more comprehensive experiments by removing the need to build one-off experimental code. MALT also provides baseline implementations of many MAL algorithms, howefully eliminating or reducing differences between algorithm implementations and increasing the reproducibility of results. Using this test suite, we ran an experiment unprecedented in size. We analyzed the results according to a variety of performance metrics including reward, maximn distance, regret, and several notions of equilibrium convergence. We confirmed several pieces of conventional wisdom, but also discovered some surprising results. For example, we found that single-agent Q-learning outperformed many more complicated and more modern MAL algorithms.

↑ This property has sometimes been called rationality (Bowling & Veloso, 2002), but we avoid that term because it has an established, different meaning in economics.