AGI Performance Measure

An AGI Performance Measure is a AI performance measure for AGIs.

• Context:
  • It can (typically) assess the generality and performance of AGI systems across a wide range of tasks and conditions.
  • It can (often) evaluate an AGI's ability to perform cognitive and meta-cognitive tasks, including learning new skills and adapting to new environments.
  • It can measure the autonomy of AGI systems, distinguishing between various levels such as AI as a tool, consultant, collaborator, expert, and agent.
  • It can focus on real-world tasks that humans value, ensuring ecological validity in performance metrics.
  • It can involve benchmarks like the "Coffee Test," which tests an AGI's flexibility and reliability by requiring it to operate competently in an arbitrary kitchen.
  • It can combine performance and generality to provide a comprehensive framework for AGI evaluation, distinguishing between narrow AI and true AGI.
  • It can use frameworks like DeepMind's Levels of AGI, which introduce five levels of AGI performance, ranging from No AI to Superhuman, based on percentile performance compared to skilled adults.
  • It can incorporate concepts like Suleyman’s Artificial Capable Intelligence (ACI), which involves economic and strategic benchmarks such as turning capital into profit.
  • It can range from being simple task-specific tests to complex, multi-dimensional evaluations involving cognitive, physical, and strategic tasks.
  • ...
• Example(s):
  • an evaluation framework like DeepMind's Levels of AGI Framework that classifies AGI systems based on performance and generality.
  • a test like Suleyman’s Modern Turing Test, which assesses an AGI's economic and strategic capabilities in real-world scenarios.
  • a benchmark like the Coffee Test that measures an AGI's ability to perform everyday tasks in varying environments.
  • an AGI Ordinal Performance Measure ...
  • ...
• Counter-Example(s):
  • Narrow AI Benchmarks, which focus on evaluating AI systems specialized in specific tasks rather than general intelligence.
  • Traditional AI Metrics, which may not adequately capture the breadth and depth required for AGI assessment.
  • ...
• See: AGI Level.

---

---

References

2023

• (Morris et al., 2023) ⇒ Meredith Ringel Morris, Jascha Sohl-dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clement Farabet, and Shane Legg. (2023). “Levels of AGI: Operationalizing Progress on the Path to AGI.” doi:10.48550/arXiv.2311.02462
  • QUOTE: "We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI performance, generality, and autonomy, providing a common language to compare models, assess risks, and measure progress along the path to AGI."
  • NOTE: This paper provides a structured approach to classify AI systems and their progress towards AGI.

2023

• (VentureBeat, 2023) ⇒ Ben Dickson. (2023). "Here is how far we are to achieving AGI, according to DeepMind." In: VentureBeat. [URL](https://venturebeat.com/ai/here-is-how-far-we-are-to-achieving-agi-according-to-deepmind/)
  • NOTE: This article discusses various AGI definitions and the challenges in creating comprehensive and robust AGI performance measures.

2023

• (McKinsey & Company, 2023) ⇒ McKinsey & Company. (2023). "What is Artificial General Intelligence (AGI)?" In: McKinsey & Company. [URL](https://www.mckinsey.com/featured-insights/artificial-intelligence/what-is-artificial-general-intelligence-agi)
  • NOTE: This article explores various definitions and evaluation methods for AGI, including economic and task-based benchmarks.

---