AI Model Interpretability Measure

A AI Model Interpretability Measure is an interpretability measure for a system model, specifically assessing how easily a human can understand the model's internal mechanics or logic and its predictions. An Interpretable Predictive Model is a predictive model with a relatively high model interpretability measure value, meaning it is designed to be comprehensible to humans.

• Context:
  • It can (typically) refer to the ability to understand the internal mechanics or logic of how a machine learning model works and makes predictions.
  • It can (often) include models like linear regression, logistic regression, and decision trees which have inherent structures that make them easier for humans to reason about.
  • It can range from being a Coarse AI Model Interpretability Measure to being a Detailed AI Model Interpretability Measure.
  • It can encompass both interpretability and explainability, where interpretability focuses on understanding the AI model itself, and explainability focuses on providing explanations for the specific predictions made by the model.
  • It can highlight the trade-off between the accuracy of complex models and the interpretability of simpler ones.
  • ...
• Example(s):
  • One based on LIME (Local Interpretable Model-agnostic Explanations).
  • One based on SHAP (SHapley Additive exPlanations) (which approximates the behavior of complex models locally around a specific prediction).
  • One for a Decision Tree Model that showcases how each decision is made based on the input features.
  • One for an Additive Model that demonstrates flexibility and interpretability in non-linear relationships.
  • ...
• Counter-Example(s):
  • AGI Controllability Measure.
• See: Interpretability, Black-Box Model, Model Translation, Predictive Model Interpretation System, Interpretable Predictive Model, Black-Box Model.

---

---

References

2024

• Perplexity: Interpretability vs. Explainability:
  • Interpretability refers to the ability to understand the internal mechanics or logic of how a machine learning model works and makes predictions. An interpretable model is one where a human can comprehend the relationship between the input features and the output, as well as the relative importance or weightings assigned to each feature in the decision-making process.
    • Some examples of interpretable models include linear regression, logistic regression, and decision trees. These models have an inherent structure or form that makes it easier for humans to reason about how they arrive at their predictions. For instance, in a linear regression model, the coefficients directly represent the change in the output for a one-unit change in the corresponding input feature, all else being equal.
  • Explainability, on the other hand, focuses on providing explanations for the specific predictions or outputs made by a model, even if the internal workings of the model itself are opaque or difficult to interpret directly.
    • Explainability methods like LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations) work by approximating the behavior of the complex model locally around a specific prediction, using interpretable surrogate models or feature importance measures. These methods aim to explain why a particular input instance received the output it did, rather than fully elucidating the global logic of the entire model.
  • While interpretability is about understanding the AI model itself, decision explainability focuses on understanding the AI model's decisions. Interpretability is about understanding the AI model itself and how it transforms inputs to outputs in a way that humans can grasp intuitively. AI decision explainability, on the other hand, focuses on providing post-hoc explanations for individual predictions made by the model, even if the model's internal mechanics are complex or opaque.
  • Citations:

[1] https://datascience.aero/explainability-interpretability-what-model-need/
[2] https://www.kdnuggets.com/2018/12/machine-learning-explainability-interpretability-ai.html
[3] https://datascience.stackexchange.com/questions/99808/an-example-of-explainable-but-not-interpretable-ml-model
[4] https://blogs.sas.com/content/hiddeninsights/2022/08/10/interpretability-vs-explainability-the-black-box-of-machine-learning/
[5] https://christophm.github.io/interpretable-ml-book/
[6] https://link.springer.com/chapter/10.1007/978-3-031-04083-2_2
[7] https://docs.aws.amazon.com/whitepapers/latest/model-explainability-aws-ai-ml/interpretability-versus-explainability.html
[8] https://www.ibm.com/topics/explainable-ai
[9] https://quiq.com/blog/explainability-vs-interpretability/
[10] https://www.datacamp.com/tutorial/explainable-ai-understanding-and-trusting-machine-learning-models 
[11] https://datascience.stackexchange.com/questions/70164/what-is-the-difference-between-explainable-and-interpretable-machine-learning

2020

• (Wikipedia, 2020) ⇒ https://en.wikipedia.org/wiki/Additive_model Retrieved:2020-10-2.
  • … Furthermore, the AM is more flexible than a standard linear model, while being more interpretable than a general regression surface at the cost of approximation errors. Problems with AM include model selection, overfitting, and multicollinearity.

2017

• (Lundberg & Lee, 2017) ⇒ Scott M. Lundberg, and Su-In Lee. (2017). “A Unified Approach to Interpreting Model Predictions.” In: Proceedings of the 31st International Conference on Neural Information Processing Systems.
  • QUOTE: ... The ability to correctly interpret a prediction model’s output is extremely important. It engenders appropriate user trust, provides insight into how a model may be improved, and supports understanding of the process being modeled. In some applications, simple models (e.g., linear models) are often preferred for their ease of interpretation, even if they may be less accurate than complex ones. However, the growing availability of big data has increased the benefits of using complex models, so bringing to the forefront the trade-off between accuracy and interpretability of a model’s output. ...

2015

• (Debray et al., 2015) ⇒ Thomas P.A. Debray, Yvonne Vergouwe, Hendrik Koffijberg, Daan Nieboer, Ewout W. Steyerberg, and Karel GM Moons. (2015). “A New Framework to Enhance the Interpretation of External Validation Studies of Clinical Prediction Models.” Journal of clinical epidemiology 68, no. 3

2015

• (Shah et al., 2015) ⇒ Neil Shah, Danai Koutra, Tianmin Zou, Brian Gallagher, and Christos Faloutsos. (2015). “TimeCrunch: Interpretable Dynamic Graph Summarization.” In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2015). ISBN:978-1-4503-3664-2 doi:10.1145/2783258.2783321

2014

• (Purushotham et al., 2014) ⇒ Sanjay Purushotham, Martin Renqiang Min, C.-C. Jay Kuo, and Rachel Ostroff. (2014). “Factorized Sparse Learning Models with Interpretable High Order Feature Interactions.” In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2014) Journal. ISBN:978-1-4503-2956-9 doi:10.1145/2623330.2623747

2012

• (Vellido et al., 2012) ⇒ Alfredo Vellido, José David Martín-Guerrero, and Paulo JG Lisboa. (2012). “Making Machine Learning Models Interpretable.” In: ESANN.
  • QUOTE: Data of different levels of complexity and of ever growing diversity of characteristics are the raw materials that machine learning practitioners try to model using their wide palette of methods and tools. The obtained models are meant to be a synthetic representation of the available, observed data that captures some of their intrinsic regularities or patterns. Therefore, the use of machine learning techniques for data analysis can be understood as a problem of pattern recognition or, more informally, of knowledge discovery and data mining. There exists a gap, though, between data modeling and knowledge extraction. Models, depending on the machine learning techniques employed, can be described in diverse ways but, in order to consider that some knowledge has been achieved from their description, we must take into account the human cognitive factor that any knowledge extraction process entails. These models as such can be rendered powerless unless they can be interpreted, and the process of human interpretation follows rules that go well beyond technical prowess. For this reason, interpretability is a paramount quality that machine learning methods should aim to achieve if they are to be applied in practice. This paper is a brief introduction to the special session on interpretable models in machine learning, It includes a discussion on the several works accepted for the session, with an overview of the context of wider research on interpretability of machine learning models.

---