Explainable AI (XAI): Where It Stands Today?

Stavros Theocharis / November 30, 2022 / Explainable AI

A quick review of its current phase

Introduction

Artificial Intelligence is quickly becoming an essential component of businesses, academia, and society, and Explainable Artificial Intelligence (XAI) is the branch of AI that explains the reasoning behind the intelligent systems’ predictions, suggestions, and conclusions. If humans want to make sure that artificial intelligence is used responsibly to benefit academia and the public interest, they should embrace and push for XAI.

There are several definitional issues with XAI. Some definitions are very general, while others are overly specific to certain technical notions. The foundations of this topic are still being researched, frequently from diverse academic viewpoints, since there is not yet a specific theory with a rigorous and widely recognized description of what explanations are. In critical algorithm studies, artificial intelligence is seen as a socio-technical-informational system. This means that any description of XAI must include not only the methods, which are essential, but also the environment in which they are used (Huvila H., 2021).

The scope and depth of the XAI field are both captured by the US Defense Advanced Research Projects Agency’s “DARPA” (Gunning D. & Aha D. W., 2019). XAI is needed so that AI systems can do the following: Help human users understand, trust, and manage the growing number of artificially intelligent systems. Actually, it should do the following:

Produce transparency, trust, and comprehension.
Make sure everything is in line with regulations and legislation.
Avoid potential danger.
Provide justifications that are responsible and credible.
Reduce the effects of bias, unfairness, and misunderstanding on the accuracy of the models, and know how to interpret them.
Verify the accuracy of XAI’s models and explanations.

Basic XAI taxonomies

There is a difference between explicitly training explainable models and explaining an unclear model after it has been trained. The first model category is referred to as a transparent model or ante-hoc explainability, whereas the second one is referred to as a post-hoc explainability model (Speith T., 2022). Linear regression, decision trees, knn, etc. are all examples of well-known transparent models. All of these models perform well across a wide range of tasks and data sets. However, such models don’t always work well in practice. The concept of a performance-explainability trade-off is relevant here (Arrieta A. B. et al., 2020). If any of these simpler models fails to provide satisfactory results, a more complex model should be used instead. These sophisticated, opaque models are known as “black boxes” due to their lack of explanation.

There are explainability approaches available to help shed light on the intricate models that have previously been trained. At this level, there is a further separation between model-agnostic techniques and model-specific ones. Another distinction is whether a model is explained locally (for a specific instance) or globally (for the whole model). This final differentiation is applicable to the transparent models discussed before as well (in the case that one of them has many parameters and tends to be a complex one).

The three types of evidence — proofs, validations, and authorizations — provide another simpler technique to classify XAI. A proof is an explanation that can be verified, traced, and shown to be true without any room for doubt by using evidence such as logical arguments, causal chains, or open procedures. Rules, decision trees, and linear regressions are all examples of “inherently interpretable” strategies used in AI systems. The authenticity of an artificial intelligence system may be validated through the use of explanations. These verifications are accomplished by using testing methodologies, repeatability, approximations, and abstractions, as well as reasons. The mechanisms through which other parties grant some type of standard, ratification, restriction, or audit constitute the basis for authorizations, which are also known as explanations. Authorizations might refer to the AI model itself, its behavior in certain scenarios, or even the method through which the AI was developed. Third parties in the public and commercial sectors, as well as professional groups, etc., may all be potential providers.

Taxonomies may be broken down into many categories, which are the “functioning-based approach,” the “result-based approach”, the “conceptual method,” and the “mixed approach”. All of the aforementioned may be further broken down into “local perturbations”, “leveraging structure” “meta-explanations”, “architecture modification”, “examples”, “feature significance”, and “surrogate models”, amongst other categories (another dedicated article will probably follow, providing more information about these categories).

XAI STRATEGY

Ridley M. (2022) supports the idea that the three main parts to be answered are about the characteristics of an effective explanation, who exactly this explanation applies to, and in what manner the explanation will be presented. The explanations given are dependent on the surrounding circumstances. As with almost all software applications, the end user’s needs and goals, as well as the XAI, are used to judge the quality of the explanation.

Generally speaking, most individuals don’t need exhaustive or even 100% correct explanations. As a consequence, who the explanation is for is important to a successful explanation. Priorities vary depending on who you’re talking to. While customers are more concerned with the system’s efficacy or effectiveness, professionals are more worried about the data’s reliability, and regulators are more curious about its policy ramifications. On the other hand, the explanations of these metrics’ performance are of primary importance to the system creators. The majority of users demand explanations that establish trust and give responsibility.

The presentation also matters if a clear explanation needs to be provided. Explanations may be provided or made available in real-time and continuously as the process unfolds, or at a post-hoc level. Interactive explanations are often favored but are not always relevant or actionable.

Institutions, businesses, governments, etc. may implement XAI that can give proofs or validations to evaluate the effectiveness of artificial intelligence in their systems, services, and collections. The important parts that have to be taken into account are the proofs (in interpretable systems), audits, approximations and abstractions, reproducibility, etc.

These methods may require knowledge of or access to the underlying machine learning model. Examples of such factors would include the algorithms used, their parameter and hyperparameter values, the optimization methods selected, and the training data. Designers of machine learning systems in highly consequential contexts should anticipate providing, and indeed be obligated to offer, access to all of these, despite the fact that they are not always readily accessible. Suppliers of library materials or systems based on machine learning should also make proofs and validations of their work available for review.

Post-hoc XAI methods

Some of the known XAI post-hoc methods are:

· Partial Dependence Plot (PDP)

· Individual Conditional Expectation (ICE)

· Accumulated Local Effects (ALE) Plot

· Permutation Feature Importance (PFI)

· Global Surrogate

· Local Surrogate (LIME)

· Scoped Rules (Anchors)

· Rule based (LORE)

· Shapley Values

· SHAP (Shapley Additive exPlanations)

· Counterfactual explanations (FACE, DiCE)

· Model distillation (LGAE)

· Summaries of Counterfactuals (AReS)

· Layer-wise Relevance Propagation (LRP)

· Gradient-weighted Class Activation Mapping (GradCAM)

Human-centered AI

The difficulty of explaining AI is fundamentally a human-centered issue. Expert system researchers have already struggled with this issue. Nonetheless, the resurgence of interest in explanations is the result of two processes happening at the same time. Firstly, the recognition of the increasing complexity and interdependence of modern AI products, and secondly, the growing recognition of the crucial roles they play as arbiters in numerous facets of human life. The current social science ideas on how individuals explain, interpret, and construct shared meaning via interaction need to form the basis for XAI procedures. In order to comprehend the social and technological needs of human-AI interaction, some researchers also discuss how to systematically apply Human-centered design (HCD) principles to the XAI development and design processes (Neerincx M. A. et al., 2019).

Real-world industrial AI initiatives include a wide variety of actors working together. Every member of the team has their own degree of technical literacy, their own specific background, and their own level of comfort with the fundamental foundations of complicated technologies like AI. The HCD process validates components such as explaining what, when, and how to humans by having them participate concurrently in the development layer. This may be accomplished via the use of surveys, interviews, and other methods.

With the support of these collaborative design methodologies, researchers and developers may be able to better appreciate the values of users as well as the social and functional context of human-AI interactions. After the various needs have been discovered, they may be broken down into their respective partial explanations and then reviewed. After completing this stage, the end users will be able to verify these explanations in a more simplified environment. This might be done in order to conduct a more in-depth analysis of the requirements and demands that have been identified.

It is important to mention that the domain plays a crucial role. Sometimes different domains may need different kinds of explanations at the end. Schoonderwoerd T. et al. (2021) provide evidence that demonstrates the need for such an approach for decision support systems, and more especially for clinical ones. They also discuss the need for an explanation framework, provide a strategy for how to incorporate end users in the development process, and address the general concerns about the design of XAI. In addition, they highlight the necessity for an explanation framework.

Discussion

So many XAI algorithms have been created, but are they all effective? Since XAI comes with a wide variety of settings, arriving at a specific solution seems to be difficult. The solution is complex because it relies on knowing how humans take in, interpret, and use AI-provided justifications. Human-computer interaction (HCI) studies, as well as human-subject research more generally, are essential for assessing XAI in the context of use, determining its limitations, and generating human-centered solutions. Actually, there are a large number of studies that reference a better understanding of the models through the XAI as it currently stands. On the other hand, XAI does not always help to bring about the desired results.

One potential disadvantage of the current XAI methods is the fact that several empirical studies have shown little proof that adding XAI components improves performance for genuine AI-assisted user activities like judgment and decision-making (Poursabzi-Sangdeh F. et al., 2021). To investigate this discrepancy, it is necessary to assume that users’ end aim with XAI is not theoretical knowledge but rather an implementable understanding that is enough to fulfill the purpose for which they are looking for answers.

In order to bridge the gap between XAI and users, it is necessary to study user interactions with XAI in contexts of use. Additionally, it is necessary to operationalize human-centered perspectives in XAI algorithms, including the development of evaluation methods that better account for the actual user’s needs in various contexts. Furthermore, everyone that is involved in such processes should be aware of what an XAI algorithm does or could do. Sometimes many teams may have a false understanding and may overestimate the knowledge that an XAI algorithm can actually give (Chromik M. et al., 2021).

Conclusion

There is a high significance to XAI, its many components, and the ways in which they might be applied to the problem of explaining deep neural network algorithms. Model-agnostic post-hoc explainability algorithms have attracted a lot of XAI attention because of their convenience and potential applicability. The current state of research in XAI assessment shows that the area of XAI is still developing, highlighting the need for caution in the creation and selection of XAI methodologies. In order to better design and evaluate XAI, recent enhancements in human-centered assessments have been made.

References

Arrieta A. B., Díaz-Rodríguez N., Del Ser J., Ben- netot A., Tabik S., Barbado A., García S., Gil-Lopez S., Molina D., Richard B., Chatila R., Herrera F. (2020). Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges Toward Responsible AI. Information Fusion, 58, 82–115.

Chromik M., Eiband M., Buchner F., Krüger A., Butz A. (2021). I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI. 26th International Conference on Intelligent User Interfaces, (pp. 307–317).

Gunning D. & Aha D. W. (2019). DARPA’s Explainable Artificial Intelligence Program. AI Magazine, 40(2).

Huvila H. (2021). Information Behavior and Practices Research Informing Information Systems Design. Journal of the Association for Information Science and Technology, 1–15.

Neerincx M.A., van Vught W., Henkemans O.B., Oleari E., Broekens J., Peters R., Kaptein F., Demiris Y., Kiefer B., Fumagalli D. (2019). Socio-cognitive engineering of a robotic partner for child’s diabetes self-management. Front. Robot, 1–16.

Poursabzi-Sangdeh F., Goldstein D. G., Hofman J., Wortman J., Wallach H. (n.d.). Manipulating and measuring model interpretability. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, (pp. 1–52).

Ridley M. (2022). Explainable Artificial Intelligence (XAI). Information Technology and Libraries, 41(2).

Schoonderwoerd T., Jorritsma W., Neerincx M., van den Bosch K. (2021). Human-centered XAI: Developing design patterns for explanations of clinical decision support systems. International Journal of Human-Computer Studies.

Speith T. (2022). A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods. Association for Computing Machinery.

Explainable AI

Explainable AI (XAI): Where It Stands Today?

Explainable AI (XAI): Where It Stands Today? Stavros Theocharis / November 30, 2022 / Explainable AI A quick review of its current […]

Stavros TheocharisNovember 30, 2022

AI ethics

European regulations in AI

European regulations in AI Megaklis Vasilakis / November 5, 2021 / AI ethics New European regulations and benchmarking will play a big […]

Megaklis VasilakisNovember 5, 2021

Explainable AI (XAI): Where It Stands Today?

Introduction

Basic XAI taxonomies

XAI STRATEGY

Post-hoc XAI methods

Human-centered AI

Discussion

Conclusion

References

Related articles

Explainable AI (XAI): Where It Stands Today?

European regulations in AI

Research

Company

Blog