From the Perspective of AI Safety: Analyzing the Impact of XAI Performance on Adversarial Attack

The outstanding performance of machine learning models across various fields enables them to be used in sensitive and high-risk activities such as healthcare, automated driving, and security services. However, the explainability of their outputs and the safety of their operation are major concerns. Although Explainable AI (XAI) can enhance the interpretability of AI models, further research is necessary to evaluate its effectiveness in explaining adversarial attacks. The use of XAI techniques and the rise of adversarial attacks are important issues related to the explainability and security of deep learning, respectively. Furthermore, the relation between the explainability and safety of deep learning, such as the vulnerability of the XAI explanation to an adversarial attack, is key to unlocking these concerns. In this paper, we use the Saliency Map as an XAI technique to explain the behavior of the Fast Gradient Sign Method (FGSM) adversarial attack on the ResNet model, and show the vulnerability related to such an explanation with respect to the attack. Extensive experiments show that as the severity of FGSM attack on the ResNet model increases, the Saliency Map gradually fails, exposing its potential vulnerability.