Doctors’ Dilemma--Understanding the Perspective of Medical Experts on AI Explanations

As a solution for the pressing issue in medicine of “black-box” artificial intelligence (AI), models that are hard to understand, explainable AI (XAI) is gaining in popularity. XAI aims at making AI more understandable by explaining its working, e.g., through human understandable explanations. However, while prior research found that such explanations must be adapted for the given expert group being addressed, we find limited work on explanations and their effect on medical experts. To address this gap, we conducted an online experiment with such medical experts (e.g., doctors, nurses) (n=204), to investigate how explanations can be utilized to achieve a causal understanding and respective usage of AI. Our results demonstrate and contribute to literature by identifying transparency and usefulness as powerful mediators, which were not known before. Additionally, we contribute to practice by depicting how these can be used by managers to improve the adoption of AI systems in medicine.