What do you base that conclusion on? Grounding explainable AI in human dialogue strategies

Abstract

When human decisions are assisted by predictions from artificial intelligence (AI) models, users' ability to understand the basis of AI outputs can be important for various reasons, such as assessing the reliability of specific predictions. One way to achieve such understanding is by letting AI systems explain their predictions. This thesis explores how AI systems can be designed to emulate how humans manage explanations in dialogue. The first part of the thesis studies expert explanations for medical judgements in clinical settings without AI assistance as well as lay explanations in an experimental setting with AI assistance. By applying a dialogue distillation methodology, collected human–human interactions are rewritten into analogous human–AI dialogues, thereby revealing capabilities that a conversationally explainable AI system would need to possess to emulate human explanatory behaviours. Based on the finding that human interlocutors often explain judgements argumentatively (as claims supported by premises) and enthymematically (by omitting one or more premises), the thesis proposes a method for extracting arguments from generalised linear models, a popular type of predictive model, and demonstrates that the method can be used to generate enthymematic explanations that invite inferences that correctly reflect the actual reasoning of the model. A design workshop with orthopaedic surgeons is also performed, indicating that users find the generated explanations informative and that they can correctly interpret generated explanations. In the final part of the thesis, observed human dialogue strategies are formally modelled in a novel framework for explanatory dialogue management based on information-state updates conceived as linear implications. The proposed framework accounts for many of the observed phenomena and strategies, including complex explanantia, chained inferences, forward expansions and signalling of presupposition violations and answer unavailability. Future research challenges associated with emulating observed human explanation strategies are also identified and discussed.